WO2024040973A1 - 一种基于堆叠沙漏网络的多尺度融合去雾方法 - Google Patents

一种基于堆叠沙漏网络的多尺度融合去雾方法 Download PDF

Info

Publication number
WO2024040973A1
WO2024040973A1 PCT/CN2023/086215 CN2023086215W WO2024040973A1 WO 2024040973 A1 WO2024040973 A1 WO 2024040973A1 CN 2023086215 W CN2023086215 W CN 2023086215W WO 2024040973 A1 WO2024040973 A1 WO 2024040973A1
Authority
WO
WIPO (PCT)
Prior art keywords
layer
module
row
hourglass
convolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/086215
Other languages
English (en)
French (fr)
Inventor
张登银
赵乾
王敬余
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Posts and Telecommunications
Original Assignee
Nanjing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Posts and Telecommunications filed Critical Nanjing University of Posts and Telecommunications
Priority to US18/312,168 priority Critical patent/US12423786B2/en
Publication of WO2024040973A1 publication Critical patent/WO2024040973A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Definitions

  • the invention relates to a multi-scale fusion defogging method based on a stacked hourglass network, and belongs to the technical field of image processing.
  • Vision is the most intuitive way for humans to obtain information. With the development of artificial intelligence, computer vision is widely used in various fields of our lives. However, due to air pollution, the clarity of the collected photos will be affected, with reduced contrast, blurred images, and a serious lack of extractable features. As the main source of information for people, the quality of images and videos seriously affects the reading and judgment of information, and high-level image processing also has high requirements for the quality of input images. Therefore, studying high-quality, fast, and universal defogging principles and methods has high theoretical significance and application value.
  • the purpose of image defogging is to eliminate the impact of the haze environment on image quality and increase the visibility of the image.
  • the methods can be divided into three main types. One is based on the image enhancement algorithm, which enhances the degraded image. Improve the quality of the image and highlight the features and valuable information of the scene in the image. However, this method does not consider the causes of image degradation. After processing, it may cause the loss of part of the image information and cause distortion.
  • the second method is based on the atmospheric degradation model, which uses the prior knowledge of the haze-free image to estimate the parameters in the model, and then substitutes the parameters into the model to restore the haze-free image.
  • the haze-free image processed by this method is clearer and clearer.
  • the third method is based on deep learning, which estimates the transmittance through the training data set, or uses the input hazy image to directly output the dehazed image.
  • the latest dehazing method currently prefers the latter, but this method has limitations such as the need to estimate parameters and the low dehazing efficiency and poor quality caused by many estimated parameters and insufficient features.
  • the present invention provides a multi-scale fusion defogging method based on stacked hourglass networks, which completes learning tasks by constructing and combining multiple networks.
  • the stacked hourglass network is used to implement feature extraction, and then the multi-scale module is used for feature fusion to obtain significantly superior generalization performance than a single network.
  • a double-layer attention module is used to improve the convolutional feature expression ability of the mobile network.
  • a multi-scale fusion dehazing method based on stacked hourglass network including the following steps:
  • the image dehazing network is a 7 ⁇ 7 convolutional layer, a stacked hourglass module, a feature fusion, a multi-scale skip connection module, a 1 ⁇ 1 convolutional layer, a 3 ⁇ 3 convolutional layer, and Hierarchical attention distillation module, a 3 ⁇ 3 convolutional layer and a 1 ⁇ 1 convolutional layer.
  • the stacked hourglass module is composed of N fourth-order hourglass modules connected in series.
  • the fourth-order hourglass module contains five parallel convolution streams: the innermost convolution stream processes the original scale, the penultimate layer to the outermost layer.
  • the layer convolution streams are down-sampled to 1/2, 1/4, 1/8 and 1/16 respectively.
  • the five convolution streams perform feature extraction in different resolution groups, and then pass the features of each resolution through the residual module. Features are finally restored to the original scale through the upsampling operation layer and fused.
  • the fourth-order hourglass module replaces the middle residual module in the fourth row of the third-order hourglass module with a first-order hourglass module.
  • the third-order hourglass module replaces the middle residual module in the third row of the second-order hourglass module with a first-order hourglass module.
  • the second-order hourglass module replaces the residual module in the middle of the second row of the first-order hourglass module with a first-order hourglass module.
  • the first-order hourglass module consists of two rows.
  • the first row contains a residual module.
  • the second row consists of a max pooling layer, three residual modules and an upsampling operation layer in sequence.
  • the first row and the second row are output after feature fusion.
  • the residual module consists of two rows.
  • the first row is a skip layer, including a 1 ⁇ 1 convolution layer.
  • the second row is the convolution layer, which is the BN layer, Relu layer, 1 ⁇ 1 convolution layer, BN layer, Relu layer, 3 ⁇ 3 convolution layer, BN layer, Relu layer and 1 ⁇ 1 convolution. layer.
  • the output ends of the skip layer and convolution layer perform feature fusion and output.
  • N is set to 8.
  • the multi-scale skip connection module includes: the first row consists of three 3 ⁇ 3 convolution layers and Relu layers connected in series, and the second row consists of three 5 ⁇ 5 convolution layers and Relu layers.
  • the third row is composed of three 7 ⁇ 7 convolutional layers and Relu layers connected in series.
  • the output of the first 3 ⁇ 3 convolutional layer and Relu layer of each row is used as the input of the second 3 ⁇ 3 convolutional layer and Relu layer of each row respectively, and the second 3 ⁇ 3 of each row is
  • the outputs of the convolutional layer and the Relu layer are used as the inputs of the third 3 ⁇ 3 convolutional layer and Relu layer of each row respectively, and the outputs of the third 3 ⁇ 3 convolutional layer and Relu layer of each row are passed through Contact Output after fusion.
  • the hierarchical attention distillation module includes a channel attention module, a spatial attention module, a channel attention module, and the output of the spatial attention module is output after fusion.
  • the channel attention module passes the input feature map F (H ⁇ W ⁇ C, H represents height, W represents width, and C represents the number of channels) through the global maximum pooling layer of H dimension and the W dimension.
  • the global average pooling layer obtains two 1 ⁇ 1 ⁇ C feature maps; the two feature maps are sent to a two-layer neural network with shared weights for channelization.
  • the features output by the MLP are added and fused, and then the final channel weighted M is generated through the sigmoid activation operation.
  • the spatial attention module passes the input feature map F (H ⁇ W ⁇ C, H: height, W: width, C: number of channels) through the maximum pooling layer and average pooling based on the C dimension respectively. layer to obtain two H ⁇ W ⁇ 1 feature maps; the two H ⁇ W ⁇ 1 feature maps are spliced based on the channel dimension, and the spliced feature map is then used for channel dimensionality reduction using a 7 ⁇ 7 convolution layer. Finally, through the sigmoid activation operation, the weight M of the spatial dimension is generated.
  • F H ⁇ W ⁇ C, H: height, W: width, C: number of channels
  • the present invention provides a multi-scale fusion defogging method based on stacked hourglass networks to solve the problems faced by existing image defogging methods based on deep learning that require estimated parameters and that there are many estimated parameters and insufficient features. Problems such as low defogging efficiency and poor quality.
  • the present invention belongs to end-to-end defogging.
  • fog-free images can be directly output.
  • a hierarchical attention distillation module is introduced to optimize the network structure and retain spatial and contextual information. and extract more useful hierarchical features. It can make significant progress in fully utilizing multi-scale fog features and restoring structural details, improving the quality of generated images.
  • This invention first uses a stacked hourglass network to repeatedly capture features at various scales from bottom to top and top down, then uses a multi-scale skip connection method to repeatedly fuse information, and finally combines a double-layer attention mechanism to avoid features. Disappear and remove unnecessary features to achieve comprehensive feature aggregation and improve defogging performance.
  • the present invention can be applied to various computer vision systems, such as image recognition, video surveillance, industrial visual inspection, etc., which can reduce a large amount of labor costs and greatly improve image quality and service efficiency. The purpose is to ensure better customer service and make the final The dehazing results can meet the requirements of high-level image processing and comply with human visual requirements.
  • Figure 1 is an overall flow chart of the implementation of the method of the present invention.
  • Figure 2 is a schematic diagram of the network architecture of the fourth-order hourglass module sampling rule.
  • Figure 3 is a schematic diagram of the network architecture of the residual module.
  • Figure 4 is a schematic diagram of the network architecture of the first-order hourglass module.
  • Figure 5 is a schematic diagram of the network architecture of the fourth-order hourglass module.
  • Figure 6 is a schematic diagram of the network architecture of the multi-scale skip connection module.
  • Figure 7 is a schematic diagram of the network architecture of the hierarchical attention distillation module.
  • Figure 8 is the implementation flow chart of the channel attention module.
  • Figure 9 is the implementation flow chart of the spatial attention module.
  • the present invention discloses a multi-scale fusion defogging method based on a stacked hourglass network.
  • a hazy image When a hazy image is input, it enters the image defogging network and directly outputs a clear image after defogging.
  • the image dehazing network consists of a 7 ⁇ 7 convolution layer, stacked hourglass module, feature fusion, multi-scale skip connection module, a 1 ⁇ 1 convolution layer, a 3 ⁇ 3 convolution layer, and hierarchical attention. Distillation module, a 3 ⁇ 3 convolutional layer and a 1 ⁇ 1 convolutional layer.
  • the 7 ⁇ 7 convolution layer is used to process the original hazy image in the first step to form the initial feature image.
  • Feature fusion is set to the addition operation, which is used for feature addition.
  • the 1 ⁇ 1 convolution layer after the multi-scale skip connection module is used to adjust the number of channels, adjust the number of channels that change after contact, and obtain low-frequency feature information.
  • the 3 ⁇ 3 convolutional layer after the multi-scale skip connection module is used to obtain high-frequency feature information.
  • the 3 ⁇ 3 convolutional layer and the 1 ⁇ 1 convolutional layer after the hierarchical attention distillation module are used to modify or assist features.
  • the stacked hourglass module is composed of N fourth-order hourglass modules connected in series.
  • N 4, 6, 8, and 10 are preferred
  • PSNR Peak Signal-to-Noise Ratio
  • SSIM Mechanism Similarity
  • the fourth-order hourglass module integrates closely connected residual block modules, maximum pooling layers, upsampling operation layers and residual fusion through a parallel structure to form a symmetric topology.
  • Each scale is first started from the top And below, there is a corresponding bottom-up layer to achieve the purpose of repeating top-down and bottom-up feature extraction.
  • the size of the box represents the size of the feature map, smaller squares represent downsampling, larger squares represent upsampling, and the plus sign represents element-wise addition.
  • the fourth-order hourglass module contains five parallel convolution streams: the innermost convolution stream processes the original scale, and the penultimate layer to the outermost convolution stream downsamples to 1/2, 1/4, and 1/4 respectively. 1/8 and 1/16, five convolution streams perform feature extraction in different resolution groups, and then pass the features of each resolution through the residual module, and finally restore to the original scale and fuse through the upsampling operation layer. That is, features of different resolutions are added according to element positions, so feature information can be extracted and retained at multiple scales, achieving the effect of retaining local features and global features at the same time.
  • the residual module is the basic component unit of the first-order hourglass module.
  • the specific network architecture is shown in Figure 3.
  • the residual module consists of two lines.
  • the first line is the skip layer, including a 1 ⁇ 1 convolution layer (Conv), which is used to retain the information of the original level.
  • the second row is the convolutional layer, used to extract features, which are BN layer, Relu layer, 1 ⁇ 1 convolutional layer, BN layer, Relu layer, 3 ⁇ 3 convolutional layer, BN layer, Relu layer and 1 ⁇ 1 convolutional layer.
  • the output ends of the skip layer and convolution layer perform feature fusion and output.
  • the BN layer In the second row of convolutional layers, the BN layer first normalizes the signal, and then adds non-linearity to the main path through the Relu layer. Linear, and then go through the 1 ⁇ 1 convolution layer to reduce the dimension, and after the dimensionality reduction, data training and feature extraction can be performed more effectively and intuitively, and then go through the BN layer and the Relu layer again, and then go through the 3 ⁇ 3
  • the convolution layer performs relatively low-dimensional calculations to increase the depth of the network and improve efficiency. Then it passes through the BN layer and the Relu layer for the third time, and then passes through the 1 ⁇ 1 convolution layer again to increase the dimension. Finally, with the skip layer Perform feature fusion without changing the data size and only increase the data depth.
  • the first-order hourglass module consists of two rows.
  • the specific network architecture is shown in Figure 4.
  • the first line only contains a residual module, and the second line is sequentially used by a maximum pooling layer (Max Pool) to downsample the feature map to obtain a feature map with a lower resolution and reduce the computational complexity.
  • Max Pool maximum pooling layer
  • Up Sample upsampling operation layer
  • the first and second lines perform feature fusion and output, so that the output result includes both the features of the original resolution and the features whose resolution is reduced to 1/2 after downsampling.
  • the second-order hourglass module replaces the residual module in the middle of the second row of the first-order hourglass module with a first-order hourglass module.
  • the third-order hourglass module replaces the residual module in the middle of the third row of the second-order hourglass module with a first-order hourglass module.
  • the fourth-order hourglass module The module is to replace the middle residual module in the fourth row of the third-order hourglass module with the first-order hourglass module, and so on, forming a recursive structure, as shown in Figure 5.
  • the multi-scale skip connection module includes: convolution operations of three different sizes of convolution kernels and activation functions.
  • the first row consists of three 3 ⁇ 3 convolution layers and Relu layers connected in series.
  • the second row is composed of three 5 ⁇ 5 convolutional layers and Relu layers connected in series, and the third row is composed of three 7 ⁇ 7 convolutional layers and Relu layers connected in series.
  • the output of the first 3 ⁇ 3 convolutional layer and Relu layer of each row is used as the input of the second 3 ⁇ 3 convolutional layer and Relu layer of each row respectively, and the second 3 ⁇ 3 of each row is The outputs of the convolutional layer and the Relu layer are used as the inputs of the third 3 ⁇ 3 convolutional layer and Relu layer of each row respectively, and the outputs of the third 3 ⁇ 3 convolutional layer and Relu layer of each row are passed through Contact Fusion.
  • the convolution operation uses zero padding.
  • the activation function is introduced to perform nonlinear operations on the output results of the convolution layer, so that the convolutional neural network can gain the ability to solve complex problems and at the same time improve the robustness of the convolutional neural network to nonlinear factors.
  • the leaky linear rectification unit Leaky ReLU is used. Its function image is a linear function in the segmented interval, a nonlinear function as a whole, and the value range is the entire real number set, which can improve the network convergence speed.
  • the present invention has some innovations in the connection method. Instead of simply performing parallel convolution operations on three groups of convolution kernels of different sizes, it adopts a skip connection method to combine a 3 ⁇ 3 convolution layer in the current row with In addition to being output to the next 3 ⁇ 3 convolution layer and Relu layer in series, the output result of the Relu layer is also output to the next 3 ⁇ 3 convolution layer and Relu layer in the other two rows, so the next 3 in each row.
  • the inputs of the ⁇ 3 convolutional layer and the Relu layer are the outputs of the previous 3 ⁇ 3 convolutional layer and the different-sized convolution kernels of the Relu layer, respectively, to achieve multi-scale information fusion.
  • 3 feature maps are obtained.
  • the three feature maps output by the third 3 ⁇ 3 convolution layer and the Relu layer are fused through Contact, that is, the channel numbers of the three feature maps are added, and each The information under one channel is not added, and the previously obtained features are combined by increasing the number of channels, and the features extracted by convolution kernels of different scales are retained to achieve better performance.
  • F in is the original image input to the multi-scale skip connection module
  • Conv n ⁇ n ( ⁇ ) is the convolution operation
  • ⁇ a n ⁇ n represents the first multi-scale convolution formation with a convolution kernel size of n ⁇ n hyperparameters.
  • the hierarchical attention distillation module is composed of a spatial attention module and a channel attention module in parallel. Its structure is shown in Figure 7. It uses dual attention units to learn the importance of space and the importance of channels respectively, and adds the output results, which retains The space retains contextual information, and reduces unnecessary features by using hierarchical attention fusion, allowing only information features to be further transferred, eliminating redundant features, achieving feature distillation, and being easily embedded into the framework.
  • the structure of the channel attention module is shown in Figure 8.
  • the specific operation is as follows: First, the input feature map F (H ⁇ W ⁇ C, H represents height, W represents width, and C represents the number of channels) is processed based on H and W uses a two-dimensional global maximum pooling layer (MaxPool) and a global average pooling layer (AvgPool) to obtain two 1 ⁇ 1 ⁇ C feature maps; then the two feature maps are fed into a double layer with shared weights.
  • the neural network (MLP) learns the dependencies between channels, and the dimensionality reduction is achieved through the compression ratio r between the two neural layers.
  • the features output by MLP are added and fused, and then through the sigmoid activation operation, the final channel weighted M is generated to mine the relationship between feature context information and hierarchical features. relation.
  • the calculation formula of the channel attention module is:
  • M(F) ⁇ (MLP(AvgPool(F))+MLP(MaxPool(F))), where ⁇ represents sigmoid.
  • the structure of the spatial attention module is shown in Figure 9.
  • the specific operation is as follows: First, the input feature map F (H ⁇ W ⁇ C, H: height, W: width, C: number of channels) is processed based on the C dimension
  • the maximum pooling layer (MaxPool) and the average pooling layer (AvgPool) are used to obtain two H ⁇ W ⁇ 1 feature maps; then the two H ⁇ W ⁇ 1 feature maps are spliced based on the channel dimension, and the spliced
  • the feature map uses a 7 ⁇ 7 convolution layer for channel dimensionality reduction.
  • the weight M of the spatial dimension is generated and the dependence between different spatial elements is learned.
  • the calculation formula of the spatial attention module is:
  • M(F) ⁇ (f 7 ⁇ 7 ([AvgPool(F); MaxPool(F)])), where ⁇ represents sigmoid and f 7 ⁇ 7 represents 7 ⁇ 7 convolution layer.
  • the invention discloses a multi-scale fusion defogging method based on stacked hourglass network in the field of image processing.
  • This method uses stacked hourglass networks to extract features from different scales to generate heat maps; then uses the skip connection method to build a new multi-scale fusion dehazing module; and finally adds hierarchical distillation with an attention mechanism.
  • the structure removes redundant information and obtains the dehazed image.
  • This invention aims to solve the problem that existing neural networks cannot effectively capture local and global features at the same time.
  • existing models have made great progress in dehazing effects, there are still problems in making full use of multi-scale fog features and restoring structural details. Inadequate, and few attempts have been made to preserve spatial features and eliminate redundant information.
  • the hourglass network in the present invention has multiple parallel prediction branches, which are stacked and combined with a multi-scale fusion module.
  • the hierarchical distillation structure reduces useless features, so it can better mix global and local information and is highly flexible.
  • the spatial continuity caused by it has better analysis capabilities for dense fog images and real scenes. It can also perform well in describing complex structures, retain texture details as completely as possible, and greatly improve image dehazing. The quality makes the visual effect of defogging more real and natural, effectively improving network performance.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Alarm Systems (AREA)

Abstract

本发明公开了一种基于堆叠沙漏网络的多尺度融合去雾方法，将有雾图像输入预先设置好的图像去雾网络；有雾图像经图像去雾网络处理后，输出去雾后的清晰图像；所述图像去雾网络为依次连接的一个7×7的卷积层、堆叠沙漏模块、特征融合、多尺度跳跃连接模块、一个1×1的卷积层、一个3×3的卷积层、分层注意力蒸馏模块、一个3×3的卷积层和1×1的卷积层。本发明可应用于各种计算机视觉系统，例如图像识别、视频监控、工业视觉检测等，可以降低大量的人工成本，大幅提升图像质量和服务效率，目的是保证更好地服务客户，使最终的去雾结果能满足高级别图像处理的要求并符合人类的视觉要求。

Description

一种基于堆叠沙漏网络的多尺度融合去雾方法 技术领域
本发明涉及一种基于堆叠沙漏网络的多尺度融合去雾方法，属于图像处理技术领域。
背景技术
视觉是人类获取信息最直观的方式，随着人工智能的发展，计算机视觉被广泛应用到我们生活的各个领域。但由于大气污染问题，采集的照片清晰度会受到影响，呈现对比度降低，图像模糊，可提取的特征严重不足的特点。而图像视频作为人们获取信息的主要来源，它的质量严重影响着信息的读取与判断，且高级别的图像处理对于输入图像的质量也有着较高的要求。因此，研究高质、快速、普适的去雾原理与方法具有很高的理论意义和应用价值。
图像去雾的目的是消除雾霾环境对图像质量的影响，增加图像的可视度，其方法主要可以分为三种，一种是基于图像增强算法，即对被降质的图像进行增强，改善图像的质量，突出图像中景物的特征和有价值的信息。但这种方法不考虑导致图像退化的原因，处理后可能会导致图像部分信息的损失，出现失真现象。第二种是基于大气退化模型的方法，即利用无雾图像的先验知识对模型中的参数进行估计，然后将参数代入模型进而恢复无雾图像，该方法处理得到的无雾图像更加清晰、自然，细节损失较少，但不同的先验知识存在着各自应用场景的局限性。第三种是基于深度学习的方法，即通过训练数据集估计透射率，或者利用输入的有雾图像，直接输出得到去雾后的图像。目前最新的去雾方法更倾向于后者，但这种方法存在需要估计参数以及估计参数多、特征不够丰富导致的去雾效率低、质量差等局限性。
发明内容
目的：为了克服现有技术中存在的不足，本发明提供一种基于堆叠沙漏网络的多尺度融合去雾方法，通过构建并结合多个网络来完成学习任务。先利用堆叠沙漏网络实现特征提取，再经过多尺度模块进行特征融合，获得比单一网络显著优越的泛化性能。在网络优化过程中，使用双层注意力模块提升移动网络的卷积特征表达能力。
技术方案：为解决上述技术问题，本发明采用的技术方案为：
一种基于堆叠沙漏网络的多尺度融合去雾方法，包括如下步骤：
将有雾图像输入预先设置好的图像去雾网络。
有雾图像经图像去雾网络处理后，输出去雾后的清晰图像。
所述图像去雾网络为依次连接的一个7×7的卷积层、堆叠沙漏模块、特征融合、多尺度跳跃连接模块、一个1×1的卷积层、一个3×3的卷积层、分层注意力蒸馏模块、一个3×3的卷积层和1×1的卷积层。
作为优选方案，所述堆叠沙漏模块由N个四阶沙漏模块串联组成，所述四阶沙漏模块包含五个并行卷积流：最内层卷积流处理原始尺度，倒数第二层至最外层卷积流分别向下采样至1/2、1/4、1/8和1/16，五个卷积流在不同分辨率组中进行特征提取，再通过残差模块传递各个分辨率的特征，最后再通过上采样操作层恢复到原始尺度并进行融合。
作为优选方案，所述四阶沙漏模块将三阶沙漏模块第四行中间残差模块替换成一阶沙漏模块，所述三阶沙漏模块是将二阶沙漏模块第三行中间残差模块替换成一阶沙漏模块，所述二阶沙漏模块是将一阶沙漏模块第二行中间的残差模块替换成一阶沙漏模块，所述一阶沙漏模块由两行组成，第一行包含一个残差模块，第二行依次由一个最大池化层、三个残差模块和一个上采样操作层组成。第一行与第二行进行特征融合后输出。
作为优选方案，所述残差模块由两行组成，第一行是跳级层，包括一个1×1卷积层。第二行是卷积层，依次为BN层、Relu层、1×1的卷积层、BN层、Relu层、3×3的卷积层、BN层、Relu层和1×1的卷积层。跳级层和卷积层输出端进行特征融合后输出。
作为优选方案，所述N设置为8。
作为优选方案，所述多尺度跳跃连接模块包括：第一行由三个3×3的卷积层和Relu层相串联而成，第二行由三个5×5的卷积层和Relu层相串联而成，第三行由三个7×7的卷积层和Relu层相串联而成。每一行的第一个3×3的卷积层和Relu层的输出，分别作为每一行的第二个3×3的卷积层和Relu层的输入，每一行的第二个3×3的卷积层和Relu层的输出，分别作为每一行的第三个3×3的卷积层和Relu层的输入，将每一行第三个3×3的卷积层和Relu层的输出通过Contact融合后输出。
作为优选方案，所述分层注意力蒸馏模块包括通道注意力模块，空间注意力模块，通道注意力模块，空间注意力模块的输出通过融合后输出。
作为优选方案，所述通道注意力模块将输入的特征图F(H×W×C，H表示高，W表示宽，C表示通道数)分别经过H维度的全局最大池化层和W维度的全局平均池化层，得到两个1×1×C的特征图；将两个特征图送入一个共享权值的双层神经网络进行通道 间依赖关系的学习，将MLP输出的特征进行相加融合，再经过sigmoid激活操作，生成最终的通道加权M。
作为优选方案，所述空间注意力模块将输入的特征图F(H×W×C，H：高，W：宽，C：通道数)分别经过基于C维度的最大池化层和平均池化层，得到两个H×W×1的特征图；将两个H×W×1的特征图基于通道维度进行拼接，拼接后的特征图再使用7×7卷积层进行通道降维。最后经过sigmoid激活操作，生成空间维度的权重M。
有益效果：本发明提供的一种基于堆叠沙漏网络的多尺度融合去雾方法，以解决现有采用基于深度学习的图像去雾方法所面临的需要估计参数以及估计参数多、特征不够丰富导致的去雾效率低、质量差等问题。
本发明属于端对端去雾，将有雾图像输入到上述网络中，可以直接输出无雾图像。由于使用堆叠沙漏模块和多尺度跳跃连接模块，可以解决现有神经网络不能同时有效地捕捉到局部和全局特征的问题；引入分层注意力蒸馏模块，对网络结构进行优化，保留空间和上下文信息并提取出更有用的层次特征。能够在充分利用多尺度雾特征和恢复结构细节方面有明显进展，提高生成图片的质量。
本发明先采用堆叠沙漏网络重复地自下而上、自上而下地在各种尺度上捕获特征，再利用多尺度跳跃连接的方法对信息进行重复融合，最后结合双层注意力机制，避免特征消失，且去除不必要的特征，以实现全面的特征聚合，提高去雾性能。本发明可应用于各种计算机视觉系统，例如图像识别、视频监控、工业视觉检测等，可以降低大量的人工成本，大幅提升图像质量和服务效率，目的是保证更好地服务客户，使最终的去雾结果能满足高级别图像处理的要求并符合人类的视觉要求。
附图说明
图1为本发明方法实施的整体流程图。
图2为四阶沙漏模块采样规律的网络架构示意图。
图3为残差模块的网络架构示意图。
图4为一阶沙漏模块的网络架构示意图。
图5为四阶沙漏模块的网络架构示意图。
图6为多尺度跳跃连接模块的网络架构示意图。
图7为分层注意力蒸馏模块的网络架构示意图。
图8为通道注意力模块的实施流程图。
图9为空间注意力模块的实施流程图。
具体实施方式
下面结合具体实施例对本发明作更进一步的说明。
如图1所示，本发明公开了一种基于堆叠沙漏网络的多尺度融合去雾方法，当有雾图像输入后，进入图像去雾网络，直接输出去雾后的清晰图像。
图像去雾网络依次为一个7×7的卷积层、堆叠沙漏模块、特征融合、多尺度跳跃连接模块、一个1×1的卷积层、一个3×3的卷积层、分层注意力蒸馏模块、一个3×3的卷积层和1×1的卷积层。
其中，7×7的卷积层用于将原始的有雾图像进行第一步处理，形成初始特征图像。特征融合设置为相加运算，用于特征相加。多尺度跳跃连接模块后的1×1的卷积层用于调整通道数，调整经过contact之后变化的通道数，并获取低频特征信息。多尺度跳跃连接模块后的3×3的卷积层用于获取高频特征信息。分层注意力蒸馏模块后的3×3的卷积层和1×1的卷积层用于实现特征的修饰或者辅助作用。
堆叠沙漏模块由N个四阶沙漏模块串联组成，本发明优选N＝4、6、8、10时，PSNR(峰值信噪比)＝27.28、27.96、28.35、28.37，SSIM(机构相似度)＝0.9122、0.9180、0.9217、0.9214。这两个指标都是越大越好，但是N从4到8时，变化明显，N从8到10时，PSNR上升不明显且SSIM降低，因此，本发明N选用最优值8。
如图2所示，所述四阶沙漏模块通过并行结构整合紧密连接的残块模块、最大池化层、上采样操作层和残差融合，形成对称拓扑结构，每个尺度都是先自上而下，再有一个相应的自底而上的层，使之达到一种重复自上而下，自下而上提取特征的目的。方块大小表示特征图大小，方块变小表示下采样，方块变大表示上采样，加号表示按元素相加。
具体流程如下：四阶沙漏模块包含五个并行卷积流：最内层卷积流处理原始尺度，倒数第二层至最外层卷积流分别向下采样至1/2、1/4、1/8和1/16，五个卷积流在不同分辨率组中进行特征提取，再通过残差模块传递各个分辨率的特征，最后再通过上采样操作层恢复到原始尺度并进行融合，即将不同分辨率的特征按元素位置进行相加，因此能够在多个尺度上提取并保留特征信息，达到同时保留局部特征和全局特征的效果。
所述残差模块是一阶沙漏模块的基本组成单元，具体网络架构见图3。残差模块由两行组成，第一行是跳级层，包括一个1×1卷积层(Conv)，用于保留原有层次的信息。第二行是卷积层，用于提取特征，依次为BN层、Relu层、1×1的卷积层、BN层、Relu层、3×3的卷积层、BN层、Relu层和1×1的卷积层。跳级层和卷积层输出端进行特征融合后输出。
第二行的卷积层，由BN层先对信号进行归一化处理，经过Relu层使主路径增加非 线性，再经过1×1的卷积层起到降维的作用，且降维之后可以更有效、更直观地进行数据训练和特征提取，接着再次经过BN层和Relu层，再经过3×3的卷积层进行相对较低维度的计算，提高网络深度且提高效率，接着第三次经过BN层和Relu层，再次经过1×1的卷积层起到升维的作用，最后与跳级层进行特征融合，不改变数据尺寸只提高数据深度。
一阶沙漏模块由两行组成，具体网络架构见图4。第一行只包含一个残差模块，第二行依次由一个最大池化层(Max Pool)用于对特征图进行下采样，以获得分辨率较低的特征图，降低计算复杂度、三个残差模块和一个上采样操作层(Up Sample)用于采用最近邻插值的方法对特征图进行下采样，使图像特征的分辨率提高，目的是保持与输入图像大小一致。第一行与第二行进行特征融合后输出，使得输出的结果既包含了原始分辨率的特征，也包含了下采样后分辨率下降至1/2的特征。
二阶沙漏模块是将一阶沙漏模块第二行中间的残差模块替换成一阶沙漏模块，三阶沙漏模块是将二阶沙漏模块第三行中间残差模块替换成一阶沙漏模块，四阶沙漏模块是将三阶沙漏模块第四行中间残差模块替换成一阶沙漏模块，以此类推，形成递归结构，如图5所示。
如图6所示，所述多尺度跳跃连接模块包括：三种不同大小卷积核的卷积运算加入激活函数组成，第一行由三个3×3的卷积层和Relu层相串联而成，第二行由三个5×5的卷积层和Relu层相串联而成，第三行由三个7×7的卷积层和Relu层相串联而成。每一行的第一个3×3的卷积层和Relu层的输出，分别作为每一行的第二个3×3的卷积层和Relu层的输入，每一行的第二个3×3的卷积层和Relu层的输出，分别作为每一行的第三个3×3的卷积层和Relu层的输入，将每一行第三个3×3的卷积层和Relu层的输出通过Contact融合。
使用不同大小的卷积核能够在不同的特征尺度进行提取，获得深层的细节信息，此外，为了保证卷积后的特征图与原始雾图大小不发生变化，其中的卷积运算采用零填充方式。在卷积运算之后引入激活函数，将卷积层的输出结果做非线性运算，使卷积神经网络获得解决复杂问题的能力，同时提高卷积神经网络对非线性因素的鲁棒性。在选择激活函数时，采用带泄露线性整流单元Leaky ReLU，其函数图像在分段区间是线性函数，整体是非线性函数，值域是全部实数集，能够提高网络收敛速度。
但在连接方式上本发明有所创新，不是简单的将三组不同大小的卷积核做并行卷积运算，而是采用跳跃连接的方式，将本行上一个3×3的卷积层和Relu层输出的结果除了输出给串联的下一个3×3的卷积层和Relu层外，还输出给另外两行下一个3×3的卷积层和Relu层，因此每行中下一个3×3的卷积层和Relu层的输入都是将上一个3×3的卷积层和Relu层不同大小卷积核的输出分别相加，以实现多尺度的信息融合。
每一行卷积核运算后得到3个特征图，将第三个3×3的卷积层和Relu层输出的三个特征图通过Contact融合，即将三个特征图的通道数相加，而每一个通道下的信息不相加，以增加通道数的方式结合前面所得到的特征，保留不同尺度卷积核提取到的特征，以实现较好的性能。
多尺度跳跃连接模块每一个卷积后的输出为：
Fa n×n为卷积大小为n×n的第一个卷积层输出，可以表示为：
Fa 3×3＝Conv3×3(Fin；θa 3×3)；
Fa 5×5＝Conv5×5(Fin；θa 5×5)；
Fa 7×7＝Conv7×7(Fin；θa 7×7)；
其中：Fin为输入多尺度跳跃连接模块的原始图像，Convn×n(·)为卷积运算，θa n×n表示卷积核大小为n×n的第一个多尺度卷积形成的超参数。
Fb n×n为卷积大小为n×n的第二个卷积层输出，可以表示为：
Fb 3×3＝Conv3×3((Fa 3×3+Fa 5×5+Fa 7×7)；θb 3×3)；
Fb 5×5＝Conv5×5((Fa 3×3+Fa 5×5+Fa 7×7)；θb 5×5)；
Fb 7×7＝Conv7×7((Fa 3×3+Fa 5×5+Fa 7×7)；θb 7×7)；
Fc n×n为卷积大小为n×n的第三个卷积层输出，可以表示为：
Fc 3×3＝Conv3×3((Fb 3×3+Fb 5×5+Fb 7×7)；θc 3×3)；
Fc 5×5＝Conv5×5((Fb 3×3+Fb 5×5+Fb 7×7)；θc 5×5)；
Fc 7×7＝Conv7×7((Fb 3×3+Fb 5×5+Fb 7×7)；θc 7×7)；
对于去雾问题，关键是要充分利用雾的特征，并将其转移到最后进行去雾。随着网络深度的增加，在传输过程中空间表达能力逐渐降低，并无目的地产生大量冗余特征，直接影响了去雾质量。分层注意力蒸馏模块由空间注意力模块和通道注意力模块并联组成，其结构见图7，利用双注意单元分别学习空间的重要性和通道的重要性，将输出结果相加，既保留了空间又保留上下文信息，又通过使用分层注意力融合减少了不必要的特征，只允许信息特征进一步传递，消除冗余特征，实现特征蒸馏，并且容易嵌入到框架中。
通道注意力模块的结构，如图8所示，具体操作如下：首先，将输入的特征图F(H×W×C，H表示高，W表示宽，C表示通道数)分别经过基于H和W两个维度的全局最大池化层(MaxPool)和全局平均池化层(AvgPool)，得到两个1×1×C的特征图；然后将两个特征图送入一个共享权值的双层神经网络(MLP)进行通道间依赖关系的学习，两层神经层之间通过压缩比r实现降维。最后，将MLP输出的特征进行相加融合，再经过sigmoid激活操作，生成最终的通道加权M，挖掘特征上下文信息和层次特征之间的 关系。
通道注意力模块计算公式为：
M(F)＝σ(MLP(AvgPool(F))+MLP(MaxPool(F)))，其中σ表示sigmoid。
空间注意力模块的结构，如图9所示，具体操作如下：首先，将输入的特征图F(H×W×C，H：高，W：宽，C：通道数)分别经过基于C维度的最大池化层(MaxPool)和平均池化层(AvgPool)，得到两个H×W×1的特征图；然后将两个H×W×1的特征图基于通道维度进行拼接，拼接后的特征图再使用7×7卷积层进行通道降维。最后经过sigmoid激活操作，生成空间维度的权重M，学习不同空间元素之间的依赖关系。
空间注意力模块计算公式为：
M(F)＝σ(f7×7([AvgPool(F)；MaxPool(F)]))，其中σ表示sigmoid，f7×7表示7×7卷积层。
本发明公开了图像处理领域的一种基于堆叠沙漏网络的多尺度融合去雾方法。该方法通过使用堆叠的沙漏网络来从不同尺度进行特征提取，从而生成热图；再利用跳跃连接的方法构建一种新的多尺度融合去雾模块；最后增加带有注意力机制的分层蒸馏结构去除冗余信息，得到去雾后的图像。
本发明旨在解决现有神经网络不能同时有效地捕捉到局部和全局特征的问题，虽然已有模型在去雾效果上有很大进展，但在充分利用多尺度雾特征和恢复结构细节方面存在不足，且很少有人尝试保存空间特征和消除冗余信息。而本发明中的沙漏网络有多个平行的预测分支，堆叠后与多尺度融合模块相结合，最后经过分层蒸馏结构减少无用特征，因此可以更好的混合全局和局部信息，具有高度的灵活性，另外其引起的空间连续性对浓雾图像和真实场景有更好的分析能力，在描述复杂结构方面同样可以表现出色，尽可能完整地保留纹理细节，很大程度上提高了图像去雾的质量，使去雾的视觉效果更加真实自然，有效提升了网络性能。
以上所述仅是本发明的优选实施方式，应当指出：对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。

Claims (9)

  1. 一种基于堆叠沙漏网络的多尺度融合去雾方法，其特征在于：包括如下步骤：
    将有雾图像输入预先设置好的图像去雾网络；
    有雾图像经图像去雾网络处理后，输出去雾后的清晰图像；
    所述图像去雾网络为依次连接的一个7×7的卷积层、堆叠沙漏模块、特征融合、多尺度跳跃连接模块、一个1×1的卷积层、一个3×3的卷积层、分层注意力蒸馏模块、一个3×3的卷积层和1×1的卷积层。
  2. 根据权利要求1所述的一种基于堆叠沙漏网络的多尺度融合去雾方法，其特征在于：
    所述堆叠沙漏模块由N个四阶沙漏模块串联组成，所述四阶沙漏模块包含五个并行卷积流：最内层卷积流处理原始尺度，倒数第二层至最外层卷积流分别向下采样至1/2、1/4、1/8和1/16，五个卷积流在不同分辨率组中进行特征提取，再通过残差模块传递各个分辨率的特征，最后再通过上采样操作层恢复到原始尺度并进行融合。
  3. 根据权利要求2所述的一种基于堆叠沙漏网络的多尺度融合去雾方法，其特征在于：
    所述四阶沙漏模块将三阶沙漏模块第四行中间残差模块替换成一阶沙漏模块，所述三阶沙漏模块是将二阶沙漏模块第三行中间残差模块替换成一阶沙漏模块，所述二阶沙漏模块是将一阶沙漏模块第二行中间的残差模块替换成一阶沙漏模块，所述一阶沙漏模块由两行组成，第一行包含一个残差模块，第二行依次由一个最大池化层、三个残差模块和一个上采样操作层组成；第一行与第二行进行特征融合后输出。
  4. 根据权利要求3所述的一种基于堆叠沙漏网络的多尺度融合去雾方法，其特征在于：
    所述残差模块由两行组成，第一行是跳级层，包括一个1×1卷积层；第二行是卷积层，依次为BN层、Relu层、1×1的卷积层、BN层、Relu层、3×3的卷积层、BN层、Relu层和1×1的卷积层；跳级层和卷积层输出端进行特征融合后输出。
  5. 根据权利要求2所述的一种基于堆叠沙漏网络的多尺度融合去雾方法，其特征在于：
    所述N设置为8。
  6. 根据权利要求1所述的一种基于堆叠沙漏网络的多尺度融合去雾方法，其特征在于：
    所述多尺度跳跃连接模块包括：第一行由三个3×3的卷积层和Relu层相串联而成，第二行由三个5×5的卷积层和Relu层相串联而成，第三行由三个7×7的卷积层和Relu层相串联而成；每一行的第一个3×3的卷积层和Relu层的输出，分别作为每一行的第二个3×3的卷积层和Relu层的输入，每一行的第二个3×3的卷积层和Relu层的输出，分别作为每一行的第三个3×3的卷积层和Relu层的输入，将每一行第三个3×3的卷 积层和Relu层的输出通过Contact融合后输出。
  7. 根据权利要求1所述的一种基于堆叠沙漏网络的多尺度融合去雾方法，其特征在于：
    所述分层注意力蒸馏模块包括通道注意力模块，空间注意力模块，通道注意力模块，空间注意力模块的输出通过融合后输出。
  8. 根据权利要求7所述的一种基于堆叠沙漏网络的多尺度融合去雾方法，其特征在于：
    所述通道注意力模块将输入的特征图F(H×W×C，H表示高，W表示宽，C表示通道数)分别经过H维度的全局最大池化层和W维度的全局平均池化层，得到两个1×1×C的特征图；将两个特征图送入一个共享权值的双层神经网络进行通道间依赖关系的学习，将MLP输出的特征进行相加融合，再经过sigmoid激活操作，生成最终的通道加权M。
  9. 根据权利要求7所述的一种基于堆叠沙漏网络的多尺度融合去雾方法，其特征在于：
    所述空间注意力模块将输入的特征图F(H×W×C，H：高，W：宽，C：通道数)分别经过基于C维度的最大池化层和平均池化层，得到两个H×W×1的特征图；将两个H×W×1的特征图基于通道维度进行拼接，拼接后的特征图再使用7×7卷积层进行通道降维。最后经过sigmoid激活操作，生成空间维度的权重M。
PCT/CN2023/086215 2022-08-22 2023-04-04 一种基于堆叠沙漏网络的多尺度融合去雾方法 Ceased WO2024040973A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/312,168 US12423786B2 (en) 2022-08-22 2023-05-04 Multi-scale fusion defogging method based on stacked hourglass network

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211007029.3 2022-08-22
CN202211007029.3A CN115330631B (zh) 2022-08-22 2022-08-22 一种基于堆叠沙漏网络的多尺度融合去雾方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/312,168 Continuation US12423786B2 (en) 2022-08-22 2023-05-04 Multi-scale fusion defogging method based on stacked hourglass network

Publications (1)

Publication Number Publication Date
WO2024040973A1 true WO2024040973A1 (zh) 2024-02-29

Family

ID=83925717

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/086215 Ceased WO2024040973A1 (zh) 2022-08-22 2023-04-04 一种基于堆叠沙漏网络的多尺度融合去雾方法

Country Status (2)

Country Link
CN (1) CN115330631B (zh)
WO (1) WO2024040973A1 (zh)

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117993442A (zh) * 2024-03-21 2024-05-07 济南大学 一种融合局部和全局信息的混合神经网络方法与系统
CN118015687A (zh) * 2024-04-10 2024-05-10 齐鲁工业大学(山东省科学院) 改进的多尺度注意残差关系感知的表情识别方法及装置
CN118169752A (zh) * 2024-03-13 2024-06-11 北京石油化工学院 一种基于多特征融合的地震相位拾取方法及系统
CN118330787A (zh) * 2024-06-11 2024-07-12 浙江工业大学 一种基于多模态域变换和自适应融合的台风生成预报方法
CN118470413A (zh) * 2024-05-16 2024-08-09 淮阴工学院 一种基于UpCPFNet模型的芒果分级识别方法
CN118469842A (zh) * 2024-05-07 2024-08-09 广东工业大学 一种基于生成对抗网络的遥感图像去雾方法
CN118521497A (zh) * 2024-07-22 2024-08-20 山东黄海智能装备有限公司 一种荧光标记细胞成像图像增强处理方法
CN118608545A (zh) * 2024-07-08 2024-09-06 济南大学 一种基于轻量级多尺度融合的双路径皮肤病变图像分割方法
CN118781018A (zh) * 2024-09-05 2024-10-15 浙江工业大学 一种基于深度学习的红外图像辅助图像去雾方法
CN118780986A (zh) * 2024-09-11 2024-10-15 济南大学 一种基于大核蒸馏网络的汽车零部件图像超分辨率方法
CN119067990A (zh) * 2024-08-17 2024-12-03 江西师范大学 基于残差注意力与多尺度特征聚合的视网膜图像分割方法
CN119151828A (zh) * 2024-11-18 2024-12-17 武汉纺织大学 一种轻量型图像去雾方法及系统
CN119359579A (zh) * 2024-11-04 2025-01-24 北京联合大学 一种暗光图像增强方法、系统及装置
CN119359573A (zh) * 2024-08-28 2025-01-24 厦门大学 一种增强空间特征优化的图像恢复方法
CN119399066A (zh) * 2024-10-14 2025-02-07 国网河北省电力有限公司信息通信分公司 雨雾天气条件下的输电线路图像处理方法及装置
CN119494794A (zh) * 2024-10-29 2025-02-21 济南深览电子科技有限公司 一种用于ct图像的增强处理的优化方法
CN119784619A (zh) * 2024-12-26 2025-04-08 四川大学 一种基于状态空间模型的图像复原方法
CN119832406A (zh) * 2024-12-20 2025-04-15 南京理工大学 基于多尺度特征融合及高校上采样算法的声呐图像水下检测方法及系统
CN119919314A (zh) * 2024-12-16 2025-05-02 三峡大学 基于条件生成对抗网络的多深度融合的图像去雾方法
CN120355575A (zh) * 2025-06-24 2025-07-22 济南大学 基于空间通道联合注意力的汽车零部件图像超分辨率方法
CN120471922A (zh) * 2025-07-15 2025-08-12 安徽大学 基于多方向特征增强网络的版图光刻热点检测方法及系统
CN120495149A (zh) * 2025-07-16 2025-08-15 中国石油大学(华东) 面向多层颜色特征的密集复用与跳跃连接的水下图像增强方法
CN120580174A (zh) * 2025-07-30 2025-09-02 昆明理工大学 基于文本引导和扰动防御的真实场景图像去雾方法
CN120634874A (zh) * 2025-05-09 2025-09-12 桂林电子科技大学 一种基于Mamba与注意力机制的多模态图像融合方法
CN120635596A (zh) * 2025-08-08 2025-09-12 思维链(天津)智能科技有限公司 一种胸部x光图像肺炎分类装置
CN120807366A (zh) * 2025-09-16 2025-10-17 武汉工程大学 一种航道图像去雾方法、系统及计算设备
CN120833279A (zh) * 2025-09-19 2025-10-24 哈尔滨工业大学（威海） 一种基于改进DehazeFormer的遥感图像去雾方法
CN120974242A (zh) * 2025-10-20 2025-11-18 浙江贵仁信息科技股份有限公司 一种基于多源数据融合的流域洪水实时预警方法和系统
CN120976063A (zh) * 2025-10-20 2025-11-18 天津工业大学 一种基于多尺度先验注入的图像去雨方法及系统
CN121437319A (zh) * 2025-10-20 2026-01-30 西安电子科技大学 一种面向浓雾场景下偏振先验双流协同清晰化方法

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115330631B (zh) * 2022-08-22 2025-07-25 南京邮电大学 一种基于堆叠沙漏网络的多尺度融合去雾方法
CN116823713A (zh) * 2023-03-27 2023-09-29 浙江大学 基于多分类任务的甲状腺癌检出系统
CN116823638B (zh) * 2023-05-12 2026-03-10 长安大学 一种基于改进gan的图像去雾方法
CN119832369B (zh) * 2024-11-14 2025-10-10 电子科技大学 一种基于多尺度信息融合的低光图像增强方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110738622A (zh) * 2019-10-17 2020-01-31 温州大学 基于多尺度卷积的轻量级神经网络单图像去雾方法
US20200273192A1 (en) * 2019-02-26 2020-08-27 Baidu Usa Llc Systems and methods for depth estimation using convolutional spatial propagation networks
CN113450273A (zh) * 2021-06-18 2021-09-28 暨南大学 一种基于多尺度多阶段神经网络的图像去雾方法及系统
CN113673590A (zh) * 2021-08-13 2021-11-19 广东工业大学 基于多尺度沙漏密集连接网络的去雨方法、系统和介质
CN113947537A (zh) * 2021-09-17 2022-01-18 南京邮电大学 图像去雾方法、装置及设备
CN114492522A (zh) * 2022-01-24 2022-05-13 四川大学 基于改进堆叠沙漏神经网络的自动调制分类方法
CN115330631A (zh) * 2022-08-22 2022-11-11 南京邮电大学 一种基于堆叠沙漏网络的多尺度融合去雾方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110349093B (zh) * 2019-06-11 2022-11-29 西安电子科技大学 基于多级沙漏结构的单幅图像去雾模型构建及去雾方法
CN113592718A (zh) * 2021-08-12 2021-11-02 中国矿业大学 基于多尺度残差网络的矿井图像超分辨率重建方法及系统
CN114202481B (zh) * 2021-12-13 2024-07-02 贵州大学 一种基于图像高频信息融合的多尺度特征去雾网络及方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200273192A1 (en) * 2019-02-26 2020-08-27 Baidu Usa Llc Systems and methods for depth estimation using convolutional spatial propagation networks
CN110738622A (zh) * 2019-10-17 2020-01-31 温州大学 基于多尺度卷积的轻量级神经网络单图像去雾方法
CN113450273A (zh) * 2021-06-18 2021-09-28 暨南大学 一种基于多尺度多阶段神经网络的图像去雾方法及系统
CN113673590A (zh) * 2021-08-13 2021-11-19 广东工业大学 基于多尺度沙漏密集连接网络的去雨方法、系统和介质
CN113947537A (zh) * 2021-09-17 2022-01-18 南京邮电大学 图像去雾方法、装置及设备
CN114492522A (zh) * 2022-01-24 2022-05-13 四川大学 基于改进堆叠沙漏神经网络的自动调制分类方法
CN115330631A (zh) * 2022-08-22 2022-11-11 南京邮电大学 一种基于堆叠沙漏网络的多尺度融合去雾方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHEN XIANG, YUFENG HUANG, LEI XU: "Multi-Scale Hourglass Hierarchical Fusion Network for Single Image Deraining", 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, 1 January 2021 (2021-01-01), pages 872 - 879, XP093142152 *

Cited By (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118169752A (zh) * 2024-03-13 2024-06-11 北京石油化工学院 一种基于多特征融合的地震相位拾取方法及系统
CN117993442A (zh) * 2024-03-21 2024-05-07 济南大学 一种融合局部和全局信息的混合神经网络方法与系统
CN118015687A (zh) * 2024-04-10 2024-05-10 齐鲁工业大学(山东省科学院) 改进的多尺度注意残差关系感知的表情识别方法及装置
CN118469842A (zh) * 2024-05-07 2024-08-09 广东工业大学 一种基于生成对抗网络的遥感图像去雾方法
CN118470413A (zh) * 2024-05-16 2024-08-09 淮阴工学院 一种基于UpCPFNet模型的芒果分级识别方法
CN118330787A (zh) * 2024-06-11 2024-07-12 浙江工业大学 一种基于多模态域变换和自适应融合的台风生成预报方法
CN118608545B (zh) * 2024-07-08 2025-01-24 济南大学 一种基于轻量级多尺度融合的双路径皮肤病变图像分割方法
CN118608545A (zh) * 2024-07-08 2024-09-06 济南大学 一种基于轻量级多尺度融合的双路径皮肤病变图像分割方法
CN118521497A (zh) * 2024-07-22 2024-08-20 山东黄海智能装备有限公司 一种荧光标记细胞成像图像增强处理方法
CN119067990A (zh) * 2024-08-17 2024-12-03 江西师范大学 基于残差注意力与多尺度特征聚合的视网膜图像分割方法
CN119359573A (zh) * 2024-08-28 2025-01-24 厦门大学 一种增强空间特征优化的图像恢复方法
CN118781018A (zh) * 2024-09-05 2024-10-15 浙江工业大学 一种基于深度学习的红外图像辅助图像去雾方法
CN118780986A (zh) * 2024-09-11 2024-10-15 济南大学 一种基于大核蒸馏网络的汽车零部件图像超分辨率方法
CN119399066A (zh) * 2024-10-14 2025-02-07 国网河北省电力有限公司信息通信分公司 雨雾天气条件下的输电线路图像处理方法及装置
CN119494794A (zh) * 2024-10-29 2025-02-21 济南深览电子科技有限公司 一种用于ct图像的增强处理的优化方法
CN119359579A (zh) * 2024-11-04 2025-01-24 北京联合大学 一种暗光图像增强方法、系统及装置
CN119151828A (zh) * 2024-11-18 2024-12-17 武汉纺织大学 一种轻量型图像去雾方法及系统
CN119919314A (zh) * 2024-12-16 2025-05-02 三峡大学 基于条件生成对抗网络的多深度融合的图像去雾方法
CN119832406A (zh) * 2024-12-20 2025-04-15 南京理工大学 基于多尺度特征融合及高校上采样算法的声呐图像水下检测方法及系统
CN119784619A (zh) * 2024-12-26 2025-04-08 四川大学 一种基于状态空间模型的图像复原方法
CN120634874A (zh) * 2025-05-09 2025-09-12 桂林电子科技大学 一种基于Mamba与注意力机制的多模态图像融合方法
CN120355575A (zh) * 2025-06-24 2025-07-22 济南大学 基于空间通道联合注意力的汽车零部件图像超分辨率方法
CN120471922A (zh) * 2025-07-15 2025-08-12 安徽大学 基于多方向特征增强网络的版图光刻热点检测方法及系统
CN120495149A (zh) * 2025-07-16 2025-08-15 中国石油大学(华东) 面向多层颜色特征的密集复用与跳跃连接的水下图像增强方法
CN120580174A (zh) * 2025-07-30 2025-09-02 昆明理工大学 基于文本引导和扰动防御的真实场景图像去雾方法
CN120635596A (zh) * 2025-08-08 2025-09-12 思维链(天津)智能科技有限公司 一种胸部x光图像肺炎分类装置
CN120635596B (zh) * 2025-08-08 2025-10-10 思维链(天津)智能科技有限公司 一种胸部x光图像肺炎分类装置
CN120807366A (zh) * 2025-09-16 2025-10-17 武汉工程大学 一种航道图像去雾方法、系统及计算设备
CN120833279A (zh) * 2025-09-19 2025-10-24 哈尔滨工业大学（威海） 一种基于改进DehazeFormer的遥感图像去雾方法
CN120974242A (zh) * 2025-10-20 2025-11-18 浙江贵仁信息科技股份有限公司 一种基于多源数据融合的流域洪水实时预警方法和系统
CN120976063A (zh) * 2025-10-20 2025-11-18 天津工业大学 一种基于多尺度先验注入的图像去雨方法及系统
CN121437319A (zh) * 2025-10-20 2026-01-30 西安电子科技大学 一种面向浓雾场景下偏振先验双流协同清晰化方法

Also Published As

Publication number Publication date
CN115330631B (zh) 2025-07-25
CN115330631A (zh) 2022-11-11

Similar Documents

Publication Publication Date Title
US12423786B2 (en) Multi-scale fusion defogging method based on stacked hourglass network
CN115330631B (zh) 一种基于堆叠沙漏网络的多尺度融合去雾方法
CN113673590B (zh) 基于多尺度沙漏密集连接网络的去雨方法、系统和介质
CN108537192B (zh) 一种基于全卷积网络的遥感影像地表覆盖分类方法
WO2023056889A1 (zh) 模型训练和场景识别方法、装置、设备及介质
CN110889895B (zh) 一种融合单帧重建网络的人脸视频超分辨率重建方法
CN114842216A (zh) 一种基于小波变换的室内rgb-d图像语义分割方法
CN111179167A (zh) 一种基于多阶段注意力增强网络的图像超分辨方法
CN113870335A (zh) 一种基于多尺度特征融合的单目深度估计方法
CN112241939B (zh) 一种基于多尺度和非局部的轻量去雨方法
CN110363068A (zh) 一种基于多尺度循环生成式对抗网络的高分辨行人图像生成方法
CN118333898B (zh) 基于改进型生成对抗网络的图像去雾方法及系统
CN118691929A (zh) 基于空频特征融合检测头的无人机目标检测方法
CN116721033A (zh) 一种基于随机掩码卷积和注意力机制的单幅图像去雾方法
CN120339779A (zh) 用于红外图像的交互式卷积与动态聚焦的特征集成方法
CN116205953B (zh) 基于分层全相关代价体聚合的光流估计方法与装置
CN115526779A (zh) 一种基于动态注意力机制的红外图像超分辨率重建方法
WO2023185284A1 (zh) 视频处理方法和装置
CN112861911A (zh) 一种基于深度特征选择融合的rgb-d语义分割方法
CN115546060A (zh) 一种可逆的水下图像增强方法
CN114972752B (zh) 一种基于轻量级交叉注意网络的实时语义分割的方法
CN115937704A (zh) 基于拓扑感知神经网络的遥感图像道路分割方法
CN115564664A (zh) 基于融合双边重组注意力的二阶段Transformer编解码器的去运动模糊方法
Hou et al. Self-supervised monocular depth estimation for all-day images based on dual-axis transformer
CN115131414B (zh) 基于深度学习的无人机图像对齐方法、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23856068

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23856068

Country of ref document: EP

Kind code of ref document: A1