CN113658200A - An edge-aware image semantic segmentation method based on adaptive feature fusion - Google Patents

An edge-aware image semantic segmentation method based on adaptive feature fusion Download PDF

Info

Publication number
CN113658200A
CN113658200A CN202110864679.9A CN202110864679A CN113658200A CN 113658200 A CN113658200 A CN 113658200A CN 202110864679 A CN202110864679 A CN 202110864679A CN 113658200 A CN113658200 A CN 113658200A
Authority
CN
China
Prior art keywords
features
feature
edge
semantic
branch
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110864679.9A
Other languages
Chinese (zh)
Other versions
CN113658200B (en
Inventor
郭军
顾文哲
董蔼萱
白经纬
崔中健
郭欣然
李威
李泽霖
张斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northeastern University China
Original Assignee
Northeastern University China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northeastern University China filed Critical Northeastern University China
Priority to CN202110864679.9A priority Critical patent/CN113658200B/en
Publication of CN113658200A publication Critical patent/CN113658200A/en
Application granted granted Critical
Publication of CN113658200B publication Critical patent/CN113658200B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

本发明提供一种基于自适应特征融合的边缘感知图像语义分割方法，是一种以残差网络为基础的新的语义分割方法，是一个双分支网络结构模型，包括边缘分支和语义分支，其中，边缘分支从语义分支的浅层部分被引出，语义分支采用了编码解码结构。在边缘分支中，加入的多尺度交叉融合操作通过叠加空洞率不同的空洞卷积获取图像多尺度特征，同时各个分支之间的交叉融合可以进一步提升多尺度特征的鲁棒性，在语义分支中基于空间注意力机制对深层特征和浅层特征进行融合，可以在获得浅层特征中含有的丰富空间信息的同时，过滤其含有的大量噪声；最后对语义分支特征和边缘分支特征进行融合，进一步优化分割效果。

Figure 202110864679

The present invention provides an edge-aware image semantic segmentation method based on adaptive feature fusion, which is a new semantic segmentation method based on residual network, and is a double-branch network structure model, including edge branch and semantic branch, wherein , the edge branch is drawn from the shallow part of the semantic branch, and the semantic branch adopts an encoding-decoding structure. In the edge branch, the added multi-scale cross-fusion operation obtains image multi-scale features by superimposing hole convolutions with different hole rates. At the same time, the cross-fusion between each branch can further improve the robustness of multi-scale features. In the semantic branch The fusion of deep features and shallow features based on the spatial attention mechanism can obtain the rich spatial information contained in the shallow features while filtering the large amount of noise contained in them; finally, the semantic branch features and edge branch features are fused to further Optimize the segmentation effect.

Figure 202110864679

Description

Edge perception image semantic segmentation method based on self-adaptive feature fusion
Technical Field
The invention relates to the technical field of semantic segmentation, in particular to an edge perception image semantic segmentation method based on self-adaptive feature fusion.
Background
Semantic segmentation refers to the realization of pixel-level automatic identification and segmentation of image contents, and is widely applied to a plurality of fields such as land utilization planning, earthquake monitoring, vegetation classification, environmental pollution monitoring and the like at present. For example, by analyzing the remote sensing images of the atmosphere, the distribution state of the atmospheric pollutants can be clarified so as to monitor the atmospheric pollution. How to accurately segment images is always a hotspot and a difficulty of domestic and foreign research.
In recent years, with the rapid development of the deep learning field, the semantic segmentation research work has made a breakthrough progress, and the semantic segmentation model based on the convolutional neural network has been significantly improved in the aspects of computational efficiency, accuracy and the like. The current popular semantic segmentation models are full convolution neural network FCN, U-Net and Deeplab series. Although these advanced models can achieve good segmentation, there are two main problems: (1) the edge features of the image are not fully extracted, so that the segmentation performance of the model in the edge region is poor. (2) The shallow feature maps are not filtered before the shallow and deep feature maps are fused, introducing a large amount of noise information.
Disclosure of Invention
Aiming at the defects of the prior art and solving the problems of the existing semantic segmentation model, the invention provides an edge perception image semantic segmentation method based on self-adaptive feature fusion, which is used for segmenting an image so as to realize a better image segmentation effect.
In order to achieve the technical effect, the invention provides an edge perception image semantic segmentation method based on self-adaptive feature fusion, which comprises the following steps:
the method comprises the following steps: making a data set, collecting N images and carrying out pixel-level classification labeling on each image, wherein each sample in the data set comprises one image and a pixel-level labeling result of the image;
step two: building an edge perception image semantic segmentation model based on self-adaptive feature fusion, wherein the edge perception image semantic segmentation model is a double-branch network structure which takes a ResNet network as a main body and comprises edge branches and semantic branches;
step three: dividing a data set into a training set and a testing set, training a model by using the training set, and verifying the trained model by using the testing set;
step four: and using the trained semantic segmentation model for image segmentation to obtain the segmentation result of the image.
The second step comprises the following steps:
step 1: using a ResNet network model as a downsampling stage of semantic branches, leading out an edge branch at the downsampling stage of the semantic branches, and performing multi-scale cross fusion operation on edge features in the edge branch;
step 2: performing up-sampling on output features of a semantic branch down-sampling stage, and performing fusion of deep features and shallow features in an up-sampling stage to obtain semantic features containing rich spatial information;
and step 3: edge feature for edge branch output
Figure BDA0003186887470000021
Semantic features and semantic branching output
Figure BDA0003186887470000022
Fusion was performed to obtain fused feature F'.
The step 1 comprises the following steps:
step 1.1: taking shallow features output by the semantic branch downsampling stage as input features of edge branches, and performing convolution processing;
step 1.2: processing the edge characteristics by using the void convolution layers with the porosity of 7, 5 and 3 respectively to obtain 3 characteristics F1、F2、F3；
Step 1.3: for feature F1Performing convolution treatment to obtain new characteristic F'1；
Step 1.4: feature F 'is spliced'1And F2Fusing to obtain new characteristic F'2；
Step 1.5: feature F 'is spliced'2And F3Fusing to obtain new characteristic F'3；
Step 1.6: feature F 'is spliced'1、F′2And F'3Carrying out fusion to obtain a new characteristic F';
step 1.7: the new feature F' is successively processed by convolution and upsampling to obtain the final edge feature
Figure BDA0003186887470000024
The step 2 comprises the following steps:
step 2.1: obtaining 4 features M from a downsampling stage of semantic branching1、M2、M3、M4；
Step 2.2: for feature M4After convolution processing, the characteristic M is compared with3Multiplying to obtain new characteristic M'3；
Step 2.3: fusion of feature M 'in a splicing manner'3And M4And obtaining an output characteristic M' through two-layer convolution processing3；
Step (ii) of2.4: for output characteristic M ″)3After convolution processing, the characteristic M is compared with2Multiplying to obtain new characteristic M'2；
Step 2.5: fusing feature map M 'in a splicing manner'2And M ″)3And obtaining an output characteristic M' through two-layer convolution processing2；
Step 2.6: for output characteristic M ″)2After convolution processing, the characteristic M is compared with1Multiplying to obtain new characteristic M'1；
Step 2.7: fusion of feature M 'in a splicing manner'1And M ″)2And obtaining semantic features of semantic branch output through two-layer convolution processing
Figure BDA0003186887470000023
The step 3 comprises the following steps:
step 3.1: edge feature by splicing
Figure BDA0003186887470000031
And semantic features
Figure BDA0003186887470000032
Fusing, and performing convolution processing to obtain a new characteristic W';
step 3.2: carrying out global pooling, convolution and Sigmoid activation processing on the feature W 'in sequence to obtain a new feature W'1；
Step 3.3: mixing characteristics W 'and W'1Multiplication to obtain new characteristic W'2；
Step 3.4: mixing characteristics W 'and W'2Fusion was performed in an additive manner to obtain fused features F'.
The invention has the beneficial effects that:
the invention provides an edge perception image semantic segmentation method based on self-adaptive feature fusion, which is a new semantic segmentation method based on a residual error network (ResNet). The model constructed based on the method is a double-branch network structure model and comprises edge branches and semantic branches, wherein the edge branches are led out from the shallow part of the semantic branches, and the semantic branches adopt coding and decoding structures. In the edge branches, the added multi-scale cross fusion operation acquires image multi-scale features by superposing the void convolutions with different void ratios, meanwhile, the cross fusion among the branches can further improve the robustness of the multi-scale features, and the deep-layer features and the shallow-layer features are fused in the semantic branches based on a space attention mechanism, so that abundant space information contained in the shallow-layer features can be obtained, and simultaneously, a large amount of noise contained in the shallow-layer features can be filtered; and finally, fusing the semantic branch features and the edge branch features to further optimize the segmentation effect.
Drawings
FIG. 1 is a flow chart of an edge perception image semantic segmentation method based on adaptive feature fusion in the present invention;
FIG. 2 is a schematic diagram of the construction of a semantic segmentation model in the present invention;
FIG. 3 is a schematic diagram of the multi-scale cross-fusion operation of the present invention;
FIG. 4 is a schematic diagram of the fusion of deep and shallow features in the present invention;
FIG. 5 is a schematic diagram of edge feature and semantic feature fusion in the present invention.
Detailed Description
The invention is further described with reference to the following figures and specific examples.
As shown in fig. 1, an edge-aware image semantic segmentation method based on adaptive feature fusion includes:
the method comprises the following steps: making a data set, collecting N images and carrying out pixel-level classification labeling on each image, wherein each sample in the data set comprises one image and a pixel-level labeling result of the image;
the data set adopted in this embodiment is an ISPRS Vaihingen data set, which includes six categories: impervious surfaces, buildings, low-rise vegetation, trees, automobiles, backgrounds; the data set contained a total of 33 images with an average size of 2494 × 2046 and a spatial resolution of 9 cm.
And (3) data preprocessing, namely, in order to further improve the segmentation precision of the model, methods such as random inversion, random cutting, random scaling and the like are adopted for the training data set.
A schematic diagram for constructing a semantic segmentation model is shown in fig. 2, wherein the network overall structure comprises semantic branches and edge branches, the semantic branches are mainly used for extracting semantic features, and a space attention fusion module (SA) is inserted into the semantic branches for fusing shallow semantic features and deep semantic features; the edge branch is mainly used for extracting edge features, and a multi-scale cross fusion Module (MSB) is inserted into the edge branch for fully digging the edge features; finally, a channel adaptation module (CA) is referenced for the fusion of semantic features and edge features. Wherein, MSB: a multi-scale cross-fusion module; and SA: a spatial attention fusion module; CA: a channel adaptation module; mul: multiplying; add: adding; UpSample: upsampling; concat: splicing; conv: convolution; BN: carrying out batch standardization; ReLU: activating; output: and outputting the characteristics.
Step two: constructing an edge perception image semantic segmentation model based on self-adaptive feature fusion, wherein the edge perception image semantic segmentation model comprises an edge branch and semantic branch double-branch network structure by taking a ResNet network as a main body; the method comprises the following steps:
step 1: using a ResNet network model as a downsampling stage of semantic branching, leading out an edge branch at the downsampling stage of the semantic branching, performing multi-scale cross fusion operation on edge features in the edge branch, and extracting more detailed edge features, as shown in FIG. 3, performing convolution on the edge features through 3 void convolution layers with different porosity to obtain 3 features with different scales, and then performing cross fusion operation (instead of simple splicing or addition) on the 3 features to more fully extract edge information; the method comprises the following steps:
step 1.1: taking shallow features output by the semantic branch downsampling stage as input features of edge branches, and performing convolution processing;
step 1.2: processing the edge characteristics by using the void convolution layers with the porosity of 7, 5 and 3 respectively to obtain 3 characteristics F1、F2、F3；
Step 1.3: for feature F1Performing convolution treatment to obtain new characteristic F'1；
Step 1.4: feature F 'is spliced'1And F2Fusing to obtain new characteristic F'2；
Step 1.5: feature F 'is spliced'2And F3Fusing to obtain new characteristic F'3；
Step 1.6: feature F 'is spliced'1、F′2And F'3Carrying out fusion to obtain a new characteristic F';
step 1.7: the new feature F' is successively processed by convolution and upsampling to obtain the final edge feature
Figure BDA0003186887470000041
Step 2: the ResNet network is a down-sampling stage of semantic branching, and performs up-sampling on output features of the down-sampling stage of semantic branching, and performs fusion of deep features and shallow features in the up-sampling stage to obtain semantic features containing rich spatial information, as shown in fig. 4, including:
step 2.1: obtaining 4 features M from a downsampling stage of semantic branching1、M2、M3、M4；
Step 2.2: for feature M4After convolution processing, the characteristic M is compared with3Multiplying to obtain new characteristic M'3；
Step 2.3: fusion of feature M 'in a splicing manner'3And M4And obtaining an output characteristic M' through two-layer convolution processing3；
Step 2.4: for output characteristic M ″)3After convolution processing, the characteristic M is compared with2Multiplying to obtain new characteristic M'2；
Step 2.5: fusing feature map M 'in a splicing manner'2And M ″)3And obtaining an output characteristic M' through two-layer convolution processing2；
Step 2.6: for output characteristic M ″)2After convolution processing, the characteristic M is compared with1Multiplying to obtain new characteristic M'1；
Step 2.7: fusion of feature M 'in a splicing manner'1And M ″)2And obtaining semantic features of semantic branch output through two-layer convolution processing
Figure BDA0003186887470000051
And step 3: edge feature for edge branch output
Figure BDA0003186887470000052
Semantic features and semantic branching output
Figure BDA0003186887470000053
The fusion is performed to obtain a fused feature F', and a finer segmentation result is obtained, as shown in fig. 5, which includes:
step 3.1: edge feature by splicing
Figure BDA0003186887470000054
And semantic features
Figure BDA0003186887470000055
Fusing, and performing convolution processing to obtain a new characteristic W';
step 3.2: carrying out global pooling, convolution and Sigmoid activation processing on the feature W 'in sequence to obtain a new feature W'1；
Step 3.3: mixing characteristics W 'and W'1Multiplication to obtain new characteristic W'2；
Step 3.4: mixing characteristics W 'and W'2Fusing in an adding mode to obtain fused characteristics F', wherein the characteristics output in the step are a group of pixel data, and the pixel data are visualized and converted into an image to obtain a segmented image;
step three: dividing a data set into a training set and a testing set, wherein the training set is used for training a model, the testing set is used for evaluating the final performance of the model, the trained model is verified by using the testing set, images in the testing data set are input into the trained semantic segmentation model, the obtained semantic segmentation result is compared with the labels of the testing set, and the segmentation precision is calculated; visualizing the segmentation result of the input image to explicitly show the segmentation effect;
step four: and using the trained semantic segmentation model for image segmentation to obtain the segmentation result of the image.
In order to verify the effect of the invention, compared with the FCN, PSPNet, scaled FCN, RotEqNet and DeepLab v3+, which have better effect, the experimental data is ISPRS Vaihingen data set widely used in semantic segmentation. Table 1 gives the results of the comparative experiments.
When the model is trained, the image is cut into 512 × 512 sub-blocks by a sliding window, considering that the image resolution in the ISPRS Vaihingen data set is too large to be input into the model at one time. In addition, the model is built by a pyrrch frame, training is completed on a GeForce RTX 2080Ti, an Adam optimizer is adopted in the training process, the initial learning rate is set to be 0.001, in order to improve the stability of the model in the training stage and jump out the local minimum value, a learning rate reduction strategy of cosine annealing is adopted, the epoch times in the three annealing processes are respectively 10, 20 and 40, and finally the model with the highest precision on the verification set is selected as the final result.
Table 1: comparative experiment results
Figure BDA0003186887470000061
As can be seen from Table 1, the proposed edge perception image semantic segmentation method based on adaptive feature fusion is verified on an ISPRS Vaihingen data set, the obtained segmentation precision is higher than that of other networks, and the effectiveness of the method is proved.

Claims (5)

1.一种基于自适应特征融合的边缘感知图像语义分割方法，其特征在于，包括：1. an edge-aware image semantic segmentation method based on adaptive feature fusion, is characterized in that, comprises: 步骤一：制作数据集，收集N张图像并对每张图像进行像素级的分类标注，数据集中的每个样本包括一张图像和该图像的像素级标注结果；Step 1: Make a data set, collect N images and perform pixel-level classification and labeling on each image. Each sample in the data set includes an image and the pixel-level labeling result of the image; 步骤二：搭建基于自适应特征融合的边缘感知图像语义分割模型，所述边缘感知图像语义分割模型是一个以ResNet网络为主体包含边缘分支和语义分支的双分支网络结构；Step 2: build an edge-aware image semantic segmentation model based on adaptive feature fusion, and the edge-aware image semantic segmentation model is a dual-branch network structure with a ResNet network as the main body and including edge branches and semantic branches; 步骤三：将数据集分为训练集和测试集，利用训练集对模型进行训练，利用测试集对训练后的模型进行验证；Step 3: Divide the data set into a training set and a test set, use the training set to train the model, and use the test set to verify the trained model; 步骤四：将训练后的语义分割模型用于图像的分割，得到图像的分割结果。Step 4: Use the trained semantic segmentation model for image segmentation to obtain image segmentation results. 2.根据权利要求1所述的一种基于自适应特征融合的边缘感知图像语义分割方法，其特征在于，所述步骤二包括：2. a kind of edge-aware image semantic segmentation method based on adaptive feature fusion according to claim 1, is characterized in that, described step 2 comprises: 步骤1：用ResNet网络模型作为语义分支的下采样阶段，在语义分支的下采样阶段引出一条边缘分支，在边缘分支中对边缘特征进行多尺度交叉融合操作；Step 1: Use the ResNet network model as the downsampling stage of the semantic branch, introduce an edge branch in the downsampling stage of the semantic branch, and perform a multi-scale cross-fusion operation on the edge features in the edge branch; 步骤2：对语义分支下采样阶段的输出特征进行上采样，在上采样阶段进行深层特征和浅层特征的融合，获得含有丰富空间信息的语义特征；Step 2: Upsampling the output features of the semantic branch downsampling stage, and fuses deep features and shallow features in the upsampling stage to obtain semantic features with rich spatial information; 步骤3：对边缘分支输出的边缘特征
Figure FDA0003186887460000012
和语义分支输出的语义特征
Figure FDA0003186887460000013
进行融合得到融合后的特征F'。Step 3: Edge features output by edge branch
Figure FDA0003186887460000012
and semantic features output by the semantic branch
Figure FDA0003186887460000013
Perform fusion to obtain the fused feature F'. 3.根据权利要求1所述的一种基于自适应特征融合的边缘感知图像语义分割方法，其特征在于，所述步骤1包括：3. a kind of edge-aware image semantic segmentation method based on adaptive feature fusion according to claim 1, is characterized in that, described step 1 comprises: 步骤1.1：将语义分支下采样阶段输出的浅层特征作为边缘分支的输入特征，并进行卷积处理；Step 1.1: Use the shallow feature output from the downsampling stage of the semantic branch as the input feature of the edge branch, and perform convolution processing; 步骤1.2：分别利用孔率为7、5、3的空洞卷积层对边缘特征进行处理，得到3个特征F1、F2、F3；Step 1.2: Use the hole convolution layers with aperture ratios of 7, 5, and 3 to process the edge features respectively, and obtain 3 features F 1 , F 2 , and F 3 ; 步骤1.3：对特征F1进行卷积处理，得到新的特征F1'；Step 1.3: Convolve the feature F 1 to obtain a new feature F 1 '; 步骤1.4：以拼接的方式对特征F1'和F2进行融合，得到新的特征F′2；Step 1.4: fuse the features F 1 ′ and F 2 in a splicing manner to obtain a new feature F′ 2 ; 步骤1.5：以拼接的方式对特征F′2和F3进行融合，得到新的特征F′3；Step 1.5: fuse the features F′ 2 and F 3 in a splicing manner to obtain a new feature F′ 3 ; 步骤1.6：以拼接的方式对特征F1'、F′2和F′3进行融合，得到新的特征F”；Step 1.6: fuse the features F 1 ', F' 2 and F' 3 in a splicing manner to obtain a new feature F"; 步骤1.7：对新的特征F”先后进行卷积和上采样处理，得到最终的边缘特征
Figure FDA0003186887460000011
Step 1.7: Perform convolution and upsampling on the new feature F" to obtain the final edge feature
Figure FDA0003186887460000011
 4.根据权利要求1所述的一种基于自适应特征融合的边缘感知图像语义分割方法，其特征在于，所述步骤2包括：4. a kind of edge-aware image semantic segmentation method based on adaptive feature fusion according to claim 1, is characterized in that, described step 2 comprises: 步骤2.1：从语义分支的下采样阶段中得到4个特征M1、M2、M3、M4；Step 2.1: Obtain 4 features M 1 , M 2 , M 3 , M 4 from the down-sampling stage of the semantic branch; 步骤2.2：对特征M4进行卷积处理后与特征M3相乘得到新的特征M′3；Step 2.2: Convolve the feature M 4 and multiply it with the feature M 3 to obtain a new feature M′ 3 ; 步骤2.3：以拼接的方式融合特征M′3和M4，并经过两层卷积处理得到输出特征M”3；Step 2.3: fuse the features M' 3 and M 4 in a splicing manner, and obtain the output feature M' 3 through two layers of convolution processing; 步骤2.4：对输出特征M”3进行卷积处理后与特征M2相乘得到新的特征M'2；Step 2.4: After performing convolution processing on the output feature M" 3 , multiply it with the feature M 2 to obtain a new feature M'2; 步骤2.5：以拼接的方式融合特征图M'2和M”3，并经过两层卷积处理得到输出特征M″2；Step 2.5: fuse the feature maps M' 2 and M" 3 in a splicing manner, and obtain the output feature M" 2 through two layers of convolution processing; 步骤2.6：对输出特征M″2进行卷积处理后与特征M1相乘得到新的特征M'1；Step 2.6: After the output feature M″ 2 is subjected to convolution processing, it is multiplied with the feature M 1 to obtain a new feature M′ 1 ; 步骤2.7：以拼接的方式融合特征M'1和M″2，并经过两层卷积处理得到语义分支输出的语义特征
Figure FDA0003186887460000021
Step 2.7: Merge the features M' 1 and M" 2 by splicing, and obtain the semantic features output by the semantic branch through two-layer convolution processing
Figure FDA0003186887460000021
 5.根据权利要求1所述的一种基于自适应特征融合的边缘感知图像语义分割方法，其特征在于，所述步骤3包括：5. a kind of edge-aware image semantic segmentation method based on adaptive feature fusion according to claim 1, is characterized in that, described step 3 comprises: 步骤3.1：以拼接的方式对边缘特征
Figure FDA0003186887460000022
和语义特征
Figure FDA0003186887460000023
进行融合，再经过卷积处理得到新的特征W'；Step 3.1: Concatenate edge features
Figure FDA0003186887460000022
and semantic features
Figure FDA0003186887460000023
Fusion is performed, and then a new feature W' is obtained through convolution processing; 步骤3.2：对特征W'依次进行全局池化、卷积和Sigmoid激活处理，得到新的特征W1'；Step 3.2: Perform global pooling, convolution and sigmoid activation on the feature W' in turn to obtain a new feature W 1 '; 步骤3.3：将特征W'和W1'相乘得到新的特征W'2；Step 3.3: Multiply the features W' and W 1 ' to obtain a new feature W'2; 步骤3.4：将特征W'和W'2以相加的方式进行融合得到融合后的特征F'。Step 3.4: Fusing the features W' and W' 2 in an additive manner to obtain a fused feature F'.
CN202110864679.9A 2021-07-29 2021-07-29 Edge perception image semantic segmentation method based on self-adaptive feature fusion Active CN113658200B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110864679.9A CN113658200B (en) 2021-07-29 2021-07-29 Edge perception image semantic segmentation method based on self-adaptive feature fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110864679.9A CN113658200B (en) 2021-07-29 2021-07-29 Edge perception image semantic segmentation method based on self-adaptive feature fusion

Publications (2)

Publication Number Publication Date
CN113658200A true CN113658200A (en) 2021-11-16
CN113658200B CN113658200B (en) 2024-01-02

Family

ID=78490847

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110864679.9A Active CN113658200B (en) 2021-07-29 2021-07-29 Edge perception image semantic segmentation method based on self-adaptive feature fusion

Country Status (1)

Country Link
CN (1) CN113658200B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113449727A (en) * 2021-07-19 2021-09-28 中国电子科技集团公司第二十八研究所 Camouflage target detection and identification method based on deep neural network
CN113936204A (en) * 2021-11-22 2022-01-14 安徽师范大学 High-resolution remote sensing image cloud and snow identification method and device fusing terrain data and deep neural network
CN114463187A (en) * 2022-04-14 2022-05-10 合肥高维数据技术有限公司 Image semantic segmentation method and system based on aggregation edge features
CN114565628A (en) * 2022-03-23 2022-05-31 中南大学 A method and system for image segmentation based on boundary-aware attention
CN114648668A (en) * 2022-05-18 2022-06-21 浙江大华技术股份有限公司 Method and apparatus for classifying attributes of target object, and computer-readable storage medium

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170132769A1 (en) * 2015-11-05 2017-05-11 Google Inc. Edge-Aware Bilateral Image Processing
WO2018076212A1 (en) * 2016-10-26 2018-05-03 中国科学院自动化研究所 De-convolutional neural network-based scene semantic segmentation method
CN110263833A (en) * 2019-06-03 2019-09-20 韩慧慧 Based on coding-decoding structure image, semantic dividing method
US20200082541A1 (en) * 2018-09-11 2020-03-12 Apple Inc. Robust Use of Semantic Segmentation for Depth and Disparity Estimation
CN111127493A (en) * 2019-11-12 2020-05-08 中国矿业大学 Remote sensing image semantic segmentation method based on attention multi-scale feature fusion
CN111797779A (en) * 2020-07-08 2020-10-20 兰州交通大学 Remote sensing image semantic segmentation method based on regional attention multi-scale feature fusion
CN112541503A (en) * 2020-12-11 2021-03-23 南京邮电大学 Real-time semantic segmentation method based on context attention mechanism and information fusion
CN112634296A (en) * 2020-10-12 2021-04-09 深圳大学 RGB-D image semantic segmentation method and terminal for guiding edge information distillation through door mechanism
CN113033570A (en) * 2021-03-29 2021-06-25 同济大学 Image semantic segmentation method for improving fusion of void volume and multilevel characteristic information
CN113034505A (en) * 2021-04-30 2021-06-25 杭州师范大学 Glandular cell image segmentation method and device based on edge perception network
CN113159051A (en) * 2021-04-27 2021-07-23 长春理工大学 Remote sensing image lightweight semantic segmentation method based on edge decoupling

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170132769A1 (en) * 2015-11-05 2017-05-11 Google Inc. Edge-Aware Bilateral Image Processing
WO2018076212A1 (en) * 2016-10-26 2018-05-03 中国科学院自动化研究所 De-convolutional neural network-based scene semantic segmentation method
US20200082541A1 (en) * 2018-09-11 2020-03-12 Apple Inc. Robust Use of Semantic Segmentation for Depth and Disparity Estimation
CN110263833A (en) * 2019-06-03 2019-09-20 韩慧慧 Based on coding-decoding structure image, semantic dividing method
CN111127493A (en) * 2019-11-12 2020-05-08 中国矿业大学 Remote sensing image semantic segmentation method based on attention multi-scale feature fusion
CN111797779A (en) * 2020-07-08 2020-10-20 兰州交通大学 Remote sensing image semantic segmentation method based on regional attention multi-scale feature fusion
CN112634296A (en) * 2020-10-12 2021-04-09 深圳大学 RGB-D image semantic segmentation method and terminal for guiding edge information distillation through door mechanism
CN112541503A (en) * 2020-12-11 2021-03-23 南京邮电大学 Real-time semantic segmentation method based on context attention mechanism and information fusion
CN113033570A (en) * 2021-03-29 2021-06-25 同济大学 Image semantic segmentation method for improving fusion of void volume and multilevel characteristic information
CN113159051A (en) * 2021-04-27 2021-07-23 长春理工大学 Remote sensing image lightweight semantic segmentation method based on edge decoupling
CN113034505A (en) * 2021-04-30 2021-06-25 杭州师范大学 Glandular cell image segmentation method and device based on edge perception network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
岑仕杰;何元烈;陈小聪;: "结合注意力与无监督深度学习的单目深度估计", 广东工业大学学报, no. 04, pages 39 - 45 *
董子昊: "多类别的边缘感知方法在图像分割中的应用", 计算机辅助设计与图形学学报, vol. 31, no. 7, pages 1075 - 1085 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113449727A (en) * 2021-07-19 2021-09-28 中国电子科技集团公司第二十八研究所 Camouflage target detection and identification method based on deep neural network
CN113449727B (en) * 2021-07-19 2025-01-28 中国电子科技集团公司第二十八研究所 A camouflaged target detection and recognition method based on deep neural network
CN113936204A (en) * 2021-11-22 2022-01-14 安徽师范大学 High-resolution remote sensing image cloud and snow identification method and device fusing terrain data and deep neural network
CN113936204B (en) * 2021-11-22 2023-04-07 安徽师范大学 High-resolution remote sensing image cloud and snow identification method and device fusing terrain data and deep neural network
CN114565628A (en) * 2022-03-23 2022-05-31 中南大学 A method and system for image segmentation based on boundary-aware attention
CN114565628B (en) * 2022-03-23 2022-09-13 中南大学 Image segmentation method and system based on boundary perception attention
CN114463187A (en) * 2022-04-14 2022-05-10 合肥高维数据技术有限公司 Image semantic segmentation method and system based on aggregation edge features
CN114463187B (en) * 2022-04-14 2022-06-17 合肥高维数据技术有限公司 Image semantic segmentation method and system based on aggregation edge features
CN114648668A (en) * 2022-05-18 2022-06-21 浙江大华技术股份有限公司 Method and apparatus for classifying attributes of target object, and computer-readable storage medium

Also Published As

Publication number Publication date
CN113658200B (en) 2024-01-02

Similar Documents

Publication Publication Date Title
CN113658200A (en) An edge-aware image semantic segmentation method based on adaptive feature fusion
CN117078943B (en) Remote sensing image road segmentation method integrating multi-scale features and dual attention mechanism
CN114359130B (en) Road crack detection method based on unmanned aerial vehicle image
CN112541503A (en) Real-time semantic segmentation method based on context attention mechanism and information fusion
CN113642390A (en) A Street View Image Semantic Segmentation Method Based on Local Attention Network
CN111860233B (en) Method and system for extracting complex buildings from SAR images based on selective attention network
CN110853057B (en) Aerial image segmentation method based on global and multi-scale fully convolutional network
CN114943963A (en) Remote sensing image cloud and cloud shadow segmentation method based on double-branch fusion network
CN110020658B (en) Salient object detection method based on multitask deep learning
CN114387512B (en) Remote sensing image building extraction method based on multi-scale feature fusion and enhancement
CN114913327A (en) A Segmentation Algorithm of Lower Limb Skeleton CT Image Based on Improved U-Net
CN116012709B (en) High-resolution remote sensing image building extraction method and system
CN117274789B (en) Underwater crack semantic segmentation method for hydraulic concrete structure
CN115544613A (en) A Multimodal Data-Driven Automated Method for Urban Road Layout Design
CN117333669A (en) Remote sensing image semantic segmentation method, system and equipment based on useful information guidance
CN109508639B (en) Road scene semantic segmentation method based on multi-scale porous convolutional neural network
CN116778318A (en) A convolutional neural network remote sensing image road extraction model and method
CN113762396A (en) A method for semantic segmentation of two-dimensional images
Wu et al. Multiscale low-frequency memory network for improved feature extraction in convolutional neural networks
CN117456191B (en) Semantic segmentation method based on three-branch network structure under complex environment
CN113850151A (en) A method, device, terminal and storage medium for identifying driver's distracted behavior
CN116778346B (en) Pipeline identification method and system based on improved self-attention mechanism
CN116721351B (en) Remote sensing intelligent extraction method for road environment characteristics in overhead line channel
CN112991398B (en) Motion Boundary-Guided Optical Flow Filtering Based on Synergistic Deep Neural Networks
CN117710348A (en) Pavement crack detection method and system based on location information and attention mechanism

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant