CN116797789A - A scene semantic segmentation method based on attention architecture - Google Patents

Abstract

The invention belongs to the technical field of computer vision, in particular to a scene semantic segmentation method based on an attention architecture, which comprises the following steps: step one: data preprocessing, namely providing data preparation for subsequent network model training; step two: training the model, namely training the constructed network model, and training parameters of the network model by utilizing mixed loss supervision in the whole training process, so as to obtain the optimal network weight of the scene semantic segmentation method based on the attention architecture by continuously reducing the loss optimization network model parameters; step three: the test of the model uses the network weight obtained through training to test the effect of semantic segmentation by inputting novel image data acquired by an external sensor. In order to enhance the characteristic representation capability of pixels, the invention utilizes the dual-attention module to model context information in the space dimension and the channel dimension respectively, thereby improving the characteristic representation capability of the whole model.

Description

A scene semantic segmentation method based on attention architecture

Technical field

The invention relates to the field of computer vision technology, specifically a scene semantic segmentation method based on attention architecture.

Background technique

In an era of rapid development in the artificial intelligence industry, autonomous driving technology is becoming more and more relevant to people's lives. In autonomous driving technology, it is very important to use computers to help the car understand the scene it is in. Only when the autonomous driving system can perceive the objects and people in the surrounding environment can it make correct and safe decisions. If the system is aware of the environment Misjudgment of people and things in the game may lead to very serious consequences.

Autonomous driving technology based on traditional algorithms first collects data on the surrounding environment through various sensors, then analyzes the data through traditional algorithms, and finally makes decisions to control the vehicle. Therefore, traditional algorithms have shortcomings such as low efficiency, inability to execute end-to-end, and low accuracy. In recent years, with the development of neural networks and improvements in computer computing power, autonomous driving technology based on deep learning has developed rapidly. First, the surrounding environment data is collected through the camera, and then the deep learning algorithm is used to perform feature extraction, image segmentation and vehicle decision-making end-to-end, which not only improves the processing speed but also greatly improves the accuracy. Compared with expensive lidar sensors, images collected by cheap cameras can significantly reduce costs and further promote the implementation of autonomous driving technology. In order to ensure vehicle driving safety, autonomous driving technology has high precision requirements for the surrounding environment.

The purpose of image semantic segmentation is to classify different pixels according to their semantic categories. Compared with traditional segmentation, semantic segmentation achieves pixel-level classification. The semantic segmentation result of the image not only contains the position information of the semantic category but also the detailed boundary and posture information. Therefore, such fine results can make the judgment of the vehicle's drivable area more accurate, and the object category and shape judgment more accurate. Nowadays, The main scene in the field of autonomous driving is urban scenes, so urban scene semantic segmentation is an important field.

The current mainstream semantic segmentation frameworks are basically evolved based on full-volume machine neural networks. However, there are still some problems when using image semantic segmentation algorithms in autonomous driving systems:

(1) The size of objects in autonomous driving scenes changes greatly. Existing algorithms have different accuracy in segmenting targets of different sizes and are not suitable for small target objects.

(2) The autonomous driving scene is complex, with problems such as large differences between light and dark, and a large number of occlusions. Target recognition is difficult and target edges are blurred. Many current algorithms are not suitable for detecting target edges.

Therefore, this work is committed to using advanced attention mechanisms to solve the above problems, thereby improving the accuracy of semantic segmentation and providing new solutions for autonomous driving technology.

Contents of the invention

(1) Technical problems solved

In view of the shortcomings of the existing technology, the present invention provides a scene semantic segmentation method based on attention architecture, which solves the problems raised in the above background technology.

(2) Technical solutions

In order to achieve the above object, the present invention specifically adopts the following technical solutions:

A scene semantic segmentation method based on attention architecture, which includes the following steps:

Step 1: Data preprocessing to provide data preparation for subsequent network model training;

Step 2: Model training. The constructed network model is trained. During the entire training process, hybrid loss is used to supervise the training of network model parameters. The network model parameters are optimized by continuously reducing the loss to obtain scene semantics based on the attention architecture. Optimal network weights for segmentation methods;

Step 3: Test the model by inputting new image data collected by external sensors and using the network weights obtained through training to test the effect of semantic segmentation.

Further, the data preprocessing in step one includes:

The original input data is randomly and arbitrarily cropped through data preprocessing work for data expansion, and then placed in a regenerated folder. The folder is full of cropped sample images used for training. The final cropped size is 768x 768 .

Further, the training of the model in step two includes the following steps:

The prepared sample images are fed into the network model for training. This network model includes three parts: one is using the residual network Resnet with expansion strategy, and the other is a lightweight network that includes channel attention and spatial attention. Symmetric dual attention module, one is an adaptive selection interaction module that fuses low-level features with high-level features.

Further, the testing of the step three model includes:

Use the trained weight parameters to test the segmentation effect in the new sensor-collected images.

(3) Beneficial effects

Compared with the existing technology, the present invention provides a scene semantic segmentation method based on attention architecture, which has the following beneficial effects:

In order to enhance the feature representation ability of pixels, the present invention uses dual attention modules to model context information in the spatial dimension and channel dimension respectively, thereby improving the overall feature expression ability of the model.

This invention uses high-level feature maps to have low resolution, but they always contain rich semantic information, so it can provide guidance for generating low-level feature maps with more semantic information; in addition, low-level feature maps have more features than high-level feature maps. Spatial information can provide spatial guidance for high-level feature maps, and the semantic segmentation effect can be further improved through effective feature fusion.

Description of the drawings

Figure 1 is an overall network structure diagram of the present invention;

Figure 2 is a structural diagram of the feature fusion module of the present invention;

Figure 3 is a structural diagram of the lightweight dual attention module of the present invention;

Figure 4 is a diagram showing the segmentation effect of each module of the present invention on the data set;

Figure 5 is an evaluation diagram of the present invention on the Cityscapes test data set.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

Example

As shown in Figures 1-5, one embodiment of the present invention proposes a scene semantic segmentation method based on attention architecture. The method includes the following steps:

Step 1: Data preprocessing to provide data preparation for subsequent network model training;

The specific operations of data preprocessing according to the present invention include the following steps:

The original input data is randomly and arbitrarily cropped for data expansion, and then placed in a regenerated folder. The folder is full of cropped sample images used for training, and the final cropped size is 768x 768.

Step 2: Model training. The constructed network model is trained. During the entire training process, hybrid loss is used to supervise the training of network model parameters. The network model parameters are optimized by continuously reducing the loss to obtain scene semantics based on the attention architecture. Optimal network weights for segmentation methods;

The training of the model in step two of the present invention includes the following steps:

The prepared sample images are fed into the network model for training. This network model includes three parts: one is using the residual network Resnet with expansion strategy, and the other is a lightweight network that includes channel attention and spatial attention. Symmetric dual attention module, one is an adaptive selection interaction module that fuses low-level features with high-level features;

The first part is the residual network Resnet used for feature extraction. This invention uses the expansion strategy based on the original residual network to remove the down-sampling operation of the last two layers of the original network to retain more details to facilitate the results of semantic segmentation. , so that the final feature extraction network output feature map is 1/8 of the original image. By extracting features from the input original image information, 4-level features from Res-1 to Res-4 are finally obtained;

The second part is a lightweight symmetric dual attention structure used to enhance high-level feature representation. The output of the highest layer of the feature extraction backbone network Resnet is passed to a hybrid attention mechanism of spatial and channel dimensions, making full use of the interaction between high-level features. Correspondence, appropriately integrating spatial information and semantic information; this model uses average pooling and maximum pooling to replace the self-attention method that calculates the correlation between pixels at all positions, avoiding the high cost of self-attention Computational costs and GPU memory usage reduce the model’s dependence on hardware;

The details are as follows:

The spatial pooling attention model first obtains two new feature vectors through parallel adaptive global average pooling and adaptive global maximum pooling.

and/>

Then the two are spliced along the channel dimension to obtain fusion features

Next, the feature channel is reduced to 1 through 1×1 convolution, and then the spatial attention weight map is obtained through the sigmoid activation function.

Finally, the discriminative features of each position are weighted through the spatial attention weight map, and then the result of the weighted operation is added to X to obtain the final output E _spatial ; the specific operation is defined as follows:

On the other hand, the channel pooling attention model effectively extracts discriminative channel information with lower computational complexity. First, input the feature map X∈R ^C×H×W , and also generate two new feature vectors by using adaptive average pooling and adaptive maximum pooling in the spatial dimension. and/>

Then, in order to reduce the amount of calculation, 1x1 convolution→Rule→1x1 convolution is used to first reduce the dimension and then increase the dimension, and finally obtain two new feature maps. and/>

Then, after superimposing the two channel feature maps, the channel attention weight map is obtained through sigmoid activation Finally, the discriminative characteristics of each channel of X are measured through the channel attention weight map, and then the result of the weight operation is added to X to obtain the final output E _channel , as shown below:

Finally, the feature map generated by the channel attention path connects the spatial attention path. Since the channel attention focuses on what it is and the spatial attention focuses on where it is, the dual attention module utilizes a hybrid attention mechanism based on the above two, This enables it to effectively utilize high-level semantic information to enhance network pixel representation.

The third part is the adaptive selection interaction structure, which aims to more effectively combine deep semantic information with shallow contour information to generate more "semantic" low-level feature maps and more accurate high-level feature maps. It contains a channel attention module that effectively bridges the gap between high-level feature maps with extensive semantic information and low-level feature maps with precise spatial information.

This module can learn the importance of each feature channel to compensate for the difference between high- and low-level features, thereby better guiding the fusion function of these features instead of simply adding them. First, upsampling is utilized to make the high-level and low-level feature maps the same size. Global average pooling provides the largest acceptable domain for global context information. Global average pooling maps feature vectors by compressing their features and feeds it back to the fully connected layer to learn the weight of each feature channel. The fused feature map is normalized to 0 and 1 using the sigmoid function to obtain the channel attention weight map. By multiplying the original feature map with the obtained channel attention weight, important information can be adaptively selected so that the low-level feature map has more high-level semantic information, thereby obtaining an enhanced low-level semantic feature map. At the same time, under the influence of the channel attention weight value, the geometric information of the low-level feature map can be used to guide high-level features, and then gradually restore the edge detail features. Finally, the enhanced high-level and low-level features are combined by pixel-wise summation.

Step 3: Test the model by inputting new image data collected by external sensors and using the network weights obtained through training to test the effect of semantic segmentation;

Testing of the model of the present invention includes;

Use the trained weight parameters to test the segmentation effect on new image data.

Due to the diversity of autonomous driving scenes, the sizes of different objects in different scenes vary greatly, they block each other a lot, and target identification is difficult. Therefore, the result performance also greatly depends on the complexity of the segmented scene, and the existing semantic segmentation algorithms are due to A series of convolution and pooling operations cause "resolution loss", which leads to insufficient segmentation accuracy of small target objects and "blurred segmentation edges". First, the present invention uses a dual attention module with spatial attention and channel attention. Contextual information is modeled in the spatial dimension and channel dimension respectively to improve the overall feature expression ability of the model. Secondly, the present invention utilizes the characteristics that high-level features are rich in semantics but lack geometric spatial detail information, while low-level features contain accurate detailed outlines but lack semantics. An adaptive selection interaction module is used to more fully guide the fusion of high-level and low-level features. Finally, in the test comparison experiment, it is proved that the present invention can not only extract long-term relevant contextual information, but also more effectively promote the combination of semantic information and contour information. , ultimately achieving a balance between segmentation accuracy and model complexity.

Cityscapes, a commonly used data set in the field of autonomous driving, is used for network model training, and the training effect of this method is tested based on the evaluation tool of the Cityscapes data set. From the data in Table 1, it can be found that the scene semantic segmentation algorithm proposed by the present invention has better performance on the test data than the weight parameters trained by other algorithms.

Figure 5 is the evaluation of the Cityscapes test data set;

In order to more intuitively feel the effectiveness of the present invention compared to existing algorithms, the segmentation results of the lightweight dual attention structure and adaptive selection interaction structure in the present invention are further visualized, as shown in Figure 4. Since the dual attention structure can model the upper and lower information more effectively and enhance the correlation between pixels, adding the dual attention structure on the basis of the original network can effectively avoid some intra-class classification errors. When the adaptive selection interaction structure is further added, low-level and high-level feature representations are enhanced by suppressing non-correlated channels, thereby achieving better fusion effects. Therefore, it can be seen from the fourth row in Figure 4 that adding an adaptive selection interaction structure to fuse low-level features allows the lost details to be restored and the edge segmentation between different categories to be more refined.

Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, it is still The technical solutions described in the foregoing embodiments may be modified, or some of the technical features may be equivalently replaced. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims (4)

1. A scene semantic segmentation method based on attention architecture, characterized in that: the method includes the following steps: Step 1: Data preprocessing to provide data preparation for subsequent network model training; Step 2: Model training. The constructed network model is trained. During the entire training process, hybrid loss is used to supervise the training of network model parameters. The network model parameters are optimized by continuously reducing the loss to obtain scene semantics based on the attention architecture. Optimal network weights for segmentation methods; Step 3: Test the model by inputting new image data collected by external sensors and using the network weights obtained through training to test the effect of semantic segmentation. 2. A scene semantic segmentation method based on attention architecture according to claim 1, characterized in that: the data preprocessing in step one includes: The original input data is randomly and arbitrarily cropped through data preprocessing work for data expansion, and then placed in a regenerated folder. The folder is full of cropped sample images used for training, and the final cropped size is 768x768. 3. A scene semantic segmentation method based on attention architecture according to claim 1, characterized in that: the training of the model in step 2 includes the following steps: The prepared sample images are fed into the network model for training. This network model includes three parts: one is using the residual network Resnet with expansion strategy, and the other is a lightweight network that includes channel attention and spatial attention. Symmetric dual attention module, one is an adaptive selection interaction module that fuses low-level features with high-level features. 4. A scene semantic segmentation method based on attention architecture according to claim 1, characterized in that: the test of the step three model includes: Use the trained weight parameters to test the segmentation effect in the new sensor-collected images.