CN111833386A - A Pyramid Binocular Stereo Matching Method Based on Multi-scale Information and Attention Mechanism - Google Patents

Abstract

The invention relates to a pyramid binocular stereo matching method based on multi-scale information and an attention mechanism, and belongs to the field of binocular vision in computer vision. The present invention first uses the channel attention mechanism with different size convolution kernels to obtain the information between pixels from the original image, then uses the empty space pyramid module to expand the receptive field to obtain multi-scale information, and finally uses the three-dimensional channel attention mechanism and the stacked The separable codec structure computes disparity to obtain depth information. Using the present invention to perform depth test on the corrected binocular image, the results show that the algorithm can not only improve the matching accuracy, but also reduce the amount of parameters and calculation of the model, and shorten the running time.

Description

A Pyramid Binocular Stereo Matching Based on Multi-scale Information and Attention Mechanism method

technical field

The invention relates to a pyramid binocular stereo matching method based on multi-scale information and an attention mechanism, and belongs to the field of binocular vision in computer vision.

Background technique

As a research hotspot in the field of computer vision, binocular stereo matching has a wide range of applications in 3D reconstruction, autonomous driving, and mobile robots. For a set of rectified stereo images captured by a binocular camera, the essence of stereo matching is to calculate the disparity of each pixel in the image. Generally speaking, stereo matching algorithms are mainly divided into two categories, one is the traditional algorithm, and the other is the method based on the convolutional neural network. The development of traditional stereo matching methods is limited due to the method of manually selecting features. With the development of deep learning, convolutional neural networks have demonstrated powerful computing power and feature extraction capabilities. Therefore, current research mainly focuses on neural network-based methods. However, it is still difficult to improve the information extraction ability of the network and obtain accurate disparity maps in ill-conditioned areas (such as weak texture areas, reflective surfaces, etc.).

SUMMARY OF THE INVENTION

In view of the above problems, the present invention proposes a pyramid binocular stereo matching method based on multi-scale information and attention mechanism. The method consists of three modules: adaptive feature extraction module, context information extraction module and disparity calculation module. For achieving the above object, the technical scheme of the present invention is:

A pyramid binocular stereo matching method based on multi-scale information and attention mechanism, comprising the following steps:

Step 1: Obtain the binocular image after parameter correction;

Step 2, using an adaptive feature extraction module to obtain the features of the weighted channel dimension. The adaptive feature extraction module is based on the residual network block, and the channel attention module with multiple convolution kernels is added, which can enable the network to obtain features of different scales, and can increase the weight occupied by the features with rich effective information, which is beneficial to Improve the subsequent matching accuracy, use the convolution layer to process the global pooled features to improve the learning ability of the network, and use the PReLU function as the activation function to retain more detailed information;

Step 3: Adopt the convolution branch with four different dilation rates and the hole space pyramid pooling structure of the global average pooling layer as the context information extraction module to obtain the multi-scale context information and global context information in the image as the information of the image dimension, In this way, the accuracy of the network in dealing with ill-conditioned regions is improved;

Step 4: Integrate the features in step 1 with the features in step 2 to construct a matching cost volume, and use the matching cost volume constructed by three-dimensional separable convolution processing in the disparity calculation module to calculate the depth information, and the disparity calculation module only retains each code. For the link between decoding structures, there are no other jump links between the encoding and decoding modules except the added channel attention module, which can effectively reduce the parameters of the network; at the same time, in order to ensure that the matching is not lost when the parameters are reduced Accuracy, and also added a 3D channel attention mechanism to the disparity calculation module.

Beneficial effects:

The present invention proposes a new end-to-end stereo matching network. By designing an adaptive feature extraction module and a multi-scale information extraction module, the extraction of local features and global features is realized. Then, a disparity calculation module is constructed by using 3D depthwise convolution and 3D channel attention mechanism, which increases the width of the network, which is beneficial to recover image details, thereby improving the matching accuracy in ill-conditioned regions.

Description of drawings

Fig. 1 is the algorithm flow chart in the present invention

2 is a schematic diagram of a feature extraction module in the present invention

3 is a schematic diagram of a disparity calculation module in the present invention

Detailed ways

The invention discloses a pyramid binocular stereo matching method based on a multi-scale context attention mechanism. The method fully considers the context information in the image while ensuring the detailed information between image pixels; meanwhile, the method is designed based on separable convolution and channel attention The disparity calculation module of the force mechanism calculates the depth information to obtain the disparity map. The method is further described below:

This method proposes an adaptive feature extraction module for binocular stereo matching. First, the fully connected layer is used to reduce the dimension of the network, and the channel attention mechanism is used to process the compressed features to generate the corresponding channel dimension weights. To make the network have better learning ability, use convolutional layers instead of fully connected layers. Since the PRELU function uses a linear operation in the negative part, it can solve the problem of neuron death in the negative part of the ReLU function. Therefore, the PRELU function is selected as the activation function to retain more feature information;

To improve the accuracy of stereo matching, it is also necessary to improve the ability of multi-scale feature extraction. A context information extraction module is constructed with a hole space pyramid structure: this module consists of four convolution branches with different hole rates and a global average pooling layer. This design can obtain information of different scales on the four branches and at the same time obtain image-level information through the global average pooling layer;

As the number of dimensions and network layers increases, the disparity calculation module generates a large number of parameters. This increases the computation time of the network. In order to reduce network parameters and save calculation time, only the links between the codec structures are retained in the disparity calculation module. There are no other jump links between the codec modules except the added channel attention module. At the same time, the use of The 3D separable convolution constructs the disparity computation unit. Adding a three-dimensional channel attention mechanism to the disparity calculation module ensures that the matching accuracy of the network will not decrease when the parameters are reduced.

Claims (3)

1. a pyramid binocular stereo matching method based on multi-scale information and attention mechanism, is characterized in that, comprises the steps: Step 1: Obtain the binocular image after parameter correction; Step 2, using an adaptive feature extraction module to obtain the features of the weighted channel dimension. The adaptive feature extraction module enables the network to obtain features of different scales by setting convolution kernels of different sizes in the same network layer. The convolution layer is used to process the global pooled features to improve the learning ability of the network, and the PReLU function is used as activation. The function retains more detailed information; Step 3, adopting the convolution branch with four different dilation rates and the hole space pyramid pooling structure of the global average pooling layer as the context information extraction module to obtain the multi-scale context information and global context information in the image as the information of the image dimension; Step 4: Integrate the features in step 1 with the features in step 2 to construct a matching cost volume, and use the three-dimensional separable convolution and three-dimensional channel attention mechanism to process the constructed matching cost volume in the disparity calculation module to calculate the depth information. The matching accuracy reduces the parameters of the network at the same time, and the disparity calculation module only retains the links between the various codec structures. There are no other jump links between the codec modules except the added channel attention module. This can effectively reduce the computational parameters of the model. 2. the pyramid binocular stereo matching method based on the multi-scale context attention mechanism as claimed in claim 1, it is characterized in that, designing adaptive feature extraction module and image-based information extraction module, has realized local feature and global feature. extract. 3. the pyramid binocular stereo matching method based on multi-scale context attention mechanism as claimed in claim 1, it is characterized in that, utilize 3D depth convolution and 3D channel attention mechanism to construct disparity calculation module, and only model does not have code. The skip link inside the decoding module can not only increase the width of the network, but also help to restore the image details, thereby improving the matching accuracy in the ill-conditioned area, and reducing the calculation parameters of the model.

Images