CN109460815A - A kind of monocular depth estimation method - Google Patents

Abstract

The invention discloses a monocular visual depth estimation method, which firstly constructs a convolutional neural network, which includes an input layer, a hidden layer and an output layer; the hidden layer includes an encoding frame, a decoding frame and an upsampling frame; and then uses a training set As the original input image, the monocular image is input into the convolutional neural network for training, and the estimated depth image corresponding to each original monocular image in the training set is obtained; then the estimated depth image corresponding to the monocular image in the training set is calculated. The loss function value between the corresponding real depth image and the convolutional neural network training model, the optimal weight vector and the optimal bias term are obtained; then the monocular image to be predicted is input into the convolutional neural network training model , and use the optimal weight vector and the optimal bias term to predict the corresponding predicted depth image; the advantage is that its prediction accuracy is high.

Description

A Monocular Vision Depth Estimation Method

technical field

The invention relates to an image signal processing technology, in particular to a monocular visual depth estimation method.

Background technique

The rapid development of the economy has brought about the continuous improvement of people's living standards. As people's requirements for a good quality of life gradually increase, the convenience of transportation is also getting better and better. As an important part of transportation, the development of automobiles has been paid more and more attention. In the age of artificial intelligence, driverless cars are also one of the hottest topics in recent years, and after Baidu announced that driverless cars will enter mass production and will be put into use, the upsurge of driverless cars continues to increase. The monocular visual depth estimation in front of the car is a part of the field of unmanned driving, which can effectively ensure the safety of the car during driving.

The methods of monocular visual depth estimation mainly include traditional methods and deep learning methods. Before the emergence of deep learning methods, the results obtained by relying on traditional methods for depth estimation are far from meeting people's minimum expectations; after the emergence of deep learning methods, end-to-end training methods are used in deep learning, using a large number of training The accuracy of the results obtained by depth estimation after learning has been greatly improved. Eigen et al. further improved the neural network proposed in the document "Depth Map Prediction from a Single Image using a Multi-Scale Deep Network" ("Depth Map Prediction of Single Image Based on Multi-Scale Deep Network"). Single Image Depth Map Prediction Based on Multi-scale Deep Networks" proposed to use two-scale neural networks for depth estimation: a coarse-scale network to predict the global depth distribution and a fine-scale network to locally refine the depth map, and Eigen et al. Based on the neural network of two scales, it is extended to three scales. The three-scale neural network architecture first uses the first scale to predict a rougher result based on the entire image area, then uses the second scale to optimize it on the basis of medium resolution, and finally uses the third scale. The scales upsample the results and refine and refine the predicted depth map. However, the three-scale neural network architecture is proposed for the joint prediction of three different computer vision tasks: depth prediction, surface normal estimation, and semantic segmentation. If it is used for depth estimation alone, the accuracy of depth estimation is not very high, and the final predicted depth map is only half the size of the original image, and the inconsistency of the size is not conducive to the direct use of depth information. .

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a monocular visual depth estimation method with high prediction accuracy.

The technical solution adopted by the present invention to solve the above technical problems is: a monocular visual depth estimation method, which is characterized in that it includes two processes: a training phase and a testing phase;

The specific steps of the training phase process are:

Step 1_1: Select N original monocular images and the real depth image corresponding to each original monocular image, and form a training set, and record the nth original monocular image in the training set as {Q ⁿ (x, y)}, denote the real depth image corresponding to {Q ⁿ (x, y)} in the training set as Among them, N is a positive integer, N≥100, n is a positive integer, 1≤n≤N, 1≤x≤R, 1≤y≤L, R represents {Q ⁿ (x, y)} and The width of , L represents {Q ⁿ (x, y)} and The height of , R and L are both divisible by 2, Q ⁿ (x, y) represents the pixel value of the pixel at the coordinate position (x, y) in {Q ⁿ (x, y)}, express The pixel value of the pixel whose middle coordinate position is (x, y);

Step 1_2: Build an end-to-end convolutional neural network: the convolutional neural network includes an input layer, a hidden layer, and an output layer; the hidden layer includes an encoding frame, a decoding frame, and an upsampling frame;

For the input layer, the input end of the input layer receives an original input image, and the output end of the input layer outputs the original input image to the hidden layer; among them, the width of the original input image received by the input end of the input layer is required to be R and the height is L ;

For the encoding framework, it consists of the first convolutional layer, the first normalization layer, the first activation layer, the first max pooling layer, the second convolutional layer, the second normalization layer, the second activation layer, 3rd Convolutional Layer, 3rd Batch Normalization Layer, 1st Concatenate Fusion Layer, 3rd Activation Layer, 2nd Max Pooling Layer, 4th Convolutional Layer, 4th Batch Normalization Layer, 4th Activation Layer, 5th Volume Convolution layer, fifth batch normalization layer, second concatenate fusion layer, fifth activation layer, third max pooling layer, first convolutional layer with holes, sixth batch normalization layer, sixth activation layer, second hole Convolutional layer, seventh batch of normalization layer, third Concatenate fusion layer, seventh activation layer, fourth maximum pooling layer, third convolutional layer with holes, eighth batch of normalization layer, eighth activation layer; The code frame is composed of the first deconvolution layer, the ninth batch of normalization layers, the fourth Concatenate fusion layer, the ninth activation layer, the sixth convolution layer, the tenth batch of normalization layers, the tenth activation layer, the fourth batch of The second deconvolution layer, the eleventh normalization layer, the fifth concatenate fusion layer, the eleventh activation layer, the seventh convolution layer, the twelfth normalization layer, the twelfth activation layer, the third deconvolution layer , the thirteenth batch of normalization layer, the sixth Concatenate fusion layer, the thirteenth activation layer, the eighth convolution layer, the fourteenth batch of normalization layer, the fourteenth activation layer, the fourth deconvolution layer, the fifteenth batch The normalization layer and the seventh Concatenate fusion layer are composed; for the upsampling framework, it consists of the first upsampling layer, the tenth convolutional layer, the seventeenth batch normalization layer, the seventeenth activation layer, and the second upsampling layer. , the eleventh convolutional layer, the eighteenth batch of normalization layers, the eighteenth activation layer, the third upsampling layer, the twelfth convolutional layer, the nineteenth batch of normalization layers, the nineteenth activation layer, the fourth upper The sampling layer, the thirteenth convolutional layer, the twentieth batch of normalization layers, and the twentieth activation layer are composed; for the output layer, it consists of the fifteenth activation layer, the ninth convolutional layer, and the sixteenth batch of normalization set in sequence. layer and the sixteenth activation layer, among which, the first convolutional layer to the thirteenth convolutional layer, the first convolutional layer with holes to the third convolutional layer with holes, the first deconvolution layer to the fourth inverse convolutional layer The convolution kernel size of each convolution layer is 3 × 3, the number of convolution kernels of the first convolution layer is 32, the number of convolution kernels of the second convolution layer and the third convolution layer is 64, and the number of convolution kernels of the fourth convolution layer is 64. The number of convolution kernels of the convolutional layer and the fifth convolutional layer is 128, the number of convolution kernels of the first convolutional layer and the second convolutional layer with holes is 256, and the number of convolutional kernels of the third convolutional layer with holes The number of convolution kernels is 512, the number of convolution kernels of the first deconvolution layer and the sixth convolution layer is 256, the number of convolution kernels of the second deconvolution layer and the seventh convolution layer is 128, The number of convolution kernels in the third deconvolution layer and the eighth convolution layer is 64, the number of convolution kernels in the fourth deconvolution layer is 32, and the number of convolution kernels in the ninth convolution layer is 1. The number of convolution kernels in the tenth convolution layer is 256, the number of convolution kernels in the eleventh convolution layer is 128, the number of convolution kernels in the twelfth convolution layer is 64, and the number of convolution kernels in the thirteenth convolution layer is 64. The number of convolution kernels is 32, the first The convolution strides from the first convolutional layer to the thirteenth convolutional layer, the first convolutional convolutional layer to the third convolutional convolutional layer with holes adopt the default values, and the first deconvolutional layer to the fourth deconvolutional layer The respective convolution step size is 2×2, the parameters of the first batch of normalization layers to the twentieth batch of normalization layers use default values, the activation functions of the first activation layer to the twentieth activation layer use ReLu, and the first maximum pooling is used. The pooling step size from the layer to the fourth maximum pooling layer is 2×2, and the sampling step size from the first upsampling layer to the fourth upsampling layer is 2×2;

For the coding framework, the input of the first convolutional layer receives the original input image output by the output of the input layer, the output of the first convolutional layer outputs 32 feature maps, and the set of all the output feature maps is denoted as J ₁ , where the width of each feature map in J ₁ is R and the height is L; the input end of the first batch of normalization layers receives all the feature maps in J ₁ , and the output end of the first batch of normalization layers outputs 32 features The set of all output feature maps is denoted as P ₁ , wherein the width of each feature map in P ₁ is R and the height is L; the input end of the first activation layer receives all feature maps in P ₁ , the output end of the first activation layer outputs 32 feature maps, and the set formed by all the output feature maps is denoted as H ₁ , wherein the width of each feature map in H ₁ is R and the height is L; the first largest The input end of the pooling layer receives all the feature maps in H ₁ , the output end of the first maximum pooling layer outputs 32 feature maps, and the set formed by all the output feature maps is denoted as Z ₁ , wherein, in Z ₁ The width of each feature map is height is The input of the second convolutional layer receives all the feature maps in Z ₁ , the output of the second convolutional layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₂ , where in J ₂ The width of each feature map is height is The input end of the _second batch of normalization layers receives all feature maps in J ₂ , and the output end of the _second batch of normalization layers outputs 64 feature maps. The width of each feature map is height is The input terminal of the _second activation layer receives all feature maps in P ₂ , and the output terminal of the _second activation layer outputs 64 feature maps. The width of the feature map is height is The input end of the third convolutional layer receives all the feature maps in H ₂ , and the output end of the _third convolution layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₃ , where in The width of each feature map is height is The input end of the third batch of normalization layers receives all feature maps in J ₃ , the output end of the third batch of normalization layers outputs 64 feature maps, and the set formed by all the output feature maps is denoted as P ₃ , where in P ₃ The width of each feature map is height is The input of the first Concatenate fusion layer receives all the feature maps in _P3 and all the feature maps in _H2 , and the output of the first Concatenate fusion layer outputs 128 feature maps, and the set composed of all the output feature maps is recorded as C ₁ , where the width of each feature map in C ₁ is height is The input terminal of the _third activation layer receives all feature maps in C ₁ , and the output terminal of the _third activation layer outputs 128 feature maps. The width of the feature map is height is The input of the second maximum pooling layer receives all the feature maps in H ₃ , the output of the second maximum pooling layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as Z ₂ , where Z The width of each feature map in ₂ is height is The input end of the fourth convolutional layer receives all the feature maps in Z ₂ , and the output end of the fourth convolution layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₄ , where in J ₄ The width of each feature map is height is The input terminal of the _fourth batch of normalization layers receives all feature maps in J ₄ , and the output terminal of the _fourth batch of normalization layers outputs 128 feature maps. The width of each feature map is height is The input end of the _fourth activation layer receives all the feature maps in P ₄ , and the output end of the _fourth activation layer outputs 128 feature maps. The width of the feature map is height is The input end of the fifth convolutional layer receives all the feature maps in H ₄ , and the output end of the fifth convolution layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₅ , where in J ₅ The width of each feature map is height is The input end of the fifth batch of normalization layers receives all feature maps in J ₅ , and the output end of the fifth batch of normalization layers outputs 128 feature maps, and the set formed by all the output feature maps is denoted as P ₅ , where in P ₅ The width of each feature map is height is The input end of the second Concatenate fusion layer receives all the feature maps in P ₅ and all the feature maps in H ₄ , and the output end of the second Concatenate fusion layer outputs 256 feature maps, and the set formed by all the output feature maps is recorded as C ₂ , where the width of each feature map in C ₂ is height is The input end of the _fifth activation layer receives all the feature maps in C ₂ , and the output end of the _fifth activation layer outputs 256 feature maps. The width of the feature map is height is The input terminal of the third maximum pooling layer receives all the feature maps in H ₅ , the output terminal of the third maximum pooling layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as Z ₃ , where Z 3 The width of each feature map in ₃ is height is The input of the first atrous convolutional layer receives all feature maps in Z ₃ , the output of the first atrous convolutional layer outputs 256 feature maps, and the set of all the output feature maps is denoted as K ₁ , where , the width _of each feature map in K1 is height is The input end of the sixth batch of normalization layers receives all the feature maps in K ₁ , the output end of the sixth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₆ , where in P ₆ The width of each feature map is height is The input end of the sixth activation layer receives all the feature maps in P ₆ , and the output end of the sixth activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H ₆ , where each of the feature maps in H ₆ The width of the feature map is height is The input end of the second atrous convolutional layer receives all the feature maps in H ₆ , the output end of the second atrous convolutional layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as K ₂ , where , the width of each feature map in _K2 is height is The input end of the seventh batch of normalization layers receives all the feature maps in K ₂ , the output end of the seventh batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₇ , where in P ₇ The width of each feature map is height is The input end of the third Concatenate fusion layer receives all the feature maps in P ₇ and all the feature maps in H ₆ , and the output end of the third Concatenate fusion layer outputs 512 feature maps, and the set composed of all the output feature maps is denoted as C ₃ , where the width of each feature map in C ₃ is height is The input terminal of the seventh activation layer receives all the feature maps in C ₃ , and the output terminal of the _seventh activation layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as H ₇ . The width of the feature map is height is The input end of the fourth maximum pooling layer receives all the feature maps in H ₇ , and the output end of the fourth maximum pooling layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as Z ₄ , where Z 4 The width of each feature map in ₄ is height is The input of the third atrous convolutional layer receives all the feature maps in Z ₄ , the output of the third atrous convolutional layer outputs 512 feature maps, and the set of all the output feature maps is denoted as K ₃ , where , the width _of each feature map in K3 is height is The input end of the eighth batch of normalization layers receives all the feature maps in K ₃ , the output end of the eighth batch of normalization layers outputs 512 feature maps, and the set formed by all the output feature maps is denoted as P ₈ , where in P ₈ The width of each feature map is height is The input end of the _eighth activation layer receives all the feature maps in P8, and the output end of the eighth activation layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as H ₈ , and H ₈ is also the coding frame. _The output of , where the width of each feature map in H8 is height is

For the decoding framework, the input end of the first deconvolution layer receives the output of the encoding framework, that is, all feature maps in _H8 , and the output end of the first deconvolution layer outputs 256 feature maps, which are composed of all the output feature maps. The set of is denoted as F ₁ , where the width of each feature map in F ₁ is height is The input end of the ninth batch of normalization layers receives all the feature maps in F ₁ , and the output end of the ninth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₉ , where in P ₉ The width of each feature map is height is The input end of the fourth Concatenate fusion layer receives all the feature maps in P ₉ and all the feature maps in P ₇ , and the output end of the fourth Concatenate fusion layer outputs 512 feature maps, and the set composed of all the output feature maps is recorded as C ₄ , where the width of each feature map in C ₄ is height is The input end of the _ninth activation layer receives all the feature maps in C ₄ , and the output end of the _ninth activation layer outputs 512 feature maps. The width of the feature map is height is The input end of the sixth convolutional layer receives all the feature maps in H ₉ , the output end of the sixth convolution layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as J ₆ , where in J ₆ The width of each feature map is height is The input end of the tenth batch of normalization layers receives all the feature maps in J ₆ , the output end of the tenth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₁₀ , where in P ₁₀ The width of each feature map is height is The input end of the tenth activation layer receives all the feature maps in P ₁₀ , and the output end of the tenth activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H ₁₀ , where each H ₁₀ The width of the feature map is height is The input end of the second deconvolution layer receives the output of the coding framework, that is, all feature maps in _H10 , and the output end of the second deconvolution layer outputs 128 feature maps, and the set of all the output feature maps is denoted as F ₂ , where the width of each feature map in _F2 is height is The input end of the eleventh batch of normalization layers receives all the feature maps in F ₂ , and the output end of the eleventh batch of normalization layers outputs 128 feature maps, and the set of all the output feature maps is denoted as P ₁₁ , where P The width of each feature map in ₁₁ is height is The input of the fifth Concatenate fusion layer receives all the feature maps in P ₁₁ and all the feature maps in P ₅ , and the output of the fifth Concatenate fusion layer outputs 256 feature maps, and the set of all the output feature maps is recorded as C ₅ , where the width of each feature map in C ₅ is height is The input end of the eleventh activation layer receives all the feature maps in _C5 , and the output end of the _{eleventh activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H 11 .} The width of each feature map is height is The input end of the seventh convolutional layer receives all the feature maps in H ₁₁ , the output end of the seventh convolution layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₇ , where in J ₇ The width of each feature map is height is The input end of the twelfth batch of normalization layers receives all feature maps in J ₇ , the output end of the twelfth batch of normalization layers outputs 128 feature maps, and the set formed by all the output feature maps is denoted as P ₁₂ , where P The width of each feature map in ₁₂ is height is The input end of the twelfth activation layer receives all the feature maps in P ₁₂ , the output end of the twelfth activation layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as H ₁₂ , where in H ₁₂ The width of each feature map is height is The input end of the third deconvolution layer receives all the feature maps in H ₁₂ , the output end of the third deconvolution layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as F ₃ , where F The width of each feature map in ₃ is height is The input end of the thirteenth batch of normalization layers receives all the feature maps in F ₃ , the output end of the thirteenth batch of normalization layers outputs 64 feature maps, and the set of all the output feature maps is denoted as P ₁₃ , where P The width of each feature map in ₁₃ is height is The input end of the sixth Concatenate fusion layer receives all the feature maps in P ₁₃ and all the feature maps in P ₃ , and the output end of the sixth Concatenate fusion layer outputs 128 feature maps, and the set composed of all the output feature maps is recorded as C ₆ , where the width of each feature map in C ₆ is height is The input end of the thirteenth activation layer receives all the feature maps in C ₆ , the output end of the thirteenth activation layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as H ₁₃ , wherein, in H ₁₃ The width of each feature map is height is The input end of the eighth convolutional layer receives all the feature maps in H ₁₃ , the output end of the eighth convolution layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₈ , where in J ₈ The width of each feature map is height is The input end of the fourteenth batch of normalization layers receives all the feature maps in J ₈ , the output end of the fourteenth batch of normalization layers outputs 64 feature maps, and the set formed by all the output feature maps is denoted as P ₁₄ , where P The width of each feature map in ₁₄ is height is The input end of the fourteenth activation layer receives all the feature maps in P ₁₄ , the output end of the fourteenth activation layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as H ₁₄ , wherein, in H ₁₄ The width of each feature map is height is The input end of the fourth deconvolution layer receives all the feature maps in H ₁₄ , the output end of the fourth deconvolution layer outputs 32 feature maps, and the set formed by all the output feature maps is denoted as F ₄ , where F The width of each feature map in ₄ is R and the height is L; the input end of the fifteenth batch of normalization layers receives all the feature maps in F ₄ , and the output end of the fifteenth batch of normalization layers outputs 32 feature maps. The set composed of all the output feature maps is denoted as P ₁₅ , wherein the width of each feature map in P ₁₅ is R and the height is L; the input end of the seventh Concatenate fusion layer receives all the feature maps in P ₁₅ , H All feature maps in ₁ , the output of the upsampling framework, the output of the seventh Concatenate fusion layer outputs 96 feature maps, and the set of all the output feature maps is denoted as C ₇ , where each feature in C ₇ The width of the graph is R and the height is L;

For the up-sampling framework, the input of the first up-sampling layer receives all the feature maps in Z ₄ , the output of the first up-sampling layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as Y ₁ , where the width of each feature map in Y ₁ is height is The input end of the tenth convolutional layer receives all the feature maps in Y ₁ , and the output end of the tenth convolution layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as J ₁₀ , where in J ₁₀ The width of each feature map is height is The input end of the seventeenth batch of normalization layers receives all feature maps in J ₁₀ , the output end of the seventeenth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₁₇ , where P The width of each feature map in ₁₇ is height is The input end of the seventeenth activation layer receives all the feature maps in P ₁₇ , the output end of the seventeenth activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H ₁₇ , wherein, in H ₁₇ The width of each feature map is height is The input end of the second upsampling layer receives all the feature maps in H ₁₇ , the output end of the second upsampling layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as Y ₂ , where in Y ₂ The width of each feature map is height is The input of the eleventh convolutional layer receives all the feature maps in Y ₂ , the output of the eleventh convolutional layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₁₁ , where J The width of each feature map in ₁₁ is height is The input end of the eighteenth batch of normalization layers receives all feature maps in J ₁₁ , the output end of the eighteenth batch of normalization layers outputs 128 feature maps, and the set formed by all the output feature maps is denoted as P ₁₈ , where P The width of each feature map in ₁₈ is height is The input end of the eighteenth activation layer receives all the feature maps in P ₁₈ , the output end of the eighteenth activation layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as H ₁₈ , where in H ₁₈ The width of each feature map is height is The input end of the third up-sampling layer receives all the feature maps in H ₁₈ , the output end of the third up-sampling layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as Y ₃ , where in Y ₃ The width of each feature map is height is The input of the twelfth convolutional layer receives all feature maps in Y ₃ , and the output of the twelfth convolutional layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₁₂ , where J The width of each feature map in ₁₂ is height is The input end of the nineteenth batch of normalization layers receives all the feature maps in J ₁₂ , the output end of the nineteenth batch of normalization layers outputs 64 feature maps, and the set formed by all the output feature maps is denoted as P ₁₉ , where P The width of each feature map in ₁₉ is height is The input end of the nineteenth activation layer receives all the feature maps in P ₁₉ , and the output end of the nineteenth activation layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as H ₁₉ , where in H ₁₉ The width of each feature map is height is The input end of the fourth upsampling layer receives all the feature maps in H ₁₉ , the output end of the fourth upsampling layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as Y ₄ , where in Y _{4 The} width of each feature map of the The set composed of all feature maps is denoted as J ₁₃ , wherein the width of each feature map in J ₁₃ is R and the height is L; the input end of the twentieth batch of normalization layers receives all the feature maps in J ₁₃ , the second The output of ten batches of normalization layers outputs 32 feature maps, and the set of all output feature maps is denoted as P ₂₀ , where the width of each feature map in P ₂₀ is R and the height is L; the twentieth activation The input end of the layer receives all the feature maps in P ₂₀ , the output end of the twentieth activation layer outputs 32 feature maps, and the set formed by all the output feature maps is denoted as H ₂₀ , where each feature in H ₂₀ The width of the graph is R and the height is L;

For the output layer, the input terminal of the fifteenth activation layer receives the output of the decoding framework, that is, all feature maps in _C7 , and the output terminal of the fifteenth activation layer outputs 96 feature maps. Denoted as H ₁₅ , wherein the width of each feature map in H ₁₅ is R and the height is L; the input end of the ninth convolution layer receives all the feature maps in H ₁₅ , and the output end of the ninth convolution layer outputs 1 feature map, the set of all output feature maps is denoted as J ₉ , where the width of the feature map in J ₉ is R and the height is L; the input end of the sixteenth batch of normalization layers receives J ₉ . Feature map, the output end of the sixteenth batch of normalization layers outputs one feature map, and the set composed of all the output feature maps is denoted as P ₁₆ , where the width of the feature map in P ₁₆ is R and the height is L; The input terminal of the sixteenth activation layer receives the feature map in P ₁₆ , the output terminal of the sixteenth activation layer outputs one feature map, and the set formed by all the output feature maps is denoted as H ₁₆ , wherein the features in H ₁₆ The width of the image is R and the height is L, and the feature map in H ₁₆ is the estimated depth image corresponding to the original input image;

Step 1_3: Take each original monocular image in the training set as the original input image, input it into the convolutional neural network for training, and obtain the estimated depth image corresponding to each original monocular image in the training set, and set {Q ⁿ The estimated depth image corresponding to (x,y)} is denoted as in, express The pixel value of the pixel whose middle coordinate position is (x, y);

Step 1_4: Calculate the loss function value between the estimated depth image corresponding to each original monocular image in the training set and the corresponding real depth image, and The loss function value between is denoted as

Step 1_5: Repeat steps 1_3 and 1_4 for a total of V times to obtain the trained convolutional neural network training model, and obtain a total of N×V loss function values; then find the smallest value from the N×V loss function values. Then take the weight vector and the bias term corresponding to the loss function value with the smallest value as the optimal weight vector and the optimal bias term of the trained convolutional neural network training model, which are recorded as W ^best and b ^best ; wherein, V>1;

The specific steps of the test phase process are:

Step 2_1: Let {Q(x',y')} denote the monocular image to be predicted; wherein, 1≤x'≤R', 1≤y'≤L', R' denotes {Q(x',y ')} width, L' means the height of {Q(x',y')}, Q(x',y') means the coordinate position in {Q(x',y')} is (x',y ') the pixel value of the pixel point;

Step 2_2: Input {Q(x',y')} into the trained convolutional neural network training model, and use W ^best and b ^best to predict, and get {Q(x',y')} corresponding The predicted depth image is recorded as {Q _depth (x',y')}; among them, Q _depth (x',y') indicates that the coordinate position in {Q _depth (x',y')} is (x',y) ') of the pixel value of the pixel point.

In the described steps 1-4, Obtained using the mean squared error function.

Compared with the prior art, the advantages of the present invention are:

1) The method of the present invention adopts the skip-layer connection mode in the process of constructing the convolutional neural network, that is, the Concatenate fusion layer is adopted, and the short-jump layer connection is used in the coding framework at the same time, that is, the first Concatenate fusion layer, The second Concatenate fusion layer and the third Concatenate fusion layer are connected; a long-hop connection is used between the encoding frame and the decoding frame, that is, the fourth Concatenate fusion layer, the fifth Concatenate fusion layer, the sixth Concatenate fusion layer, The seventh Concatenate fusion layer is connected. The use of skip layer connection is beneficial to multi-scale feature fusion and boundary preservation. Short skip layer connection enriches the diversity of information in the encoding process, and long skip layer connection solves the lack of original boundary information in the decoding part. , so that the depth estimation using the trained convolutional neural network training model is more accurate.

2) The method of the present invention uses an end-to-end convolutional neural network training framework, and uses three atrous convolutional layers after the third maximum pooling layer of the coding frame to extract feature information, and the atroused convolutional layer can On the premise of increasing the number of training parameters, the receptive field of neurons can be expanded and more feature information can be obtained.

3) The hidden layer of the convolutional neural network created by the method of the present invention includes an encoding frame, a decoding frame and an upsampling frame, and the combination of the three frames enables the convolutional neural network training model obtained by training to extract features with rich information , so that the depth information with high accuracy can be obtained, thereby improving the accuracy of the depth estimation result.

4) The size of the predicted depth image obtained by the method of the present invention is the same as the size of the original monocular image, which is beneficial to the direct use of the depth information therein.

Description of drawings

Fig. 1 is the composition structure schematic diagram of the coding frame in the hidden layer of the convolutional neural network created in the method of the present invention;

Fig. 2 is the respective composition structure diagram of the decoding frame in the hidden layer of the convolutional neural network created in the method of the present invention and the output layer of the created convolutional neural network;

FIG. 3 is a schematic diagram of the composition and structure of the upsampling framework in the hidden layer of the convolutional neural network created in the method of the present invention.

Detailed ways

The present invention will be further described in detail below with reference to the embodiments of the accompanying drawings.

A monocular visual depth estimation method proposed by the present invention is characterized in that it includes two processes: a training phase and a testing phase.

The specific steps of the training phase process are:

Step 1_1: Select N original monocular images and the real depth image corresponding to each original monocular image, and form a training set, and record the nth original monocular image in the training set as {Q ⁿ (x, y)}, denote the real depth image corresponding to {Q ⁿ (x, y)} in the training set as Among them, N is a positive integer, N≥100, if N=1000, n is a positive integer, 1≤n≤N, 1≤x≤R, 1≤y≤L, R represents {Q ⁿ (x,y) }and The width of , L represents {Q ⁿ (x, y)} and The height of , R and L are both divisible by 2, Q ⁿ (x, y) represents the pixel value of the pixel at the coordinate position (x, y) in {Q ⁿ (x, y)}, express The pixel value of the pixel whose mid-coordinate position is (x, y); here, the original monocular image and its corresponding real depth image are directly provided by the KITTI official website.

Step 1_2: Build an end-to-end convolutional neural network: The convolutional neural network includes an input layer, a hidden layer, and an output layer; the hidden layer includes an encoding frame, a decoding frame, and an upsampling frame.

For the input layer, the input end of the input layer receives an original input image, and the output end of the input layer outputs the original input image to the hidden layer; among them, the width of the original input image received by the input end of the input layer is required to be R and the height is L .

For the encoding framework, as shown in Figure 1, it consists of the first convolutional layer, the first normalization layer, the first activation layer, the first max pooling layer, the second convolutional layer, and the second batch of normalization layers. , second activation layer, third convolution layer, third batch normalization layer, first concatenate fusion layer, third activation layer, second max pooling layer, fourth convolution layer, fourth batch normalization layer, fourth batch Activation layer, fifth convolution layer, fifth batch normalization layer, second concatenate fusion layer, fifth activation layer, third max pooling layer, first atrous convolutional layer, sixth batch normalization layer, sixth activation layer layer, second atrous convolutional layer, seventh batch normalization layer, third concatenate fusion layer, seventh activation layer, fourth max pooling layer, third atrous convolutional layer, eighth batch normalization layer, eighth batch The activation layer consists of the activation layer; for the decoding framework, as shown in Figure 2, it consists of the first deconvolution layer, the ninth batch of normalization layers, the fourth Concatenate fusion layer, the ninth activation layer, the sixth convolution layer, The tenth normalization layer, the tenth activation layer, the second deconvolution layer, the eleventh normalization layer, the fifth Concatenate fusion layer, the eleventh activation layer, the seventh convolution layer, the twelfth batch normalization layer, Twelfth Activation Layer, Third Deconvolution Layer, Thirteenth Batch Normalization Layer, Sixth Concatenate Fusion Layer, Thirteenth Activation Layer, Eighth Convolutional Layer, Fourteenth Batch Normalization Layer, Fourteenth Activation Layer , the fourth deconvolution layer, the fifteenth batch normalization layer, and the seventh Concatenate fusion layer; for the upsampling framework, as shown in Figure 3, it consists of the first upsampling layer, the tenth convolutional layer, Batch 17 Normalization Layer, Activation Layer 17, Upsampling Layer 2, Convolution Layer 11, Normalization Layer 18, Activation Layer 18, Upsampling Layer 3, Convolution 12 layer, the nineteenth batch normalization layer, the nineteenth activation layer, the fourth upsampling layer, the thirteenth convolution layer, the twentieth batch normalization layer, and the twentieth activation layer; for the output layer, as shown in Figure 2 It is composed of the fifteenth activation layer, the ninth convolution layer, the sixteenth batch of normalization layers, and the sixteenth activation layer, which are set in sequence. The size of the convolution kernels from the atrous convolutional layer to the third atrous convolutional layer and the first deconvolutional layer to the fourth deconvolutional layer is 3×3, and the number of convolution kernels of the first convolutional layer is 32. The number of convolution kernels of the second convolutional layer and the third convolutional layer is 64, the number of convolutional kernels of the fourth convolutional layer and the fifth convolutional layer is 128, the first convolutional layer with holes and The number of convolution kernels in the second convolutional layer with holes is 256, the number of convolution kernels in the third convolutional layer with holes is 512, and the number of convolution kernels in the first deconvolution layer and the sixth convolutional layer is 256, the number of convolution kernels of the second deconvolution layer and the seventh convolution layer is 128, the number of convolution kernels of the third deconvolution layer and the eighth convolution layer is 64, and the number of convolution kernels of the fourth deconvolution layer is 64. The number of convolution kernels of the layer is 32, the number of convolution kernels of the ninth convolution layer is 1, the number of convolution kernels of the tenth convolution layer is 256, and the number of convolution kernels of the eleventh convolution layer For 128, the convolution of the twelfth convolutional layer The number of kernels is 64, the number of convolution kernels of the thirteenth convolutional layer is 32, the first convolutional layer to the thirteenth convolutional layer, the first convolutional layer to the third convolutional layer with holes are respectively The convolution step size of the first deconvolution layer to the fourth deconvolution layer is 2 × 2, and the parameters of the first batch of normalization layers to the twentieth batch of normalization layers use the default values. , the activation functions from the first activation layer to the twentieth activation layer use ReLu, the pooling step size from the first maximum pooling layer to the fourth maximum pooling layer is 2×2, and the first upsampling layer to the fourth upsampling The sampling stride of the layer is 2×2.

For the coding framework, the input of the first convolutional layer receives the original input image output by the output of the input layer, the output of the first convolutional layer outputs 32 feature maps, and the set of all the output feature maps is denoted as J ₁ , where the width of each feature map in J ₁ is R and the height is L; the input end of the first batch of normalization layers receives all the feature maps in J ₁ , and the output end of the first batch of normalization layers outputs 32 features The set of all output feature maps is denoted as P ₁ , wherein the width of each feature map in P ₁ is R and the height is L; the input end of the first activation layer receives all feature maps in P ₁ , the output end of the first activation layer outputs 32 feature maps, and the set formed by all the output feature maps is denoted as H ₁ , wherein the width of each feature map in H ₁ is R and the height is L; the first largest The input end of the pooling layer receives all the feature maps in H ₁ , the output end of the first maximum pooling layer outputs 32 feature maps, and the set formed by all the output feature maps is denoted as Z ₁ , wherein, in Z ₁ The width of each feature map is height is The input of the second convolutional layer receives all the feature maps in Z ₁ , the output of the second convolutional layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₂ , where in J ₂ The width of each feature map is height is The input end of the _second batch of normalization layers receives all feature maps in J ₂ , and the output end of the _second batch of normalization layers outputs 64 feature maps. The width of each feature map is height is The input terminal of the _second activation layer receives all feature maps in P ₂ , and the output terminal of the _second activation layer outputs 64 feature maps. The width of the feature map is height is The input end of the third convolutional layer receives all the feature maps in H ₂ , and the output end of the _third convolution layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₃ , where in The width of each feature map is height is The input end of the third batch of normalization layers receives all feature maps in J ₃ , the output end of the third batch of normalization layers outputs 64 feature maps, and the set formed by all the output feature maps is denoted as P ₃ , where in P ₃ The width of each feature map is height is The input of the first Concatenate fusion layer receives all the feature maps in _P3 and all the feature maps in _H2 , and the output of the first Concatenate fusion layer outputs 128 feature maps, and the set composed of all the output feature maps is recorded as C ₁ , where the width of each feature map in C ₁ is height is The input terminal of the _third activation layer receives all feature maps in C ₁ , and the output terminal of the _third activation layer outputs 128 feature maps. The width of the feature map is height is The input of the second maximum pooling layer receives all the feature maps in H ₃ , the output of the second maximum pooling layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as Z ₂ , where Z The width of each feature map in ₂ is height is The input end of the fourth convolutional layer receives all the feature maps in Z ₂ , and the output end of the fourth convolution layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₄ , where in J ₄ The width of each feature map is height is The input terminal of the _fourth batch of normalization layers receives all feature maps in J ₄ , and the output terminal of the _fourth batch of normalization layers outputs 128 feature maps. The width of each feature map is height is The input end of the _fourth activation layer receives all the feature maps in P ₄ , and the output end of the _fourth activation layer outputs 128 feature maps. The width of the feature map is height is The input end of the fifth convolutional layer receives all the feature maps in H ₄ , and the output end of the fifth convolution layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₅ , where in J ₅ The width of each feature map is height is The input end of the fifth batch of normalization layers receives all feature maps in J ₅ , and the output end of the fifth batch of normalization layers outputs 128 feature maps, and the set formed by all the output feature maps is denoted as P ₅ , where in P ₅ The width of each feature map is height is The input end of the second Concatenate fusion layer receives all the feature maps in P ₅ and all the feature maps in H ₄ , and the output end of the second Concatenate fusion layer outputs 256 feature maps, and the set formed by all the output feature maps is recorded as C ₂ , where the width of each feature map in C ₂ is height is The input end of the _fifth activation layer receives all the feature maps in C ₂ , and the output end of the _fifth activation layer outputs 256 feature maps. The width of the feature map is height is The input terminal of the third maximum pooling layer receives all the feature maps in H ₅ , the output terminal of the third maximum pooling layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as Z ₃ , where Z 3 The width of each feature map in ₃ is height is The input of the first atrous convolutional layer receives all feature maps in Z ₃ , the output of the first atrous convolutional layer outputs 256 feature maps, and the set of all the output feature maps is denoted as K ₁ , where , the width _of each feature map in K1 is height is The input end of the sixth batch of normalization layers receives all the feature maps in K ₁ , the output end of the sixth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₆ , where in P ₆ The width of each feature map is height is The input end of the sixth activation layer receives all the feature maps in P ₆ , and the output end of the sixth activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H ₆ , where each of the feature maps in H ₆ The width of the feature map is height is The input end of the second atrous convolutional layer receives all the feature maps in H ₆ , the output end of the second atrous convolutional layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as K ₂ , where , the width of each feature map in _K2 is height is The input end of the seventh batch of normalization layers receives all the feature maps in K ₂ , the output end of the seventh batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₇ , where in P ₇ The width of each feature map is height is The input end of the third Concatenate fusion layer receives all the feature maps in P ₇ and all the feature maps in H ₆ , and the output end of the third Concatenate fusion layer outputs 512 feature maps, and the set composed of all the output feature maps is denoted as C ₃ , where the width of each feature map in C ₃ is height is The input terminal of the seventh activation layer receives all the feature maps in C ₃ , and the output terminal of the _seventh activation layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as H ₇ . The width of the feature map is height is The input end of the fourth maximum pooling layer receives all the feature maps in H ₇ , and the output end of the fourth maximum pooling layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as Z ₄ , where Z 4 The width of each feature map in ₄ is height is The input of the third atrous convolutional layer receives all the feature maps in Z ₄ , the output of the third atrous convolutional layer outputs 512 feature maps, and the set of all the output feature maps is denoted as K ₃ , where , the width _of each feature map in K3 is height is The input end of the eighth batch of normalization layers receives all the feature maps in K ₃ , the output end of the eighth batch of normalization layers outputs 512 feature maps, and the set formed by all the output feature maps is denoted as P ₈ , where in P ₈ The width of each feature map is height is The input end of the _eighth activation layer receives all the feature maps in P8, and the output end of the eighth activation layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as H ₈ , and H ₈ is also the coding frame. _The output of , where the width of each feature map in H8 is height is

For the decoding framework, the input end of the first deconvolution layer receives the output of the encoding framework, that is, all feature maps in _H8 , and the output end of the first deconvolution layer outputs 256 feature maps, which are composed of all the output feature maps. The set is denoted as F ₁ , where the width of each feature map in F ₁ is height is The input end of the ninth batch of normalization layers receives all the feature maps in F ₁ , and the output end of the ninth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₉ , where in P ₉ The width of each feature map is height is The input end of the fourth Concatenate fusion layer receives all the feature maps in P ₉ and all the feature maps in P ₇ , and the output end of the fourth Concatenate fusion layer outputs 512 feature maps, and the set composed of all the output feature maps is recorded as C ₄ , where the width of each feature map in C ₄ is height is The input end of the _ninth activation layer receives all the feature maps in C ₄ , and the output end of the _ninth activation layer outputs 512 feature maps. The width of the feature map is height is The input end of the sixth convolutional layer receives all the feature maps in H ₉ , the output end of the sixth convolution layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as J ₆ , where in J ₆ The width of each feature map is height is The input end of the tenth batch of normalization layers receives all the feature maps in J ₆ , the output end of the tenth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₁₀ , where in P ₁₀ The width of each feature map is height is The input end of the tenth activation layer receives all the feature maps in P ₁₀ , and the output end of the tenth activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H ₁₀ , where each H ₁₀ The width of the feature map is height is The input end of the second deconvolution layer receives the output of the coding framework, that is, all feature maps in _H10 , and the output end of the second deconvolution layer outputs 128 feature maps, and the set of all the output feature maps is denoted as F ₂ , where the width of each feature map in _F2 is height is The input end of the eleventh batch of normalization layers receives all the feature maps in F ₂ , and the output end of the eleventh batch of normalization layers outputs 128 feature maps, and the set of all the output feature maps is denoted as P ₁₁ , where P The width of each feature map in ₁₁ is height is The input of the fifth Concatenate fusion layer receives all the feature maps in P ₁₁ and all the feature maps in P ₅ , and the output of the fifth Concatenate fusion layer outputs 256 feature maps, and the set of all the output feature maps is recorded as C ₅ , where the width of each feature map in C ₅ is height is The input end of the eleventh activation layer receives all the feature maps in _C5 , and the output end of the _{eleventh activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H 11 .} The width of each feature map is height is The input end of the seventh convolutional layer receives all the feature maps in H ₁₁ , the output end of the seventh convolution layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₇ , where in J ₇ The width of each feature map is height is The input end of the twelfth batch of normalization layers receives all feature maps in J ₇ , the output end of the twelfth batch of normalization layers outputs 128 feature maps, and the set formed by all the output feature maps is denoted as P ₁₂ , where P The width of each feature map in ₁₂ is height is The input end of the twelfth activation layer receives all the feature maps in P ₁₂ , the output end of the twelfth activation layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as H ₁₂ , where in H ₁₂ The width of each feature map is height is The input end of the third deconvolution layer receives all the feature maps in H ₁₂ , the output end of the third deconvolution layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as F ₃ , where F The width of each feature map in ₃ is height is The input end of the thirteenth batch of normalization layers receives all the feature maps in F ₃ , the output end of the thirteenth batch of normalization layers outputs 64 feature maps, and the set of all the output feature maps is denoted as P ₁₃ , where P The width of each feature map in ₁₃ is height is The input end of the sixth Concatenate fusion layer receives all the feature maps in P ₁₃ and all the feature maps in P ₃ , and the output end of the sixth Concatenate fusion layer outputs 128 feature maps, and the set composed of all the output feature maps is recorded as C ₆ , where the width of each feature map in C ₆ is height is The input end of the thirteenth activation layer receives all the feature maps in C ₆ , the output end of the thirteenth activation layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as H ₁₃ , wherein, in H ₁₃ The width of each feature map is height is The input end of the eighth convolutional layer receives all the feature maps in H ₁₃ , the output end of the eighth convolution layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₈ , where in J ₈ The width of each feature map is height is The input end of the fourteenth batch of normalization layers receives all the feature maps in J ₈ , the output end of the fourteenth batch of normalization layers outputs 64 feature maps, and the set formed by all the output feature maps is denoted as P ₁₄ , where P The width of each feature map in ₁₄ is height is The input end of the fourteenth activation layer receives all the feature maps in P ₁₄ , the output end of the fourteenth activation layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as H ₁₄ , wherein, in H ₁₄ The width of each feature map is height is The input end of the fourth deconvolution layer receives all the feature maps in H ₁₄ , the output end of the fourth deconvolution layer outputs 32 feature maps, and the set formed by all the output feature maps is denoted as F ₄ , where F The width of each feature map in ₄ is R and the height is L; the input end of the fifteenth batch of normalization layers receives all the feature maps in F ₄ , and the output end of the fifteenth batch of normalization layers outputs 32 feature maps. The set composed of all the output feature maps is denoted as P ₁₅ , wherein the width of each feature map in P ₁₅ is R and the height is L; the input end of the seventh Concatenate fusion layer receives all the feature maps in P ₁₅ , H All feature maps in ₁ , the output of the upsampling framework, the output of the seventh Concatenate fusion layer outputs 96 feature maps, and the set of all the output feature maps is denoted as C ₇ , where each feature in C ₇ The width of the graph is R and the height is L.

For the up-sampling framework, the input of the first up-sampling layer receives all the feature maps in Z ₄ , the output of the first up-sampling layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as Y ₁ , where the width of each feature map in Y ₁ is height is The input end of the tenth convolutional layer receives all the feature maps in Y ₁ , and the output end of the tenth convolution layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as J ₁₀ , where in J ₁₀ The width of each feature map is height is The input end of the seventeenth batch of normalization layers receives all feature maps in J ₁₀ , the output end of the seventeenth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₁₇ , where P The width of each feature map in ₁₇ is height is The input end of the seventeenth activation layer receives all the feature maps in P ₁₇ , the output end of the seventeenth activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H ₁₇ , wherein, in H ₁₇ The width of each feature map is height is The input end of the second upsampling layer receives all the feature maps in H ₁₇ , the output end of the second upsampling layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as Y ₂ , where in Y ₂ The width of each feature map is height is The input of the eleventh convolutional layer receives all the feature maps in Y ₂ , the output of the eleventh convolutional layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₁₁ , where J The width of each feature map in ₁₁ is height is The input end of the eighteenth batch of normalization layers receives all feature maps in J ₁₁ , the output end of the eighteenth batch of normalization layers outputs 128 feature maps, and the set formed by all the output feature maps is denoted as P ₁₈ , where P The width of each feature map in ₁₈ is height is The input end of the eighteenth activation layer receives all the feature maps in P ₁₈ , the output end of the eighteenth activation layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as H ₁₈ , where in H ₁₈ The width of each feature map is height is The input end of the third up-sampling layer receives all the feature maps in H ₁₈ , the output end of the third up-sampling layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as Y ₃ , where in Y ₃ The width of each feature map is height is The input of the twelfth convolutional layer receives all feature maps in Y ₃ , and the output of the twelfth convolutional layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₁₂ , where J The width of each feature map in ₁₂ is height is The input end of the nineteenth batch of normalization layers receives all the feature maps in J ₁₂ , the output end of the nineteenth batch of normalization layers outputs 64 feature maps, and the set formed by all the output feature maps is denoted as P ₁₉ , where P The width of each feature map in ₁₉ is height is The input end of the nineteenth activation layer receives all the feature maps in P ₁₉ , and the output end of the nineteenth activation layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as H ₁₉ , where in H ₁₉ The width of each feature map is height is The input end of the fourth upsampling layer receives all the feature maps in H ₁₉ , the output end of the fourth upsampling layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as Y ₄ , where in Y _{4 The} width of each feature map of the The set composed of all feature maps is denoted as J ₁₃ , wherein the width of each feature map in J ₁₃ is R and the height is L; the input end of the twentieth batch of normalization layers receives all the feature maps in J ₁₃ , the second The output of ten batches of normalization layers outputs 32 feature maps, and the set of all output feature maps is denoted as P ₂₀ , where the width of each feature map in P ₂₀ is R and the height is L; the twentieth activation The input end of the layer receives all the feature maps in P ₂₀ , and the output end of the _twentieth activation layer outputs ₃₂ feature maps. The width of the graph is R and the height is L.

For the output layer, the input terminal of the fifteenth activation layer receives the output of the decoding framework, that is, all the feature maps in _C7 , and the output terminal of the fifteenth activation layer outputs 96 feature maps. It is denoted as H ₁₅ , wherein the width of each feature map in H ₁₅ is R and the height is L; the input end of the ninth convolution layer receives all the feature maps in H ₁₅ , and the output end of the ninth convolution layer outputs 1 feature map, the set of all output feature maps is denoted as J ₉ , where the width of the feature map in J ₉ is R, and the height is L; the input of the sixteenth batch of normalization layers receives J ₉ . Feature map, the output end of the sixteenth batch of normalization layers outputs one feature map, and the set composed of all the output feature maps is denoted as P ₁₆ , where the width of the feature map in P ₁₆ is R and the height is L; The input terminal of the sixteenth activation layer receives the feature map in P ₁₆ , and the output terminal of the sixteenth activation layer outputs one feature map, and the set formed by all the output feature maps is denoted as H ₁₆ , wherein the features in H ₁₆ The width of the map is R and the height is L, and the feature map in H ₁₆ is the estimated depth image corresponding to the original input image.

Step 1_3: Take each original monocular image in the training set as the original input image, input it into the convolutional neural network for training, and obtain the estimated depth image corresponding to each original monocular image in the training set, and set {Q ⁿ The estimated depth image corresponding to (x,y)} is denoted as in, express The pixel value of the pixel whose middle coordinate position is (x, y).

Step 1_4: Calculate the loss function value between the estimated depth image corresponding to each original monocular image in the training set and the corresponding real depth image, and The loss function value between is denoted as Obtained using the mean squared error function.

Step 1_5: Repeat steps 1_3 and 1_4 for a total of V times to obtain a trained convolutional neural network training model, and obtain a total of N×V loss function values; then find the smallest value from the N×V loss function values. Then take the weight vector and the bias term corresponding to the loss function value with the smallest value as the optimal weight vector and the optimal bias term of the trained convolutional neural network training model, corresponding to W ^best and b ^best ; where V>1, in this embodiment, V=20.

The specific steps of the described testing phase process are:

Step 2_1: Let {Q(x',y')} denote the monocular image to be predicted; wherein, 1≤x'≤R', 1≤y'≤L', R' denotes {Q(x',y ')} width, L' means the height of {Q(x',y')}, Q(x',y') means the coordinate position in {Q(x',y')} is (x',y ') of the pixel value of the pixel point.

Step 2_2: Input {Q(x',y')} into the trained convolutional neural network training model, and use W ^best and b ^best to predict, and get {Q(x',y')} corresponding The predicted depth image is recorded as {Q _depth (x', y')}; among them, Q _depth (x', y') indicates that the coordinate position in {Q _depth (x', y')} is (x', y ') of the pixel value of the pixel point.

In order to verify the feasibility and effectiveness of the method of the present invention, experiments were carried out on the method of the present invention.

Here, the monocular images constituting the training set and the monocular images used for testing in the method of the present invention are given by the official website of KITTI, so the test data set given by the official website of KITTI is directly used to analyze and test the accuracy of the method of the present invention sex. Input each monocular image in the test data set as the monocular image to be predicted into the trained deep convolutional neural network training model, and then load the optimal weight W ^best obtained in the training stage to obtain the corresponding predicted depth image .

Here, six common objective parameters of the monocular visual depth prediction evaluation method are used as evaluation indicators, namely: root mean squared error (rms), logarithmic root mean squared error (log_rms), average logarithmic error (average log ₁₀ error, log 10), threshold accuracy (thr): δ ₁ , δ ₂ , δ ₃ . The lower the values of root mean square error, logarithmic root mean square error, and average logarithmic error, the closer the predicted depth image is to the real depth image, and the higher the values of δ ₁ , δ ₂ , and δ ₃ , the accuracy of the predicted depth image. higher. Table 1 lists the results of root mean square error, logarithmic root mean square error, average logarithmic error, and δ ₁ , δ ₂ , and δ ₃ reflecting the evaluation performance indicators of the present invention. From the data listed in Table 1, it can be seen that the difference between the predicted depth image obtained by the method of the present invention and the real depth image is very small, which shows that the accuracy of the prediction result of the method of the present invention is very high, and reflects the feasibility of the method of the present invention. sex and effectiveness.

Table 1 Comparison and evaluation index between the predicted depth image and the real depth image predicted by the method of the present invention

Claims (2)

1. a monocular vision depth estimation method is characterized in that comprising two processes of training stage and testing stage; The specific steps of the training phase process are: Step 1_1: Select N original monocular images and the real depth image corresponding to each original monocular image, and form a training set, and record the nth original monocular image in the training set as {Q ⁿ (x, y)}, denote the real depth image corresponding to {Q ⁿ (x, y)} in the training set as Among them, N is a positive integer, N≥100, n is a positive integer, 1≤n≤N, 1≤x≤R, 1≤y≤L, R represents {Q ⁿ (x, y)} and The width of , L represents {Q ⁿ (x, y)} and The height of , R and L are both divisible by 2, Q ⁿ (x, y) represents the pixel value of the pixel at the coordinate position (x, y) in {Q ⁿ (x, y)}, express The pixel value of the pixel whose middle coordinate position is (x, y); Step 1_2: Build an end-to-end convolutional neural network: the convolutional neural network includes an input layer, a hidden layer, and an output layer; the hidden layer includes an encoding frame, a decoding frame, and an upsampling frame; For the input layer, the input end of the input layer receives an original input image, and the output end of the input layer outputs the original input image to the hidden layer; among them, the width of the original input image received by the input end of the input layer is required to be R and the height is L ; For the encoding framework, it consists of the first convolutional layer, the first normalization layer, the first activation layer, the first max pooling layer, the second convolutional layer, the second normalization layer, the second activation layer, 3rd Convolutional Layer, 3rd Batch Normalization Layer, 1st Concatenate Fusion Layer, 3rd Activation Layer, 2nd Max Pooling Layer, 4th Convolutional Layer, 4th Batch Normalization Layer, 4th Activation Layer, 5th Volume Convolution layer, fifth batch normalization layer, second concatenate fusion layer, fifth activation layer, third max pooling layer, first convolutional layer with holes, sixth batch normalization layer, sixth activation layer, second hole Convolutional layer, seventh batch of normalization layer, third Concatenate fusion layer, seventh activation layer, fourth maximum pooling layer, third convolutional layer with holes, eighth batch of normalization layer, eighth activation layer; The code frame is composed of the first deconvolution layer, the ninth batch of normalization layers, the fourth Concatenate fusion layer, the ninth activation layer, the sixth convolution layer, the tenth batch of normalization layers, the tenth activation layer, the fourth batch of The second deconvolution layer, the eleventh normalization layer, the fifth concatenate fusion layer, the eleventh activation layer, the seventh convolution layer, the twelfth normalization layer, the twelfth activation layer, the third deconvolution layer , the thirteenth batch of normalization layer, the sixth Concatenate fusion layer, the thirteenth activation layer, the eighth convolution layer, the fourteenth batch of normalization layer, the fourteenth activation layer, the fourth deconvolution layer, the fifteenth batch The normalization layer and the seventh Concatenate fusion layer are composed; for the upsampling framework, it consists of the first upsampling layer, the tenth convolutional layer, the seventeenth batch normalization layer, the seventeenth activation layer, and the second upsampling layer. , the eleventh convolutional layer, the eighteenth batch of normalization layers, the eighteenth activation layer, the third upsampling layer, the twelfth convolutional layer, the nineteenth batch of normalization layers, the nineteenth activation layer, the fourth upper The sampling layer, the thirteenth convolutional layer, the twentieth batch of normalization layers, and the twentieth activation layer are composed; for the output layer, it consists of the fifteenth activation layer, the ninth convolutional layer, and the sixteenth batch of normalization set in sequence. layer and the sixteenth activation layer, among which, the first convolutional layer to the thirteenth convolutional layer, the first convolutional layer with holes to the third convolutional layer with holes, the first deconvolution layer to the fourth inverse convolutional layer The convolution kernel size of each convolution layer is 3 × 3, the number of convolution kernels of the first convolution layer is 32, the number of convolution kernels of the second convolution layer and the third convolution layer is 64, and the number of convolution kernels of the fourth convolution layer is 64. The number of convolution kernels of the convolutional layer and the fifth convolutional layer is 128, the number of convolution kernels of the first convolutional layer and the second convolutional layer with holes is 256, and the number of convolutional kernels of the third convolutional layer with holes The number of convolution kernels is 512, the number of convolution kernels of the first deconvolution layer and the sixth convolution layer is 256, the number of convolution kernels of the second deconvolution layer and the seventh convolution layer is 128, The number of convolution kernels in the third deconvolution layer and the eighth convolution layer is 64, the number of convolution kernels in the fourth deconvolution layer is 32, and the number of convolution kernels in the ninth convolution layer is 1. The number of convolution kernels in the tenth convolution layer is 256, the number of convolution kernels in the eleventh convolution layer is 128, the number of convolution kernels in the twelfth convolution layer is 64, and the number of convolution kernels in the thirteenth convolution layer is 64. The number of convolution kernels is 32, the first The convolution strides from the first convolutional layer to the thirteenth convolutional layer, the first convolutional convolutional layer to the third convolutional convolutional layer with holes adopt the default values, and the first deconvolutional layer to the fourth deconvolutional layer The respective convolution step size is 2×2, the parameters of the first batch of normalization layers to the twentieth batch of normalization layers use default values, the activation functions of the first activation layer to the twentieth activation layer use ReLu, and the first maximum pooling is used. The pooling step size from the layer to the fourth maximum pooling layer is 2×2, and the sampling step size from the first upsampling layer to the fourth upsampling layer is 2×2; For the coding framework, the input of the first convolutional layer receives the original input image output by the output of the input layer, the output of the first convolutional layer outputs 32 feature maps, and the set of all the output feature maps is denoted as J ₁ , where the width of each feature map in J ₁ is R and the height is L; the input end of the first batch of normalization layers receives all the feature maps in J ₁ , and the output end of the first batch of normalization layers outputs 32 features The set of all output feature maps is denoted as P ₁ , wherein the width of each feature map in P ₁ is R and the height is L; the input end of the first activation layer receives all feature maps in P ₁ , the output end of the first activation layer outputs 32 feature maps, and the set formed by all the output feature maps is denoted as H ₁ , wherein the width of each feature map in H ₁ is R and the height is L; the first largest The input end of the pooling layer receives all the feature maps in H ₁ , the output end of the first maximum pooling layer outputs 32 feature maps, and the set formed by all the output feature maps is denoted as Z ₁ , wherein, in Z ₁ The width of each feature map is height is The input of the second convolutional layer receives all the feature maps in Z ₁ , the output of the second convolutional layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₂ , where in J ₂ The width of each feature map is height is The input end of the _second batch of normalization layers receives all feature maps in J ₂ , and the output end of the _second batch of normalization layers outputs 64 feature maps. The width of each feature map is height is The input terminal of the _second activation layer receives all feature maps in P ₂ , and the output terminal of the _second activation layer outputs 64 feature maps. The width of the feature map is height is The input end of the third convolutional layer receives all the feature maps in H ₂ , and the output end of the _third convolution layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₃ , where in The width of each feature map is height is The input end of the third batch of normalization layers receives all feature maps in J ₃ , the output end of the third batch of normalization layers outputs 64 feature maps, and the set formed by all the output feature maps is denoted as P ₃ , where in P ₃ The width of each feature map is height is The input of the first Concatenate fusion layer receives all the feature maps in _P3 and all the feature maps in _H2 , and the output of the first Concatenate fusion layer outputs 128 feature maps, and the set composed of all the output feature maps is recorded as C ₁ , where the width of each feature map in C ₁ is height is The input terminal of the _third activation layer receives all feature maps in C ₁ , and the output terminal of the _third activation layer outputs 128 feature maps. The width of the feature map is height is The input of the second maximum pooling layer receives all the feature maps in H ₃ , the output of the second maximum pooling layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as Z ₂ , where Z The width of each feature map in ₂ is height is The input end of the fourth convolutional layer receives all the feature maps in Z ₂ , and the output end of the fourth convolution layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₄ , where in J ₄ The width of each feature map is height is The input terminal of the _fourth batch of normalization layers receives all feature maps in J ₄ , and the output terminal of the _fourth batch of normalization layers outputs 128 feature maps. The width of each feature map is height is The input end of the _fourth activation layer receives all the feature maps in P ₄ , and the output end of the _fourth activation layer outputs 128 feature maps. The width of the feature map is height is The input end of the fifth convolutional layer receives all the feature maps in H ₄ , and the output end of the fifth convolution layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₅ , where in J ₅ The width of each feature map is height is The input end of the fifth batch of normalization layers receives all feature maps in J ₅ , and the output end of the fifth batch of normalization layers outputs 128 feature maps, and the set formed by all the output feature maps is denoted as P ₅ , where in P ₅ The width of each feature map is height is The input end of the second Concatenate fusion layer receives all the feature maps in P ₅ and all the feature maps in H ₄ , and the output end of the second Concatenate fusion layer outputs 256 feature maps, and the set formed by all the output feature maps is recorded as C ₂ , where the width of each feature map in C ₂ is height is The input end of the _fifth activation layer receives all the feature maps in C ₂ , and the output end of the _fifth activation layer outputs 256 feature maps. The width of the feature map is height is The input terminal of the third maximum pooling layer receives all the feature maps in H ₅ , the output terminal of the third maximum pooling layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as Z ₃ , where Z 3 The width of each feature map in ₃ is height is The input of the first atrous convolutional layer receives all feature maps in Z ₃ , the output of the first atrous convolutional layer outputs 256 feature maps, and the set of all the output feature maps is denoted as K ₁ , where , the width _of each feature map in K1 is height is The input end of the sixth batch of normalization layers receives all the feature maps in K ₁ , the output end of the sixth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₆ , where in P ₆ The width of each feature map is height is The input end of the sixth activation layer receives all the feature maps in P ₆ , and the output end of the sixth activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H ₆ , where each of the feature maps in H ₆ The width of the feature map is height is The input end of the second atrous convolutional layer receives all the feature maps in H ₆ , the output end of the second atrous convolutional layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as K ₂ , where , the width of each feature map in _K2 is height is The input end of the seventh batch of normalization layers receives all the feature maps in K ₂ , the output end of the seventh batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₇ , where in P ₇ The width of each feature map is height is The input end of the third Concatenate fusion layer receives all the feature maps in P ₇ and all the feature maps in H ₆ , and the output end of the third Concatenate fusion layer outputs 512 feature maps, and the set composed of all the output feature maps is denoted as C ₃ , where the width of each feature map in C ₃ is height is The input terminal of the seventh activation layer receives all the feature maps in C ₃ , and the output terminal of the _seventh activation layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as H ₇ . The width of the feature map is height is The input end of the fourth maximum pooling layer receives all the feature maps in H ₇ , and the output end of the fourth maximum pooling layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as Z ₄ , where Z 4 The width of each feature map in ₄ is height is The input of the third atrous convolutional layer receives all the feature maps in Z ₄ , the output of the third atrous convolutional layer outputs 512 feature maps, and the set of all the output feature maps is denoted as K ₃ , where , the width _of each feature map in K3 is height is The input end of the eighth batch of normalization layers receives all the feature maps in K ₃ , the output end of the eighth batch of normalization layers outputs 512 feature maps, and the set formed by all the output feature maps is denoted as P ₈ , where in P ₈ The width of each feature map is height is The input end of the _eighth activation layer receives all the feature maps in P8, and the output end of the eighth activation layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as H ₈ , and H ₈ is also the coding frame. _The output of , where the width of each feature map in H8 is height is For the decoding framework, the input end of the first deconvolution layer receives the output of the encoding framework, that is, all feature maps in _H8 , and the output end of the first deconvolution layer outputs 256 feature maps, which are composed of all the output feature maps. The set is denoted as F ₁ , where the width of each feature map in F ₁ is height is The input end of the ninth batch of normalization layers receives all the feature maps in F ₁ , and the output end of the ninth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₉ , where in P ₉ The width of each feature map is height is The input end of the fourth Concatenate fusion layer receives all the feature maps in P ₉ and all the feature maps in P ₇ , and the output end of the fourth Concatenate fusion layer outputs 512 feature maps, and the set composed of all the output feature maps is recorded as C ₄ , where the width of each feature map in C ₄ is height is The input end of the _ninth activation layer receives all the feature maps in C ₄ , and the output end of the _ninth activation layer outputs 512 feature maps. The width of the feature map is height is The input end of the sixth convolutional layer receives all the feature maps in H ₉ , the output end of the sixth convolution layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as J ₆ , where in J ₆ The width of each feature map is height is The input end of the tenth batch of normalization layers receives all the feature maps in J ₆ , the output end of the tenth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₁₀ , where in P ₁₀ The width of each feature map is height is The input end of the tenth activation layer receives all the feature maps in P ₁₀ , and the output end of the tenth activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H ₁₀ , where each H ₁₀ The width of the feature map is height is The input end of the second deconvolution layer receives the output of the coding framework, that is, all feature maps in _H10 , and the output end of the second deconvolution layer outputs 128 feature maps, and the set of all the output feature maps is denoted as F ₂ , where the width of each feature map in _F2 is height is The input end of the eleventh batch of normalization layers receives all the feature maps in F ₂ , and the output end of the eleventh batch of normalization layers outputs 128 feature maps, and the set of all the output feature maps is denoted as P ₁₁ , where P The width of each feature map in ₁₁ is height is The input of the fifth Concatenate fusion layer receives all the feature maps in P ₁₁ and all the feature maps in P ₅ , and the output of the fifth Concatenate fusion layer outputs 256 feature maps, and the set of all the output feature maps is recorded as C ₅ , where the width of each feature map in C ₅ is height is The input end of the eleventh activation layer receives all the feature maps in _C5 , and the output end of the _{eleventh activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H 11 .} The width of each feature map is height is The input end of the seventh convolutional layer receives all the feature maps in H ₁₁ , the output end of the seventh convolution layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₇ , where in J ₇ The width of each feature map is height is The input end of the twelfth batch of normalization layers receives all feature maps in J ₇ , the output end of the twelfth batch of normalization layers outputs 128 feature maps, and the set formed by all the output feature maps is denoted as P ₁₂ , where P The width of each feature map in ₁₂ is height is The input end of the twelfth activation layer receives all the feature maps in P ₁₂ , the output end of the twelfth activation layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as H ₁₂ , where in H ₁₂ The width of each feature map is height is The input end of the third deconvolution layer receives all the feature maps in H ₁₂ , the output end of the third deconvolution layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as F ₃ , where F The width of each feature map in ₃ is height is The input end of the thirteenth batch of normalization layers receives all the feature maps in F ₃ , the output end of the thirteenth batch of normalization layers outputs 64 feature maps, and the set of all the output feature maps is denoted as P ₁₃ , where P The width of each feature map in ₁₃ is height is The input end of the sixth Concatenate fusion layer receives all the feature maps in P ₁₃ and all the feature maps in P ₃ , and the output end of the sixth Concatenate fusion layer outputs 128 feature maps, and the set composed of all the output feature maps is recorded as C ₆ , where the width of each feature map in C ₆ is height is The input end of the thirteenth activation layer receives all the feature maps in C ₆ , the output end of the thirteenth activation layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as H ₁₃ , wherein, in H ₁₃ The width of each feature map is height is The input end of the eighth convolutional layer receives all the feature maps in H ₁₃ , the output end of the eighth convolution layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₈ , where in J ₈ The width of each feature map is height is The input end of the fourteenth batch of normalization layers receives all the feature maps in J ₈ , the output end of the fourteenth batch of normalization layers outputs 64 feature maps, and the set formed by all the output feature maps is denoted as P ₁₄ , where P The width of each feature map in ₁₄ is height is The input end of the fourteenth activation layer receives all the feature maps in P ₁₄ , the output end of the fourteenth activation layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as H ₁₄ , wherein, in H ₁₄ The width of each feature map is height is The input end of the fourth deconvolution layer receives all the feature maps in H ₁₄ , the output end of the fourth deconvolution layer outputs 32 feature maps, and the set formed by all the output feature maps is denoted as F ₄ , where F The width of each feature map in ₄ is R and the height is L; the input end of the fifteenth batch of normalization layers receives all the feature maps in F ₄ , and the output end of the fifteenth batch of normalization layers outputs 32 feature maps. The set composed of all the output feature maps is denoted as P ₁₅ , wherein the width of each feature map in P ₁₅ is R and the height is L; the input end of the seventh Concatenate fusion layer receives all the feature maps in P ₁₅ , H All feature maps in ₁ , the output of the upsampling framework, the output of the seventh Concatenate fusion layer outputs 96 feature maps, and the set of all the output feature maps is denoted as C ₇ , where each feature in C ₇ The width of the graph is R and the height is L; For the up-sampling framework, the input of the first up-sampling layer receives all the feature maps in Z ₄ , the output of the first up-sampling layer outputs 512 feature maps, and the set formed by all the output feature maps is denoted as Y ₁ , where the width of each feature map in Y ₁ is height is The input end of the tenth convolutional layer receives all the feature maps in Y ₁ , and the output end of the tenth convolution layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as J ₁₀ , where in J ₁₀ The width of each feature map is height is The input end of the seventeenth batch of normalization layers receives all feature maps in J ₁₀ , the output end of the seventeenth batch of normalization layers outputs 256 feature maps, and the set formed by all the output feature maps is denoted as P ₁₇ , where P The width of each feature map in ₁₇ is height is The input end of the seventeenth activation layer receives all the feature maps in P ₁₇ , the output end of the seventeenth activation layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as H ₁₇ , wherein, in H ₁₇ The width of each feature map is height is The input end of the second upsampling layer receives all the feature maps in H ₁₇ , the output end of the second upsampling layer outputs 256 feature maps, and the set formed by all the output feature maps is denoted as Y ₂ , where in Y ₂ The width of each feature map is height is The input of the eleventh convolutional layer receives all the feature maps in Y ₂ , the output of the eleventh convolutional layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as J ₁₁ , where J The width of each feature map in ₁₁ is height is The input end of the eighteenth batch of normalization layers receives all feature maps in J ₁₁ , the output end of the eighteenth batch of normalization layers outputs 128 feature maps, and the set formed by all the output feature maps is denoted as P ₁₈ , where P The width of each feature map in ₁₈ is height is The input end of the eighteenth activation layer receives all the feature maps in P ₁₈ , the output end of the eighteenth activation layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as H ₁₈ , where in H ₁₈ The width of each feature map is height is The input end of the third up-sampling layer receives all the feature maps in H ₁₈ , the output end of the third up-sampling layer outputs 128 feature maps, and the set formed by all the output feature maps is denoted as Y ₃ , where in Y ₃ The width of each feature map is height is The input of the twelfth convolutional layer receives all feature maps in Y ₃ , and the output of the twelfth convolutional layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as J ₁₂ , where J The width of each feature map in ₁₂ is height is The input end of the nineteenth batch of normalization layers receives all the feature maps in J ₁₂ , the output end of the nineteenth batch of normalization layers outputs 64 feature maps, and the set formed by all the output feature maps is denoted as P ₁₉ , where P The width of each feature map in ₁₉ is height is The input end of the nineteenth activation layer receives all the feature maps in P ₁₉ , and the output end of the nineteenth activation layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as H ₁₉ , where in H ₁₉ The width of each feature map is height is The input end of the fourth upsampling layer receives all the feature maps in H ₁₉ , the output end of the fourth upsampling layer outputs 64 feature maps, and the set formed by all the output feature maps is denoted as Y ₄ , where in Y _{4 The} width of each feature map of the The set formed by all feature maps is denoted as J ₁₃ , wherein the width of each feature map in J ₁₃ is R and the height is L; the input end of the twentieth batch of normalization layers receives all feature maps in J ₁₃ , the second The output of ten batches of normalization layers outputs 32 feature maps, and the set of all output feature maps is denoted as P ₂₀ , where the width of each feature map in P ₂₀ is R and the height is L; the twentieth activation The input end of the layer receives all the feature maps in P ₂₀ , and the output end of the _twentieth activation layer outputs ₃₂ feature maps. The width of the graph is R and the height is L; For the output layer, the input terminal of the fifteenth activation layer receives the output of the decoding framework, that is, all the feature maps in _C7 , and the output terminal of the fifteenth activation layer outputs 96 feature maps. It is denoted as H ₁₅ , wherein the width of each feature map in H ₁₅ is R and the height is L; the input end of the ninth convolution layer receives all the feature maps in H ₁₅ , and the output end of the ninth convolution layer outputs 1 feature map, the set of all output feature maps is denoted as J ₉ , where the width of the feature map in J ₉ is R, and the height is L; the input of the sixteenth batch of normalization layers receives J ₉ . Feature map, the output end of the sixteenth batch of normalization layers outputs one feature map, and the set composed of all the output feature maps is denoted as P ₁₆ , where the width of the feature map in P ₁₆ is R and the height is L; The input terminal of the sixteenth activation layer receives the feature map in P ₁₆ , and the output terminal of the sixteenth activation layer outputs one feature map, and the set formed by all the output feature maps is denoted as H ₁₆ , wherein the features in H ₁₆ The width of the image is R and the height is L, and the feature map in H ₁₆ is the estimated depth image corresponding to the original input image; Step 1_3: Take each original monocular image in the training set as the original input image, input it into the convolutional neural network for training, and obtain the estimated depth image corresponding to each original monocular image in the training set, and set {Q ⁿ The estimated depth image corresponding to (x,y)} is denoted as in, express The pixel value of the pixel whose middle coordinate position is (x, y); Step 1_4: Calculate the loss function value between the estimated depth image corresponding to each original monocular image in the training set and the corresponding real depth image, and The loss function value between is denoted as Step 1_5: Repeat steps 1_3 and 1_4 for a total of V times to obtain the trained convolutional neural network training model, and obtain a total of N×V loss function values; then find the smallest value from the N×V loss function values. Then take the weight vector and the bias term corresponding to the loss function value with the smallest value as the optimal weight vector and the optimal bias term of the trained convolutional neural network training model, which are recorded as W ^best and b ^best ; wherein, V>1; The specific steps of the test phase process are: Step 2_1: Let {Q(x',y')} denote the monocular image to be predicted; wherein, 1≤x'≤R', 1≤y'≤L', R' denotes {Q(x',y ')} width, L' means the height of {Q(x',y')}, Q(x',y') means the coordinate position in {Q(x',y')} is (x',y ') the pixel value of the pixel point; Step 2_2: Input {Q(x',y')} into the trained convolutional neural network training model, and use W ^best and b ^best to predict, and get {Q(x',y')} corresponding to The predicted depth image is recorded as {Q _depth (x',y')}; among them, Q _depth (x',y') indicates that the coordinate position in {Q _depth (x',y')} is (x',y) ') of the pixel value of the pixel point. 2. a kind of monocular vision depth estimation method according to claim 1, is characterized in that in described step 1-4, Obtained using the mean squared error function.