CN114882091A - Depth estimation method combined with semantic edge - Google Patents

Depth estimation method combined with semantic edge Download PDF

Info

Publication number
CN114882091A
CN114882091A CN202210476348.2A CN202210476348A CN114882091A CN 114882091 A CN114882091 A CN 114882091A CN 202210476348 A CN202210476348 A CN 202210476348A CN 114882091 A CN114882091 A CN 114882091A
Authority
CN
China
Prior art keywords
edge
semantic
depth
module
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210476348.2A
Other languages
Chinese (zh)
Other versions
CN114882091B (en
Inventor
朱冬晨
吴德明
张广慧
石文君
李嘉茂
王磊
张晓林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Institute of Microsystem and Information Technology of CAS
Original Assignee
Shanghai Institute of Microsystem and Information Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Institute of Microsystem and Information Technology of CAS filed Critical Shanghai Institute of Microsystem and Information Technology of CAS
Priority to CN202210476348.2A priority Critical patent/CN114882091B/en
Publication of CN114882091A publication Critical patent/CN114882091A/en
Application granted granted Critical
Publication of CN114882091B publication Critical patent/CN114882091B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a depth estimation method combined with semantic edges, which comprises the following steps: acquiring an image to be subjected to depth estimation; inputting the image into a trained deep learning network to obtain a depth prediction image and a semantic edge prediction image; the deep learning network includes: the system comprises a shared feature extraction module, a depth estimation module, an edge enhancement weight module, a depth edge semantic classification module and a semantic edge detection module; the shared feature extraction module is used for extracting feature information in the image and transmitting the feature information to the depth estimation module and the semantic edge detection module; the depth estimation module guides parallax smoothing through the semantic edge output by the semantic edge detection module and carries out depth estimation in an image double reconstruction mode; the edge enhancement weighting module forms a feature result required to be fused by the semantic edge detection module based on the depth edge of the depth prediction image output by the depth estimation module; the depth edge semantic classification module is used for performing depth edge semantic classification prediction; the semantic edge detection module is used for outputting semantic edge classification prediction of the image. The invention can improve the accuracy.

Description

Depth estimation method combined with semantic edge
Technical Field
The invention relates to the technical field of computer vision, in particular to a depth estimation method combined with semantic edges.
Background
The depth estimation and semantic edge extraction are used as basic problems of computer vision, and the task results can be deployed in practical sciences such as automatic driving, virtual reality, robots and the like to assist in achieving better results. The depth estimation refers to analyzing three-dimensional perception information from an image, a semantic edge extraction task is a task combining an edge extraction task and a classification task of the image, the semantic information of the edge and the boundary of an object can be obtained simultaneously, and the two tasks are processed by a deep learning method at present.
Depth estimation is further divided into monocular depth estimation and multi-ocular depth estimation, wherein monocular depth estimation has the advantages of fast processing speed, low cost and the like to a great extent due to the fact that the monocular depth estimation uses few sensors. The multi-view depth estimation uses two or more sensors to acquire information, so that the defects of information redundancy, complicated labeling and the like exist to a great extent. The use of monocular depth estimation is therefore also the mainstream choice in current research and markets.
Monocular depth estimation has a problem of low accuracy because depth estimation is performed using only a single piece of image information. The current mainstream method is to guide training of a depth estimation network by using a true value of additional information, such as edge information, semantic information, and the like, so as to generate a depth map with higher precision. The problem with the above methods is that there is no truth value for the extra information at the time of inference, so that only the trained method can be used to generate the required extra information, thereby losing the reliability of using truth value to guide at the time of inference. Another irrationale existing in the current mainstream monocular depth estimation schemes is to use one left view to generate both left and right disparity maps simultaneously, but it is considered irrational to use the left view to generate right disparity.
The main difficulty of semantic edge extraction is to suppress the influence of non-semantic edges, and in order to obtain finer semantic edges, the mainstream method adopted at present is to add non-maximum suppression Loss (NMS _ Loss) or use dynamic weights to enhance edge response and suppress non-edge response when training a network. The above method using dynamic weights has problems in that the weights are obtained by forcibly learning from features, theoretical support is lacked, and a learning layer with more parameters is added, so that the network becomes complicated.
Currently, the association of semantics and depth and the association of edges and depth are the main methods for joint learning of exploration and depth estimation tasks. However, there is little research on how to correlate depth estimates with semantic edges. Two simple strategies are possible. The first is that given semantic edge labels, we can do depth estimation through the guidance of the labels. Otherwise, given a depth label, semantic edge detection may be performed by directing the depth label. However, both of the above approaches depend largely on the accuracy of a given tag and are both step-wise modes, which can be suboptimal and inefficient.
Disclosure of Invention
The invention aims to provide a depth estimation method combined with semantic edges, which can improve the accuracy.
The technical scheme adopted by the invention for solving the technical problems is as follows: a depth estimation method combined with semantic edges is provided, which comprises the following steps:
acquiring an image to be subjected to depth estimation;
inputting the image into a trained deep learning network to obtain a depth prediction image and a semantic edge prediction image;
wherein the deep learning network comprises: the system comprises a shared feature extraction module, a depth estimation module, an edge enhancement weight module, a depth edge semantic classification module and a semantic edge detection module; the shared feature extraction module is used for extracting feature information in the image and transmitting the feature information to the depth estimation module and the semantic edge detection module; the depth estimation module guides parallax smoothing through the semantic edge output by the semantic edge detection module and carries out depth estimation in an image double reconstruction mode; the edge enhancement weighting module forms a feature result required to be fused by the semantic edge detection module based on the depth edge of the depth prediction image output by the depth estimation module; the depth edge semantic classification module is used for performing depth edge semantic classification prediction; the semantic edge detection module is used for outputting semantic edge classification prediction of the image.
The depth estimation module guides parallax smoothing through the semantic edge output by the semantic edge detection module, and the formula is as follows:
Figure BDA0003625727340000021
wherein the content of the first and second substances,
Figure BDA0003625727340000022
representing the gradient solution in the X and Y directions, respectively, d i,j Representing the parallax and the value of (i, j) in an RGB image, N representing the number of pixels, S i,j Represents the value of (i, j) in the semantic edge image, ε is the hyper-parameter.
The image double-construction means that the pixel value of a certain point on the left view is moved by the parallax value pixel of the left parallax and assigned to the position to obtain the right view, the parallax value of the certain point of the left parallax is used for searching the pixel value of the point on the right view and assigned to the point to reconstruct the left view.
The edge enhancement weighting module comprises: extracting a depth edge of the depth prediction map through an edge detection operator, inputting the depth edge into an EEW unit, outputting dynamic weight information by the EEW unit, wherein the weight information needs to satisfy F ═ W × F, wherein F ═ { A { (A } W } F { (A } F } W } F (1) ,A (2) ,A (3) }，A (1) ,A (2) ,A (3) Respectively representing the feature information of different depths extracted by the shared feature extraction module, and W representing dynamic weight information.
The depth edge semantic classification module combines the depth edge extracted by the Laplace operator and the feature information extracted by the shared feature extraction module, carries out depth edge semantic classification prediction through the CASSENet, and supervises by adopting the multi-label loss of the CASSENet.
The depth edge semantic classification module generates a basic true value of depth edge semantic classification by using the predicted depth edge and the basic true value of the semantic edge; and taking the intersection part of the depth edge and the ground truth value of the semantic edge as the ground truth value of the depth edge semantic classification task.
Advantageous effects
Due to the adoption of the technical scheme, compared with the prior art, the invention has the following advantages and positive effects: the invention realizes mutual benefit and benefit of the tasks of depth estimation and semantic edge extraction by sharing the feature extraction module. The invention utilizes semantic edge guided disparity smoothing, image dual reconstruction to improve depth estimation at the edges. In semantic edge detection, an edge enhancement weight strategy is proposed, which enhances edge pixels by learning weights from deep edges and assigning them to edge features, thereby improving the accuracy of semantic edges. The invention also provides a depth edge semantic classification model to realize semantic edge and depth edge consistency constraint so as to realize implicit loss monitoring.
Drawings
FIG. 1 is a schematic structural diagram of a deep learning network in an embodiment of the present invention;
FIG. 2 is a schematic diagram of image double reconstruction in an embodiment of the present invention;
FIG. 3 is a schematic diagram of an edge enhancement weighting module according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a deep edge semantic classification module according to an embodiment of the invention.
Detailed Description
The invention will be further illustrated with reference to the following specific examples. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and such equivalents may fall within the scope of the present invention as defined in the appended claims.
The embodiment of the invention relates to a depth estimation method combined with semantic edges, which comprises the following steps: acquiring an image to be subjected to depth estimation; inputting the image into a trained deep learning network to obtain a depth prediction image and a semantic edge prediction image;
as shown in fig. 1, the deep learning network includes: the system comprises a shared feature extraction module, a depth estimation module, an edge enhancement weight module, a depth edge semantic classification module and a semantic edge detection module; the shared feature extraction module is used for extracting feature information in the image and transmitting the feature information to the depth estimation module and the semantic edge detection module; the depth estimation module guides parallax smoothing through the semantic edge output by the semantic edge detection module and carries out depth estimation in an image double reconstruction mode; the edge enhancement weighting module forms a feature result required to be fused by the semantic edge detection module based on the depth edge of the depth prediction image output by the depth estimation module; the depth edge semantic classification module is used for performing depth edge semantic classification prediction; the semantic edge detection module is used for outputting semantic edge classification prediction of the image.
According to the embodiment, the strong consistency relationship between the depth edge and the semantic edge is utilized, so that task synchronization joint learning, edge enhancement weight, depth edge semantic classification, semantic edge guide parallax smoothing and image double-structure loss are realized.
In common disparity estimation models, the original RGB image is usually used to smoothly guide the disparity, i.e., the local smoothing of the disparity is optimized by using L1 loss at the disparity gradient. Since the nature of discontinuities in depth often manifests at the gradients of an RGB image, this loss can be weighted with an edge perception term. In this embodiment, the depth estimation module guides disparity smoothing through the semantic edge output by the semantic edge detection module, and the formula is as follows:
Figure BDA0003625727340000041
wherein the content of the first and second substances,
Figure BDA0003625727340000042
representing the gradient solution in the X and Y directions, respectively, d i,j Denotes the value of (i, j) of the parallax in the RGB image, N denotes the number of pixels, S i,j The value (i, j) in the semantic edge image is represented, and ∈ is a hyper-parameter and is set to 0.001 in the present embodiment.
Since there is no reason to generate right parallax using left view, the present embodiment cannot train in a way of generating right view using left parallax and right view. This embodiment proposes a dual-configuration approach, i.e. using left parallax and left view to generate right view instead of the above unreasonable approach. As shown in fig. 2, in this embodiment, the pixel value of a certain point of the left view is shifted by parallax value by a pixel and assigned to the position to obtain the right view. In the case of a perfect left-right attempt and left parallax, it can be considered that a right view of an object without occlusion can be reconstructed using the left view and left parallax; and searching the pixel value of a certain point on the right view by using the parallax value of the certain point of the left parallax, and assigning the pixel value to the point to reconstruct a left view, thereby obtaining a left view with completely dense pixel values.
The edge enhancement weighting module in the embodiment is based on the strong consistency relationship between the depth edge and the semantic edge, and utilizes the Sobel operator to extract the depth edge of the depth map as the input of the EEW module in the figure 3 so as to learn a dynamic weighting information W belonging to R h×w×8 . By utilizing the characteristics of side1-3Organization F ═ A (1) ,A (2) ,A (3) And assigning a weight to each pixel to ensure that the edge information is strengthened and F is obtained, so as to form a final feature result to be fused of the semantic edge extraction branch, as shown in the following formula.
Figure BDA0003625727340000043
Figure BDA0003625727340000044
Is the ith class output of side5, where A f The method refers to the feature combination of the fused classification layer input to the CASSEN, and then carries out classification convolution of 1 x 1 of K-groups to generate an activation graph of a K channel, and the activation graph is also the final output of the semantic edge extraction branch.
The depth edge semantic classification module in the present embodiment combines the depth edge D extracted by the laplacian operator edge And side4 feature, depth edge semantic classification prediction is performed through sharedcollocation and fusedClassification structures of CASSET, and multi-label loss of CASSET is adopted for supervision, as shown in FIG. 4. Specifically, a depth-classified ground truth prediction is first generated. And generating a basic truth value of the depth edge semantic classification by using the predicted depth edge and the basic truth value of the semantic edge. Since the ground truth values of the depth edge and the semantic edge are both binary mappings of 0-1, the intersection of the two can be regarded as the ground truth value of the depth edge semantic classification task, as follows:
Figure BDA0003625727340000051
wherein, Gt sem-edge Refers to the truth value of the semantic edge extraction,
Figure BDA0003625727340000052
refers to the edge D by giving depth edge Setting a hyper-parameter thresholdIn the obtained binary image, the hyper-parameter threshold is set to 0.5 in the present embodiment, that is, the obtained binary image is obtained
Figure BDA0003625727340000053
The embodiment achieves the optimal effect in monocular depth estimation at present, and is improved by 5.1% in the semantic edge extraction task compared with the reference network.
As can be easily found, the invention realizes the mutual benefit and benefit of the tasks of depth estimation and semantic edge extraction by sharing the feature extraction module. The invention utilizes semantic edge guided disparity smoothing, image dual reconstruction to improve depth estimation at the edges. In semantic edge detection, an edge enhancement weight strategy is proposed, which enhances edge pixels by learning weights from deep edges and assigning them to edge features, thereby improving the accuracy of semantic edges. The invention also provides a depth edge semantic classification model to realize semantic edge and depth edge consistency constraint so as to realize implicit loss monitoring.

Claims (6)

1. A depth estimation method combined with semantic edges is characterized by comprising the following steps;
acquiring an image to be subjected to depth estimation;
inputting the image into a trained deep learning network to obtain a depth prediction image and a semantic edge prediction image;
wherein the deep learning network comprises: the system comprises a shared feature extraction module, a depth estimation module, an edge enhancement weight module, a depth edge semantic classification module and a semantic edge detection module; the shared feature extraction module is used for extracting feature information in the image and transmitting the feature information to the depth estimation module and the semantic edge detection module; the depth estimation module guides parallax smoothing through the semantic edge output by the semantic edge detection module and carries out depth estimation in an image double reconstruction mode; the edge enhancement weighting module forms a feature result required to be fused by the semantic edge detection module based on the depth edge of the depth prediction image output by the depth estimation module; the depth edge semantic classification module is used for performing depth edge semantic classification prediction; the semantic edge detection module is used for outputting semantic edge classification prediction of the image.
2. The depth estimation method combined with semantic edge according to claim 1, wherein the depth estimation module guides disparity smoothing through the semantic edge output by the semantic edge detection module, and the formula is as follows:
Figure FDA0003625727330000011
wherein the content of the first and second substances,
Figure FDA0003625727330000012
representing the gradient solution in the X and Y directions, respectively, d i,j Denotes the value of (i, j) of the parallax in the RGB image, N denotes the number of pixels, S i,j Represents the value of (i, j) in the semantic edge image, ε is the hyper-parameter.
3. The method of claim 1, wherein the image reconstruction includes shifting a pixel value of a point on the left view by a disparity value of the point and assigning the pixel value to the position to obtain a right view, and searching the right view for the pixel value of the point by using the disparity value of the point on the left view and assigning the pixel value to the point to reconstruct the left view.
4. The method for depth estimation in conjunction with semantic edges as defined in claim 1, wherein the edge enhancement weighting module comprises: extracting a depth edge of the depth prediction map through an edge detection operator, inputting the depth edge into an EEW unit, outputting dynamic weight information by the EEW unit, wherein the weight information needs to satisfy F ═ W × F, wherein F ═ { A { (A } W } F { (A } F } W } F (1) ,A (2) ,A (3) }，A (1) ,A (2) ,A (3) Respectively representing the feature information of different depths extracted by the shared feature extraction module, and W representing dynamic weight information.
5. The depth estimation method combined with semantic edges according to claim 1, wherein the depth edge semantic classification module performs depth edge semantic classification prediction by using CASENEt in combination with the depth edge extracted by the Laplacian and the feature information extracted by the shared feature extraction module, and performs supervision by using multi-label loss of CASENEt.
6. The depth estimation method combined with semantic edges as claimed in claim 5, wherein the depth edge semantic classification module generates a basic true value of depth edge semantic classification using the predicted depth edge and the basic true value of semantic edge; and taking the intersection part of the depth edge and the ground truth value of the semantic edge as the ground truth value of the depth edge semantic classification task.
CN202210476348.2A 2022-04-29 2022-04-29 Depth estimation method combining semantic edges Active CN114882091B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210476348.2A CN114882091B (en) 2022-04-29 2022-04-29 Depth estimation method combining semantic edges

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210476348.2A CN114882091B (en) 2022-04-29 2022-04-29 Depth estimation method combining semantic edges

Publications (2)

Publication Number Publication Date
CN114882091A true CN114882091A (en) 2022-08-09
CN114882091B CN114882091B (en) 2024-02-13

Family

ID=82674259

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210476348.2A Active CN114882091B (en) 2022-04-29 2022-04-29 Depth estimation method combining semantic edges

Country Status (1)

Country Link
CN (1) CN114882091B (en)

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106204522A (en) * 2015-05-28 2016-12-07 奥多比公司 The combined depth of single image is estimated and semantic tagger
CN109829929A (en) * 2018-12-30 2019-05-31 中国第一汽车股份有限公司 A kind of level Scene Semantics parted pattern based on depth edge detection
CN110120049A (en) * 2019-04-15 2019-08-13 天津大学 By single image Combined estimator scene depth and semantic method
CN110781897A (en) * 2019-10-22 2020-02-11 北京工业大学 Semantic edge detection method based on deep learning
US20200160533A1 (en) * 2018-11-15 2020-05-21 Samsung Electronics Co., Ltd. Foreground-background-aware atrous multiscale network for disparity estimation
CN111401380A (en) * 2020-03-24 2020-07-10 北京工业大学 RGB-D image semantic segmentation method based on depth feature enhancement and edge optimization
CN112150493A (en) * 2020-09-22 2020-12-29 重庆邮电大学 Semantic guidance-based screen area detection method in natural scene
CN112950645A (en) * 2021-03-24 2021-06-11 中国人民解放军国防科技大学 Image semantic segmentation method based on multitask deep learning
CN113096176A (en) * 2021-03-26 2021-07-09 西安交通大学 Semantic segmentation assisted binocular vision unsupervised depth estimation method
CN113822919A (en) * 2021-11-24 2021-12-21 中国海洋大学 Underwater image relative depth estimation method based on semantic information constraint
CN114241210A (en) * 2021-11-22 2022-03-25 中国海洋大学 Multi-task learning method and system based on dynamic convolution
CN114359361A (en) * 2021-12-28 2022-04-15 Oppo广东移动通信有限公司 Depth estimation method, depth estimation device, electronic equipment and computer-readable storage medium

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106204522A (en) * 2015-05-28 2016-12-07 奥多比公司 The combined depth of single image is estimated and semantic tagger
US20200160533A1 (en) * 2018-11-15 2020-05-21 Samsung Electronics Co., Ltd. Foreground-background-aware atrous multiscale network for disparity estimation
CN109829929A (en) * 2018-12-30 2019-05-31 中国第一汽车股份有限公司 A kind of level Scene Semantics parted pattern based on depth edge detection
CN110120049A (en) * 2019-04-15 2019-08-13 天津大学 By single image Combined estimator scene depth and semantic method
CN110781897A (en) * 2019-10-22 2020-02-11 北京工业大学 Semantic edge detection method based on deep learning
CN111401380A (en) * 2020-03-24 2020-07-10 北京工业大学 RGB-D image semantic segmentation method based on depth feature enhancement and edge optimization
CN112150493A (en) * 2020-09-22 2020-12-29 重庆邮电大学 Semantic guidance-based screen area detection method in natural scene
CN112950645A (en) * 2021-03-24 2021-06-11 中国人民解放军国防科技大学 Image semantic segmentation method based on multitask deep learning
CN113096176A (en) * 2021-03-26 2021-07-09 西安交通大学 Semantic segmentation assisted binocular vision unsupervised depth estimation method
CN114241210A (en) * 2021-11-22 2022-03-25 中国海洋大学 Multi-task learning method and system based on dynamic convolution
CN113822919A (en) * 2021-11-24 2021-12-21 中国海洋大学 Underwater image relative depth estimation method based on semantic information constraint
CN114359361A (en) * 2021-12-28 2022-04-15 Oppo广东移动通信有限公司 Depth estimation method, depth estimation device, electronic equipment and computer-readable storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JING LIU 等: "Collaborative Deconvolutional Neural Networks for Joint Depth Estimation and Semantic Segmentation", 《IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS》, vol. 29, no. 11, pages 5655 - 5666, XP011692883, DOI: 10.1109/TNNLS.2017.2787781 *
张 鹏 等: "基于语义边缘驱动的实时双目深度估计算法", 《计算机科学》, vol. 48, no. 9, pages 216 - 222 *
张浩东 ，宋嘉菲，张广慧: "边缘引导特征融合和代价聚合的立体匹配算法", 《计算机工程与应用》, pages 182 - 188 *

Also Published As

Publication number Publication date
CN114882091B (en) 2024-02-13

Similar Documents

Publication Publication Date Title
US20210142095A1 (en) Image disparity estimation
US10353271B2 (en) Depth estimation method for monocular image based on multi-scale CNN and continuous CRF
Lin et al. Depth estimation from monocular images and sparse radar data
US20210065393A1 (en) Method for stereo matching using end-to-end convolutional neural network
CN110706269B (en) Binocular vision SLAM-based dynamic scene dense modeling method
CN109117691A (en) Drivable region detection method, device, equipment and storage medium
CN112084859A (en) Building segmentation method based on dense boundary block and attention mechanism
CN112651423A (en) Intelligent vision system
Zhang et al. Robust stereo matching with surface normal prediction
CN113536920A (en) Semi-supervised three-dimensional point cloud target detection method
Huang et al. Measuring the absolute distance of a front vehicle from an in-car camera based on monocular vision and instance segmentation
CN114693744A (en) Optical flow unsupervised estimation method based on improved cycle generation countermeasure network
CN112686952A (en) Image optical flow computing system, method and application
Yang et al. SAM-Net: Semantic probabilistic and attention mechanisms of dynamic objects for self-supervised depth and camera pose estimation in visual odometry applications
CN112184731A (en) Multi-view stereo depth estimation method based on antagonism training
Zhang et al. Local stereo matching: An adaptive weighted guided image filtering-based approach
CN110766609B (en) Depth-of-field map super-resolution reconstruction method for ToF camera
CN114882091A (en) Depth estimation method combined with semantic edge
CN113284221B (en) Target detection method and device and electronic equipment
Pan et al. An automatic 2D to 3D video conversion approach based on RGB-D images
CN114445618A (en) Cross-modal interaction RGB-D image salient region detection method
Tang et al. Cross-modality depth estimation via unsupervised stereo RGB-to-infrared translation
Zhou et al. Underwater occluded object recognition with two-stage image reconstruction strategy
CN111784680A (en) Detection method based on consistency of key points of left and right eye views of binocular camera
Liu Scalable Vision-Based 3D Object Detection and Monocular Depth Estimation for Autonomous Driving

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant