KR20220079125A - System and method for semi-supervised single image depth estimation and computer program for the same - Google Patents

Abstract

A single image depth estimation system includes: a depth estimation unit configured to calculate pseudo depth information for an input image; a confidence calculation unit configured to calculate confidence information for the pseudo depth information; and a threshold value determining unit configured to determine a threshold of reliability for filtering a part of the pseudo-depth information through learning using the reliability information. The depth estimation unit is further configured to generate a depth estimation model for a single image by using the pseudo-depth information filtered by the threshold value. The single-image depth estimation system exhibits superior performance compared to systems using known single-image depth estimation methods, and by using a threshold network, it prevents performance degradation due to errors existing in the pseudo-correction depth image and uses the existing reliability estimation method. There is an advantage that the performance of the method can be improved.

Description

A semi-supervised single image depth estimation system and method, and a computer program for the same

Embodiments relate to a single image depth estimation system and method and a computer program therefor. More specifically, the embodiments provide a novel method for monocular depth estimation based on a semi-supervised learning method using a pseudo ground truth generated through stereo matching. It is about the technology that provides the framework.

Monocular depth estimation or single-image depth estimation technology that predicts depth information for each pixel in a single image such as one RGB image plays an important role in various fields such as robotics and autonomous driving. Early research on single-image depth estimation is mainly based on supervised learning using training data with depth images corresponding to the correct answers. However, for this, a huge amount of training data must be built, which is expensive and labor-intensive.

Due to this problem, most recent studies are based on a self-supervised learning method that calculates the reconstruction loss using the pixel-by-pixel similarity of the image. The self-supervised learning method seems to be an alternative to the absence of training data with correct answers, but it blurs the result of the depth map at the boundary side of the object, or does not consider the pixels of the occluded area of the stereo image. There is a problem in that the performance of the estimated depth information is deteriorated because the region cannot be processed.

Unexamined Patent Publication No. 10-2017-0082794

According to an aspect of the present invention, it is by a new approach that utilizes a depth map generated through stereo matching as a pseudo ground truth, and thresholding depth information of a pseudo correct answer It is possible to provide a single image depth estimation system and method, and a computer program for the same, that can prevent performance degradation due to errors existing in pseudo-answers by filtering by the obtained confidence map and learning the depth estimation network through this.

A single image depth estimation system according to an aspect of the present invention includes: a depth estimation unit configured to calculate pseudo depth information for an input image; a confidence calculation unit configured to calculate confidence information for the pseudo depth information; and a threshold value determining unit configured to determine a threshold of reliability for filtering a part of the pseudo-depth information through learning using the reliability information.

In this case, the depth estimator is further configured to generate a depth estimation model for a single image by using the pseudo-depth information filtered by the threshold value.

In an embodiment, the depth estimator may include: a stereo matching unit configured to calculate the pseudo-depth information from the input image using a pre-stored stereo matching model; and a depth learning unit configured to train a depth estimation network using the pseudo-depth information filtered by the threshold value.

In an embodiment, the depth estimation network includes one or more encoder layers for extracting feature values from an image and one or more decoder layers configured to convert the feature values into depth information. In this case, the threshold value determining unit is further configured to determine the threshold value through adaptive learning using the feature values extracted by the one or more encoder layers.

In an embodiment, the threshold value determining unit is further configured to generate the reliability information thresholded by a differential soft-thresholding function defined using the reliability information and the threshold value.

In an embodiment, the threshold determining unit is further configured to determine the threshold by learning a threshold network using a loss function defined by the thresholded reliability information and the reference reliability information.

In an embodiment, the depth estimator is further configured to train the depth estimation network by a regression loss function defined using the thresholded reliability information and the pseudo depth information.

A single image depth estimation method according to an aspect of the present invention comprises: calculating, by a single image depth estimation system, pseudo depth information for an input image; calculating, by the single image depth estimation system, reliability information for the pseudo depth information; determining, by the single image depth estimation system, a threshold of reliability for filtering a part of the pseudo depth information through learning using the reliability information; and generating, by the single image depth estimation system, a depth estimation model for a single image by using the pseudo-depth information filtered by the threshold value.

In an embodiment, the calculating of the pseudo-depth information includes calculating, by the single image depth estimation system, the pseudo-depth information from the input image using a pre-stored stereo matching model.

In an embodiment, generating the depth estimation model includes training, by the single image depth estimation system, a depth estimation network using the pseudo-depth information filtered by the threshold value.

In an embodiment, the depth estimation network includes one or more encoder layers for extracting feature values from an image and one or more decoder layers configured to convert the feature values into depth information. In this case, the calculating of the threshold includes, by the single image depth estimation system, determining the threshold through adaptive learning using the feature values extracted by the one or more encoder layers.

In an embodiment, the calculating of the threshold value comprises: generating, by the single image depth estimation system, reliability information thresholded by a differential soft-thresholding function defined using the reliability information and the threshold value includes steps.

In an embodiment, the calculating of the threshold may further include, by the single image depth estimation system, learning the threshold network using a loss function defined by the thresholded reliability information and the reference reliability information. include

In an embodiment, the generating of the depth estimation model comprises: training, by the single image depth estimation system, a depth estimation network using a regression loss function defined using the thresholded reliability information and the pseudo depth information. includes steps.

In one aspect of the present invention, the computer program is combined with hardware to execute the single image depth estimation method according to the above-described embodiments, and may be stored in a computer-readable recording medium.

A single image depth estimation system and method according to an aspect of the present invention uses three sub-networks: a monocular depth network, a confidence network, and a threshold network, and a pseudo correct answer by the monocular depth network. It is configured to train the depth estimation network in a semi-supervised learning method using ground truth.

The single-image depth estimation system and method according to an aspect of the present invention exhibit superior performance compared to known single-image depth estimation methods, and use a threshold network to prevent performance degradation due to errors existing in the pseudo correct depth image, and The performance of the method using the existing reliability estimation method can be improved, which has the advantage of providing a base technology that can be used in various fields such as autonomous vehicles and virtual reality.

1 is a schematic block diagram of a single image depth estimation system according to an embodiment.
2 is a flowchart illustrating each step of a method for estimating depth of a single image according to an exemplary embodiment.
3 is a conceptual diagram illustrating sub-networks included in a single image depth estimation system according to an embodiment.
4 is a graph illustrating a reliability value thresholded by a single image depth estimation method according to an exemplary embodiment.
5 is an image illustrating depth information obtained by step-by-step application of the single image depth estimation method according to an embodiment to an original image.
6 and 7 are images illustrating the performance of a single image depth estimation method according to an exemplary embodiment in comparison with the related art.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a schematic block diagram of a single image depth estimation system according to an embodiment.

Referring to FIG. 1 , the single image depth estimation system 1 according to the present embodiment includes a depth estimation unit 10 , a reliability calculation unit 20 , and a threshold value determination unit 30 . In an embodiment, the single image depth estimation system 3 further includes a database (DB) 40 in which a depth estimation model and/or an input image, etc. are stored. Also in one embodiment, the single image depth estimation system 3 further includes an output unit 50 for providing the user with depth information generated from the input image. Further, in an embodiment, the depth estimation unit 10 includes a stereo matching unit 11 and a depth learning unit 12 .

The devices described herein may be wholly hardware, or may have aspects that are partly hardware and partly software. For example, each unit ( 10 , 20 , 30 , 40 , 50 ) and its sub-units included in the single image depth estimation system 1 are devices for transmitting and receiving data in a specific format and content in an electronic communication manner. and software related thereto. As used herein, terms such as “unit”, “module”, “server”, “system”, “platform”, “device” or “terminal” are intended to refer to a combination of hardware and software driven by the hardware. do. For example, the hardware herein may be a data processing device including a CPU or other processor. In addition, software driven by hardware may refer to a running process, an object, an executable file, a thread of execution, a program, and the like.

In addition, in this specification, each part constituting the single image depth estimation system 1 is not necessarily intended to refer to physically distinct separate components. That is, in FIG. 1 , each part 10 , 20 , 30 , 40 , 50 of the single image depth estimation system 1 is shown as separate blocks separated from each other, but this is a single image depth estimation system 1 by this. It is functionally classified according to the action being executed. Depending on the embodiment, some or all of the aforementioned units 10, 20, 30, 40, and 50 may be integrated in the same single device, or one or more units may be physically separated from other units as separate devices. may be implemented. For example, each part of the single image depth estimation system 1 may be components communicatively connected to each other under a distributed computing environment.

The depth estimator 10 is configured to calculate pseudo ground truth depth information (also referred to as pseudo depth information in this specification) for the input image. Also, the depth estimator 10 is configured to generate a depth estimation model capable of estimating depth information from a single image by performing learning on the depth estimation network.

In the present specification, the network refers to extracting a feature value from an input image through one or more layers and/or restoring a feature value to an image through one or more other layers while learning parameters related to the extraction or restoration process. It refers to a machine learning model configured to be updated using

The reliability calculation unit 20 is configured to calculate reliability information from the pseudo depth information through learning about the reliability network. In this case, the reliability is a function of the probability density of the image, and in the present specification, the reliability information means that the probability of occurrence of a value of the corresponding area in the entire object is assigned to each unit area (eg, pixel) of the image. For example, each of the pseudo-depth information and the reliability information may have the form of a map.

The threshold value determining unit 30 determines a threshold value for filtering a part of the pseudo-depth information, and serves to generate a thresholded reliability map by learning using the reliability information. In this case, the depth estimator 10 may generate a depth estimation model for a single image by using the pseudo-depth information filtered by the thresholded reliability map.

Also, the depth estimator 10 may generate depth information for an unknown single image by using the depth estimation model generated through learning as described above. The output unit 50 processes the generated depth information in a form that can be viewed by a user and transmits it to a user device (not shown) through a communication method through a network, or an output means of the single image depth estimation system 1 ( not shown) can be provided.

2 is a flowchart illustrating each step of a method for estimating depth of a single image according to an embodiment, and FIG. 3 is a conceptual diagram illustrating sub-networks included in the system for estimating depth of a single image according to an embodiment. Hereinafter, for convenience of description, a single image depth estimation method according to the present embodiment will be described with reference to FIGS. 1 to 3 .

First, the single image depth estimation system 1 may receive training data for generating a depth estimation model ( S1 ). The training data may be a stereo image including a left eye image and a right eye image, and depth information corresponding to each region (eg, pixel) of the image may be previously labeled in the training data.

Next, the stereo matching unit 11 of the depth estimator 10 performs the stereo matching method in any one of the stereo images, for example, the pseudo-depth information 302 d ^Pgt for the monocular image 301 I ^l corresponding to the left eye. can be generated (S2). The generation of pseudo-depth information 302 through stereo matching is a known method using a pre-trained stereo matching network, for example, “Learning from scratch a confidence measure” by Poggi, M. and Mattoccia, S. (BMVC, 2016). ), so a detailed description thereof will be omitted in order to clarify the gist of the invention.

The depth learning unit 12 of the depth estimator 10 converts the monocular image 301 I ^l into a depth estimation network including one or more encoding layers 351 and one or more decoding layers 352 ( 305) can be used as an input image to learn about depth information. For example, the depth estimation network 305 is disclosed in Ronneberger, O. et al., "U-net: Convolutional networks for biomedical image segmentation" (International Conference on Medical image computing and computer-assisted intervention, 234-241, 2015). It may have an encoder-decoder architecture known as U-net, and may have an encoder network corresponding to 13 convolution layers and a decoder network symmetric thereto. However, the shape of the depth estimation network 305 is not limited thereto.

Meanwhile, in an embodiment, at this time, the feature value extracted from the monocular image 301 I ¹ through the encoding layer 351 may be used for adaptive learning of the threshold value network 306 M _T to be described later.

The reliability calculator 20 may generate the reliability information 304 c through learning using the pseudo depth information 302 d ^Pgt as an input image for the reliability network 303 M _c ( S4 ). The reliability network 303 M _c may be constructed by any known reliability estimation method or to be developed in the future. For example, the reliability network 303 M _c may be configured by the CCNN method proposed by M. Poggi and S. Mattoccia, but is not limited thereto.

Next, the threshold value determination unit 30 can trust only the depth value with the reliability exceeding the threshold T through learning using the reliability information 304 c as an input image for the threshold network 306 M _T It is possible to determine the threshold value T to determine that (S5). Threshold network 306 M _T may be configured the same as or similar to one or more encoding layers 351 of depth estimation network 305 . At this time, how to set the threshold value T depends on the characteristics of the image. For example, in an image in which pseudo-depth information is difficult to obtain through stereo matching, the threshold value T should be high, and the image in which pseudo-depth information is easy to obtain through stereo matching. In , the threshold value T may be low.

In an embodiment, the threshold value T may be adaptively learned by reflecting the characteristics of the image. To this end, the threshold value determining unit 30 uses a feature value (eg, a convolutional feature value) extracted from the monocular image 301 I ^l through the encoding layer 351 to the threshold value network 306 M _T . learning can be performed (S3, S5).

In an embodiment, the threshold value determiner 30 may generate thresholded reliability information by applying a threshold using a differential soft-thresholding function (S6). For example, in an embodiment, the reliability map 307 C ^T corresponding to the thresholded reliability information may be calculated as in Equation 1 below.

In Equation 1, p represents a pixel of the image, and c _p represents the reliability of the pixel. At this time, the slope of the thresholded reliability map 307 C ^T is controlled by the value of the hyper parameter ε set by the user. ε may be, for example, a constant having a positive value.

4 is a graph illustrating a reliability value thresholded by a method for estimating a single image depth according to an embodiment. In each of the four graphs 401 to 404 shown in FIG. 4, the value of the parameter ε in Equation 1 is 5. , 10, 25, and 90 indicate the thresholded reliability C ^T according to the reliability c value. As shown, the larger the value of the parameter ε, the sharper the value of the thresholded reliability C ^T is mapped to 0 or 1. In the test examples described herein, the value of the parameter ε is set to 10, but is not limited thereto.

Referring back to FIGS. 1 to 3 , the threshold determining unit 30 is a loss function L defined by the thresholded reliability map 307 C ^T and preset ground truth reliability information 308 C ^gt . _T may be used to train the threshold network 306 M _T . In this case, the reference reliability information 308 C ^gt is "Unsupervised domain adaptation for depth prediction from images" (EEE transactions on pattern analysis and machine intelligence, 2019) by Tonioni, A. et al. and by Kim, S. et al. "Laf-net: Locally adaptive fusion networks for stereo confidence estimation" (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 205-214, 2019) can be obtained using sparse ground truth depth data from a known method. have.

In addition, in this process, the reliability calculator 20 may also learn the reliability network 303 M _C using the loss function L _T . For example, in an embodiment, the loss function L _T may be defined as in Equation 2 below.

When the thresholded reliability information is determined through the threshold network 306 M _T and the reliability network 303 M _C learned according to Equation 2, the depth learning unit 12 of the depth estimator 10 is Depth information 310 for monocular image 301 I ^l by training a depth estimation network 305 M _D using pseudo depth information 309 d ^Pgt filtered using localized reliability information 307 C _T . d can be generated (S8).

For example, in one embodiment, the depth estimation network 305 M _D may be learned by calculating a value of a regression loss function L _D induced by reliability, in which case the regression loss function L _D is expressed as in Equation 3 below. can be defined.

In Equation 3, d _p denotes the estimated depth information 310 , d ^Pgt denotes the pseudo-depth information 309 , and Ω denotes a set of all pixels included in the monocular image 301 I ^l . Also, in an embodiment, the loss calculated by Equation 3 may be normalized by Equation 4 below.

Through the learning process described above, a depth estimation model for estimating depth information from a single image may be generated. As a result, it is possible to estimate depth information by applying the depth estimation model to an unknown input image of which depth information is not known, and provide it to the user (S9).

The present inventors used a stereo image set and sparse depth map provided by the KITTI dataset to train the threshold network 306 M _T and the reliability network 303 M _C , and LIDAR ( LiDAR) depth maps were used.

Figure 5 is an image showing the result, Figure 5 (a) shows the original image, Figure 5 (b) is monocular depth estimation without the threshold network M _T and the reliability network M _C in the embodiment of the present invention. Depth information obtained through In addition, Fig. 5 (c) shows depth information estimated by learning only the reliability network M _C while fixing the threshold value T to 0.3, and Fig. 5 (d) is the threshold value network M _T according to an embodiment of the present invention. and depth information obtained by the depth estimation network M _D while learning all of the reliability networks M _C .

As shown, it can be seen that the depth estimation result of the image is gradually improved while changing from (a) to (d) of FIG. 5 . In particular, it can be seen that there is a limit to learning the depth estimation network MD through only the pseudo-depth information from the point that the pseudo-depth information shown in FIG. 5B is inaccurate _. It can be seen that depth estimation performance can be improved except for pixels with low reliability in FIG. 5B by using the obtained reliability information.

6 is an image showing the performance of a single image depth estimation method according to an embodiment in comparison with the prior art. FIG. 6 (a) is an original image, and FIG. 6 (f) is an embodiment of the present invention. Depth information estimated by On the other hand, FIGS. 6 (b) to (e) show depth information estimated by the prior art, and FIG. 6 (b) is "Semisupervised deep learning for monocular depth map prediction" (Proceedings) by Kuznietsov, Y. et al. of the IEEE conference on computer vision and pattern recognition, 6647-6655, 2017) represents depth information estimated by the method disclosed, and Figure 6 (c) is a co-author of Godard, C. et al. "Unsupervised monocular depth estimation with left- right consistency" (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 270-279, 2017) represents depth information estimated by the method disclosed, (d) of FIG. 6 is Godard, C. et al. co-author "Digging into Self-supervised monocular depth estimation" (Proceedings of the IEEE international conference on computer vision, 3828-3838, 2019) represents depth information estimated by the method disclosed, (e) of FIG. 6 is a co-author of Watson, J. et al. Self-supervised monocular depth hints", it represents depth information estimated by the method disclosed in Proceedings of the IEEE International Conference on Computer Vision, 2162-2171, 2019).

As shown, it can be seen that by estimating depth information according to an embodiment of the present invention, it is possible to completely detect an object up to a fine boundary without blurring the boundary of the object as in the prior art.

In addition, the present inventors compared the detection accuracy according to the embodiments of the present invention for the KITTI eigen split data set with the known prior art, and the results are shown in Table 1. S and L of the data column represent a stereo image and a left-eye image, respectively, Sem represents a map model trained through a semantic segmentation network, and PGT represents a model using pseudo correct depth information according to an embodiment of the present invention. indicates.

evaluation value accuracy Way data map
method Abs Rel Sqr
Rel RM
SE RMSE
Log δ<
1.25 δ<
1.25 ² δ<
1.25 ³ One Monodepth S self 0.138 1.186 5.650 0.234 0.813 0.930 0.969 2 Uncensorship S self 0.107 0.811 4.796 0.200 0.866 0.952 0.978 3 MonoResMatch S self 0.111 0.867 4.714 0.199 0.864 0.954 0.979 4 Kuznietsov S+LiDAR map 0.122 0.762 4.815 0.194 0.845 0.957 0.987 5 Kuznietsov S+LiDAR jun map 0.113 0.741 4.621 0.189 0.862 0.960 0.986 6 Monodepth2 S self 0.108 0.842 4.891 0.207 0.866 0.949 0.976 7 DepthHint S self 0.102 0.762 4.602 0.189 0.880 0.960 0.981 8 Guizilimi S+
Sem self 0.102 0.698 4.381 0.178 0.896 0.964 0.984 9 Example 1 L+
PGT jun map 0.099 0.657 4.289 0.185 0.884 0.962 0.982 10 Example 2 L+
PGT jun map 0.098 0.647 4.253 0.186 0.884 0.960 0.981

In addition, Example 1 of Table 1 shows the depth estimation network M _D and the threshold network M using (ii) loss functions L _D and L _T in a state where (i) the reliability network M _C is fixed by the pre-trained reliability network. It corresponds to the result of learning the parameters of _T. In addition, in Example 2, in addition to the methods (i) and (ii) of Example 1, (iii) a reliability network M _C and a threshold network using (iv) a loss function L _T in a state where the depth estimation network M _D is fixed. An embodiment in which the process of learning M _T and (v ₎ the process of learning the depth estimation network _{MD by using the loss function L D in} a state where the reliability network MC and the threshold network _MT are fixed is further shown.

In Table 1, the higher the accuracy value, the better the performance. Other evaluation values indicate that the lower the value, the better the performance. Among the performance evaluation values in Table 1, Abs Rel and Sqr Rel represent the absolute relationship error and the square relationship error between the predicted value and the correct answer, respectively. Also in Table 1, RMSE and RMSE log (log) represent root mean square error and log root mean square error, respectively. Furthermore, in Table 1, δ<1.25 ⁿ means a ratio value of pixels in which the ratio between the predicted value and the correct answer is less than 1.25 to the nth power. In addition, in Table 1, the bold and underlined figures indicate the 1st and 2nd ranks in the order of superior performance in each evaluation value. As can be seen, it can be seen that the embodiments of the present invention have better performance in most items while having at least the same performance as the prior art.

7 is another image showing the performance of a method for estimating a single image depth according to an embodiment in comparison with the prior art. Depth information estimated by example is shown. Meanwhile, FIGS. 7(b) to 7(d) show depth information estimated according to the prior art, and FIG. 7(b) is a co-author of Godard, C. et al., "Unsupervised monocular depth estimation with left-right consistency." "Shows depth information estimated by the method disclosed in ", (c) of FIG. 7 is a co-author of Tosi, F., Aleotti, F., Poggi, M., and Mattoccia, S. "Learning monocular depth estimation infusing traditional stereo knowledge" (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 9799-9809, 2019) indicates the depth information estimated by the method disclosed, (d) of FIG. It represents depth information estimated by the method disclosed in "monocular depth hints".

The results shown in FIG. 7 represent depth estimation results obtained using the published Cityscapes dataset, and represent qualitative analysis results for 500 validation sets. As shown, it can be seen that the embodiments of the present invention have superior performance even when compared with the recently disclosed technology.

The operation by the single image depth estimation method according to the embodiments described above may be implemented at least partially as a computer program and recorded in a computer-readable recording medium. A computer-readable recording medium in which a program for implementing the operation according to the method according to the embodiments is recorded includes all types of recording devices in which computer-readable data is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable recording medium may be distributed in a network-connected computer system, and the computer-readable code may be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present embodiment may be easily understood by those skilled in the art to which the present embodiment belongs.

Although the present invention as described above has been described with reference to the embodiments shown in the drawings, it will be understood that these are merely exemplary, and that various modifications and variations of the embodiments are possible therefrom by those of ordinary skill in the art. However, such modifications should be considered to be within the technical protection scope of the present invention.

Claims (14)

a depth estimator configured to calculate pseudo-depth information for the input image;
a reliability calculation unit configured to calculate reliability information for the pseudo depth information; and
A threshold value determining unit configured to determine a threshold value of reliability for filtering a part of the pseudo depth information through learning using the reliability information,
The depth estimation unit is further configured to generate a depth estimation model for a single image by using the pseudo-depth information filtered by the threshold value.
The method of claim 1,
The depth estimation unit,
a stereo matching unit configured to calculate the pseudo-depth information from the input image using a pre-stored stereo matching model; and
and a depth learning unit configured to train a depth estimation network using the pseudo-depth information filtered by the threshold value.
3. The method of claim 2,
The depth estimation network includes one or more encoder layers for extracting feature values from an image and one or more decoder layers configured to convert the feature values into depth information,
The single image depth estimation system further configured to determine the threshold value through adaptive learning using the feature value extracted by the one or more encoder layers, wherein the threshold value determination unit is configured to determine the threshold value.
According to claim 1,
The threshold value determining unit is further configured to generate the reliability information thresholded by a differential soft-thresholding function defined using the reliability information and the threshold value.
5. The method of claim 4,
The threshold value determining unit is further configured to determine the threshold value by learning a threshold value network using a loss function defined by the thresholded reliability information and the reference reliability information.
6. The method of claim 5,
The depth estimation unit is further configured to train the depth estimation network using a regression loss function defined using the thresholded reliability information and the pseudo depth information.
calculating, by a single image depth estimation system, pseudo-depth information for an input image;
calculating, by the single image depth estimation system, reliability information for the pseudo depth information;
determining, by the single image depth estimation system, a threshold of reliability for filtering a part of the pseudo depth information through learning using the reliability information; and
and generating, by a single image depth estimation system, a depth estimation model for a single image by using the pseudo-depth information filtered by the threshold value.
8. The method of claim 7,
The calculating of the pseudo-depth information includes calculating, by the single-image depth estimation system, the pseudo-depth information from the input image using a pre-stored stereo matching model.
8. The method of claim 7,
The generating of the depth estimation model includes, by the single-image depth estimation system, training a depth estimation network using the pseudo-depth information filtered by the threshold value.
10. The method of claim 9,
The depth estimation network includes one or more encoder layers for extracting feature values from an image and one or more decoder layers configured to convert the feature values into depth information,
The calculating of the threshold value includes determining, by the single image depth estimation system, the threshold value through adaptive learning using the feature values extracted by the one or more encoder layers. Way.
8. The method of claim 7,
The step of calculating the threshold value includes, by the single image depth estimation system, generating reliability information thresholded by a differential soft-thresholding function defined using the reliability information and the threshold value. Image depth estimation method.
12. The method of claim 11,
Calculating the threshold may include, by the single image depth estimation system, learning a threshold network using a loss function defined by the thresholded reliability information and the reference reliability information. Estimation method.
13. The method of claim 12,
The generating of the depth estimation model may include training, by the single image depth estimation system, a depth estimation network using a regression loss function defined using the thresholded reliability information and the pseudo depth information. Image depth estimation method.
A computer program stored in a computer-readable recording medium in combination with hardware to execute the single image depth estimation method according to any one of claims 7 to 13.

Images