WO2018127007A1

WO2018127007A1 - Depth image acquisition method and system

Info

Publication number: WO2018127007A1
Application number: PCT/CN2017/119992
Authority: WO
Inventors: 周剑; 龙学军
Original assignee: 成都通甲优博科技有限责任公司
Priority date: 2017-01-03
Filing date: 2017-12-29
Publication date: 2018-07-12
Also published as: CN106780590B; CN106780590A

Abstract

A depth image acquisition method and system, said method comprising: using a parallax acquisition method to obtain the parallax of a left image and a right image; according to the parallax, calculating the matching cost difference between each pixel point in a depth image and two adjacent pixel points homologous thereto; using a continuous matching cost fitting function on an integer parallax space to perform fitting a plurality of times to obtain a continuous parallax space, and calculating a pixel coordinate having sub-pixel level accuracy to obtain a sub-pixel parallax space; calculating a depth value according to the sub-pixel parallax space, and obtaining the depth image. In the present method, sub-pixel fitting is performed directly on a parallax space-based depth image, relative to a depth image algorithm based on stereo matching of the sub-pixel space, algorithm operating time is shortened while required storage space is greatly reduced; a continuous function fitting method is used to perform fitting and interpolation on a discrete parallax space to obtain the continuous parallax space, thus eliminating a layering effect of the depth image such that the accuracy of parallax-based three-dimensional measurements is increased.

Description

Depth map acquisition method and system

The present application claims priority to Chinese Patent Application No. 201710001725.6, entitled "A Method and System for Obtaining a Depth Map", which is filed on Jan. 3, 2017, the entire contents of which is incorporated herein by reference. In the application.

Technical field

The present invention relates to the field of computer vision technology, and in particular, to a method and system for acquiring a depth map.

Background technique

Obtaining accurate depth information is an important part of the automatic system environment perception. Depth precision is one of the most important characteristics of sensors used for distance estimation. Depth maps are a very common application in the positioning and dimensioning of automated systems. The stereo camera system uses pixel-level correspondence between two images taken from different angles to achieve image depth estimation. However, for long-distance systems, the depth map accuracy based on integer parallax is not enough, because the depth map based on the stereo matching of integer pixels is discretely distributed in the parallax space, and the layering effect is obvious, so that some high precision cannot be achieved. The measurement accuracy requirements of the application scenario of the demand. To this end, it is necessary to optimize the depth map in units of integer parallax, so that the depth map information is continuous, so that accurate three-dimensional measurement information can be obtained in the application.

At present, the optimization method for depth map mainly focuses on depth map processing based on image information. These methods rely on the pixel information, edge, etc. of the depth map, and basically process the two-dimensional depth image by filtering, classical interpolation and the like. To some extent, it is possible to improve the imaging of the depth map, but it does not meet the requirements of measurement applications with very high precision requirements. Therefore, how to obtain a depth map with accurate depth accuracy is a technical problem that a person skilled in the art needs to solve.

Summary of the invention

The object of the present invention is to provide a method and a system for acquiring a depth map, which can solve the pixel locking problem, realize accurate estimation of sub-pixels, and obtain high depth precision. At the same time, the algorithm requires less memory, simple calculation, and takes less time. , real-time is good.

To solve the above technical problem, the present invention provides a method for obtaining a depth map, including:

Obtaining a disparity of the left image and the right image by using a parallax acquisition method;

Calculating, according to the disparity, a matching cost difference between each pixel point in the depth map and two pixels adjacent to the same name point;

A continuous matching cost fitting function based on the integer pixel sampling determined by the matching cost difference, and the integer parallax space is multi-fitted by the continuous matching cost fitting function to obtain a continuous parallax space, and the calculation sub- Pixel-level precision pixel coordinates to obtain a sub-pixel parallax space;

A depth map is obtained by calculating a depth value from the sub-pixel parallax space.

Optionally, the disparity of the left image and the right image is obtained by using a parallax acquisition method, including:

The parallax of the left image and the right image is obtained by the fast matching method of the same name image point.

Optionally, calculating, according to the disparity, a matching cost difference between two adjacent pixels in the depth map, including:

use

Calculating the matching cost difference between two adjacent pixels in the depth map;

Where d is the disparity after the 3D reconstruction of the current pixel point integer, C _d is the matching cost after the aggregation in the stereo matching corresponding to the disparity _d of the current pixel point, and C _d-1 is the current pixel point corresponding to the disparity d-1 The matching cost, C _d+1 is the matching cost of the current pixel point at the parallax d+1, LeftDif is the matching cost difference between the current pixel point and the pixel point to the left of the same name point, RightDif is the current pixel point and the pixel point to the right of the same name point The matching cost is poor.

Optionally, the continuous matching cost fitting function based on the integer pixel sampling determined by the matching cost difference includes:

use

Determining the fitted variable h;

Determining, according to the fitting variable h, a continuous matching cost fitting function f(h) based on integer pixel sampling, wherein

Optionally, calculating pixel coordinates of sub-pixel precision to obtain a sub-pixel disparity space, including:

use

Calculate the pixel coordinate d _{Refine of} sub-pixel precision and determine the sub-pixel disparity space D _New ={d _Refine }.

Optionally, the depth value is calculated according to the sub-pixel parallax space, and the depth map is obtained, including:

use

Calculate the world coordinates of the depth map;

use

Unitizing the world coordinates to obtain point cloud data of the depth map;

Where E=(x, y, z) is the world coordinate, E1=(X, Y, Z) is the unitized world coordinate, e=(u, v) is the image coordinate, Q is the re-projection matrix, and Z is the sub- The depth value after pixel optimization.

Optionally, after obtaining the depth map, the method further includes:

The depth map is output through a display.

The invention also provides a depth map acquisition system, comprising:

a disparity calculation module, configured to acquire a disparity of a left image and a right image by using a parallax acquisition method;

a matching cost difference calculation module, configured to calculate, according to the disparity, a matching cost difference between each pixel point in the depth map and two pixels adjacent to the same name point;

a sub-pixel disparity space obtaining module, configured to perform a continuous matching cost fitting function based on the integer pixel sampling determined by the matching cost difference, and use the continuous matching cost fitting function to perform multiple times A continuous parallax space is obtained, and pixel coordinates of sub-pixel precision are calculated to obtain a sub-pixel parallax space;

a depth map obtaining module, configured to calculate a depth value according to the sub-pixel parallax space, to obtain a depth map.

Optionally, the sub-pixel disparity space acquiring module includes:

Continuous matching cost fit function determining unit for utilizing

Determining a fitted variable h; determining a continuous matching cost fit function f(h) based on the integer pixel sample according to the fitting variable h;

Sub-pixel parallax space acquisition unit for utilizing

Calculate the pixel coordinate d _{Refine of} sub-pixel precision and determine the sub-pixel disparity space D _New ={d _Refine };

among them,

LeftDif is the matching cost difference between the current pixel point and the left pixel point, RightDif is the matching cost difference between the current pixel point and the right pixel point, d is the parallax after the current pixel point integer 3D reconstruction, and C _d is the parallax _d of the current pixel point. The matching cost after aggregation in the corresponding stereo matching.

Optionally, the system further includes:

And an output module, configured to output the depth map through a display.

A method for obtaining a depth map according to the present invention includes: obtaining a disparity of a left image and a right image by using a disparity acquisition method; and calculating, between the pixel points of each pixel point adjacent to the same name point in the depth map according to the disparity Matching cost difference; a continuous matching cost fitting function based on integer pixel sampling determined by matching cost difference, and fitting the integer parallax space to the continuous parallax space by using the continuous matching cost fitting function to obtain a continuous parallax space, Calculating pixel coordinates of sub-pixel precision to obtain a sub-pixel disparity space; calculating a depth value according to the sub-pixel disparity space to obtain a depth map;

It can be seen that the method directly performs sub-pixel fitting on the depth map based on the integer parallax space, and the depth map algorithm based on the sub-pixel space-based stereo matching greatly reduces the required memory space and shortens the time. The running time of the algorithm is fitted by the continuous function fitting method to fit the discrete parallax space to obtain a continuous parallax space, thus eliminating the layering effect of the depth map, so that the accuracy of the three-dimensional measurement based on parallax is improved. The three-dimensional measurement scenes that require different measurement accuracy, especially the three-dimensional measurement scenes that require very high measurement accuracy; the present invention also provides a depth map acquisition system, which has the above-mentioned beneficial effects, and will not be described herein.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is an embodiment of the present invention, and those skilled in the art can obtain other drawings according to the provided drawings without any creative work.

FIG. 1 is a flowchart of a method for acquiring a depth map according to an embodiment of the present invention;

2 is a schematic diagram of a stereo matching integer parallax space according to an embodiment of the present invention;

3 is a schematic diagram of a schematic diagram of a depth information space according to an embodiment of the present invention;

4 is a schematic diagram of matching cost curve fitting according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a pixel of a sensor according to an embodiment of the present invention; FIG.

FIG. 6 is a structural block diagram of an acquisition system of a depth map according to an embodiment of the present invention;

FIG. 7 is a structural block diagram of an acquisition system of another depth map according to an embodiment of the present invention.

detailed description

The core of the present invention is to provide a method and system for acquiring a depth map, which can solve the pixel locking problem, realize accurate estimation of sub-pixels, and obtain high depth precision. At the same time, the algorithm requires less memory, simple calculation, and takes less time. , real-time is good.

The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

At present, the research on the acquisition method of depth map usually focuses on a simple window solution based on stereo matching. The original classification proposed by Scharstein and Szeliski divides the stereo algorithm into two main groups: the local method and the global method. The local algorithm class uses the limited support area around each point to calculate the disparity. This method is based on the selected matching window, and usually uses matching cost aggregation to achieve a smoothing effect. Large windows reduce the number of unsuccessful matches and reduce the mismatch rate of deep discontinuities. The main advantage of the local method is that the computational complexity is small and can be realized in real time. The main disadvantage is that only local information near the pixel is used at each step, resulting in the inability of these methods to handle uncharacterized regions or repeated texture regions. Global methods have high computational complexity, so they are not suitable for automated systems that require high real-time requirements. It has also been proposed to replace the existing solution with a semi-global matching (SGM) stereo matching algorithm, which can effectively reduce the execution time. These stereo algorithms are able to get pixel-level errors, but they ignore sub-pixel results.

Recently, it has been proposed to use a simple parabolic interpolation method in the stereo matching algorithm to achieve sub-pixel estimation. The method uses a minimum matching cost and a matching cost corresponding to two points of its neighborhood for parabolic interpolation, the position of the smallest point of the parabola will represent the sub-pixel point. The solution is able to achieve precise accuracy. However, this method has a serious problem for a simple window based on a stereo algorithm, and a pixel locking phenomenon ("pixel-locking") occurs, that is, the number of point clouds corresponding to the parallax is Gaussian with the symmetric parallax as the axis of symmetry. Shimitzu and Okutomi recently proposed to solve the pixel locking problem by modeling error and modeling correction. They found that the error is symmetrical and can be eliminated by transforming the image. The transformed image will make the error function transposed to achieve and be simple. The match matches. Although this solution is quite effective, there are still some shortcomings. The main disadvantage is that the stereo matching needs to be performed three times, which causes a large waste of computing resources. Finally, modern stereo matching methods such as semi-global methods are almost impossible to implement in order to estimate a perfect sub-pixel interpolation mathematical model that requires the use of multiple nonlinear transformations.

Aiming at the complexity of the existing depth map optimization technology, the optimization effect is not good enough, and some pixel locking problems existing in the depth map sub-pixel optimization method based on the stereo matching algorithm, and the stereo matching method are complicated and cumbersome, the implementation is insufficient. The method provides a method for acquiring a depth map; the method can solve the pixel locking problem, realize accurate estimation of sub-pixels, and obtain high depth precision. At the same time, the algorithm requires less memory, simple calculation, less time, and good real-time performance. . For details, please refer to FIG. 1 , which is a flowchart of a method for acquiring a depth map according to an embodiment of the present invention;

S100. Acquire a disparity of a left image and a right image by using a parallax acquisition method;

Specifically, the main purpose of this step is to obtain the parallax of pixel-level precision, that is, to obtain the disparity d of the left image and the right image by using the parallax acquisition method. This embodiment does not limit the calculation method of the specific parallax. Since the parallax is the basis of subsequent calculations, in order to ensure the reliability and accuracy of the subsequent calculated values, a highly accurate parallax calculation method, such as a fast matching method of the same name image point, can be selected here. Here, when selecting the parallax calculation method, the user should not only consider the accuracy of the parallax calculation, but also the calculation speed of the hardware and the requirements of the real-time performance of the system.

After comprehensively considering the above indicators, it is preferable to obtain the disparity of the left image and the right image by using the same-name image point fast matching method.

Specifically, the camera system calibration is first performed.

Wherein, the internal parameters and external parameters of the camera in the binocular camera system are first calibrated, and the camera matrix K, the distortion matrix D, the rotation matrix R, the translation vector T and the re-projection matrix Q of the binocular camera system are obtained.

Where the camera matrix is

Where f _x and f _{y are the} main distance parameters, and (c _x , c _y ) is the main point coordinates.

The distortion matrix is D = [k ₁ k ₂ p ₁ p ₂ k ₃ [k ₄ k ₅ k ₆ ]], where k _i , p _j , i = 1, 2, ..., 6, j = 1, 2 Distortion parameters.

The re-projection matrix is

Wherein, x T _x of a component T, T is the translation vector.

Where the rotation matrix

The translation vector T = [T _x T _y T _z ].

Second, the pixel-level parallax d is calculated.

Among them, the parallax is the difference between the left and right cameras to observe the same target. The stereoscopic vision is described as the distance of the same name in the left image and the right image on the X axis. The mathematical description is:

d=x _l -x _r ; where x _l is the distance of the point of the same name in the left image on the X axis, and x _r is the distance of the point of the same name in the right image on the X axis.

S110. Calculate, according to the parallax, a matching cost difference between each pixel point in the depth map and two pixels adjacent to the same name point;

S120: A continuous matching cost fitting function based on the integer pixel sampling determined by the matching cost difference, and fitting the integer parallax space by the continuous matching cost fitting function to obtain a continuous parallax space. Calculating pixel coordinates of sub-pixel precision to obtain a sub-pixel parallax space;

Wherein, since each pixel of the sensor has a certain area, the precise position of an ideal point on the sensor pixel in the world coordinate system cannot be reflected on the image. Generally, the pixel coordinate position obtained on the image is only a part of the pixel. The position information (such as the center point), the image information represented by this coordinate cannot reflect the image information of the entire pixel, which fundamentally causes the pixel point coordinate positioning error, that is, the image recognition error, to the image. Then, the integer parallax obtained by stereo matching is discontinuous. The stereo matching integer parallax space as shown in FIG. In Fig. 2, d represents a parallax, x is an image plane x-axis, and y is an image plane y-axis. It can be seen that the parallax d is discontinuous, and in the d-layer parallax, from the near to the far, the 0th layer, the 1st layer, the 2nd layer, ..., the d-1th layer, the dth layer. The conversion relationship between integer parallax and depth obtained by stereo matching is:

Where b is the baseline length, f is the principal distance, z is the depth value, and D = {d}.

As described above, the integer parallax is discontinuous. Then, after three-dimensional recovery, the depth information transformed by the integer parallax is also distributed in discrete layers, that is, the corresponding depth information space is also discontinuous, as shown in FIG. Shown. In Figure 3, the distance between the d+1 layer and the d layer is:

Where δ is the pixel size, that is, the spatial size occupied by each pixel, b is the baseline length, and z is the depth value. It can be seen that the larger the depth value, the more obvious the stratification effect, and the smaller the parallax, the greater the error brought in. The smaller the depth value, the larger the parallax and the smaller the error.

For this reason, in order to reduce the measurement error, it is necessary to restore the continuous depth space, that is, to restore the continuous parallax space, and to obtain an accurate measurement size, which is achieved by obtaining sub-pixel size. Sub-pixel optimization based on depth map is generally implemented using a matching cost curve fitting method based on shaped sampling. Briefly described as: curve fitting using the point to be optimized and the matching cost of two points adjacent to it. As shown in Fig. 4, the minimum value point C _s (minimum point) can be obtained by curve fitting of points C ₁ , C ₂ , and C ₃ . Point C _s is the matching cost function corresponding to the parallax of the sub-pixel level, that is, the point optimized for C ₂ . Steps S110 to S120 are sub-pixel optimization processes by curve fitting. That is, step S110 to step S120 mainly include a matching cost difference calculation process, a fitting variable determination process, a continuous matching cost fitting function determining process, and a pixel coordinate calculation process of sub-pixel level precision. This embodiment does not limit the specific implementation forms of these several processes. As long as it is guaranteed to use the continuous matching cost fit function to perform the matching cost fit, the sub-pixel disparity space can be determined.

Preferably, according to the disparity, calculating a matching cost difference between two adjacent pixel points in the depth map may include, that is, the matching cost difference calculation process may include:

use

In this embodiment, the method of continuous function fitting is used to fit and interpolate the discrete parallax spaces to obtain a continuous parallax space, thereby weakening the layering effect of the depth map, so that the accuracy of the three-dimensional measurement based on parallax is improved, and is suitable for Measurement accuracy requires different 3D measurement scenarios, especially 3D measurement scenarios where measurement accuracy is very high. Further, in order to reduce the complexity of the calculation process, it is preferable to use a cosine function as a continuous matching cost fitting function for matching cost fitting. That is, the continuous matching cost fitting function based on the integer pixel sampling determined by the matching cost difference may include, that is, the fitting variable and the continuous matching cost fitting function determining process may include:

use

Determining the fitted variable h;

Specifically, in this embodiment, the cosine function is used to perform the matching cost fitting of the integer pixel sampling, and the pixel locking phenomenon can be substantially eliminated. That is, the cosine function is used to interpolate the time difference space, which reduces the complexity of the parallax space fitting, and eliminates the pixel locking effect of the parallax space fitting, and improves the interpolation precision of the parallax space.

Specifically, in the depth map corresponding to the integer parallax space, the layering is obvious, which reflects that the integer pixel precision is insufficient to describe accurate image information, however, the image information acquired by the image acquisition sensor is pixel-based. Image information. In order to obtain accurate image depth information, it is necessary to sub-pixel optimize the depth map based on the parallax space of the integer, that is, optimize the parallax space of the integer. The image pixel obtained by the sensor occupies a certain space. The pixel position in the stereo matching based on the integer pixel is only a certain point of the pixel, and generally takes the geometric center point coordinate of the pixel as the position coordinate of the integer pixel, as shown in FIG. 5 . As shown, a is the coordinates of the pixel A in the sensor, but in reality, the pixel A also contains points such as b, c, d, etc., therefore, only the point a is image information that cannot represent the entire pixel. In order to obtain a continuous parallax space, it is necessary to make some adjustments based on the parallax obtained by integer pixel matching. This adjustment range is within a half pixel range of the integer parallax, that is, a pixel with sub-pixel precision is preferably calculated. Coordinates, resulting in subpixel parallax space can include:

use

That is, sub-pixel optimization is a process of fitting a floating-point parallax space in an integer parallax space, using a continuous matching cost fitting function to perform multiple fittings, and the process of obtaining discrete data by curve fitting to obtain continuous data. .

S130. Calculate a depth value according to the sub-pixel parallax space to obtain a depth map.

Wherein, step S110 to step S120 are sub-pixel optimization processes. In this step, the sub-pixel parallax space is converted to obtain a depth value, thereby obtaining a depth map after sub-pixel optimization based on the depth map. This embodiment does not limit the conversion process from the sub-pixel parallax space to the depth map. Specifically, the depth value is calculated according to the sub-pixel disparity space, and the obtaining the depth map may include:

use

Calculate the world coordinates of the depth map;

use

Unitizing the world coordinates to obtain point cloud data of the depth map;

Where E=(x, y, z) is the world coordinate, E1=(X, Y, Z) is the point cloud data of the depth map, e=(u, v) is the image coordinate, Q is the re-projection matrix, Z The depth value after optimization for subpixels.

Based on the foregoing technical solution, the method for obtaining the depth map provided by the embodiment directly performs sub-pixel fitting on the depth map based on the integer parallax space, and the depth map algorithm based on the stereo matching based on the sub-pixel space is greatly While reducing the required memory space, the running time of the algorithm is also shortened, and the discrete parallax space is fitted and fitted by the continuous function fitting method to obtain a continuous parallax space, thereby eliminating the layering effect of the depth map. The accuracy of the three-dimensional measurement based on parallax is improved, and is suitable for a three-dimensional measurement scene with different measurement accuracy requirements, especially a three-dimensional measurement scene with very high measurement accuracy; further, a cosine function is used for matching of integer pixel sampling. The cost fit can substantially eliminate pixel locking. That is, the cosine function is used to interpolate the time difference space, which reduces the complexity of the parallax space fitting, and eliminates the pixel locking effect of the parallax space fitting, and improves the interpolation precision of the parallax space.

Based on the foregoing embodiment, the embodiment may further include: after acquiring the depth map:

The depth map is output through a display.

Specifically, the depth map after optimization is displayed. From the method in the above embodiment, the depth map obtained is very dense, that is, the depth information is continuous and highly accurate.

Wherein, the display here may be a display device such as a display screen.

The acquisition system of the depth map provided by the embodiment of the present invention is described below. The acquisition system of the depth map described below and the acquisition method of the depth map described above may refer to each other.

Please refer to FIG. 6. FIG. 6 is a structural block diagram of a system for acquiring a depth map according to an embodiment of the present invention; the acquiring system may include:

a disparity calculation module 100, configured to acquire disparity of a left image and a right image by using a disparity acquisition method;

The matching cost difference calculation module 200 is configured to calculate, according to the disparity, a matching cost difference between each pixel point in the depth map and two pixels adjacent to the same name point;

The sub-pixel disparity space obtaining module 300 is configured to use a continuous matching cost fitting function based on the integer pixel sampling determined by the matching cost difference, and perform the integer disparity space multiple times by using the continuous matching cost fitting function Fitting to obtain a continuous parallax space, calculating pixel coordinates of sub-pixel precision, and obtaining a sub-pixel parallax space;

The depth map obtaining module 400 is configured to calculate a depth value according to the sub-pixel parallax space to obtain a depth map.

Based on the foregoing embodiment, the sub-pixel disparity space obtaining module 300 may include:

Continuous matching cost fit function determining unit for utilizing

Sub-pixel parallax space acquisition unit for utilizing

among them,

Based on any of the above embodiments, referring to FIG. 7, the acquiring system may further include:

The output module 500 is configured to output the depth map through a display.

The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same similar parts between the various embodiments may be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

A person skilled in the art will further appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of both, in order to clearly illustrate the hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein can be implemented directly in hardware, a software module executed by a processor, or a combination of both. The software module can be placed in random access memory (RAM), memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or technical field. Any other form of storage medium known.

The method and system for acquiring the depth map provided by the present invention are described in detail above. The principles and embodiments of the present invention have been described herein with reference to specific examples, and the description of the above embodiments is only to assist in understanding the method of the present invention and its core idea. It should be noted that those skilled in the art can make various modifications and changes to the present invention without departing from the spirit and scope of the invention.

Claims

A method for obtaining a depth map, comprising:

Obtaining a disparity of the left image and the right image by using a parallax acquisition method;

Calculating, according to the disparity, a matching cost difference between each pixel point in the depth map and two pixels adjacent to the same name point;

A continuous matching cost fitting function based on the integer pixel sampling determined by the matching cost difference, and the integer parallax space is multi-fitted by the continuous matching cost fitting function to obtain a continuous parallax space, and the calculation sub- Pixel-level precision pixel coordinates to obtain a sub-pixel parallax space;

A depth map is obtained by calculating a depth value from the sub-pixel parallax space.
The method for acquiring a depth map according to claim 1, wherein the disparity of the left image and the right image is obtained by using a parallax acquisition method, including:

The parallax of the left image and the right image is obtained by the fast matching method of the same name image point.
The method for acquiring a depth map according to claim 2, wherein calculating a matching cost difference between two adjacent pixel points in the depth map according to the disparity includes:

use
Calculating the matching cost difference between two adjacent pixels in the depth map;

Where d is the disparity after the 3D reconstruction of the current pixel point integer, C d is the matching cost after the aggregation in the stereo matching corresponding to the disparity d of the current pixel point, and C d-1 is the current pixel point corresponding to the disparity d-1 The matching cost, C d+1 is the matching cost of the current pixel point at the parallax d+1, LeftDif is the matching cost difference between the current pixel point and the pixel point to the left of the same name point, RightDif is the current pixel point and the pixel point to the right of the same name point The matching cost is poor.
The method for obtaining a depth map according to claim 3, wherein the matching cost matching function based on the integer pixel sampling determined by the matching cost difference comprises:

use
Determining the fitted variable h;

Determining, according to the fitting variable h, a continuous matching cost fitting function f(h) based on integer pixel sampling, wherein
The method for acquiring a depth map according to claim 4, wherein calculating the pixel coordinates of the sub-pixel level precision to obtain the sub-pixel parallax space comprises:

use
Calculate the pixel coordinate d Refine of sub-pixel precision and determine the sub-pixel disparity space D New ={d Refine }.
The method for obtaining a depth map according to claim 5, wherein the depth value is calculated according to the sub-pixel parallax space, and the depth map is obtained, including:

use
Calculate the world coordinates of the depth map;

use
Unitizing the world coordinates to obtain point cloud data of the depth map;

Where E=(x, y, z) is the world coordinate, E1=(X, Y, Z) is the point cloud data of the depth map, e=(u, v) is the image coordinate, Q is the re-projection matrix, Z The depth value after optimization for subpixels.
The method for obtaining a depth map according to any one of claims 1 to 6, wherein after obtaining the depth map, the method further comprises:

The depth map is output through a display.
A depth map acquisition system, comprising:

a disparity calculation module, configured to acquire a disparity of a left image and a right image by using a parallax acquisition method;

a matching cost difference calculation module, configured to calculate, according to the disparity, a matching cost difference between each pixel point in the depth map and two pixels adjacent to the same name point;

a sub-pixel disparity space obtaining module, configured to perform a continuous matching cost fitting function based on the integer pixel sampling determined by the matching cost difference, and use the continuous matching cost fitting function to perform multiple times A continuous parallax space is obtained, and pixel coordinates of sub-pixel precision are calculated to obtain a sub-pixel parallax space;

a depth map obtaining module, configured to calculate a depth value according to the sub-pixel parallax space, to obtain a depth map.
The acquiring system of the depth map according to claim 8, wherein the sub-pixel disparity space obtaining module comprises:

Continuous matching cost fit function determining unit for utilizing
Determining a fitted variable h; determining a continuous matching cost fit function f(h) based on the integer pixel sample according to the fitting variable h;

Sub-pixel parallax space acquisition unit for utilizing
Calculate the pixel coordinate d Refine of sub-pixel precision and determine the sub-pixel disparity space D New ={d Refine };

among them,
LeftDif is the matching cost difference between the current pixel point and the left pixel point, RightDif is the matching cost difference between the current pixel point and the right pixel point, d is the parallax after the current pixel point integer 3D reconstruction, and C d is the parallax d of the current pixel point. The matching cost after aggregation in the corresponding stereo matching.
The method for obtaining a depth map according to claim 8 or 9, further comprising:

And an output module, configured to output the depth map through a display.