CN108564536B - A global optimization method for depth maps - Google Patents

Abstract

A global optimization method of depth map, which fully utilizes the difference information of left and right viewing angle disparity data and the edge gradient information of color data to optimize the depth map globally. First, based on the region growing method, the initial left and right viewing angle disparity data are regionally filtered to remove the isolated small block-shaped erroneous disparity; then, the optimized left and right disparity data difference information is used to

The model is used to calculate the parallax confidence coefficient data, and the experiment proves that the method is simple and effective; finally, the left perspective parallax data and confidence coefficient data are converted into the initial depth data and confidence data under the perspective of the color camera through perspective projection, making full use of the color The edge information of the image is used to construct a linear equation system about the depth data, and the optimized depth data can be obtained by solving the over-relaxation iterative method. This method can obtain high-precision depth data in real time, and the optimized depth map is smooth, retains edges, and can fill large holes well.

Description

A global optimization method for depth maps

technical field

The invention relates to the technical fields of computer vision and image processing, in particular to a global optimization method of a depth map.

Background technique

Two images of the scene are acquired from different viewpoints, and the depth information of the scene can be estimated by the position offset of the scene in the two images. This position offset corresponds to the disparity of image pixels, which can be directly converted into scene depth, which is generally represented by a depth map. However, when the scene has texture missing and texture duplication, the calculated depth map may have large holes in the corresponding area. The existing method, on the one hand, enriches the texture by artificially compensating the scene (such as pasting marker points, projecting light spots, etc.), but it is inconvenient, inoperable, and ineffective. Optimized, but the method is complex, over-optimized, or unrealistic.

SUMMARY OF THE INVENTION

In order to solve the deficiencies in the prior art, the present invention provides a global optimization method for a depth map, which realizes the filtering and denoising of the depth map and the filling of large holes, and converts the left and right perspective parallax data to the RGB camera perspective. Using RGB image edge information, it is concise and efficient.

In order to achieve the above object, the specific scheme adopted in the present invention is: a global optimization method for a depth map, the method comprising the following steps:

Step 1: Perform regional filtering on the initial left-perspective disparity data and the initial right-perspective disparity data based on the region growing method to remove the erroneous parallax in isolated blocky regions, and obtain optimized left-perspective disparity data and optimized right-perspective disparity data ; The specific process of removing the erroneous parallax in the block region based on the region growing method is as follows:

S1. Create two new images Buff and Dst whose size is equal to the initial left-view disparity data and the initial right-view disparity data and whose initial value is zero. Buff is used to record the grown pixels, and Dst is used to mark the image blocks that meet the conditions. area;

S2, setting a first threshold value and a second threshold value; the first threshold value is the parallax difference value, and the second threshold value is the area value of the block region with erroneous parallax;

S3, traverse each ungrown pixel point, take the current point as the seed point, and press into the regional growth function;

S4. Create a stack vectorGrowPoints and a stack resultPoints, take the end point from the stack vectorGrowPoints, and then press the point in eight directions: {-1, -1}, {0, -1}, {1, -1}, {1, 0}, {1, 1}, {0, 1}, {-1, 1}, {-1, 0} Take out the disparity value of the ungrown pixel point and compare it with the disparity value of the seed point. If a threshold is reached, it is considered that the conditions are met, and they are pushed into the stack vectorGrowPoints and the stack resultPoints respectively, and the grown points are marked in the Buff, and the above process is repeated until there are no points in the stack vectorGrowPoints; if the number of points in the stack resultPoints is less than the first point The second threshold is marked in Dst;

S5, repeating steps S3 and S4, removing the marked area in Dst from the disparity data, to obtain optimized left-view disparity data and optimized right-view disparity data;

Step 2: Calculate the left-view confidence coefficient data from the optimized left-view parallax data and the optimized right-view parallax data in step 1; the specific method for calculating the left-view confidence coefficient data is: O _p =e ^{-|ld-rd |} , wherein, ld is the left-view parallax data optimized in step 1, rd is the right-view parallax data optimized by the corresponding step 1, and _Op is the left-view confidence coefficient data;

Step 3: Calculate the left-view depth data from the left-view parallax data and camera parameters optimized in step 1; convert the left-view depth data and the left-view confidence coefficient data obtained in step 2 through perspective projection at the same time to obtain the RGB camera view angle. Initial depth data and confidence coefficient data;

Step 4: Use the RGB image edge information to calculate edge constraint coefficient data, and then use the edge constraint coefficient data, the initial depth data and confidence coefficient data from the perspective of the RGB camera in step 3 to generate optimized depth data using the global optimization objective function.

Preferably, an acquisition device is used in the process of acquiring the depth image, and the acquisition device includes two near-infrared cameras and one RGB camera.

Preferably, in step 3, the specific calculation process of the initial depth data from the perspective of the RGB camera is as follows:

T1, traverse the image pixels, know the baseline and focal length of the left and right near-infrared cameras, and convert the parallax value into the depth data of the left perspective;

T2. Calculate the three-dimensional coordinates of the corresponding spatial point in the corresponding coordinate system from the depth data of the left perspective and the internal parameters of the left near-infrared camera or the near-infrared right camera;

T3. Calculate the three-dimensional coordinates of the corresponding space point in the RGB camera coordinate system from the relative positional relationship between the left near-infrared camera or the right near-infrared camera coordinate system and the RGB camera coordinate system and the stereo correction matrix between the left and right near-infrared cameras; T4, by The internal parameters of the RGB camera are calculated by calculating the projection and depth value of the corresponding spatial point on the RGB image plane, that is, the initial depth data under the perspective of the RGB camera.

Preferably, the global optimization objective function adopted in step 4 is:

为图像上像素点p的初始深度数据，D_p为待求深度数据，α_p为像素点p在RGB相机视角下的置信度系数数据，ω_qp为边缘约束系数数据，q为p的四邻域像素点；当ε(D)最小时，优化结束；假设图像有n个像素点，为使ε(D)达到最小，令全局优化目标函数等号右侧部分对每一个D_p求导等于零，得到n个方程，整理得AX＝B的线性方程组，其中A为n×n的系数矩阵，与α_p和ω_qp有关，B为n×1的常数矩阵，与α_p和

有关，X即为待求深度数据列向量[D₁，D₂，…，D_n]^T，通过迭代计算，得优化后的深度数据。in,

is the initial depth data of the pixel p on the image, D _p is the depth data to be obtained, α _p is the confidence coefficient data of the pixel p in the RGB camera perspective, ω _qp is the edge constraint coefficient data, and q is the four neighborhoods of p Pixel point; when ε(D) is the smallest, the optimization ends; assuming that the image has n pixels, in order to minimize ε(D), the right part of the equal sign of the global optimization objective function is made equal to zero for each D _p derivation, Obtain n equations, and arrange a linear equation system of AX=B, where A is a coefficient matrix of n × n, which is related to α _p and ω _qp , B is a constant matrix of n × 1, and α _p and

Relevant, X is the depth data column _vector [ ^D ₁ , D ₂ , .

计算得系数矩阵A和常数矩阵B。Preferably, for any pixel p, the p-th behavior in AX=B:

The coefficient matrix A and the constant matrix B are calculated.

Preferably, the specific calculation process of the coefficient matrix A and the constant matrix B is as follows:

为像素点q和p的灰度差值，然后

其取值范围在[0，1]之间，其中β为调优参数，且β＝20；(1), first find the gradient of the RGB image

is the grayscale difference between pixels q and p, and then

Its value range is between [0, 1], where β is a tuning parameter, and β=20;

(2) Calculate the coefficient matrix A from α _p and ω _qp , where the p-th row of A: (α _p + ∑ _{(p, q)∈E} (ω _pq + ω _qp ))D _p -∑ _{(p, q )∈E} (ω _pq +ω _qp )D _q , the row has 5 non-zero values, and the 5 non-zero values are the corresponding elements of the pixel point p and the four neighboring pixels of the pixel point p, wherein, The element corresponding to the pixel point p is α _p +∑ _{(p, q)∈E} (ω _pq +ω _qp ), and the element corresponding to the pixel point q in the four neighborhoods of the pixel point p is -(ω _pq +ω _qp );

计算常数矩阵B，其中B的第p行为

(3), by α _p and the initial depth data

Computes a constant matrix B where the pth row of B

Preferably, an over-relaxation iterative method is used to solve the linear equation system to obtain the optimized depth data.

Beneficial effects:

(1) The present invention provides a global optimization method for a depth map. The method is based on an acquisition device. The acquisition device includes two near-infrared cameras (NIR) and one visible light (RGB) camera. The near-infrared cameras constitute a dual The stereo vision system can obtain the depth map in real time and register it with the RGB image collected by the visible light camera; make full use of the global information of the left and right perspective parallax data and the edge constraints of the color data to optimize the depth map globally, and convert the left and right perspective parallax data into In the RGB camera perspective, the edge information of the RGB image is used; when calculating the confidence coefficient data, the e- ^x model is used to directly use the left and right perspective disparity data. The experiment proves that the method is simple and effective. The simplicity is reflected in: in the existing method, the confidence coefficient is determined by fitting the matching cost quadratic curve of three adjacent integer disparity values of the pixel point. The three matching cost values of α are subjected to quadratic fitting, and the positive and negative values of α _p are determined by judging the direction of the curve. Therefore, the method of the present invention is relatively simple compared with the prior art; it is effectively reflected in: the optimized depth map is smooth, Retain edges and large voids can be better filled;

(2) The present invention provides a global optimization method for a depth map. The region growing method is used to perform regional filtering on the initial left-view disparity data and the initial right-view disparity data. Experiments have shown that the method can complete the marking by traversing the image once. And it can effectively remove the erroneous disparity in small isolated areas with similar disparity values and obviously different from the surrounding disparity values.

Description of drawings

Fig. 1 is the flow chart of the present invention;

Figure 2 is the depth map of a large hole in the head before optimization;

FIG. 3 is a depth map after optimization by the global optimization method of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Please refer to FIG. 1 of the flow chart of the present invention. The internal and external parameters of all cameras in the present invention are known, and the initial left-view disparity data and the initial right-view disparity data are calculated by the prior art, which will not be repeated here. A global optimization method for a depth map, the method uses an acquisition device in the process of acquiring a depth image based on an acquisition device, the acquisition device includes two near-infrared cameras and an RGB camera, and the method includes the following steps:

Step 1: Perform regional filtering on the initial left-perspective disparity data and the initial right-perspective disparity data based on the region growing method to remove the erroneous parallax in isolated blocky regions, and obtain optimized left-perspective disparity data and optimized right-perspective disparity data Generally, the generated disparity data has been checked left and right, and a large number of incorrectly matched point disparities have been removed, but there are still erroneous disparities in small blocks. The small isolated areas with similar values further improve the parallax quality. The specific process of removing erroneous parallax in blocky areas based on the region growing method is as follows:

S1. Create two new images Buff and Dst whose size is equal to the initial left-view disparity data and the initial right-view disparity data and whose initial value is zero. Buff is used to record the grown pixels, and Dst is used to mark the image blocks that meet the conditions. area;

S2. Set a first threshold value and a second threshold value; the first threshold value is the parallax difference value, and the second threshold value is the area value of the block region with erroneous parallax; The threshold is 60;

S3, traverse each ungrown pixel point, take the current point as the seed point, and press into the regional growth function;

S4. Create a stack vectorGrowPoints and a stack resultPoints, take the end point from the stack vectorGrowPoints, and then press the point in eight directions: {-1, -1}, {0, -1}, {1, -1}, {1, 0}, {1, 1}, {0, 1}, {-1, 1}, {-1, 0} Take out the disparity value of the ungrown pixel point and compare it with the disparity value of the seed point. If a threshold is reached, it is considered that the conditions are met, and they are pushed into the stack vectorGrowPoints and the stack resultPoints respectively, and the grown points are marked in the Buff, and the above process is repeated until there are no points in the stack vectorGrowPoints; if the number of points in the stack resultPoints is less than the first point The second threshold is marked in Dst;

S5, repeating steps S3 and S4, removing the marked area in Dst from the disparity data, to obtain optimized left-view disparity data and optimized right-view disparity data;

Step 2: Calculate the left-view confidence coefficient data from the optimized left-view parallax data and the optimized right-view parallax data in step 1; the specific method for calculating the left-view confidence coefficient data is: O _p =e ^-|ld-rd |, where ld is the left-view disparity data after step 1 optimization, rd is the right-view disparity data after the corresponding step 1 optimization, and _Op is the left-view confidence coefficient data; The method for determining the parallax confidence coefficient data of the point is complicated to implement, and the method for calculating the confidence coefficient data of the present invention is simple and efficient. The left-view confidence coefficient data plays a decisive role in the optimization effect, and the reliability of the value of O _p is closely related to the accuracy of the parallax data. A small block in the parallax data has erroneous parallax, which will cause the corresponding area to appear large after optimization. The block has erroneous depth data, therefore, the present invention proposes a method for removing block-shaped erroneous parallax based on the region growing method to improve the parallax quality;

Step 3: Calculate the left-view depth data from the left-view parallax data and camera parameters optimized in step 1; convert the left-view depth data and the left-view confidence coefficient data obtained in step 2 through perspective projection at the same time to obtain the RGB camera view angle. Initial depth data and confidence coefficient data; the specific calculation process of initial depth data from the perspective of the RGB camera is as follows:

T1, traverse the image pixels, know the baseline and focal length of the left and right near-infrared cameras, and convert the parallax value into the depth data of the left perspective;

T2. Calculate the three-dimensional coordinates of the corresponding spatial point in the corresponding coordinate system from the depth data of the left perspective and the internal parameters of the left near-infrared camera or the near-infrared right camera;

T3. Calculate the three-dimensional coordinates of the corresponding space point in the RGB camera coordinate system from the relative positional relationship between the left near-infrared camera or the right near-infrared camera coordinate system and the RGB camera coordinate system and the stereo correction matrix between the left and right near-infrared cameras; T4, by Calculate the internal parameters of the RGB camera, calculate the projection and depth value of the corresponding spatial point on the RGB image plane, and obtain the initial depth data under the RGB camera perspective;

Step 4: Use the edge information of the RGB image to calculate the edge constraint coefficient data, and then use the edge constraint coefficient data, the initial depth data and confidence coefficient data from the perspective of the RGB camera in step 3 to generate the optimized depth data by using the global optimization objective function. The global optimization objective function is:

为图像上像素点p的初始深度数据，D_p为待求深度数据，α_p为像素点p在RGB相机视角下的置信度系数数据，ω_qp为边缘约束系数数据，q为p的四邻域像素点；当ε(D)最小时，优化结束；假设图像有n个像素点，为使ε(D)达到最小，令全局优化目标函数等号右侧部分对每一个D_p求导等于零，得到n个方程，整理得AX＝B的线性方程组，其中A为n×n的系数矩阵，与α_p和ω_qp有关，B为n×1的常数矩阵，与α_p和

有关，X即为待求深度数据列向量[D₁，D₂，…，D_n]^T，通过迭代计算，得优化后的深度数据。in,

is the initial depth data of the pixel p on the image, D _p is the depth data to be obtained, α _p is the confidence coefficient data of the pixel p in the RGB camera perspective, ω _qp is the edge constraint coefficient data, and q is the four neighborhoods of p Pixel point; when ε(D) is the smallest, the optimization ends; assuming that the image has n pixels, in order to minimize ε(D), the right part of the equal sign of the global optimization objective function is made equal to zero for each D _p derivation, Obtain n equations, and arrange a linear equation system of AX=B, where A is a coefficient matrix of n × n, which is related to α _p and ω _qp , B is a constant matrix of n × 1, and α _p and

Relevant, X is the depth data column _vector [ ^D ₁ , D ₂ , .

计算得系数矩阵A和常数矩阵B。For any pixel p, the pth row in AX=B:

The coefficient matrix A and the constant matrix B are calculated.

In step 3, the initial depth data has been obtained, and the coefficient matrix and the constant matrix are calculated below. For an image with a resolution of one million, the amount of depth data can reach one million, and the amount of coefficient matrix data is square. In order to meet the real-time realization of GPU, the present invention adopts The over-relaxation iterative method (SOR) solves the linear equation system and completes the optimization of depth data, as shown in Figure 2 and Figure 3, Figure 2 is the depth map of a large cavity in the head before optimization; Figure 3 is after optimization using the global optimization method of the present invention. depth map. The specific calculation process of coefficient matrix A and constant matrix B is as follows:

为像素点q和p的灰度差值，然后

其取值范围在[0，1]之间，其中β为调优参数，且β＝20，通过此步骤求解ω_qp，ω_qp对深度效果的影响是保持深度边缘，使其不被过度平滑；(1), first find the gradient of the RGB image

is the grayscale difference between pixels q and p, and then

Its value range is between [0, 1], where β is a tuning parameter, and β=20, through this step to solve ω _qp , the influence of ω _qp on the depth effect is to keep the depth edge so that it is not over-smoothed ;

(2) Calculate the coefficient matrix A from α _p and ω _qp , where the p-th row of A: (α _p + ∑ _{(p, q)∈E} (ω _pq + ω _qp ))D _p -∑ _{(p, q )∈E} (ω _pq +ω _qp )D _q , the row has 5 non-zero values, and the 5 non-zero values are the corresponding elements of the pixel point p and the four neighboring pixels of the pixel point p, wherein, The element corresponding to the pixel point p is α _p +∑ _{(p, q)∈E} (ω _pq +ω _qp ), and the element corresponding to the pixel point q in the four neighborhoods of the pixel point p is -(ω _pq +ω _qp );

计算常数矩阵B，其中B的第p行为

(3), by α _p and the initial depth data

Computes a constant matrix B where the pth row of B

(4), solve the linear equation system by the SOR method, and obtain the optimized depth data.

The invention provides a global optimization method for a depth map. The method performs global optimization on the initial depth of the scene, realizes real-time high-precision acquisition of the depth, and mainly solves the problem of a large number of holes in the calculated parallax data when the scene texture is missing or repeated. , such as hair, the texture is single, and even if an active light source is used to project structured light, it is easily absorbed and lacks features. It can be used in 3D reconstruction, somatosensory interaction and other cases. In 3D reconstruction, high-quality depth data from various perspectives is provided for real-time high-precision reconstruction, which simplifies subsequent optimization processing operations. In somatosensory interaction, through the establishment of different interactor models, the real picture is displayed in front of each other.

It should also be noted that in this document, relational terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply those entities or operations There is no such actual relationship or order between them. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (7)

1. a global optimization method of depth map, is characterized in that, this method comprises the steps: Step 1: Perform regional filtering on the initial left-perspective disparity data and the initial right-perspective disparity data based on the region growing method to remove the erroneous parallax in isolated blocky regions, and obtain optimized left-perspective disparity data and optimized right-perspective disparity data ; The specific process of removing the erroneous parallax in the block region based on the region growing method is as follows: S1. Create two new images Buff and Dst whose size is equal to the initial left-view disparity data and the initial right-view disparity data and whose initial value is zero. Buff is used to record the grown pixels, and Dst is used to mark the image blocks that meet the conditions. area; S2, set a first threshold value and a second threshold value; the first threshold value is the difference value of the parallax size, and the second threshold value is the area with erroneous parallax in the block region; S3, traverse each ungrown pixel point, take the current point as the seed point, and press into the regional growth function; S4. Create a stack vectorGrowPoints and a stack resultPoints, take the end point from the stack vectorGrowPoints, and then press the point in eight directions: {-1, -1}, {0, -1}, {1, -1}, {1, 0}, {1, 1}, {0, 1}, {-1, 1}, {-1, 0} Take out the disparity value of the ungrown pixel point and compare it with the disparity value of the seed point. If a threshold is reached, it is considered that the conditions are met, and they are pushed into the stack vectorGrowPoints and the stack resultPoints respectively, and the grown points are marked in the Buff, and the above process is repeated until there are no points in the stack vectorGrowPoints; if the number of points in the stack resultPoints is less than the first point The second threshold is marked in Dst; S5, repeating steps S3 and S4, removing the marked area in Dst from the disparity data, to obtain optimized left-view disparity data and optimized right-view disparity data; Step 2: Calculate the left-view confidence coefficient data from the optimized left-view parallax data and the optimized right-view parallax data in step 1; the specific method for calculating the left-view confidence coefficient data is: O _p =e ^{-|ld-rd |} , wherein, ld is the left-view parallax data optimized in step 1, rd is the right-view parallax data optimized by the corresponding step 1, and _Op is the left-view confidence coefficient data; Step 3: Calculate the left-view depth data from the left-view parallax data and camera parameters optimized in step 1; convert the left-view depth data and the left-view confidence coefficient data obtained in step 2 through perspective projection at the same time to obtain the RGB camera view angle. Initial depth data and confidence coefficient data; Step 4: Use the RGB image edge information to calculate edge constraint coefficient data, and then use the edge constraint coefficient data, the initial depth data and confidence coefficient data from the perspective of the RGB camera in step 3 to generate optimized depth data using the global optimization objective function. 2 . The global optimization method of a depth map according to claim 1 , wherein an acquisition device is used in the process of acquiring the depth image, and the acquisition device comprises two near-infrared cameras and one RGB camera. 3 . 3. the global optimization method of a kind of depth map as claimed in claim 1, is characterized in that: in step 3, the concrete calculation process of initial depth data under RGB camera angle of view is as follows: T1. Traverse the optimized left perspective parallax data, know the baseline and focal length of the left and right near-infrared cameras, and convert the parallax value into left perspective depth data; T2. Calculate the three-dimensional coordinates of the corresponding spatial point in the corresponding coordinate system from the depth data of the left perspective and the internal parameters of the left near-infrared camera or the right near-infrared camera; T3. Calculate the three-dimensional coordinates of the corresponding spatial point in the RGB camera coordinate system from the relative positional relationship between the left near-infrared camera or the right near-infrared camera coordinate system and the RGB camera coordinate system and the stereo correction matrix between the left and right near-infrared cameras; T4. Calculate the projection and depth value of the corresponding spatial point on the RGB image plane from the internal parameters of the RGB camera, that is, to obtain the initial depth data under the perspective of the RGB camera. 4. the global optimization method of a kind of depth map as claimed in claim 1, is characterized in that: the global optimization objective function that step 4 adopts is:

in,

is the initial depth data of pixel p on the image from the perspective of the RGB camera, D _p is the depth data to be found, α _p is the confidence coefficient data of the pixel p in the perspective of the RGB camera, ω _qp is the edge constraint coefficient data, q is the four neighborhood pixels of p; when ε(D) is the smallest, the optimization ends; assuming that the image has n pixels, in order to minimize ε(D), let the right part of the equal sign of the global optimization objective function for each D The derivative of _p is equal to zero, and n equations are obtained, and the linear equation system of AX=B is arranged, where A is an n×n coefficient matrix, which is related to α _p and ω _qp , B is an n×1 constant matrix, and α _p and

Relevant, X is the depth data column _vector [ ^D ₁ , D ₂ , . 5. the global optimization method of a kind of depth map as claimed in claim 4 is characterized in that: to any pixel p, the behavior of pixel p in AX=B:

The coefficient matrix A and the constant matrix B are calculated. 6. the global optimization method of a kind of depth map as claimed in claim 5 is characterized in that: the concrete calculation process of coefficient matrix A and constant matrix B is as follows: (1), first find the gradient of the RGB image

is the grayscale difference between pixels q and p, and then

Its value range is between [0, 1], where β is a tuning parameter, and β=20; (2) Calculate the coefficient matrix A from α _p and ω _qp , where the behavior of the pixel p of A is: (α _p +∑ _{(p, q)∈E} (ω _pq +ω _qp ))D _p -∑ _{(p , q)∈E} (ω _pq +ω _qp )D _q , the row has 5 non-zero values, and the 5 non-zero values are the corresponding elements of the pixel p and the four neighboring pixels of the pixel p, Among them, the element corresponding to the pixel point p is α _p +∑ _{(p, q)∈E} (ω _pq +ω _qp ), and the element corresponding to the pixel point q in the four neighborhoods of the pixel point p is -(ω _pq + ω _qp ); (3), by α _p and the initial depth data

Calculate the constant matrix B, where the pixel p of B is in the behavior

7 . The global optimization method of a depth map according to claim 6 , wherein: an over-relaxation iterative method is used to solve the linear equation system to obtain the optimized depth data. 8 .

Images