CN115294188A - Two-stage segmentation unit-based patch matching parallel method - Google Patents

Abstract

The invention discloses a two-stage segmentation unit-based patch matching parallel method, which comprises the following steps: step 1: acquiring a left image and a right image of a stereo matching pair; step 2: constructing a computation layer S composed of segmentation units by using an input image as a graph structure ₁ And S ₂ (ii) a And step 3: randomly initializing a tag value for each pixel; and 4, step 4: at the computing layer S ₁ Realizing label space propagation in a medium parallel mode; and 5: at the computing layer S ₂ Realizing label plane refinement in the middle parallel mode; step 6: repeating the step 4) and the step 5) until the disparity map is converged to obtain a left image sub-pixel disparity map and a right image sub-pixel disparity map; and 7: detecting pixel point disparity values which do not meet consistency constraint by using the left disparity map and the right disparity map, and performing post-processing operation on the pixel point disparity values; and 8: converting the disparity map into a depth map by using parameters of a binocular camera to obtain a fieldAnd (4) three-dimensional appearance of the scene. The method and the device achieve the purposes of reducing the computation time complexity of the patch matching algorithm, realizing efficient parallel computation and quickly converging the optimized result.

Description

Two-stage segmentation unit-based patch matching parallel method

Technical Field

The invention belongs to the technical field of visual non-contact measurement, and particularly relates to a patch matching parallel method based on two stages of segmentation units.

Background

The stereoscopic vision technology is always a hotspot of research in the field of machine vision and computer vision, and is widely applied to the aspects of unmanned driving, robot navigation, earth observation, virtual reality, cultural heritage protection and the like. The depth perception technology has the characteristics of simple system, high automation degree and dense non-contact and point cloud. The stereo matching is used as the most core technology of stereo vision, and aims to calculate homonymous pixel points in images at different visual angles and then recover the three-dimensional appearance of an object according to a triangular geometric relationship by utilizing the difference of imaging coordinates at different visual angles. Early stereo matching algorithms generally employed an integer disparity space model of the forward-looking plane, which reduced the size of the label space and reduced the computation time of the disparity optimization algorithm. However, the front-view plane model and the integer parallax space can only obtain discrete parallax maps, and a late-stage interpolation algorithm is required to further obtain sub-pixel parallax maps, so that the accuracy of the sub-pixel parallax maps is difficult to guarantee.

In order to obtain the sub-pixel disparity map, a plane label model is widely used, and the stereo matching algorithm aims to calculate an optimal plane label for each pixel. The plane label model can obtain the sub-pixel parallax value and the normal vector of the pixel plane, and is beneficial to subsequent point cloud splicing and three-dimensional reconstruction. The patch matching algorithm can efficiently calculate the nearest neighbor field of each pixel, only one or a plurality of labels and cost values of the pixels are reserved, the memory occupation of the algorithm is reduced, and meanwhile, the patch matching algorithm has high calculation efficiency, so that the patch matching algorithm is widely applied to binocular stereo matching, multi-view stereo matching and optical flow tasks.

Bleyer et al were the earliest applications of patch Matching algorithms to the Stereo Matching task (Bleyer M, rhemann C, rother C. PatchMatch Stereo-Stereo Matching with Slated Support Windows [ C ]. 2011Procedents of the British Machine Vision conference, 2011). In the algorithm, a single pixel is taken as a calculation unit, and only adaptive weight can be adopted for cost aggregation, so that the calculation time complexity is high; meanwhile, the cost aggregation values of the adjacent pixels to the same label cannot be reused, so that a large amount of repeated calculation is caused.

Lu et al consider that local area pixels of an image have the same tag value, and perform superpixel segmentation on the image in a patch Filtering algorithm (Lu J, li Y, yang H, et al. PatchMatch Filter: edge-Aware Filtering means oriented Search for Visual reconstruction [ J ]. IEEE Transactions on Pattern Analysis and Machine Analysis, 2017,39 (9): 1866-1879.) to complete tag space propagation and tag refinement with the segmentation unit as a computation unit. The super-pixel is used as a calculation unit, so that cost aggregation can adopt a linear time complexity cost filtering algorithm, the calculation time complexity of cost filtering is reduced, the number of calculation units is reduced, and a large amount of repeated calculation is avoided. However, the label updating regions constructed by the super-pixel unit in the patch filtering algorithm are overlapped, so that the algorithm cannot realize parallel calculation; in addition, the representative pixels are used for thinning the labels of the segmentation unit pixels, the utilization rate of label calculation is reduced by the strategy, and the label convergence speed of the pixels in the texture-rich area is low.

Disclosure of Invention

Aiming at the defects that the current patch matching algorithm is high in computational complexity, cannot perform parallel computation and the like in a stereo matching task, the invention aims to provide a patch matching algorithm which is low in computational complexity and can perform efficient parallel computation on a CPU (central processing unit) and a GPU (graphics processing unit). Similar to the traditional algorithm, the method still adopts the ideas of space propagation and plane refinement of a patch matching algorithm; the algorithm is characterized in that the division units in regular shapes are used as calculation units, optimization layers of the two scale division units are constructed to respectively finish the purposes of space propagation and random search, and efficient parallel calculation is realized. Meanwhile, a plane thinning strategy based on the segmentation unit is provided, and the convergence speed of the label in the optimization process is accelerated.

In order to achieve the purpose, the invention adopts the following technical scheme:

a patch matching parallel method based on two-stage segmentation units comprises the following steps:

step 1: acquiring a left image and a right image of a stereoscopic vision matching pair, and calculating a pixel parallax search range;

step 2: regarding the left image and the right image as graph structures, and constructing a calculation layer S consisting of two stages of segmentation units ₁ And S ₂ Constructing corresponding subgraphs by the segmentation units;

and 3, step 3: randomly generating a plane label for each pixel by adopting a random initialization method;

and 4, step 4: at the computing layer S ₁ Performing spatial propagation of pixel labels by taking a segmentation unit as a calculation unit;

and 5: at the computing layer S ₂ In the method, a segmentation unit is used as a calculation unit to finish the plane thinning of a pixel label;

step 6: repeating the step 4) and the step 5) until the disparity map is converged, and obtaining the label of each pixel of the right image in the same process;

and 7: judging an error parallax value in the parallax image by using the left and right parallax images, and performing post-processing on the error parallax value;

and 8: and converting the parallax map into a depth map by using parameters of the binocular camera to obtain the three-dimensional appearance of the scene.

The further improvement of the invention is that the specific implementation method of the step 1) is as follows:

step 1.1: arranging a left camera and a right camera according to scene characteristics to form a binocular system, and calibrating internal parameters and external parameters of the cameras by adopting Zhangyingyou checkerboards;

step 1.2: the calibrated binocular system is used for shooting left and right visual angle images of a scene, and epipolar correction is carried out on the images according to system parameters, so that the homonymous pixel points of the left and right images only change in image coordinates in the X direction.

The further improvement of the invention is that the specific implementation method of the step 2 is as follows:

step 2.1: viewing the input image as a graph, the pixels representing nodes in the graph; with side length of k ₁ The rectangles of the pixels divide the image, and the whole image is divided into K ₁ A rectangular area; each of the divided units is used as a calculation unit to form a calculation layer S ₁ (ii) a Extending r pixels outwards from the segmentation unit to construct an arbitrary segmentation unit S ₁ (k) Subgraph of size (k) ₁ +2r)×(k ₁ +2 r), where r represents the filter window radius;

step 2.1: with side length of l ₂ The rectangles of the pixels divide the image, and the whole image is divided into K ₂ A rectangular area; each of the divided units is used as a calculation unit to form a calculation layer S ₂ (ii) a For all the segmentation units, extending r pixels outwards to construct any segmentation unit S ₂ (k) Subgraph of size (k) ₂ +2r)×(k ₂ +2 r), where r represents the filter window radius.

The further improvement of the invention is that the specific implementation method of the step 3 is as follows:

randomly generating a parallax value d in the parallax range for each pixel ₀ Simultaneously randomly initializing a unit vector

The pixel p being calculated from the pixel disparity value and the normal vector as the normal vector of the planeLabel l _p ＝(a _p ,b _p ,c _p ) Wherein a is _p ＝-n _x /n _y ；b _p ＝-n _y /n _z ；c _p ＝(n _x x ₀ +n _y y ₀ +n _z d ₀ )/n _z 。

The further improvement of the invention is that the specific implementation method of the step 4 is as follows:

step 4.1: computing layer S ₁ Middle arbitrary division unit S ₁ (k) Internal random sampling N _s Each pixel, the label values of the pixels form a recommended label sequence

Step 4.2: by dividing the unit S ₁ (k) As a calculation unit, the recommended tags l are sequentially extracted from the recommended tag sequence _s (ii) a In a dividing unit S ₁ (k) Calculating the matching cost of all pixels under the recommended label in the subgraph;

step 4.3: in a dividing unit S ₁ (k) Completing cost aggregation in the subgraph; the method is realized by adopting a derivative filtering or cross filtering linear cost filtering algorithm;

step 4.4: completing label update of pixel, i.e. to the segmentation unit S ₁ (k) Any pixel p, when C (p, l) is satisfied _p )＞C(p,l _s ) When the current pixel is used, the label and the cost value of the current pixel are recommended, otherwise, the label value and the cost value are not updated; where C (-) represents the matching cost after cost aggregation, l _s Indicates a recommendation tag,/ _p Representing the current label of the pixel.

The further improvement of the invention is that the specific implementation method of the step 5 is as follows:

step 5.1: computing layer S ₂ Upper, arbitrarily dividing the unit S ₂ (k) Internally and randomly sampling a pixel to obtain the parallax value d of the pixel ₀ Sum normal vector

Simultaneously randomly sampling to obtain a parallax value and an offset of a normal vector

And Δ _n The disparity value and normal vector of the generated recommendation label are expressed as

And

wherein

And

the space is sampled for the offset amount of time,

and

respectively representing the maximum sampling range values of the parallax value and the normal vector; an initial spatial range is set as

The spatial extent decreasing with the number of calculations, i.e.

And

the termination condition is set as

Step 5.2: disparity value according to recommended label

Sum normal vector

Calculating plane parameters

Step 5.3: the unit S is divided by adopting the steps 4.2 to 4.3 ₂ (k) Calculating matching cost and cost aggregation according to the recommended tags in the subgraph and updating the tags of the pixels;

step 5.4: repeating the step 5.1 to the step 5.3 until the calculation meets the termination condition

The further improvement of the invention is that the specific implementation method of the step 6 is as follows:

repeating the step 4 and the step 5, and stopping calculation when the mismatching rate change of the parallax image after the adjacent optimization times is smaller than a set threshold value to obtain the parallax image of the current image; and when the right image calculates the homonymous pixel points in the left image, the parallax value takes a negative number, and the parallax image of the right image is obtained by adopting the calculation process of the parallax image of the left image.

The further improvement of the invention is that the specific implementation method of the step 7 is as follows:

step 7.1: finding out pixels which do not meet left and right consistency constraints in the disparity map by utilizing left and right consistency detection, and setting the pixels as error disparity values;

step 7.2: traversing the pixels with the wrong parallax values back and forth along the X direction to obtain a first pixel with a correct parallax value, and selecting the minimum value of the two parallax values to fill the parallax value of the current pixel;

step 7.3: and filtering the disparity map by adopting median filtering.

The further improvement of the invention is that the specific implementation method of the step 8 is as follows:

converting the disparity map into a depth map under a camera coordinate system by using binocular system parameters,

wherein f is the focal length of the camera, B represents the binocular baseline distance, and the image coordinate of the pixel point p is (x) _p ,y _p ) The parallax value is d _p Corresponding to the three-dimensional coordinate (X) in the camera coordinate system _p ,Y _p ,Z _p )。

The invention has at least the following beneficial technical effects:

the invention provides a patch matching parallel method based on two-stage segmentation units, which can efficiently search label values of pixels in parallel in CPU (central processing unit) and GPU (graphic processing unit) hardware. The algorithm firstly adopts two scales to segment an image to construct two optimization layers, wherein a segmentation unit adopts a regular shape, so that label updating can be independently completed in the segmentation unit, and the parallel computing capability of the algorithm is ensured; in the two-stage segmentation unit calculation layer, the large-size segmentation unit calculation layer is mainly used for completing space propagation, the segmentation units adopt large sizes to reduce the number of the segmentation units, and meanwhile, the space propagation distance of the pixel labels is also enlarged; and the label refinement is completed by a calculation layer with a smaller segmentation unit, and the segmentation unit with a smaller size is adopted to meet the condition that pixels in the segmentation unit have the same label value as much as possible, so that the effectiveness of the label refinement is improved. The design of the two-stage segmentation unit calculation layer meets the parallel calculation requirements of space propagation and plane refinement, and meanwhile, the convergence rate of the label can be increased. The method has the advantages that the recommended label is generated by sampling each time based on the plane refinement of the segmentation unit, the label offset is sampled, the label to be optimized is sampled, the optimized utilization rate can be improved, and the convergence rate of the label is increased.

Drawings

FIG. 1 is a flow chart of the calculation of the present invention.

Fig. 2 is a schematic diagram of the regular shape division and calculation units of the graph, in which fig. 2 (a) is a schematic diagram of a division graph, and fig. 2 (b) is an arbitrary division unit and a subgraph.

Fig. 3 is a schematic diagram of two-level different scale partition unit computation layers, where fig. 3 (a) is a large scale partition unit computation layer and fig. 3 (b) is a small scale partition unit computation layer.

Fig. 4 is a time comparison of parallel and non-parallel computations of the proposed algorithm on the CPU.

Fig. 5 shows the effect of the disparity map of the proposed algorithm, where fig. 5 (a) shows the left image in the matched pair image, fig. 5 (b) shows the standard disparity map of fig. (a), and fig. 5 (c) shows the disparity map obtained by the algorithm.

Fig. 6 three-dimensional point cloud reconstruction results.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. The invention will be described in detail below with reference to the accompanying drawings in conjunction with an embodiment example.

The invention provides a two-stage segmentation unit-based patch matching parallel method, which adopts a regular shape to segment an image, and takes a segmentation unit as a calculation unit and a label updating area to ensure the parallel calculation capability of an algorithm; designing a calculation layer of two stages of segmentation units, wherein the layer with the smaller size of the segmentation unit is used for plane refinement calculation, and the layer with the larger size of the segmentation unit is used for space propagation calculation, so that the two processes can be efficiently and parallelly operated alternately, and the space propagation in the large-size segmentation unit is equivalent to the space propagation between the small-size segmentation units; and in the process of carrying out plane refinement in the segmentation unit, randomly sampling the label to be optimized and the label offset, and improving the convergence speed of the label. The calculation flow is shown in fig. 1, and specifically comprises the following steps:

step 1: acquiring a left image and a right image of a stereoscopic vision matching pair, and calculating a pixel parallax search range;

in the embodiment, stereo matching disclosed by a Middlebury data set is selected to explain images, the images in the data set comprise left and right matching pair images, and the maximum parallax search range d _max The focal length f of the binocular system and the baseline distance B.

Step 2: constructing regular-shaped segmentation units from the input image, the two segmentation units forming two computation layers S ₁ And S ₂ Simultaneously constructing a corresponding subgraph for each segmentation unit, as shown in fig. 2 (b);

step 2.1: viewing the input image as a graph, the pixels representing nodes in the graph; using side lengths of k ₁ The rectangles of the pixels divide the image, and the whole image is divided into K ₁ And a dividing unit. FIG. 2 (a) shows the segmentation of the image, with a segmentation unit S within the dashed rectangle ₁ (k) (ii) a Simultaneously, the segmentation unit is extended outwards by r pixels to construct a segmentation unit S ₁ (k) Is used for cost calculation and cost aggregation, as shown in fig. 2 (b), where r represents the filter window radius; the above operation constructs S ₁ And calculating the layer.

Step 2.2: construction of S with step 2.1 ₂ Computing layers with the difference that the rectangle size of the partition unit is k ₂ ×k ₂ . S in algorithm design ₁ Layers for spatial propagation, S ₂ Layers are used for plane refinement. Therefore, k is usually set ₁ ≥1.5k ₂ 。

And 3, step 3: the matching point calculation is started. Firstly, randomly initializing a plane label for each pixel;

there are 3 variables per pixel: disparity value d, plane normal vector

Planar ginseng _p ＝(a _p ,b _p ,c _p ). Wherein the conversion relationship between parallax and plane parameter is d _p ＝a _p p _x +b _p p _y +c _p . The disparity value and the plane normal vector are used in random search of plane refinement, and the plane parameters are used as labels for neighborhood propagation.

In the random initialization process of the algorithm, the algorithm parallax space [0, d ] is initialized for any pixel p _max ]An intra-random sampling of a disparity value d is performed while the sampling generates a unit vector

As a normal vector of the plane, thereby calculating the label parameter l of the pixel p _p ＝(a _p ,b _p ,c _p ) Wherein a is _p ＝-n _x /n _y ；b _p ＝-n _y /n _z ；c _p ＝(n _x x ₀ +n _y y ₀ +n _z d _p )/n _z 。

And 4, step 4: at the computing layer S ₁ In the method, a segmentation unit is used as a calculation unit to realize the spatial propagation of the label;

step 4.1: at the computing layer S ₁ Middle arbitrary division unit S ₁ (k) Internal random sampling N _s Pixel, the label values of the pixels constitute a recommended label sequence L = { L = { (L) } _s |n＝1,...,N _s }；

And 4.2: in a dividing unit S ₁ (k) And performing cost calculation, cost aggregation and tag updating on all tags in the tag sequence. Arbitrary tag in internal alignment sequence l _s Calculating unit R ₁ (k) All pixels in the block are worth. From the label l _s Calculating to obtain the parallax value d of the left image pixel p _p It then corresponds to the matching point in the right image

Has a coordinate of x _R ＝x _L -d _p . The color values of the sub-pixel positions are obtained by carrying out bilinear interpolation on the image. (ii) a Here, the operator is described using a cost of "color + gradient":

in the formula: α represents a weight ratio of the color value and the gradient value; tau is _col And τ _grad Respectively, color values and cutoff thresholds for gradient values. The parameters are set to { α, τ _col ,τ _grad }:＝{0.9,10,2}

Step 4.3: in the diagram R ₁ (k) Internally completing cost aggregation; using classical guided filteringThe linear cost filtering algorithms such as word cross filtering and the like can quickly finish cost aggregation operation;

step 4.4: completing label update of pixel, i.e. to the segmentation unit S ₁ (k) Any pixel p, when C (p, l) is satisfied _p )＞C(p,l _s ) Time-current pixel fetching recommendation label l _s And its cost value, otherwise, not updating label value and cost value;

in the process, recommended labels are obtained through random sampling, unified cost calculation, cost aggregation and label updating are carried out on the recommended labels in the segmentation units, and the propagation of sampling point pixel labels in the whole segmentation units is realized; therefore, the distance of label propagation is enlarged, and the algorithm is not easy to fall into local minimum.

And 5: at the computing layer S ₂ In the method, a segmentation unit is taken as a calculation unit to finish the plane thinning of the pixel label;

step 5.1: computing layer S ₂ Upper, arbitrarily divide the unit S ₂ Internally and randomly sampling a pixel to obtain the parallax value d of the pixel ₀ Sum normal vector

Simultaneously randomly sampling to obtain a parallax value and an offset of a normal vector

And Δ _n The disparity value and normal vector of the generated recommendation label are expressed as

And

then calculating plane parameters according to the parallax value and the normal vector;

wherein

And

to be the offset range,

and

maximum sampling spaces respectively representing the disparity value and the normal vector; an initial spatial range is set as

Wherein d is _max Searching for a maximum disparity value;

and step 5.2: disparity value according to recommended label

Normal vector

Calculating plane parameters

Step 5.3: the step 4.2 to the step 4.3 are adopted to divide the unit S ₂ (k) And subfigure R thereof ₂ (k) Calculating matching cost according to the recommended label, gathering cost and updating the label of the pixel;

step 5.4: repeating the step 5.1 to the step 5.3, wherein the adopted space range is reduced along with the times of calculation in multiples, namely

And

the termination condition is set as

In the calculation process, in a round of random search process, the pixel label is randomly sampled in the segmentation unit when the label offset is obtained by each random sampling. After each recommended label is generated, the label is updated, and the recommended label is generated by performing the next random search. This is intended to increase the probability of sampling a representative label in a segment, and to increase the amount of label refinement.

And 6: repeating the step 4) and the step 5) until the disparity map is converged, and obtaining the label of each pixel of the right image in the same process; FIG. 4 shows a time comparison of the proposed algorithm on three sets of data (Tsukuba, venus and Conses) with and without parallel computations using the CPU. Fig. 5 shows the initial left disparity map obtained by the algorithm, and the distribution map of the mismatching points calculated from the standard disparity map (where the mismatching points of the non-occlusion region are marked in gray, the mismatching points of the occlusion region are marked in black, and the threshold of the mismatching points is 0.5 pixels).

And 7: detecting an error parallax value by using the parallax images corresponding to the obtained left image and the right image and adopting consistency constraint, and processing the error parallax value by adopting post-processing methods such as parallax value filling, median filtering and the like;

and 8: and converting the parallax map into a depth map by using parameters of the binocular camera to obtain the three-dimensional appearance of the scene.

Converting the disparity map into a depth map under a camera coordinate system by using binocular system parameters,

wherein f is the focal length of the camera, B represents the binocular baseline distance, and the image coordinate of the pixel point p is (x) _p ,y _p ) The parallax value is d _p Corresponding to the three-dimensional coordinate (X) in the camera coordinate system _p ,Y _p ,Z _p ). Fig. 6 shows the result of converting the post-processing disparity map in fig. 5 into a three-dimensional point cloud under the camera parameters.

Although the invention has been described in detail with respect to the general description and the specific embodiments thereof, it will be apparent to those skilled in the art that modifications and improvements can be made based on the invention. Accordingly, such modifications and improvements are intended to be within the scope of the invention as claimed.

Claims (9)

1. A patch matching parallel method based on two-stage segmentation units is characterized by comprising the following steps:

step 1: acquiring a left image and a right image of a stereoscopic vision matching pair, and calculating a pixel parallax search range;

step 2: regarding the left image and the right image as graph structures, and constructing a calculation layer S consisting of two stages of segmentation units ₁ And S ₂ Constructing corresponding subgraphs by the segmentation units;

and 3, step 3: randomly generating a plane label for each pixel by adopting a random initialization method;

and 4, step 4: at the computing layer S ₁ Performing spatial propagation of pixel labels by taking a segmentation unit as a calculation unit;

and 5: at the computing layer S ₂ In the method, a segmentation unit is taken as a calculation unit to finish the plane thinning of the pixel label;

step 6: repeating the step 4) and the step 5) until the disparity map is converged, and obtaining the label of each pixel of the right image in the same process;

and 7: judging an error parallax value in the parallax image by using the left and right parallax images, and filling the error parallax value;

and step 8: and converting the parallax map into a depth map by using parameters of a binocular camera to obtain the three-dimensional appearance of the scene.

2. The two-stage segmentation unit-based patch matching parallel method according to claim 1, wherein the specific implementation method of step 1) is as follows:

step 1.1: arranging a left camera and a right camera according to scene characteristics to form a binocular system, and calibrating internal parameters and external parameters of the cameras by adopting Zhang Zhengyou checkerboard;

step 1.2: the calibrated binocular system is used for shooting left and right visual angle images of a scene, and epipolar correction is carried out on the images according to system parameters, so that the homonymous pixel points of the left and right images only change in image coordinates in the X direction.

3. The method according to claim 2, wherein the step 2 is implemented as follows:

step 2.1: viewing the input image as a graph, the pixels representing nodes in the graph; with side length of k ₁ The rectangles of each pixel divide the graph, the whole graph is divided into K ₁ A rectangular area; each of the divided units is used as a calculation unit to form a calculation layer S ₁ (ii) a Extending r pixels outwards from the segmentation unit to construct an arbitrary segmentation unit S ₁ (k) Subfigure R ₁ (k) Subpicture size of (k) ₁ +2r)×(k ₁ +2 r), where r represents the filter window radius;

step 2.2: with side length of l ₂ The rectangles of each pixel divide the graph, the whole graph is divided into K ₂ A rectangular area; each of the divided units is used as a calculation unit to form a calculation layer S ₂ (ii) a Extending r pixels outwards from all the segmentation units to construct an arbitrary segmentation unit S ₂ (k) Subgraph of size (k) ₂ +2r)×(k ₂ +2 r), where r represents the filter window radius.

4. The two-stage segmentation unit-based patch matching parallel method according to claim 3, wherein the specific implementation method of step 3 is as follows:

randomly generating a parallax value d in the parallax range for each pixel ₀ Simultaneously randomly initializing a unit vector

As a normal vector of the plane, a label l of the pixel p is calculated from the pixel parallax value and the normal vector _p ＝(a _p ,b _p ,c _p ) Wherein a is _p ＝-n _x /n _y ；b _p ＝-n _y /n _z ；c _p ＝(n _x x ₀ +n _y y ₀ +n _z d ₀ )/n _z 。

5. The two-stage segmentation unit-based patch matching parallel method according to claim 4, wherein the specific implementation method of step 4 is as follows:

step 4.1: computing layer S ₁ Middle arbitrary division unit S ₁ (k) Internal random sampling N _s Each pixel, the label values of the pixels form a recommended label sequence

And 4.2: by dividing the unit S ₁ (k) As a calculation unit, the recommended tags l are sequentially extracted from the recommended tag sequence _s (ii) a In a dividing unit S ₁ (k) Calculating the matching cost of all pixels under the recommended label in the subgraph;

step 4.3: in a dividing unit S ₁ (k) Completing cost aggregation in the subgraph; the method is realized by adopting a derivative filtering or cross filtering linear cost filtering algorithm;

step 4.4: completing label update of pixel, i.e. to the segmentation unit S ₁ (k) Inner arbitrary pixel p, when C (p, l) is satisfied _p )＞C(p,l _s ) If the current pixel is the recommended label and the cost value of the recommended label, otherwise, the label value and the cost value are not updated; where C (-) denotes the matching cost after cost aggregation, l _s Represents a recommended label,/ _p Representing the current label of the pixel.

6. The two-stage segmentation unit-based patch matching parallel method according to claim 5, wherein the specific implementation method of step 5 is as follows:

step 5.1: computing layer S ₂ Upper, arbitrarily dividing the unit S ₂ (k) Internally and randomly sampling a pixel to obtain the parallax value d of the pixel ₀ Sum normal vector

Simultaneously and randomly sampling to obtain a parallax value and a normal vectorAn offset amount

And Δ _n The disparity value and normal vector of the regenerated new recommendation label are

And

wherein

And

the space is sampled for the offset amount of time,

and

respectively representing the parallax value and the maximum sampling value of the normal vector; an initial spatial range is set as

The spatial extent decreasing with the number of calculations, i.e.

And

the termination condition is set as

Step 5.2: disparity value according to recommended label

Sum normal vector

Calculating plane parameters

Step 5.3: the unit S is divided by adopting the steps 4.2 to 4.3 ₂ (k) And subfigure R thereof ₂ (k) Calculating matching cost according to the recommended label, gathering cost and updating the label of the pixel;

step 5.4: repeating the step 5.1 to the step 5.3 until the calculation meets the termination condition

7. The two-stage segmentation unit-based patch matching parallel method according to claim 6, wherein the specific implementation method of step 6 is as follows:

repeating the step 4 and the step 5, and stopping calculation when the mismatching rate change of the parallax image is smaller than a set threshold value after adjacent optimization times to obtain the parallax image of the current image; and when the right image calculates the homonymous pixel points in the left image, the parallax value takes a negative number, and the parallax image of the right image is obtained by adopting the calculation process of the parallax image of the left image.

8. The method according to claim 7, wherein the step 7 is implemented as follows:

step 7.1: utilizing left and right disparity map consistency detection to find out pixels which do not meet consistency constraint in the disparity map, and judging that the pixel disparity value is an error disparity value;

step 7.2: traversing the pixels with the error parallax values back and forth along the X direction to obtain a first pixel with a correct parallax value, and selecting the minimum value of the two parallax values to fill the parallax value of the current pixel;

step 7.3: and filtering the disparity map by adopting median filtering.

9. The two-stage segmentation unit-based patch matching parallel method according to claim 8, wherein the step 8 is implemented as follows:

converting the disparity map into a depth map under a camera coordinate system by using binocular system parameters,

wherein f is the focal length of the camera, B represents the binocular baseline distance, and the image coordinate of the pixel point p is (x) _p ,y _p ) With a disparity value of d _p Corresponding to the three-dimensional coordinate (X) in the camera coordinate system _p ,Y _p ,Z _p )。

Images