CN106257537A - A kind of spatial depth extracting method based on field information - Google Patents

Abstract

The invention discloses a kind of spatial depth extracting method based on field information, comprise the steps: step 1, selected reference view in four-dimensional light field data, calculate the spatial depth of image border composition at reference view；Step 2, carries out region segmentation computing to image at reference view, is classified as some regions according to color or brightness homogeneity；Step 3, revises the spatial depth of each edges of regions after segmentation, according to revised spatial depth, the spatial depth of regional center part is made interpolation arithmetic.The present invention is by the method such as TABU search, region segmentation, degree of depth interpolation, it is to avoid the described function faced by spatial depth described function optimization method institute selects difficulty and the problem such as operation time is uncertain, it is achieved that quick, the exact arithmetic to spatial depth extraction.

Description

A kind of spatial depth extracting method based on field information

Technical field

The present invention relates to computer vision and calculate photography field, particularly relating to a kind of space based on field information deep Degree extracting method.

Background technology

The acquisition of information of optical field imaging belongs to calculating photography field, and spatial depth extracts and belongs to computer vision field. The imagination of record light field is proposed in the Latin paper delivered for 1908 by Gabriel Lippmann the earliest.1996, M.Levoy and P.Hanrahan proposes light field and renders theory, and plenoptic function tapers to the four-dimension, and referred to as light field letter Number.The acquisition mode of field information has two kinds, a kind of form adding microlens array for traditional camera, and another kind is by several Camera arrangement becomes the form of array.Two kinds of methods are reached the same goal by different routes, and all can obtain the sampling of field information.With tradition imaging mode phase Ratio, the method for optical field imaging not only have recorded the intensity of sensor pixel, and have recorded the direction of incident ray.Thus light field Imaging can obtain the anaglyph horizontally and vertically of imaging object, thus containing object space depth information.

The existing method extracting spatial depth in field information is required for greatly by optimizing spatial depth described function Realize, and due to needs loop optimization, its operation time is difficult to accurately estimate, and the time is longer.By optimizing spatial depth Described function realizes the method that spatial depth extracts, and needs to select strong effective described function, its operation result accurate Property have the biggest association with the matching degree of described function and actual scene, be difficult to find the description that all scenes are all adapted to optimize letter Number.

The patent documentation of Publication No. 104899870A discloses a kind of depth estimation method based on light field data distribution. The method reference light field data characteristic, from a series of refocusing light field image changing input light field image pixel distribution gained Extract and estimate scene depth with focal length correlation tensor.Further, also utilize this tensor with the variation tendency of the degree of depth and scene center The gradient information of sub-aperture texture maps is set up polynary credibility model and is weighed each point ID quality, optimizes and ties according to a preliminary estimate Really.The method is computationally intensive, and operation time is long.

Summary of the invention

In order to solve time complexity and the calculating accuracy that spatial depth extracting method based on function optimization is faced Problem, the present invention combines field information, proposes a kind of spatial depth extracting method, extracts entire image by once calculating Spatial depth, it is not necessary to iterative computation.

A kind of spatial depth extracting method based on field information, comprises the steps:

Step 1, selected reference view in four-dimensional light field data, calculate the space of image border composition at reference view deep Degree；

Step 2, carries out region segmentation computing to image at reference view, if being classified as according to color or brightness homogeneity Dry region；

Step 3, revises, according to revised spatial depth to region the spatial depth of each edges of regions after segmentation The spatial depth of core makees interpolation arithmetic.

In step 1, reference view can be central viewpoint, it is also possible to be edge viewpoint.For obtaining figure at reference view As marginal element, only ginseng need to can be obtained in conjunction with being in the image information of collinear two groups of other viewpoints with reference view Examine the spatial depth of image border composition at viewpoint.

Spatial depth refer in visual field on the object corresponding to a certain pixel a little to the distance of light field recording equipment.

Being different from traditional method, the present invention is not required to K × K viewpoint forming array, but only needs 2K-1 viewpoint.Wherein, K Individual viewpoint is positioned on straight line, and additionally K viewpoint is positioned at another straight line, and two straight-line intersections are reference view, therefore are total to Need 2K-1 viewpoint.Further, herein to the viewpoint group being in two lines, its included angle of straight line is without particular/special requirement.

Specifically comprising the following steps that of step 1

Step 1-1, obtains field information, and four-dimensional field information is decomposed into the image of each viewpoint.To figure at reference view As doing gradient algorithm, thus extract image border composition at reference view；

Step 1-2, in conjunction with reference view with a line with other viewpoint image-forming informations of string, calculate reference view Place's image border composition spatial depth characteristic curve slope；

Step 1-3, can calculate the space of image border composition at reference view according to spatial depth characteristic curve slope The degree of depth.

In step 1-1, the inventive method acquisition mode no requirement (NR) to field information, all satisfied four-dimensional light field feature shapes The data of formula are the most effective.

In step 1-2, Grad is more than to the marginal position of noise threshold limit value, by preimage element and the phase of its correspondence Adjacent pixel, scans for coupling by viewpoint；Two viewpoint direction of X and Y all can find one group of position matched, thus The characteristic curve of Special composition depth information；The computational methods of spatial depth characteristic curve, first use TABU search to reduce on characteristic curve The hunting zone of point, finds out the point minimum with point of proximity difference on fixed characteristic curve in hunting zone, is added into spy Levy line point range, then further determined that next step hunting zone by new point range set.

Noise threshold limit value can be manually set, and typically chooses 0≤T≤0.25G_max, G_maxFor maximum gray scale.

TABU search refers to the result searched according to a upper viewpoint position, reduces this hunting zone, such that it is able to greatly Width improves operation efficiency.

In step 2, carry out the border between the region of region segmentation computing to be in step 1 at the reference view obtained Image border composition.Region segmentation computing uses the mode of quadtree decomposition and polymerization, can be effectively improved arithmetic speed Robustness with program.

Quaternary tree separates and the discrimination standard of polymerization is that whether the concordance of pixel exceedes marginal value, for gray scale in region Image, concordance is the maximum difference of pixel grey scale；For coloured image, concordance is the maximum of color and average color Difference.

In step 3, obtain according to spatial depth and the step 2 of image border composition at the reference view of step 1 acquisition Region segmentation situation, the spatial depth of each zone boundary is revised, with adjust produce because of the spatial occlusion relationship between object Raw boundary space deep errors.Then according to the spatial depth of revised zone boundary, non-borderline region is done space deep Degree interpolation arithmetic, to obtain the spatial depth in whole region.

Specifically comprising the following steps that of step 3

Step 3-1, it is thus achieved that each region, travel through its border, in order and the position of record delimitation pixel, and according to The result of step 1 is sequentially recorded the spatial depth value of its correspondence.

Step 3-2, does calculus of differences to the spatial depth value of the zone boundary arranged in order,

Step 3-3, does integral operation to the result of calculus of differences.

When step 3-3 does integral operation, the difference component of threshold value will be exceeded by threshold calculations, thus by the fortune of step 3 Calculate the acute variation of the spatial depth eliminating zone boundary.

The present invention not specific dependency is in a certain kind of two kinds of light field samplings as described in the background art, and the present invention is by prohibiting Avoid the methods such as search, region segmentation, degree of depth interpolation, it is to avoid spatial depth described function optimization method faced by described function The problems such as selection difficulty is uncertain with operation time, it is achieved that quick, the exact arithmetic that spatial depth is extracted.

Accompanying drawing explanation

Fig. 1 is the schematic flow sheet of spatial depth extracting method based on field information；

Fig. 2 is the schematic diagram of field information, and wherein, Fig. 2 a Fig. 2 b is that the light of different points of view reception Same Scene shows respectively It is intended to；

Fig. 3 be reference view choose schematic diagram, wherein, Fig. 3 a is two line orthogonal the choosing of reference view when dividing equally Schematic diagram, Fig. 3 b be two straight lines the most orthogonal time reference view choose schematic diagram, Fig. 3 c is two straight lines ginsengs when arbitrarily intersecting That examines viewpoint chooses schematic diagram；

Fig. 4 is TABU search schematic diagram.

Detailed description of the invention

In conjunction with specific embodiments and the drawings, the inventive method is described in detail.

As it is shown in figure 1, it is as follows to carry out the step of spatial depth extraction based on field information:

(1) at reference view, the spatial depth of image border composition calculates

The reference view of selected four-dimensional light field data.The information of optical field imaging record is 4 D data, and its data record former Reason is as in figure 2 it is shown, (x, y), at the diverse location of image, (x, y) through different viewpoints, (u, v) record, can for piece image I (u, v, x, y), (u, v, x y) are (u, v, x, y) light luminance put to R to obtain four-dimensional light field data R.

The invention is not restricted to the acquisition mode of field information, the available light field data represented such as Fig. 2 of all acquisitions, the present invention Method all can realize it is extracted spatial depth.

Assume u_min≤u≤u_max,v_min≤v≤v_max,x_min≤x≤x_max,y_min≤y≤y_max, choose viewpoint (u, v) definition In the range of a viewpoint (u₀,v₀) it is reference view.The present invention is not required to K × K viewpoint forming array, but only needs 2K-1 Viewpoint, wherein, K viewpoint is positioned on straight line, and additionally K viewpoint is positioned at another straight line, and two straight-line intersections are ginseng Examining viewpoint, therefore, the present invention only needs to arrange 2K-1 viewpoint, reference view choose three kinds of situations as shown in Figure 3 of requirement.

Image at reference view is gradient algorithm G (u₀,v₀, x, y),

$G(u_0,v_0,x,y)=\underset{(x_n,y_n)\∈N(x,y)}{\Σ}\[R(u_0,v_0,x,y)-R(u_0,v_0,x_n,y_n)\]/4,$

Wherein (x y) is (x to N_n,y_n) four neighborhood territory pixels, and

N (x, y)={ (x_n,y_n)||x_n-x|-|y_n-y |=1},

Marginal area E (u₀,v₀) definition be

E(u₀,v₀)={ (u₀,v₀,x,y)||G(u₀,v₀, x, y) | ＞ T},

Wherein T is noise threshold limit value, and its value can be manually set, and typically chooses 0≤T≤0.25G_max, G_maxFor maximum Gray scale.

Edge region, for (the v determined₀,y_e), (u, x) can form a stack features line, and (u x) crosses point (u₀,x_e)；Right In (the u determined₀,x_e), (v, y) can form a stack features line, and (v y) crosses point (v₀,y_e)。

Above, a bit (u of marginal area₀,v₀,x_e,y_e)∈E(u₀,v₀)。

A bit (u of note marginal area₀,v₀,x_e,y_e) two stack features line set be respectively

With Characteristic curve can be tried to achieve by the way of iterationWith

Wherein,

$x\left(u\right){|}_{\underset{y=y_e}{\underset{v=v_0}{u,u-1}}\∈(u_{min},u_{max})}=\underset{x\left(u\right)}{\mathrm{argmin}}\[2R(u,v_0,x(x),y_e)-R(u-1,v_0,x(u-1),y_e)-R(u_0,v_0,x_e,y_e)\],$

$y\left(v\right){|}_{\underset{x=x_e}{\underset{u=u_0}{v,v-1}}\∈(v_{min},v_{max})}=\underset{y\left(u\right)}{\mathrm{argmin}}\[2R(u_0,v,x_e,y(v\left)\right)-R(u_0,v-1,x_e,y(v-1\left)\right)-R(u_0,v_0,x_e,y_e)\].$

As it is shown on figure 3, the process of TABU search process iteration is without the traversal overall situation, after often determining on characteristic curve a bit, A hunting zone containing 3 to 5 pixels will be locked at next iterative position, thus can significantly compressive features line The calculating time.

After determining characteristic curve, it may be determined that the slope S of characteristic curve_ux(u₀,v₀,x_e,y_e) and S_vy(u₀,v₀,x_e,y_e)。

$S_{ux}(u_0,v_0,x_e,y_e)=\frac{\mathrm{\Σ}(u-\stackrel{\‾}{u})(x-\stackrel{\‾}{x})}{\mathrm{\Σ}{(u-\stackrel{\‾}{u})}^2},\∀(u,v,x,y)\∈R_{ux}(u,v_0,x,y_e),$

$S_{vy}(u_0,v_0,x_e,y_e)=\frac{\mathrm{\Σ}(v-\stackrel{\‾}{v})(y-\stackrel{\‾}{y})}{\mathrm{\Σ}{(v-\stackrel{\‾}{v})}^2},\∀(u,v,x,y)\∈R_{vy}(u_0,v,x_e,y).$

Finally can determine that edge pixel spatial depth D (μ₀,v₀,x_e,y_e) it is

$D(u_0,v_0,x_e,y_e)=\frac{D_0}{1-0.5\[S_{ux}(u_0,v_0,x_e,y_e)+S_{vy}(u_0,v_0,x_e,y_e)\]},$

Wherein D₀For normalization spatial depth.

(2) image at reference view is carried out region segmentation computing

The present invention uses the mode of quadtree decomposition and polymerization to implement the region segmentation of image at reference view.First will ginseng Examine image at viewpoint and carry out quadtree decomposition, it is desirable in the region after decomposition, pixel threshold is less than T, herein threshold value and first step threshold It is worth identical.

If R_iFor current rectangle region, then

$P\left(R_i\right)=\left\{\begin{array}{c}1,max\left(R_i\right)-min\left(R_i\right)\≤T\\ 0,max\left(R_i\right)-min\left(R_i\right)>T\end{array}\right.,$

P(R_i) for judge region R_iWhether carry out the logic split, if value is 1, do not split, if value is 0, Segmentation；max(R_i) it is the maximum in current rectangle region, min (R_i) it is the minima in current rectangle region.

If P is (R_i)=0, then by R_iIt is divided into mutually disjoint 4 regions, is allowed to satisfied

Repeat above step until each subregion is the most indivisible.The region of the arbitrary neighborhood after then decomposing is tasted Examination polymerization, if meeting max (R_i∪R_j)-min(R_i∪R_j)≤T, then by merging.

(3) depth fill-in of cut zone core pixel

(3-1) first marginal element's spatial depth of each cut zone is revised.

Arbitrary cut zone R after being combined_i, retrieve its edge degree of depth counterclockwise, it is thus achieved that marginal position ordered series of numbers C_i With edge degree of depth ordered series of numbers D_i, wherein i=1,2 ..., n.Ask for ordered series of numbers D_iDifference d_i,

$d_i=\left\{\begin{array}{c}{D}_{i+1}-D_i,i\≠n\\ D_0-D_i,i=n\end{array}\right.,$

Revised difference d_i' it is

${d_i}^{\&prime;}=\left\{\begin{array}{c}{d}_i,d_i<\mathrm{\&Delta;}{D}_{max}\\ {\mathrm{\&Delta;D}}_{max},d_i\&GreaterEqual;{\mathrm{\&Delta;D}}_{max}\end{array}\right.,$

Wherein, Δ D_maxFor spatial depth change threshold.If D_iMinima isThe most revised spatial depth D_i' it is

${D_i}^{\&prime;}=\left\{\begin{array}{c}{D}_{i-1}+{d_i}^{\&prime;},i>i_0\\ D_{min},i=i_0\\ D_{i+1}-{d_i}^{\&prime;},i<i_0\end{array}\right..$

(3-2) according to the spatial depth at revised cut zone edge, deep to the space of cut zone non-edge part Degree is filled.

Fill the most in the x-direction.For any point in cut zone, (x y), finds the left hand edge of identical y-coordinate (x_left, y) with right hand edge (x_right, y), according to the spatial depth at revised cut zone edge to (x, y) spatial depth enters Line linearity interpolation.The spatial depth D in the x direction after interpolation_x(u₀,v₀, x, y) be

$D_x(u_0,v_0,x,y)=\frac{x-x_{left}}{x_{right}-x_{left}}{D}^{\&prime;}(u_0,v_0,x_{right},y)+\frac{x_{right}-x}{x_{right}-x_{left}}{D}^{\&prime;}(u_0,v_0,x_{left},y),$

Wherein D'(u₀,v₀,x_right, y) it is pixel (u₀,v₀, x, y) the corresponding revised spatial depth of region right margin, D'(u₀,v₀,x_left, y) it is pixel (u₀,v₀, x, y) the corresponding revised spatial depth of region left margin.

For data with a line in region, it is only necessary to retrieve a left and right edges, thus can significantly compress calculating time Between complexity.

In the same manner, the spatial depth D in the y direction obtained after interpolation in the y-direction_y(u₀,v₀, x, y) be

$D_y(u_0,v_0,x,y)=\frac{y-y_{down}}{y_{up}-y_{dowm}}D(u_0,v_0,x,y_{up})+\frac{y_{up}-y}{y_{up}-y_{dowm}}D(u_0,v_0,x,y_{down}),$

Wherein D (u₀,v₀,x,y_up) it is pixel (u₀,v₀, x, y) the corresponding revised spatial depth in coboundary, region, D (u₀,v₀,x,y_down) it is pixel (u₀,v₀, x, y) the corresponding revised spatial depth of region lower boundary.

Final interpolation result is D_x(u₀,v₀, x, y) and D_y(u₀,v₀, x, meansigma methods D (u y)₀,v₀, x, y), i.e.

$D(u_0,v_0,x,y)=\frac{1}{2}\&lsqb;D_x(u_0,v_0,x,y)+D_y(u_0,v_0,x,y)\&rsqb;.$

Embodiment of the present invention adaptability is extensive, and the present invention does not relies on certain types of optical field acquisition device, all meets Fig. 1 The light field data described, all can realize the extraction to spatial depth by the present invention.

Advantages of the present invention is as follows:

(1) present invention is without setting central viewpoint, and prior art depends on central viewpoint more and extracts spatial depth.Difference In prior art, using and be directed to the method for reference view to extract spatial depth, reference view may be located at center, it is also possible to It is positioned at side.

(2) operation time of the present invention is short, it is assumed that at reference view, the data volume of image is N, and total viewpoint number is constant, then originally Inventing step 1, the time complexity of 2,3 described in explanation and be respectively N, NlogN, N, total time complexity is T (N)=O (2N+NlogN)=O (NlogN).Thus the operation time of the present invention is the shortest, possesses the operability of actual realization.

Technical scheme and beneficial effect have been described in detail by above-described detailed description of the invention, Ying Li Solve is to the foregoing is only presently most preferred embodiment of the invention, is not limited to the present invention, all principle models in the present invention Enclose interior done any amendment, supplement and equivalent etc., should be included within the scope of the present invention.

Claims (10)

1. a spatial depth extracting method based on field information, it is characterised in that comprise the steps:

Step 1, selected reference view in four-dimensional light field data, calculate the spatial depth of image border composition at reference view；

Step 2, carries out region segmentation computing to image at reference view, is classified as some districts according to color or brightness homogeneity Territory；

Step 3, revises, according to revised spatial depth to regional center the spatial depth of each edges of regions after segmentation The spatial depth of part makees interpolation arithmetic.

2. spatial depth extracting method based on field information as claimed in claim 1, it is characterised in that reference view is 2K-1 viewpoint, wherein, K viewpoint is positioned on straight line, and additionally K viewpoint is positioned at another straight line, two straight-line intersections For reference view, constitute 2K-1 viewpoint altogether.

3. spatial depth extracting method based on field information as claimed in claim 1, it is characterised in that step 1 concrete Step is as follows:

Step 1-1, obtains field information, and four-dimensional field information is decomposed into the image of each viewpoint.Image at reference view is done Gradient algorithm, thus extract image border composition at reference view；

Step 1-2, in conjunction with reference view with a line with other viewpoint image-forming informations of string, calculate figure at reference view As marginal element spatial depth characteristic curve slope；

Step 1-3, can calculate the spatial depth of image border composition at reference view according to spatial depth characteristic curve slope.

4. spatial depth extracting method based on field information as claimed in claim 3, it is characterised in that in step 1-2, For Grad more than the marginal position of noise threshold limit value, by preimage element and the neighbor of its correspondence, search by viewpoint Rope mates；Two viewpoint direction of X and Y all can find one group of position matched, thus Special composition depth information Characteristic curve；

Noise threshold limit value chooses 0≤T≤0.25G_max, G_maxFor maximum gray scale.

5. spatial depth extracting method based on field information as claimed in claim 3, it is characterised in that in step 1-2, The computational methods of spatial depth characteristic curve, first use TABU search to reduce the hunting zone of point on characteristic curve, in hunting zone Find out the point minimum with point of proximity difference on fixed characteristic curve, be added into characteristic curve point range, then by new point range Set further determines that next step hunting zone.

6. spatial depth extracting method based on field information as claimed in claim 1, it is characterised in that in step 2, enter Border between the region of row region segmentation computing is image border composition at the reference view obtained in step 1.

A kind of spatial depth extracting method based on field information, it is characterised in that in step 2, Region segmentation computing uses the mode of quadtree decomposition and polymerization；In the discrimination standard of quaternary tree separation and polymerization is region Whether the concordance of pixel exceedes marginal value, and for gray level image, concordance is the maximum difference of pixel grey scale；For cromogram Picture, concordance is the maximum difference of color and average color.

8. spatial depth extracting method based on field information as claimed in claim 1, it is characterised in that in step 3, root The region segmentation situation obtained according to the spatial depth of image border composition at the reference view that step 1 obtains and step 2, to each district The spatial depth on border, territory is revised；

According to the spatial depth of revised zone boundary, non-borderline region is done spatial depth interpolation arithmetic, it is thus achieved that whole district The spatial depth in territory.

9. spatial depth extracting method based on field information as claimed in claim 1, it is characterised in that step 3 concrete Step is as follows:

Step 3-1, it is thus achieved that each region, travel through its border, in order and the position of record delimitation pixel, and according to step The result of 1 is sequentially recorded the spatial depth value of its correspondence；

Step 3-2, does calculus of differences to the spatial depth value of the zone boundary arranged in order；

Step 3-3, does integral operation to the result of calculus of differences.

10. spatial depth extracting method based on field information as claimed in claim 9, it is characterised in that in step 3-3 In, when doing integral operation, the difference component of threshold value will be exceeded by threshold calculations.