CN105374039A - Monocular image depth information estimation method based on contour acuity - Google Patents

Abstract

The invention provides a monocular image depth information estimation method based on contour acuity. The method uses edge contour acuity as a fuzzy information estimation characteristic, and through low-level cue information, depth information extraction is carried out. Edge detection is firstly carried out on the image; edge energy and contour acuity are calculated for the edge in the image, the edge energy and the contour acuity serve as outer energy of the contour, inner energy, contour line characteristic energy and contour line distance energy, of the contour is combined to build a contour tracking model, the minimal value of an energy function is solved, and an image contour is searched; then, depth gradient hypothesis serves as a priori assumed gradient model, depth value filling is carried out on an area with different contour lines, and depth distribution is obtained through calculation; and finally, the original image information and the obtained depth image information are used for optimization on the obtained depth image, and a final disparity map is obtained. As is proved by an experimental result, the depth estimation algorithm is simple, and the depth image for the monocular image can be quickly and accurately estimated.

Description

Based on the monocular image depth information method of estimation of contour acuity

Technical field

The present invention relates to one and can carry out depth information method of estimation to monocular image, the method can obtain the depth information of object shown in monocular image by image low level clue.

Background technology

Depth information perception is the prerequisite producing stereoscopic vision, plays a significant role in the fields such as the practical application based on depth information is heavily separated in scene, three-dimensional reconstruction, pattern-recognition, target following.In actual applications, monocular-camera, multi-lens camera or depth camera can be adopted to carry out extraction of depth information.Wherein, based on the depth information method of estimation of monocular-camera because its reality is simple to operate, hardware cost is low, can from existing monocular image material extracting directly depth information and there is certain advantage.

Monocular image depth information estimates the two dimension such as color, shape, coplanarity, the three-dimensional geometric information that are extracted target by single image, thus utilizes a small amount of known conditions to obtain the space three-dimensional information of this target.Algorithm is all adopt high-level, the middle-level clue of image to realize mostly at present.As obtained the semantic label of known image through study reference picture, utilize it to realize target image definition semantic label, thus obtain relative depth information.Utilizing training the image structure information obtained in true anaglyph, as color, texture, shape etc., carrying out over-segmentation to target image, application has the Markov random field distinguishing training to derive depth information.And utilize low level hint information not need to carry out image the analysis of content, only needing directly to apply local message can recover depth information from image, and algorithm is relatively simple.

Fuzzy message can be utilized in the depth information estimation procedure of monocular image as a key character of estimation of Depth.Mostly fuzzy message is that in imaging process the target that there is different depth in the inaccurate or imaging region of focusing causes.According to this characteristic, monocular image depth information is estimated by defocusing process to image, estimates the degree of depth of scene according to the prospect of the ambiguity determination image of image and background.As using the fuzzy value of marginal position as initially, application delustring and Markov random field make fuzzyly to be diffused into entire image, realize relative depth and extract.Or utilize heat expansion from the modeling of process implementation defocus blur process, utilize uneven backheating diffusion equation estimated edge location fuzzy amount to carry out the restoration scenario degree of depth.And due to fuzzy method of diffusion complexity, efficiency is lower, and its practicality is poor.

Therefore, the present invention utilizes image low level clue and new fuzzy message characteristic, simplifies monocular image depth information method of estimation.

Summary of the invention

The present invention proposes a kind of monocular image depth information method of estimation based on contour acuity, the method utilizes image low level hint information, estimate that using the contour acuity information at edge as fuzzy message feature carries out contour of object extraction, according to contour of object and contacting of Object Depth edge, depth assignment is carried out to different objects, thus obtain the depth information of different objects in image.

The object of the invention is to be achieved through the following technical solutions:

Based on the monocular image depth information method of estimation of contour acuity, it is characterized in that, utilize the characteristic in gradient amplitude and contour acuity message reflection image with the contour of object of different depth, when carrying out estimation of Depth, not only consider the gradient amplitude at edge, add the spatial information at edge simultaneously, embody object edge smear out effect trend more fully.Comprise the steps:

(1) for input picture, carry out rim detection, obtain the marginal point P={p of objects in images ₁, p ₂... p _n;

(2) carry out the definition of initial profile line according to priori concentration gradient model, define one group of parallel to each other and outline line that spacing is equal as initial profile line V={v ₀, v ₁v _m; Wherein, v (s)=[x (s), y (s)], x (s), y (s) they are the coordinate of the some s in initial profile line;

The gross energy of outline line is defined as:

E _total＝w ₁E _edge+w ₂E _sharp+w ₃E _s+w ₄E _d

Wherein E _edgefor contour edge energy function, E _sharpcontour acuity energy function, E _sfor outline line characteristic energies function, E _dfor outline line distance energy function, w ₁, w ₂, w ₃, w ₄for each function weight controling parameters, can according to concrete picture characteristics assignment.

Contour edge energy function E _edgerepresent the gradient amplitude of profile, be defined as the gradient amplitude size of image I (x, y) along gradient direction, be expressed as:

$E_{edge}=\exp (-\frac{{\left|g\left(I\right(x,y\left)\right)\right|}^2}{a})$

Wherein, $g\left(I\right(x,y\left)\right)=\sqrt{{\left({\∂}_{x}I\right(x,y\left)\right)}^2+{\left({\∂}_{y}I\right(x,y\left)\right)}^2}$ That gradient amplitude is asked to I (x, y);

Contour acuity energy function E _sharprepresenting the fog-level of profile, obtaining by solving gradient profile acutance.Be defined as:

$E_{sharp}=\exp (-\frac{\mathrm{\σ}{\left(p\right(q_0\left)\right)}^2}{b});$

Wherein, gradient profile acutance σ (p (q ₀)) be the root mean square of gradient profile variable variance.Here gradient profile is edge pixel q in image ₀(x ₀, y ₀) as starting point, along the extrorse boundary tracking of gradient direction, until the gradient amplitude at edge no longer changes, institute obtains one dimension path p (q ₀) the gradient amplitude curve that formed.Its gradient profile acutance is defined as

$\mathrm{\σ}\left(p\right(q_0\left)\right)=\sqrt{\underset{x\∈p\left(q_0\right)}{\Σ}\frac{g\left(q\right)}{G\left(q_0\right)}{d_c}^2(q,q_0)};G\left(q_0\right)=\underset{s\∈p\left(q_0\right)}{\Σ}g\left(s\right)$

Wherein, d _c(q, q ₀) be gradient profile mid point q and q ₀between length of curve, g (q) is the gradient amplitude at q point place, G (q ₀) in gradient profile gradient amplitude a little and, s is any point in gradient profile, and parameter b is control of right parameter.

Outline line characteristic energies function E _sfor retraining the smoothness of profile;

Outline line distance energy function E _dregion of search can not be exceeded for controlling profile aircraft pursuit course;

(3) to each initial profile line in image from the left side starting point of image, upgraded the position of each point by image base to top, according to outline line gross energy definition in step (2), Contour extraction energy is calculated to each point;

(4) minimum value of energy function is solved; Search for the pixel in the pixel column adjacent with current outline point, search meets the marginal point P={p of energy function definition ₁, p ₂... p _nin there is the point of minimum energy value, select the pixel position with least energy as new point; To right side on the left of image, repeat search minimum value, obtains final Contour searching result, namely obtains target wheel profile V;

(5) using concentration gradient hypothesis as a priori assumption gradient former, depth value filling is carried out to the region with different outline lines, calculates depth profile.

Outline line concentrates outline line V={v ₀, v ₁..., v _m, corresponding distribution depth value Depth is:

$Depth\left(v_i\right)=255\×\frac{m-i}{m},i=0...m$

(6) original image half-tone information and gained deep image information is utilized to be optimized process to the depth image obtained:

${{\mathrm{Depth}}_f}^{bf}\left(x_i\right)=\frac{1}{W\left(x_i\right)}\underset{x_j\∈\mathrm{\Ω}\left(x_i\right)}{\Σ}\exp \left(G_{{\mathrm{\σ}}_s}\right(\left|\right|x_i-x_j\left|\right|)G_{{\mathrm{\σ}}_r}\left(I\right(x_i)-I(x_j\left)\right))Depth\left(x_j\right)$

$W\left(x_i\right)=\underset{q\∈S}{\Σ}\exp \left(G_{{\mathrm{\σ}}_s}\right(\left|\right|x_i-x_j\left|\right|\left)G_{{\mathrm{\σ}}_r}\right(I\left(x_i\right)-I\left(x_j\right)))$

Wherein Depth (x _i) be input depth image, Ω (x _i) be with pixel x _icentered by neighborhood, I (x _i) be pixel x _ibrightness I component, x _jfor pixel x _iat neighborhood Ω (x _i) neighborhood territory pixel, W (x _i) be the normalized factor of filter parameter; || x _i-x _j|| be the space Euclidean distance of two pixels; I (x _i)-I (x _j) represent the brightness similarity of two pixels, pixel x _iand x _jvolume coordinate be respectively (x _ix, x _iy) and (x _jx, x _jy).Space weight coefficient with colourity weight coefficient be defined as:

$G_{{\mathrm{\σ}}_s}\left(\right||x_i-x_j|\left|\right)=\frac{{\[\sqrt{{(x_{jx}-x_{ix})}^2+{(x_{jy}-x_{iy})}^2}\]}^2}{2{{\mathrm{\σ}}_s}^2}$

$G_{{\mathrm{\σ}}_r}\left(I\right(x_i)-I(x_j\left)\right)=\frac{{|I\left(x_j\right)-I\left(x_i\right)|}^2}{2{{\mathrm{\σ}}_r}^2};$

Wherein σ _sfor variance, the σ of space weight _rfor the variance of colourity weight.

(7) obtain to input monocular image carry out the disparity map after depth information estimation.

Advantage of the present invention is, proposes a kind of monocular image depth information method of estimation based on contour acuity.Traditional monocular image depth information method of estimation needs to utilize figure image height, middle-level clue carries out the step such as learning training, image understanding, and algorithm is complicated.And our rule utilizes image low level clue, calculate simple.With it tradition utilize fuzzy message distinguish Object Depth method unlike, the present invention, when calculating fuzzy message, utilizes contour acuity information effectively to distinguish contour of object, thus obtains the contour of object with different depth, avoid the steps such as fuzzy diffusion, improve the experimental of method.This method utilizes the characteristic in gradient amplitude and contour acuity message reflection image with the contour of object of different depth, when carrying out estimation of Depth, not only consider the gradient amplitude at edge, add the spatial information at edge simultaneously, embody object edge smear out effect trend more fully.

Accompanying drawing explanation

Fig. 1 is this method process flow diagram.

Fig. 2 shows the definition of contour acuity.

The estimation of Depth relativeness schematic diagram of Fig. 3 for utilizing outline line to distribute.

Embodiment

Below in conjunction with accompanying drawing and instantiation, give detailed description to implementation process of the present invention.

(1) for input picture, adopt Canny edge detection algorithm to carry out rim detection, obtain the marginal point P={p of objects in images ₁, p ₂... p _n;

(2) carry out the definition of initial profile line according to priori concentration gradient model, define one group of parallel to each other and outline line that spacing is equal as initial profile line V={v ₀, v ₁..., v _m.Wherein, v (s)=[x (s), y (s)], x (s), y (s) they are the coordinate of the some s in initial profile line;

The gross energy of outline line is defined as

E _total＝w ₁E _edge+w ₂E _sharp+w ₃E _s+w ₄E _d

Wherein E _edgefor contour edge energy function, E _sharpcontour acuity energy function, E _sfor outline line characteristic energies function, E _dfor outline line distance energy function, w ₁, w ₂, w ₃, w ₄for each function weight controling parameters, w can be set to for general image ₁=0.25, w ₂=0.5, w ₃=0.125, w ₄=0.125.

(3) contour edge energy function E _edgefor image I (x, y) is along the gradient amplitude size of gradient direction, be defined as

$E_{edge}=\exp (-\frac{{\left|g\left(I\right(x,y\left)\right)\right|}^2}{a})$

Wherein, $g\left(I\right(x,y\left)\right)=\sqrt{{\left({\∂}_{x}I\right(x,y\left)\right)}^2+{\left({\∂}_{y}I\right(x,y\left)\right)}^2}$ That gradient amplitude is asked to I (x, y);

Wherein parameter a is control of right parameter.

(4) the gradient profile acutance at edge is utilized to represent the fog-level of profile in the method, so be defined by gradient profile acutance to solve the contour acuity energy function E obtained _sharpas the characterization value of soft edge degree.As shown in Figure 2, with edge pixel q in scheming ₀(x ₀, y ₀) as starting point, follow the trail of along the extrorse both sides of gradient direction, until the gradient amplitude at edge no longer changes, obtain path p (q like this ₀).Along one dimension path p (q ₀) the gradient amplitude curve that formed is called as gradient profile.The root mean square of application gradient profile variable variance defines contour acuity, is expressed as:

$\mathrm{\σ}\left(p\right(q_0\left)\right)=\sqrt{\underset{x\∈p\left(q_0\right)}{\Σ}\frac{g\left(q\right)}{G\left(q_0\right)}{d_c}^2(q,q_0)}$

$G\left(q_0\right)=\underset{s\∈p\left(q_0\right)}{\Σ}g\left(s\right)$

Here d _c(q, q ₀) be gradient profile mid point q and q ₀between length of curve, g (q) is the gradient amplitude at q point place, G (q ₀) in gradient profile gradient amplitude a little and, s is any point in gradient profile.

Acutance energy function E _sharpbe defined as:

$E_{sharp}=\exp (-\frac{\mathrm{\σ}{\left(p\right(q_0\left)\right)}^2}{b})$

Wherein parameter b is control of right parameter.

(5) outline line characteristic energies function E _scontrol Contour extraction as level and smooth item constraint, guarantee that it is level and smooth for following the tracks of the curve generated, and is absorbed in local extremum when preventing from solving simultaneously.

In definition outline line, point is N={n ₀, n ₁..., n _n, wherein n ₀for profile starting point, then outline line characteristic energies function is defined as

$E_s\left(n_i\right)=\frac{d_s(n_i,n_{i-1})}{c}=\frac{\left|n_{iy}-n_{i-1y}\right|}{c},i=0,1,...n$

Wherein d _s(n _i, n _i-1) represent some n _iwith a n _i-1between length of curve, parameter c is control of right parameter.

(6) outline line distance energy function E _dthe elastic restraint item of Contour extraction, it for retrain Contour extraction process outline line in the distance of each point, make Contour extraction curve exceed region of search, ensure that outline line does not intersect mutually.

$E_d\left(n_i\right)=\frac{d_e(n_i,n_0)}{d}=\frac{\left|n_{iy}-n_{0y}\right|}{d},i=0,1,...n$

Wherein d _e(n _i, n ₀) represent Parametric Representation point n _iwith a n ₀between vertical range, d is control of right parameter.

(7) to each initial profile line in image from the left side starting point of image, upgraded the position of each point by image base to top, according to outline line gross energy computing method in step (2 ~ 6), Contour extraction energy is calculated to each point;

(8) minimum value of energy function is solved.Search for the pixel in the pixel column adjacent with current outline point, search meets the marginal point P={p of energy function definition ₁, p ₂... p _nin there is the point of minimum energy value, select the pixel position with least energy as new point; To right side on the left of image, repeat search minimum value, obtains final Contour searching result, namely obtains target wheel profile V.

(9) using the concentration gradient progressively deepened from the bottom to top hypothesis as a priori assumption gradient former, according to Fig. 3, depth value filling is carried out to the region with different outline lines, calculate depth profile.

Outline line concentrates outline line V={v ₀, v ₁..., v _m, corresponding distribution depth value is.

$Depth\left(v_i\right)=255\×\frac{m-i}{m},i=0...m$

(10) original image information and gained deep image information is utilized to be optimized process to the depth image obtained:

${{\mathrm{Depth}}_f}^{bf}\left(x_i\right)=\frac{1}{W\left(x_i\right)}\underset{x_j\∈\mathrm{\Ω}\left(x_i\right)}{\Σ}\exp \left(G_{{\mathrm{\σ}}_s}\right(\left|\right|x_i-x_j\left|\right|\left)G_{{\mathrm{\σ}}_r}\right(I\left(x_i\right)-I\left(x_j\right)\left)\right)Depth\left(x_j\right)$

$W\left(x_i\right)=\underset{q\∈S}{\Σ}\exp \left(G_{{\mathrm{\σ}}_s}\right(\left|\right|x_i-x_j\left|\right|\left)G_{{\mathrm{\σ}}_r}\right(I\left(x_i\right)-I\left(x_j\right)\left)\right)$

Wherein Depth (x _i) be input depth image, Ω (x _i) be with pixel x _icentered by neighborhood, I (x _i) be pixel x _ibrightness I component, pixel x _jfor pixel x _iat neighborhood Ω (x _i) neighborhood territory pixel, W (x _i) be the normalized factor of filter parameter; || x _i-x _j|| be the space Euclidean distance of two pixels; I (x _i)-I (x _j) represent the brightness similarity of two pixels, pixel x _iand x _jvolume coordinate be respectively (x _ix, x _iy) and (x _jx, x _jy).Space weight coefficient with colourity weight coefficient be defined as:

$G_{{\mathrm{\σ}}_s}\left(\right||x_i-x_j|\left|\right)=\frac{{\[\sqrt{{(x_{jx}-x_{ix})}^2+{(x_{jy}-x_{iy})}^2}\]}^2}{2{{\mathrm{\σ}}_s}^2}$

$G_{{\mathrm{\σ}}_r}\left(I\right(x_i)-I(x_j\left)\right)=\frac{{|I\left(x_j\right)-I\left(x_i\right)|}^2}{2{{\mathrm{\σ}}_r}^2}$

Wherein σ _sfor variance, the σ of space weight _rfor the variance of colourity weight.

(11) obtain to input monocular image carry out the disparity map after depth information estimation.

Claims (3)

1., based on the monocular image depth information method of estimation of contour acuity, it is characterized in that, comprise the steps:

(1) for input picture, carry out rim detection, obtain the marginal point P={p of objects in images ₁, p ₂... p _n;

(2) carry out the definition of initial profile line according to priori concentration gradient model, define one group of parallel to each other and outline line that spacing is equal as initial profile line V={v ₀, v ₁... v _m; Wherein, v (s)=[x (s), y (s)], x (s), y (s) they are the coordinate of the some s in initial profile line;

The gross energy of outline line is defined as

E _total＝w ₁E _edge+w ₂E _sharp+w ₃E _s+w ₄E _d

Wherein E _edgefor contour edge energy function, represent the gradient amplitude of profile;

E _sharpcontour acuity energy function, represents the fog-level of profile;

E _sfor outline line characteristic energies function, for retraining the smoothness of profile;

E _dfor outline line distance energy function, region of search can not be exceeded for controlling profile aircraft pursuit course;

W ₁, w ₂, w ₃, w ₄for each function weight controling parameters;

(3) to each initial profile line in image from the left side starting point of image, upgraded the position of each point by image base to top, according to outline line gross energy definition in step (2), Contour extraction energy is calculated to each point;

(4) minimum value of energy function is solved; Search for the pixel in the pixel column adjacent with current outline point, search meets the marginal point P={p of energy function definition ₁, p ₂... p _nin there is the point of minimum energy value, select the pixel position with least energy as new point; To right side on the left of image, repeat search minimum value, obtains final Contour searching result, namely obtains target wheel profile V;

(5) using concentration gradient hypothesis as a priori assumption gradient former, depth value filling is carried out to the region with different outline lines, calculates depth profile;

Outline line concentrates outline line V={v ₀, v ₁..., v _m, for i=0 ... the distribution depth value Depth that m is corresponding is:

$Depth\left(v_i\right)=255\×\frac{m-i}{m}$

(6) original image half-tone information and gained deep image information is utilized to be optimized process to the depth image obtained:

${{\mathrm{Depth}}_f}^{bf}\left(x_i\right)=\frac{1}{W\left(x_i\right)}\underset{x_j\∈\mathrm{\Ω}\left(x_i\right)}{\Σ}\exp \left(G_{{\mathrm{\σ}}_s}\right(\left|\right|x_i-x_j\left|\right|\left)G_{{\mathrm{\σ}}_r}\right(I\left(x_i\right)-I\left(x_j\right)\left)\right)Depth\left(x_j\right)$

$W\left(x_i\right)=\underset{q\∈S}{\Σ}\exp \left(G_{{\mathrm{\σ}}_s}\right(\left|\right|x_i-x_j\left|\right|\left)G_{{\mathrm{\σ}}_r}\right(I\left(x_i\right)-I\left(x_j\right)\left)\right)$

Wherein Depth (x _i) be input depth image, Ω (x _i) be with pixel x _icentered by neighborhood, I (x _i) be pixel x _ibrightness I component, x _jfor pixel x _iat neighborhood Ω (x _i) neighborhood territory pixel, W (x _i) be the normalized factor of filter parameter; || x _i-x _j|| be the space Euclidean distance of two pixels; I (x _i)-I (x _j) represent the brightness similarity of two pixels, pixel x _iand x _jvolume coordinate be respectively (x _ix, x _iy) and (x _jx, x _jy); Space weight coefficient with colourity weight coefficient be defined as:

$G_{{\mathrm{\σ}}_s}\left(\right||x_i-x_j|\left|\right)=\frac{{\[\sqrt{{(x_{jx}-x_{ix})}^2+{(x_{jy}-x_{iy})}^2}\]}^2}{2{{\mathrm{\σ}}_s}^2}$

$G_{{\mathrm{\σ}}_r}\left(I\right(x_i)-I(x_j\left)\right)=\frac{|I\left(x_j\right)-I\left(x_i\right){|}^2}{2{{\mathrm{\σ}}_r}^2};$

Wherein σ _sfor variance, the σ of space weight _rfor the variance of colourity weight.

(7) obtain to input monocular image carry out the disparity map after depth information estimation.

2. the monocular image depth information method of estimation based on contour acuity according to claim 1, is characterized in that, described contour edge energy function E _edgefor image I (x, y) is along the gradient amplitude size of gradient direction, be defined as:

$E_{edge}=\exp (-\frac{\left|g\left(I\right(x,y)\right){|}^2}{a})$

Wherein, $g\left(I\right(x,y\left)\right)=\sqrt{{\left({\∂}_{x}I\right(x,y\left)\right)}^2+{\left({\∂}_{y}I\right(x,y\left)\right)}^2}$ That gradient amplitude is asked to I (x, y);

Wherein parameter a is control of right parameter.

3. the monocular image depth information method of estimation based on contour acuity according to claim 1, is characterized in that, described contour acuity energy function E _sharpobtain to solve gradient profile acutance, be defined as:

$E_{sharp}=\exp (-\frac{\mathrm{\σ}{\left(p\right(q_0\left)\right)}^2}{b});$

Wherein, gradient profile acutance σ (p (q ₀)) be the root mean square of gradient profile variable variance.Here gradient profile is edge pixel q in image ₀(x ₀, y ₀) as starting point, along the extrorse boundary tracking of gradient direction, until the gradient amplitude at edge no longer changes, institute obtains one dimension path p (q ₀) the gradient amplitude curve that formed; Its gradient profile acutance is defined as

$\mathrm{\σ}\left(p\right(q_0\left)\right)=\sqrt{\underset{x\∈p\left(q_0\right)}{\Σ}\frac{g\left(q\right)}{G\left(q_0\right)}{d_c}^2(q,q_0)};G\left(q_0\right)=\underset{s\∈p\left(q_0\right)}{\Σ}g\left(s\right)$

Wherein, d _c(q, q ₀) be gradient profile mid point q and q ₀between length of curve, g (q) is the gradient amplitude at q point place, G (q ₀) in gradient profile gradient amplitude a little and, s is any point in gradient profile, and parameter b is control of right parameter.