CN104463853A - Shadow detection and removal algorithm based on image segmentation - Google Patents

Abstract

The invention discloses a shadow detection and removal algorithm based on image segmentation, and relates to the technical field of image processing. The algorithm mainly solves the problems about how to judge whether shadows exist in a region or not or whether an edge is a shadow or not and how to remove corresponding shadows. The algorithm includes the steps that firstly, through texture and brightness characteristics, the probability that each pixel point is a shadow edge is estimated through the combination of local information and overall information; an image is segmented through a watershed algorithm and contour information shown in the specification; a shadow region and a non-shadow region in the image are segmented through a region fusion algorithm based on edges, and meanwhile the shadow region and the non-shadow region are segmented into multiple sub-regions respectively; then classifiers SVM are trained respectively for recognizing shadows; then, a shadow detection energy equation is solved through an image segmentation algorithm, and then a final shadow detection result is acquired; finally, according to the shadow detection result, shadow labels are calculated through an image matting algorithm, the shadow region is lightened through the marks, and illumination of the shadow region is restored.

Description

A kind of shadow Detection based on Iamge Segmentation and removal algorithm

Technical field

The present invention is specifically related to a kind of shadow Detection based on Iamge Segmentation and removes algorithm, removes, belong to technical field of image processing for image shadow Detection and penumbra region.

Background technology

Shadow Detection is one of study hotspot of image processing field always.Existence due to shade can increase the difficulty of object identification, Video segmentation scheduling algorithm, by shadow Detection out and its removal can be significantly improved the performance of the many algorithms of image processing field.

The method of a lot of shadow Detection is based on illumination model, or color model proposition.As the shadow detection method under color space HSI, the method utilizes the ratio of H and I in HSI space to detect shade.But the method is more suitable for taking photo by plane, figure or the obvious image of shade, then barely satisfactory for the shadow Detection effect under complex scene.But owing to can not judge that a pixel belongs to shade and still belongs to the darker non-shadow of color well, the method automatically detecting shade remains a major challenge of shadow Detection.

Up to now, there is a large amount of shadow Detection algorithms in image processing field.According to the difference of detection means, existing shadow Detection algorithm can be divided into based on the shadow Detection at edge and the shadow Detection based on study.

First shadow Detection algorithm based on edge needs the gray-scale map of scene cromogram and illumination invariant, and gray-scale map is obtained by the video camera calibrated, and detects shade (G.D.Finlayson, 2006) by the edge at the edge and original image that compare gray-scale map.The method has splendid treatment effect to high quality graphic, and general to normal image effect.Shadow Detection algorithm based on study considers the complicacy of Shadow edge, then from the angle of experience, the method for data-driven is incorporated into shadow Detection, as based on the Information Pull such as intensity of illumination, gradient condition random field (Conditional random field, CRF), judge whether a region is shade; Or utilize CRF to judge an edge whether to be shade.Although utilize the method for condition random field CRF can detect shade well under certain condition this, its training process is tediously long, and higher to the dependency degree of training set.Even to this day, under the impact of the factors such as illumination, object reflectance and shade geometric configuration, shadow Detection remains a very challenging problem.

Summary of the invention

For above-mentioned prior art, the object of the invention is to how a kind of shadow Detection based on Iamge Segmentation is provided and removes algorithm, it can split shade and non-hatched area effectively, can also detect better and remove in scene from shade and cast shadow.

In order to solve the problems of the technologies described above, the present invention adopts following technical scheme:

Based on shadow Detection and the removal algorithm of Iamge Segmentation, it is characterized in that, comprise the steps:

In conjunction with local message and global information, S100: utilize texture and brightness, estimates that each pixel is the probability of Shadow edge;

S200: use watershed algorithm to utilize profile information gPb to split image;

S300: utilize based on edge area merges algorithm by shadow region in image and non-hatched area separated, respectively shadow region and non-hatched area are divided into several subregions simultaneously; Then, utilize the information of single area information and matching area, train a sorter SVM respectively, shade is identified; Use figure cuts the energy equation that Algorithm for Solving detects shade subsequently, obtains final shadow detection result.

S400, according to the result of shadow Detection, uses stingy nomography computational shadowgraph label, utilizes the label obtained to light shadow region, recovers the illumination of shadow region, itself and the unshaded area light of surrounding is taken a picture same.

The main the following steps composition of described step S100:

S101: by calculated direction gradient information G (x, y, θ) build Shadow edge detecting device Pb, Shadow edge detecting device calculates the gradient information G (x, y, θ) of brightness and texture primitive two passages respectively, the construction method of Shadow edge detecting device is (x more in the picture, y), centered by, be that radius draws a circle with r, this circle is divided into two semicircles by the diameter that direction is θ;

S102: by the χ between the histogram that calculates these two semicircles ²distance obtains direction gradient G:

${\mathrm{\χ}}^2(g,h)=\frac{1}{2}\underset{i}{\mathrm{\Σ}}\frac{{(g\left(i\right)-h\left(i\right))}^2}{g\left(i\right)+h\left(i\right)}---\left(1\right)$

Wherein, g and h represents two semicircles, the value in i representative image codomain;

S103: combined by the detecting device Pb calculated, obtains different scale, local message (local cues) mPb that different combination of channels is got up:

$\mathrm{mPb}(x,\mathrm{y\θ})=\underset{s}{\mathrm{\Σ}}\underset{i}{\mathrm{\Σ}}{\mathrm{\α}}_{i,s}{G}_{i,\mathrm{\σ}(i,s)}(x,y,\mathrm{\θ})---\left(2\right)$

Wherein, behalf radius of a circle, i representative feature passage (brightness, texture primitive); G _{i, σ (i, s)}difference between two semicircles that (x, y, θ) compares centered by (x, y), σ (i, s) is radius size, θ is radial direction; α _i,sit is then the weight representated by each gradient information;

On each point, choose the mPb value of maximal value as this point of gradient information G, this value represents final local message:

$\mathrm{mPb}(x,y,\mathrm{\θ})=\underset{\mathrm{\θ}}{\max }\left\{\mathrm{mPb}(x,y,\mathrm{\θ})\right\}---\left(3\right)$

Step S104: build a sparse matrix with mPb, by calculating eigenwert and the proper vector of this matrix, obtains required global information; Sparse matrix is in the region of radius r=5 pixel, each pixel link is got up:

$W_{\mathrm{ij}}=\exp (-\underset{p\∈\stackrel{\‾}{\mathrm{ij}}}{\max }\frac{\left\{\mathrm{mPb}\left(p\right)\right\}}{\mathrm{\ρ}})---\left(4\right)$

Wherein, representative be contact between i and j, ρ=0.1, then defines D _ii=∑ _jw _ij, pass through

(D-W)v＝λDv (5)

Calculate proper vector { v ₀, v ₁..., v _n, and eigenwert 0=λ ₀≤ λ ₁≤ ... ≤ λ _n.

Step S105: regard each proper vector in step S104 as a sub-picture, by calculating the gaussian filtering under different directions, obtains directional information then, the directional information linear superposition under different characteristic vector is got up to obtain global information sPb:

$\mathrm{sPb}(x,y,\mathrm{\θ})=\underset{k=1}{\overset{n}{\mathrm{\Σ}}}\frac{1}{\sqrt{{\mathrm{\λ}}_k}}\·{\▿}_{\mathrm{\θ}}{v}_k(x,y)---\left(6\right)$

Step S106: local message mPb and global information sPb is organically combined analysis chart as profile information gPb:

$\mathrm{gPb}(x,y,\mathrm{\θ})=\underset{s}{\mathrm{\Σ}}\underset{i}{\mathrm{\Σ}}\{{\mathrm{\β}}_{i,s}{G}_{i,\mathrm{\σ}}(x,y,\mathrm{\θ})+\mathrm{\γ}\·\mathrm{sPb}(x,y,\mathrm{\θ})\}---\left(7\right)$

Wherein, β _i,sthe coefficient of mPb and sPb is represented respectively with γ.

The main the following steps composition of described step S200:

Step S201: estimate any point (x, y) in image is the probability of profile on the θ of direction, obtains the maximal value that this dot profile detects:

$E(x,y)=\underset{\mathrm{\θ}}{\max }E(x,y,\mathrm{\θ})---\left(8\right)$

Step S202: utilize mathematical morphology, calculates each region with the minimum value of E (x, y) in region for " retaining basin ", and each " retaining basin " corresponding region, is designated as P ₀; Liang Ge retaining basin intersection is " watershed divide ", is designated as K ₀;

Step S203: watershed algorithm can produce over-segmentation problem, the place that should not be limit by this is labeled as watershed divide, utilizes region merging algorithm to solve over-segmentation problem;

Described region merging algorithm is as follows: define a non-directed graph G=(P ₀, K ₀, W (K ₀), E (P ₀)), wherein W (K ₀) represent and the weights of every bar watershed divide obtained, E (P by the gross energy that watershed divide is put divided by the number that watershed divide is put ₀) representing the energy value of each retaining basin, the zero energy in each basin is zero, W (K ₀) describe diversity between adjacent two regions; By watershed divide according to its weights, ascending stored in queue.

Region merging algorithm comprise the following steps into:

One, the limit C finding weight minimum ^*=argminW (C)

Assuming that R ₁and R ₂by limit C ^*segmentation, and R=R ₁∪ R ₂if, min{E (R ₁), E (R ₂) ≠ 0, judge whether to merge, merging condition is: then W (K ₀)≤τ min{E (R ₁), E (R ₂) (9)

Or min{E (R ₁), E (R ₂)=0 (10),

Wherein τ is a constant;

If two merge, then upgrade E (R), P ₀and K ₀, E (R), P ₀and K ₀update method be:

E(R)＝max{E(R ₁),E(R ₂),W(C ^*)} (11)

P ₀←P ₀\{R ₁,R ₂}∪R (12)

K ₀←K ₀\{C ^*} (13)

More a step ground, adjusts merging condition by adjustment τ, enters the size that face controls final area, and τ is larger, and the final region area merged is larger.

Obtain final shadow detection result by separating following energy equation in described step S300, energy equation cuts algorithm by figure to solve:

$\begin{array}{c}\hat{y}=\arg \underset{y}{\min }\underset{k}{\mathrm{\Σ}}{\cos t}_k^{\mathrm{unary}}\left(y_k\right)\\ +{\mathrm{\α}}_2\underset{\{i,j\}\∈E_{\mathrm{same}}}{\mathrm{\Σ}}{c}_{\mathrm{ij}}^{\mathrm{same}}1(y_i\≠y_j)\end{array}---\left(15\right)$

Meanwhile,

$\begin{array}{c}{\cos t}_k^{\mathrm{unary}}\left(y_k\right)=-c_k^{\mathrm{shadow}}{y}_k\\ -{\mathrm{\α}}_1\underset{\{i=k,j\}\∈E_{\mathrm{diff}}}{\mathrm{\Σ}}{c}_{\mathrm{ij}}^{\mathrm{diff}}{y}_k\\ +{\mathrm{\α}}_1\underset{\{i,j=k\}\∈E_{\mathrm{diff}}}{\mathrm{\Σ}}{c}_{\mathrm{ij}}^{\mathrm{diff}}{y}_k\end{array}---\left(16\right)$

Wherein, represent that Region Matching sorter is for two area light photograph estimations together, represent Region Matching sorter for two area light according to different estimations, the estimation of whether be single region classifier to single regions be shade, { i, j} ∈ E _samerepresent two regions of same light photograph, { i, j} ∈ E _diffrepresent two regions of different light; Y={-1,1} ⁿ, when for 1 time represent that this region is shadow region.

Further be described as described step S400: use and scratch nomography computational shadowgraph label, this algorithm thinks an image I _ican by prospect F _iwith background B _imix, its formula is as follows:

I _i＝k _iF _i+(1-k _i)B _i(18)

＝k _i(L _dR _i+L _eR _i)+(1-k _i)L _eR _i(19)

Wherein, L _ddirect light, L _esurround lighting, k _ishade label, R _iit is the reflection coefficient of an i.

Prospect is labeled as non-shadow, and context marker is shade, obtains k by the minimum value calculating following energy equation _isize, k _irepresent the label of some i, obtained by (20), k is k _ithe vector of composition, obtains k and just obtains k _i.

$E\left(k\right)=k^T\mathrm{Lk}+\mathrm{\λ}{(k-\hat{k})}^{T}D(k-\hat{k})---\left(20\right)$

K ^tbe the transposition of k, k is k _ithe vector of composition, λ is a very large numerical value, specifically determines by putting into practice. the label y obtained by the energy equation of step S300 _kthe vector of composition, in the value of each element be exactly the y obtained in formula _k, each element value is not 0 is here exactly 1.Please note: this formula is exactly that the k calculated is a vector, the label of each pixel of each element representation in vector in order to calculate k; But the span of the value of each element becomes [0,1], and that is the label of one part of pixel has been become the value of 0 to 1 interval range from 1 or 0; Label is still just expression shadow region and the non-hatched area of 0 or 1, label value expression penumbra region between zero and one.

Wherein, L is the Laplacian Matrix of stingy figure, and D (i, i) is a diagonal matrix, and D (i, i)=1 represents that pixel i is the edge of shadow region, and D (i, i)=0 item represents other institute a little;

Utilize the label obtained to light shadow region, recover the illumination of shadow region:

According to shadow model, if a pixel is lit, then

$\begin{array}{cc}{I}_i^{\mathrm{shadow}-\mathrm{free}}=(L_d+L_e)R_i& \left(21\right)\\ =(k_i\·L_d+L_e)R_i\frac{(L_d+L_e)}{(k_i\·L_d+L_e)}& \left(22\right)\\ =\frac{r+1}{k_{i}r+1}{I}_i& \left(23\right)\end{array}$

Wherein, r=L _d/ L _edirect light L _dwith surround lighting L _eratio, I _irepresent pixel i value originally, so calculating r just can by shadow removal;

Known

I _i＝(k _i·L _d+L _e)R _i(24)

I _j＝(k _j·L _d+L _e)R _j(25)

If the reflection coefficient of two pixels is identical, i.e. R _i=R _j, then

$r=\frac{I_j-I_i}{k_j{I}_i-k_i{I}_j}$

The object of formula (24) and (25) calculates r, and the thought calculating this value is exactly, and finding material, identical (namely reflection coefficient is identical, R _i=R _j), illumination is different from (L _dand L _ethe same, but shade label k is different) two somes i, j, utilization mathematical relation between the two just can obtain r.I _jjust represent the pixel value of such some j.

Compared with prior art, the present invention has following beneficial effect:

One, utilize identical shadow character segmentation image and detection shade organically to be combined, split the shade in image and non-hatched area well, make Iamge Segmentation more accurate, more accurate, better effects if is detected in shadow region;

Two, the profile of shade is more intactly detected, the antijamming capability of detection algorithm provided by the invention to complex background is better, the image removed not only saves good textural characteristics in inside, shadow region, and comparatively smooth at Shadow edge place, show that algorithm achieves illumination transition effect well in penumbra region herein.

Three, according to profile information, a kind of algorithm of region merging technique is proposed, and the parameter that can be merged by control area, control the size of combined region easily, these advantages improve the effect of shadow removal jointly.

Accompanying drawing explanation

Fig. 1 is algorithm flow chart of the present invention;

A () is original image; B () is the image after segmentation; C shade that () is detected for single area identifier and non-hatched area, white represents shadow region; D shade that () is detected for matching area recognizer and non-hatched area; (e) shade label k for calculating through stingy nomography _igray-scale map; F () is the image after shadow removal.

Fig. 2 is region merging algorithm process flow diagram;

Fig. 3 is region matching graph, and in figure, red line represents the matching area of same light photograph, and blue line represents the matching area of different light.

Embodiment

Below in conjunction with the drawings and the specific embodiments, the invention will be further described.

The present invention, according to the feature of shade, is merged Shadow edge and is detected and shadow region detection, segmentation shade and non-hatched area.Algorithm is divided into two steps: Shadow edge detects and Shadow segmentation.

Embodiment Shadow edge detects

First, Shadow edge detecting device Pb is built by calculated direction gradient information G (x, y, θ).The method built is more in the picture centered by (x, y), and be that radius draws a circle with r, this circle is divided into two semicircles by the diameter that direction is θ.Finally by calculate these two semicircles histogram between χ ²distance obtains direction gradient G:

${\mathrm{\χ}}^2(g,h)=\frac{1}{2}\underset{i}{\mathrm{\Σ}}\frac{{(g\left(i\right)-h\left(i\right))}^2}{g\left(i\right)+h\left(i\right)}---\left(1\right)$

Wherein, g and h represents two semicircles, the value in i representative image codomain.

Detecting device calculates the gradient information G (x, y, θ) of brightness and texture primitive two passages respectively.The account form of texture primitive is that the odd even Gaussian filter in selection 16 directions carries out filtering, and the texture primitive through k average algorithm cluster is divided into 32 groups.

Secondly, the detecting device Pb calculated is combined, obtains different scale, local message (local cues) mPb that different combination of channels is got up:

$\mathrm{mPb}(x,y,\mathrm{\θ})=\underset{s}{\mathrm{\Σ}}\underset{i}{\mathrm{\Σ}}{\mathrm{\α}}_{i,s}{G}_{i,\mathrm{\σ}(i,s)}(x,y,\mathrm{\θ})---\left(2\right)$

Wherein, behalf radius of a circle, i representative feature passage (brightness, texture primitive); G _{i, σ (i, s)}difference between two semicircles that (x, y, θ) compares centered by (x, y), σ (i, s) is radius size, θ is radial direction, α _i,sit is then the weight representated by each gradient information.

On each point, choose the mPb value of maximal value as this point of gradient information G, the final local message that this value represents:

$\mathrm{mPb}(x,y,\mathrm{\θ})=\underset{\mathrm{\θ}}{\max }\left\{\mathrm{mPb}(x,y,\mathrm{\θ})\right\}---\left(3\right)$

Then, in order to extract profile information better, the method using standard normalization figure to cut obtains global information (global cues).Building a sparse matrix with mPb, by calculating eigenwert and the proper vector of this matrix, obtaining required global information.Sparse matrix is in the region of radius r=5 pixel, each pixel link is got up:

$W_{\mathrm{ij}}=\exp (-\underset{p\∈\stackrel{\‾}{\mathrm{ij}}}{\max }\frac{\left\{\mathrm{mPb}\left(p\right)\right\}}{\mathrm{\ρ}})---\left(4\right)$

Wherein, representative be contact between i and j, ρ=0.1, then defines D _ii=∑ _jw _ij, pass through

(D-W)v＝λDv (5)

Calculate proper vector { v ₀, v ₁..., v _n, and eigenwert 0=λ ₀≤ λ ₁≤ ... ≤ λ _n.

Although normalization figure cuts can not split image well, the profile of image can be reflected well.So, regarding each proper vector as a sub-picture, by calculating the gaussian filtering under different directions, obtaining directional information then, the directional information linear superposition under different characteristic vector is got up:

$\mathrm{sPb}(x,y,\mathrm{\θ})=\underset{k=1}{\overset{n}{\mathrm{\Σ}}}\frac{1}{\sqrt{{\mathrm{\λ}}_k}}\·{\▿}_{\mathrm{\θ}}{v}_k(x,y)---\left(6\right)$

MPb and sPb represents different marginal informations respectively.MPb covers marginate information, is local message; SPb then illustrates the information on limit the most outstanding, is global information.Just because of this, both organically being combined can analysis chart be as profile information effectively, and the present invention is defined as gPb:

$\mathrm{gPb}(x,y,\mathrm{\θ})=\underset{s}{\mathrm{\Σ}}\underset{i}{\mathrm{\Σ}}\{{\mathrm{\β}}_{i,s}{G}_{i,\mathrm{\σ}}(x,y,\mathrm{\θ})+\mathrm{\γ}\·\mathrm{sPb}(x,y,\mathrm{\θ})\}---\left(7\right)$

In formula, β _i,sthe coefficient of mPb and sPb is represented respectively with γ.

Embodiment Iamge Segmentation

GPb effectively can represent profile, but these profiles neither be complete totally enclosed, therefore can not be used as segmentation image.In order to split image better, watershed algorithm is used to utilize profile information gPb to carry out region segmentation herein.

In order to better describe watershed algorithm, first consider any one contour detecting device E (x, y, θ).It is the probability of profile that this detecting device can estimate any point (x, y) in image on the θ of direction, and this value is larger, represents that this point is that the probability of profile is higher.Each point is asked to the maximal value of contour detecting:

$E(x,y)=\underset{\mathrm{\θ}}{\max }E(x,y,\mathrm{\θ})---\left(8\right)$

Then, utilize mathematical morphology, calculate each region with the minimum value of E (x, y) in region for " retaining basin ".Each " retaining basin " corresponding region, is designated as P ₀; Liang Ge retaining basin intersection is " watershed divide ", is designated as K ₀.But watershed algorithm can produce over-segmentation problem, should not that the place on limit is labeled as watershed divide by this.In order to address this problem, utilize a kind of new region merging algorithm.The prerequisite of this algorithm is: watershed algorithm is bound to produce undue problem, and namely initial each retaining basin needs to merge.

Region merging algorithm is described below: define a non-directed graph G=(P ₀, K ₀, W (K ₀), E (P ₀)), wherein W (K ₀) represent the weights of every bar watershed divide, sentence by the gross energy that watershed divide is put number that watershed divide is put and obtain; E (P ₀) representing the energy value of each retaining basin, the zero energy in each basin is zero.While it is noted that in the drawings, every bar splits two regions all just, W (K ₀) describe diversity between adjacent two regions.By watershed divide according to its weights, ascending stored in queue, the process flow diagram of algorithm as shown in Figure 2.

Assuming that R ₁and R ₂by limit C ^*segmentation, and R=R ₁∪ R ₂.The condition merged is: if min{E is (R ₁), E (R ₂) ≠ 0, then

W(K ₀)≤τ·min{E(R ₁),E(R ₂)} (9)

Or min{E (R ₁), E (R ₂)=0 (10)

Wherein, τ represents a constant.Just can adjust merging condition by adjustment τ, thus control the size of final area, τ is larger, and the final region area merged is larger.

E (R), P ₀and K ₀update method be:

E(R)＝max{E(R ₁),E(R ₂),W(C ^*)} (11)

P ₀←P ₀\{R ₁,R ₂}∪R (12)

K ₀←K ₀\{C ^*} (13)

Embodiment shadow Detection

Single region recognition: train a SVM classifier to judge that single region is the probability of shade; Training set manual markings has gone out dash area, and sorter utilizes the brightness of shade and texture primitive to classify, and exports as representing that region is the probability of shade.

Matching area identification: whether be shadow region, region that should be similar to its texture compares if detecting a region.If with brightness similar, then under the two is in same intensity of illumination; And if brightness different, then assert that the darker region of brightness is shade.

The present invention utilizes sorter to train four features to judge shadow region:

1. the χ of brightness and texture primitive ²distance;

2. average RGB ratio

When the matching area contrasted is identical material, the triple channel value of non-hatched area is higher.Computing formula is as follows:

${\mathrm{\ρ}}_R=\frac{R_{\mathrm{avg}1}}{R_{\mathrm{avg}2}}{\mathrm{\ρ}}_G=\frac{G_{\mathrm{avg}1}}{G_{\mathrm{avg}2}}{\mathrm{\ρ}}_B=\frac{B_{\mathrm{avg}1}}{B_{\mathrm{avg}2}}---\left(14\right)$

Wherein, R _avg1represent the mean value of first region R passage.

3. color degree of registration

Shade/the non-shadow of identical material keeps alignment to the color on rgb space.This parameter is by calculating ρ _r/ ρ _gand ρ _g/ ρ _bobtain.

4. region normalized cumulant

Whether whether the material due to matching area not necessarily identical and adjacent relevant, so this parameter is also classified as one of training characteristics by this.This value be by matching area between Euclidean distance calculated.

Build following energy equation, use figure cuts algorithm and obtains final shadow detection result:

$\hat{y}=\arg \underset{y}{\min }\underset{k}{\mathrm{\Σ}}{\cos t}_k^{\mathrm{unary}}\left(y_k\right)+{\mathrm{\α}}_2\underset{\{i,j\}\∈E_{\mathrm{same}}}{\mathrm{\Σ}}{c}_{\mathrm{ij}}^{\mathrm{same}}1(y_i\≠y_j)---\left(15\right)$

Meanwhile,

$\begin{array}{c}{\cos t}_k^{\mathrm{unary}}\left(y_k\right)=-c_k^{\mathrm{shadow}}{y}_k\\ -{\mathrm{\α}}_1\underset{\{i=k,j\}\∈E_{\mathrm{diff}}}{\mathrm{\Σ}}{c}_{\mathrm{ij}}^{\mathrm{diff}}{y}_k\\ +{\mathrm{\α}}_1\underset{\{i,j=k\}\∈E_{\mathrm{diff}}}{\mathrm{\Σ}}{c}_{\mathrm{ij}}^{\mathrm{diff}}{y}_k\end{array}---\left(16\right)$

Wherein, represent that Region Matching sorter is for two area light photograph estimations together, represent Region Matching sorter for two area light according to different estimations, the estimation of whether be single region classifier to single regions be shade; { i, j} ∈ E _samerepresent two regions of same light photograph, { i, j} ∈ E _di ^frepresent two regions of different light; Y={-1,1} ⁿ, when for 1 time represent that this region is shadow region.

Embodiment shadow removal

Removing for effectively carrying out shade, a suitable shadow model must be set up.The illumination of shadow model is determined jointly by direct light and surround lighting:

I _i＝(k _i·L _d+L _e)R _i(17)

For pixel i, k _irepresent shade label.Work as k _iwhen=0, represent that this pixel is in shadow region; Work as k _iwhen=1, represent that this pixel is in non-hatched area; Work as 0<k _i<1, represents that this pixel is in penumbra region.

Although shadow Detection imparts a label (0 or 1) for each pixel, however in actual scene Shadow edge by non-shadow to shade progressively transition.In order to remove Shadow edge better, use herein and scratching nomography calculating penumbra region:

I _i＝k _iF _i+(1-k _i)B _i(18)

＝k _i(L _dR _i+L _eR _i)+(1-k _i)L _eR _i(19)

Prospect is labeled as non-shadow, and context marker is shade.K is obtained by the minimum value calculating following energy equation _isize:

$E\left(k\right)=k^T\mathrm{Lk}+\mathrm{\&lambda;}{(k-\hat{k})}^{T}D(k-\hat{k})---\left(20\right)$

Wherein, L is the Laplacian Matrix of stingy figure, and D (i, i) is a diagonal matrix.In experiment, D (i, i)=1 represents that pixel i is the edge of shadow region, and D (i, i)=0 item represents other institute a little.

According to shadow model, if a pixel is lit, then

$\begin{array}{cc}{I}_i^{\mathrm{shadow}-\mathrm{free}}=(L_d+L_e)R_i& \left(21\right)\\ =(k_i\&CenterDot;L_d+L_e)R_i\frac{(L_d+L_e)}{(k_i\&CenterDot;L_d+L_e)}& \left(22\right)\\ =\frac{r+1}{k_{i}r+1}{I}_i& \left(23\right)\end{array}$

Wherein, r=L _d/ L _edirect light L _dwith surround lighting L _eratio, I _irepresent pixel i value originally.So calculating r just can by shadow removal.

Known

I _i＝(k _i·L _d+L _e)R _i(24)

I _j＝(k _j·L _d+L _e)R _j(25)

If the reflection coefficient of two pixels is identical, i.e. R _i=R _j, then

$r=\frac{I_j-I_i}{k_j{I}_i-k_i{I}_j}$

Choose the most close pixel at adjacent shade and non-hatched area both sides, then calculate r value according to formula (24).

Above content is only some detailed descriptions carried out the present invention in conjunction with concrete scheme, can not assert that the concrete enforcement of invention is only limited to these explanations.Concerning general technical staff of the technical field of the invention, do not departing under concept thereof of the present invention, simple deduction and replacement can also made, all should be considered as in protection scope of the present invention.

Claims (7)

1., based on shadow Detection and the removal algorithm of Iamge Segmentation, it is characterized in that, comprise the steps:

In conjunction with local message and global information, S100: utilize texture and brightness, estimates that each pixel is the probability of Shadow edge;

S200: use watershed algorithm to utilize profile information gPb to split image;

S300: utilize based on edge area merges algorithm by shadow region in image and non-hatched area separated, respectively shadow region and non-hatched area are divided into several subregions simultaneously; Then, utilize the information of single area information and matching area, train a sorter SVM respectively, shade is identified; Use figure cuts the energy equation that Algorithm for Solving detects shade subsequently, obtains final shadow detection result.

S400: according to the result of shadow Detection, uses stingy nomography computational shadowgraph label, utilizes the label obtained to light shadow region, recovers the illumination of shadow region, itself and the unshaded area light of surrounding is taken a picture same.

2. the shadow Detection based on Iamge Segmentation according to claim 1 and removal algorithm, is characterized in that, the main the following steps composition of described step S100:

S101: by calculated direction gradient information G (x, y, θ) build Shadow edge detecting device Pb, Shadow edge detecting device calculates the gradient information G (x, y, θ) of brightness and texture primitive two passages respectively, the construction method of Shadow edge detecting device is (x more in the picture, y), centered by, be that radius draws a circle with r, this circle is divided into two semicircles by the diameter that direction is θ;

S102: by the χ between the histogram that calculates these two semicircles ²distance obtains direction gradient G:

Wherein, g and h represents two semicircles, the value in i representative image codomain;

S103: combined by the detecting device Pb calculated, obtains different scale, local message (local cues) mPb that different combination of channels is got up:

Wherein, behalf radius of a circle, i representative feature passage (brightness, texture primitive); G _{i, σ (i, s)}difference between two semicircles that (x, y, θ) compares centered by (x, y), σ (i, s) is radius size, θ is radial direction; α _{i, s}it is then the weight representated by each gradient information;

On each point, choose the mPb value of maximal value as this point of gradient information G, this value represents final local message:

mPb(x，y，θ)＝max _θ{mPb(x，y，θ)} (3)；

Step S104: build a sparse matrix with mPb, by calculating eigenwert and the proper vector of this matrix, obtains required global information; Sparse matrix is in the region of radius r=5 pixel, each pixel link is got up:

Wherein, representative be contact between i and j, ρ=0.1, then defines D _ii=∑ _jw _ij, pass through

(D-W)v＝λDv (5)

Calculate proper vector { v ₀, v ₁..., v _n, and eigenwert 0=λ ₀≤ λ ₁≤ ... ≤ λ _n;

Step S105: regard each proper vector in step S104 as a sub-picture, by calculating the gaussian filtering under different directions, obtains directional information then, the directional information linear superposition under different characteristic vector is got up to obtain global information sPb:

Step S106: local message mPb and global information sPb is organically combined analysis chart as profile information gPb:

Wherein, β _{i, s}the coefficient of mPb and sPb is represented respectively with γ.

3. the shadow Detection based on Iamge Segmentation according to claim 1 and removal algorithm, is characterized in that, the main the following steps composition of described step S200:

Step S201: estimate any point (x, y) in image is the probability of profile on the θ of direction, obtains the maximal value that this dot profile detects:

Step S202: utilize mathematical morphology, calculates each region with the minimum value of E (x, y) in region for " retaining basin ", and each " retaining basin " corresponding region, is designated as P ₀; Liang Ge retaining basin intersection is " watershed divide ", is designated as K ₀;

Step S203: watershed algorithm can produce over-segmentation problem, the place that should not be limit by this is labeled as watershed divide, utilizes region merging algorithm to solve over-segmentation problem;

Described region merging algorithm is as follows: define a non-directed graph G=(P ₀, K ₀, W (K ₀), E (P ₀)), wherein W (K ₀) represent and the weights of every bar watershed divide obtained, E (P by the gross energy that watershed divide is put divided by the number that watershed divide is put ₀) representing the energy value of each retaining basin, the zero energy in each basin is zero, W (K ₀) describe diversity between adjacent two regions; By watershed divide according to its weights, ascending stored in queue.

4. the shadow Detection based on Iamge Segmentation according to claim 3 with remove algorithm, it is characterized in that, region merging algorithm comprise the following steps into:

One, the limit C finding weight minimum ^*=argminW (C)

Assuming that R ₁and R ₂by limit C ^*segmentation, and R=R ₁∪ R ₂if, min{E (R ₁), E (R ₂) ≠ 0, judge whether to merge, merging condition is: then W (K ₀)≤τ min{E (R ₁), E (R ₂) (9)

Or min{E (R ₁), E (R ₂)=0 (10),

Wherein τ is a constant;

If two merge, then upgrade E (R), P ₀and K ₀, E (R), P ₀and K ₀update method be:

E(R)＝max{E(R ₁)，E(R ₂)，W(C ^*)} (11)

P ₀←P ₀\{R ₁，R ₂}∪R (12)

K ₀←K ₀\{C ^*} (13)。

5. the shadow Detection based on Iamge Segmentation according to claim 1 and removal algorithm, is characterized in that, obtain final shadow detection result by separating following energy equation in described step S300, energy equation cuts algorithm by figure to solve:

Meanwhile,

Wherein, represent that Region Matching sorter is for two area light photograph estimations together, represent Region Matching sorter for two area light according to different estimations, the estimation of whether be single region classifier to single regions be shade, { i, j} ∈ E _samerepresent two regions of same light photograph, { i, j} ∈ E _diffrepresent two regions of different light; Y={-1,1} ⁿ, when for 1 time represent that this region is shadow region.

6. the shadow Detection based on Iamge Segmentation according to claim 1 with remove algorithm, it is characterized in that, use in described step S400 and scratch nomography computational shadowgraph label, this algorithm thinks an image I _ican by prospect F _iwith background B _imix, its formula is as follows:

I _i＝k _iF _i+(1-k _i)B _i(18)

＝k _i(L _dR _i+L _eR _i)+(1-k _i)L _eR _i(19)

Wherein, L _ddirect light, L _esurround lighting, k _ishade label, R _iit is the reflection coefficient of an i.

Prospect is labeled as non-shadow, and context marker is shade, obtains k by the minimum value calculating following energy equation _isize, k _irepresent the label of some i, obtained by (20), k is k _ithe vector of composition, obtains k and just obtains k _i.

K ^tbe the transposition of k, k is k _ithe vector of composition, λ is a very large numerical value, specifically determines by putting into practice. the label y obtained by the energy equation of step S300 _kthe vector of composition, in the value of each element be exactly the y obtained in formula _k, each element value is not 0 is here exactly 1.Please note: this formula is exactly that the k calculated is a vector, the label of each pixel of each element representation in vector in order to calculate k; But the span of the value of each element becomes [0,1], and that is the label of one part of pixel has been become the value of 0 to 1 interval range from 1 or 0; Label is still just expression shadow region and the non-hatched area of 0 or 1, label value expression penumbra region between zero and one; L is the Laplacian Matrix of stingy figure, and D (i, i) is a diagonal matrix, and D (i, i)=1 represents that pixel i is the edge of shadow region, and D (i, i)=0 item represents other institute a little.

7. the shadow Detection based on Iamge Segmentation according to claim 6 and removal algorithm, is characterized in that, utilize the label obtained to light shadow region, recover the illumination of shadow region:

According to shadow model, if a pixel is lit, then

Wherein, r=L _d/ L _edirect light L _dwith surround lighting L _eratio, I _irepresent pixel i value originally, so calculating r just can by shadow removal;

Known

I _i＝(k _i·L _d+L _e)R _i(24)

I _j＝(k _j·L _d+L _e)R _j(25)

If the reflection coefficient of two pixels is identical, i.e. R _i=R _j, then