CN1224242C - Method for fast picking up picture with any image as background in digital image process - Google Patents

Abstract

The present invention discloses a method for quickly picking up images in digital image processing by using any image as a background, which comprises the steps: 1) region segmentation is manually carried out for images; each image is divided into three parts: a foreground region, an unknown region and a background region; 2) according to the divided regions, the color of any point c in the unknown region is set as C, and the foreground color component F and the background color component B of the point c are calculated; 3) the values of F and B are adjusted by the measure of color adjustment according to the estimated foreground color component F and the background color component B; finally, the alpha value of the point c is estimated. Images can be picked up and processed effectively by the method for picking up the images; simultaneously, the speed of picking up images is greatly improved, the calculation cost is low, and the present invention has good practicability value.

Description

In the Digital Image Processing be with the arbitrary image background scratch drawing method fast

Technical field

The present invention relates in a kind of Digital Image Processing with the arbitrary image be background scratch drawing method fast.

Background technology

Stingy diagram technology is a kind of a kind of image processing techniques that the part of prospect in the arbitrary image is separated from background.It has extensive and deep application at aspects such as film and television special efficacy stunt making.According to unrestricted to having of image background, stingy diagram technology roughly can be divided into two big classes:

Can be divided into the stingy figure of blue screen (blue screen matting) and difference sectional drawing (difference matting) again to display foreground or the restricted class of background.The stingy diagram technology of blue screen has certain limitation to the background of image, background is constant color normally, is generally blueness or green, or prepares many images that identical prospect is arranged, utilize the difference of the background color in these images, reach the purpose of the prospect that accurately takes.The difference sectional drawing Technology Need is prepared a background image in addition, estimates the alpha value by the color distortion that compares respective point among background image and the former figure.

To the unrestricted class natural image matting of image background (natural image matting).It does not do requirement to the background of image, and only needs an image.

Scratching the figure problem can be defined as: to any point c on the given image, and foreground F that the color C that asks c to order is contained and alpha value α.The difficulty of scratching the figure problem is that to any point c on the image, separating of its F and α is not unique, and we will find out the most rational separating from countless right separating.

The stingy diagram technology of blue screen is simple, and amount of calculation is little, and it is effective to scratch figure.But it has its fatal weakness, and being exactly it has certain limitation to the color of the background of image.Context request is blue or green generally speaking, the background that needs a people to hold the piece blueness when using this technology goes from place to place, it generally also requires the RGB component of the color of prospect to distribute in certain ratio simultaneously, and these make blue screen scratch diagram technology and brought very big inconvenience in the middle of concrete the application.

The natural image matting technology has Knockout method, Ruzon﹠amp; Tomasi method, Hillman method and Chuang method.Natural image matting generally can be divided into three steps:

1. Region Segmentation.Generally speaking to be divided into the master by hand.Because the levels of precision of Region Segmentation has very big influence to scratching the figure effect.Manual cut zone can have better accuracy, about the Region Segmentation of general image 2-3 consuming time minute.

2. prospect and background color are estimated.The Knockout method utilizes the weighted average of the point of adjacent domain to estimate that method is simple, and amount of calculation is less; Ruzon﹠amp; Tomasi method, Hillman method and Chuang method have all been utilized statistical rule, the method complexity, and amount of calculation is big.

3.Alpha value is estimated.Prospect that utilization estimates and background color are estimated the alpha value.In known method, except the Knockout method, all the other methods all do not adjust to prospect and background color.And the adjustment that the Knockout method is done is too coarse, causes its stingy figure poor effect.

Summary of the invention

The purpose of this invention is to provide in a kind of Digital Image Processing with the arbitrary image be background scratch drawing method fast.

In the Digital Image Processing be with the arbitrary image background to scratch drawing method fast be that image is carried out area dividing, it is divided into three parts: foreground area, zone of ignorance and background area is characterized in that: the steps include:

1) according to the zone of cutting apart, to any 1 c in the zone of ignorance, establishing its color is C, calculates its initial prospect and background color component F and B; Said initial prospect and the calculating of background color component F and B are: for any 1 c in the zone of ignorance, find out on prospect profile line and the background outline line from c point nearest some f ' and b ', the distance that some f ' and b ' are ordered from c is respectively d ₁And d ₂, a given arithmetic number θ (1.0＜θ≤10.0) is the center of circle with a c, respectively with θ d ₁With θ d ₂Long for radius, be two concentric circles C ₁And C ₂, be located at round C ₁Inner and have a few that be positioned on the prospect profile line is f ₁, f ₂..., f _k, these distances of ordering from c are d ₁₁, d ₁₂..., d _1k, at circle C ₂Inner and have a few that be positioned on the background outline line is b ₁, b ₂..., b ₁, these distances of ordering from c are d ₂₁, d ₂₂..., d ₂₁, calculate f _i(i=1,2 ..., the k) weighted average of some color $\stackrel{\‾}{F}=\underset{i=1}{\overset{k}{\mathrm{\Σ}}}{w}_{1i}{f}_i,b_j(j=\mathrm{1,2},\·\·\·,l)$ The weighted average of some color $\stackrel{\‾}{B}=\underset{j=1}{\overset{l}{\mathrm{\Σ}}}{w}_{2j}{b}_j,$ Wherein $w_{1i}=\frac{\mathrm{\θ}}{\mathrm{\θ}-1}-\frac{1}{\mathrm{\θ}-1}\·\frac{d_{1i}}{d_1}(i=\mathrm{1,2},\·\·\·,k),w_{2j}=\frac{\mathrm{\θ}}{\mathrm{\θ}-1}-\frac{1}{\mathrm{\θ}-1}\·\frac{d_{2j}}{d_2}(j=\mathrm{1,2},\·\·\·,l),$ F and B are exactly estimated initial prospect and background color component;

2) the initial prospect that goes out according to estimates and background color component F and B, take color adjustment measure to be adjusted into the value of F and B, estimate the alpha value that c is ordered at last, said alpha value is estimated: the steps include: 1) according to the colour brightness difference of foreground object in the image and background object, the value of decision aberration parameter ρ, 2) according to the relation of colour brightness ratio λ and the aberration parameter ρ of F and B, the value of adjusting F and B is F and B, 3) according to adjusted F and B, by formula $\mathrm{\α}=\frac{(C-B)\·(F-B)}{{\left|\right|F-B\left|\right|}^2}$ Calculate the alpha value α of 1 c in the zone of ignorance.

The present invention has the speed piece, effective advantage.Natural image matting technology in the past all has been placed on second step to most energy, and also promptly how to estimate prospect and background color, and ignored the 3rd step, also be the estimation of alpha.The result who pays close attention to step 2 omit step 3 has caused occurring the color estimation model of various complexity, though improved stingy figure effect to a certain extent, but also increased amount of calculation greatly simultaneously, this makes the application in practice of natural image matting technology be subjected to very big restriction.

The present invention has broken through this framework, and it is little to have designed a kind of amount of calculation, scratches the effective stingy diagram technology of figure and satisfies actual needs.Its color estimation model is very simple, and the estimation scheme of alpha value is then creative.

Table 1 has been listed different stingy figure examples required processing time on different machines.Compare with the current best Chuang method of figure effect of scratching in the world, speed of the present invention has improved 10～12 times than it, does not reduce and scratch the figure effect.The Chuang method is P3 1.0G at CPU, and RAM is under the environment of 512M, and treatment S yringe image needs about 120 seconds.Use method of the present invention machine conditions than the slightly poor condition of Chuang method under, speed is about 11 times of Chuang method.

Form 1 different scratching the stingy figure time of figure example under varying environment

	CPU：Celeron 400 RAM：320M	CPU：P3 600 RAM：512M	CPU：P4 1.8G RAM：256M
				Syringe	16.484 second	10.485 second	5.680 second
Feather_edge	25.696 second	16.474 second	8.844 second
				Galadriel	4.266 second	2.754 second	1.406 second
Gandalf	5.618 second	3.635 second	1.891 second
				Tiger	5.658 second	3.666 second	1.922 second
Water	9.303 second	5.988 second	3.281 second

Application the present invention can fast and effeciently pluck out the prospect part in the arbitrary image.A large amount of examples prove, the present invention has solved the speed that exists among the stingy figure and the contradiction between the effect well, has good practical value.

Description of drawings

Fig. 1 is a schematic flow sheet of the present invention;

What Fig. 2 (a) (b) described is the estimation schematic diagram of area dividing and prospect background color component;

Fig. 3 (a) (b) (c) is color F, B and the C several relative position schematic diagrames in color space (d);

Fig. 4 (a) is that color among the present invention is adjusted the scheme schematic diagram (b);

Fig. 5 is that the effect about example Syringe image of the present invention and other stingy drawing methods compares schematic diagram;

Fig. 6 is that the effect about example Feather_edge image of the present invention and Chuang method compares schematic diagram;

Fig. 7 is some other effect schematic diagram of the present invention.

Embodiment

The principle of scratching drawing method fast that in the Digital Image Processing with the arbitrary image is background is: for any 1 c in the transitional region between prospect and the background (also being the zone of ignorance in the Region Segmentation), with in the image space from the color-weighted mean value F of these nearest some foreground points of point and background dot and B as the prospect and the background color component of c point color, adopt suitable alpha value estimation scheme to estimate the alpha value α that c is ordered then.This method need not to set up complicated model and asks prospect and background color, and the adjustment scheme of color is also very simple, and therefore, this method is a kind of drawing method of scratching fast.

At first, in second step of this stingy drawing method, the model that calculates foreground color component F and background color component B is simple and practical.This method is only calculated the point on the outline line that is included in two round intra-zones, and the distance that the calculating of weight and these point and c are ordered is linear, and the weight maximum of nearest point is along with the increase weight linearity of distance reduces.Stingy drawing method of the present invention is simplified as much as possible at the color estimation model in second step.

Secondly, in the 3rd step, the color of this method is adjusted the scheme simple, intuitive.This scheme keeps the constant rate of the RGB component of color, utilize the human eye characteristics very inresponsive to the minor variations of colour brightness, the brightness of adjustment prospect and background color on minimum degree utilizes adjusted prospect and background color to estimate the alpha value that C is ordered at last.This scheme specific as follows:

In three-dimensional RGB color space, the color of any can be expressed as a point or a vector.In the color three-dimensional coordinate, some O is the origin of coordinates, and it represents black.Any 2 P in the given color space ₁, P ₂, | P ₁P ₂| expression line segment P ₁P ₂Length.If any point c in the zone of ignorance, its color is C, and initial prospect that estimates and background color component are B, F.In three-dimensional color space, in order to express easily, we do as giving a definition:

Scratch the figure deviation: the distance between some C and the line segment BF is called scratches the figure deviation.In Fig. 3 (a), some C is positioned on the line segment BF, and scratch the figure deviation this moment is 0; Among Fig. 2 (b), some C is line segment CC ' perpendicular to line segment BF and meet at a C ' outside line segment BF, and scratch the figure deviation this moment is the length of line segment CC ' | CC ' |.

Rule point and irregular point: when the distance of C point off-line section BF was not far, we claimed that the C point of this moment is regular point, otherwise are irregular point.The ratio of the length of line segment CC ' and BF has reacted the systematicness that C is ordered to a certain extent.

When the C point just dropped on the line segment BF, this situation was the optimal situation that C is ordered.Can release from the synthetic equation of color:

$\mathrm{\α}=\frac{(C-B)\·(F-B)}{{\left|\right|F-B\left|\right|}^2}---\left(1\right)$

In the formula (1), vector (C-B) is corresponding to directed line segment BC, and vector (F-B) is corresponding to directed line segment BF, and α then is the ratio of the length of the length of the projection of line segment BC on BF and BF.When the C point was on line segment BF, scratching the figure deviation was 0, and the length of the projection of line segment BC on BF is exactly the length of line segment BC.In most cases, scratching the figure deviation is not 0, and (1) formula of using is calculated α and had certain deviation.So problem is converted into the value that how as far as possible reasonably reduces to scratch the figure deviation.Color of the present invention is adjusted scheme based on following several observations:

Observe 1: when C point be regular putting, the error of the alpha value that is calculated by formula (3) can be ignored.(shown in Fig. 3 (b));

Observe 2: when angle ∠ BCF was the obtuse angle, the distance of some C off-line section BF was very not far away.

Observe 3: when the length of line segment OC was between line segment OB and OF, the distance of some C off-line section BF was also not far.

The C point that satisfies observation 2 and 3 is regular point.Be not that all rule points all satisfy observation 2 or 3.After estimating initial prospect and background color component F and B, the method of introducing by the present invention is adjusted the position of F and B in color space, can be converted into regular point to the irregular C point of major part.

The present invention's definition: F=(F _r, F _g, F _b), B=(B _r, B _g, B _b), C=(C _r, C _g, C _b), A wherein _r, A _g, A _bBe the component of color A on three passages of RGB, A=F here, B, C calculates l _F=F _r+ F _g+ F _b, l _B=B _r+ B _g+ B _b, l _C=C _r+ C _g+ C _b, ratio $\mathrm{\λ}=l_{\stackrel{\‾}{F}}/l_{\stackrel{\‾}{H}},$ Be divided into three kinds of situation discussion this moment: if 1. $\mathrm{\λ}>1/\mathrm{\ρ}$ And l _C＞l _F, perhaps $\mathrm{\λ}\≤1/\mathrm{\ρ}$ And l _C＜l _F, then F is adjusted into F=(Fr * l _C/ l _F, F _g* l _C/ l _F, F _b* l _C/ l _F), B does not adjust, and is B=B, if 2. yet $\mathrm{\λ}>1/\mathrm{\ρ}$ And l _C＜l _B, perhaps $\mathrm{\λ}\≤1/\mathrm{\ρ}$ And l _C＞l _B, B is adjusted into B=(B _r* l _C/ l _B, Bg * l _C/ l _B, B _b* l _C/ l _B), F does not adjust, and is F=F yet, and 3. under remaining situation, F and B all do not adjust, and are F=F yet, B=B.Adjusted F and B substitution formula (1), calculate alpha value α.

Color of the present invention is adjusted scheme and only the brightness of initial prospect and background color component F and B is revised on minimum degree, and does not change their color.Generally speaking, human eye is not too responsive to the minor variations of the brightness of color, therefore can replace F and B to come alpha value calculated with F and B fully.This replacement has kept the continuity of α estimated value, can not cause very big impact to human eye.The more approaching real α value of the α value of calculating with F and B.

Fig. 1 is a detail flowchart of the present invention.At first, input picture is carried out area dividing, being divided into is three parts: foreground area, background area and zone of ignorance; Secondly, the zone according to dividing to the every bit in the zone of ignorance, goes out its prospect and background color component according to a preliminary estimate; At last, the present invention adopts color to adjust scheme the prospect and the background color component that estimate is adjusted, and estimates its alpha value.

Fig. 2 (a) is an example of area dividing among the present invention; Fig. 2 (b) has described prospect and background color component how to estimate the point in the zone of ignorance in the present invention.Wherein less round C ₁Point on the prospect profile line of that section redness that inside comprises be exactly calculate the prospect component required have a few bigger round C ₂Point on the background outline line of inner that section green that is comprised be exactly calculate background component required have a few.

Fig. 3 (a) mid point C is just on line segment BF; Fig. 3 (b) mid point C outside line segment BF, but and line segment BF distance very near; L among Fig. 3 (c) _CNot between l _BAnd l _FBetween, but ∠ BC F is the obtuse angle; L among Fig. 3 (d) _CBetween l _BAnd l _FBetween, but ∠ BC F is an acute angle.

Fig. 4 is that the color among the present invention is adjusted the scheme schematic diagram.Here only discuss and work as l _B＜l _FThe time situation.If this moment l _CNot between l _BAnd l _FBetween, two kinds of situation: l are then arranged _C＜l _B＜l _F(shown in Fig. 4 (a)) and l _B＜l _F＜l _C(shown in Fig. 4 (b)).In Fig. 4 (a), on the extended line of BF, look for 1 F, make l _F=l _C, l is so just arranged _B＜l _C≤ l _F, satisfy and observe 3, can think that the C point is regular point at this moment, can application of formula (1) calculate the alpha value.The processing of Fig. 4 (b) is similar with it.

This specification has been lifted 6 examples of implementation altogether.In the examples of implementation that this specification is lifted, parameter value all is θ=2.0, ρ=1.428.Fig. 5, Fig. 6 are examples of implementation 1,2, and Fig. 7 comprises examples of implementation 3～6.

Embodiment 1

Enlarged drawing 1 among Fig. 5 is to be combined by three parts, and wherein the left side is former figure, and the centre is a gray-scale map, and the right is that background is the composite diagram of black.The Knockout method does not pluck out some hairlines come in enlarged drawing 1, and the distortion as a result of some points is arranged on the enlarged drawing 1 in addition.Ruzon ﹠amp; The Tomasi method then has some hairlines more serious phenomenon of rupture to occur, and tangible discontinuous phenomenon is arranged in the composite diagram in addition.The effect of Chuang method is better, but examines, and can find that slight non-continuous event is arranged at the bottom of enlarged drawing 2, and slightly discontinuous also arranged in enlarged drawing 1.Method of the present invention does not then have above-mentioned shortcoming in the present example.

Embodiment 2

Scratch the figure erroneous results or impurity occurs in two oval inner zones of Chuang method in enlarged drawing 1 among Fig. 6.In the little oval inside on top, a part of hair is sliced off by force, and the stingy figure result here is wrong, and a slice impurity has then appearred in big oval inside below.Method of the present invention then good treatment these problems.

Embodiment 3～6

(a) figure is the Gandalf example among Fig. 7, and (b) figure is the Galadriel example, and (c) figure is the Tiger example, and (d) figure is the Water example.

The main points of scratching drawing method fast that in the Digital Image Processing with the arbitrary image are background are:

1. carry out area dividing by hand, guarantee outline line through suitable zone, and have enough precision;

2. the sample point of using in the color estimation is the point on the inner outline line of circle, and the length of two radius of a circles is generally 1.5～3.0 times of beeline;

3. adjust prospect and the background color that estimates, make the C point transfer regular point to;

4. utilize adjusted color to estimate the alpha value that c is ordered.

The whether accurate precision to natural image matting of area dividing has very big influence.Should note when scratching figure following some: guarantee that 1. the point in the prospect profile line all be the foreground point, background outline line point outward all is a background dot, can not allow intersection, otherwise will influence the stingy figure accuracy of intersection region greatly; 2. owing to illumination and air influence, the boundary of object always has a point fuzziness in the human eye.Prospect and background outline line be the border of too close object not, leave certain leeway, guarantees that transitional region does not put prospect or background area under.The distance of border to two outline line about equally; 3. make outline line not pass the violent zone of color change as far as possible.

The sample point that the estimation of prospect and background color is used among the present invention only is the point on the outline line, and this reduces the number of sample point greatly, has improved computational speed.

Color adjustment scheme of the present invention is the emphasis of entire method.Its core is converted into regular point to the C point by irregular point exactly, makes finding the solution of α can utilize formula (1).Under the situation of the RGB of color constant rate, the brightness of color is adjusted on minimum degree ground, utilize the among a small circle variation insensitive characteristics of human eye,, try to achieve more rational alpha value adjusting the color that the replacement of back prospect and background color estimated originally to colour brightness.

Claims (4)

1. in the Digital Image Processing be with the arbitrary image background scratch drawing method fast, image is carried out area dividing, it is divided into three parts: foreground area, zone of ignorance and background area is characterized in that: the steps include:

1) according to the zone of cutting apart, to any 1 c in the zone of ignorance, establishing its color is C, calculates its initial prospect and background color component F and B; Said initial prospect and the calculating of background color component F and B are: for any 1 c in the zone of ignorance, find out on prospect profile line and the background outline line from c point nearest some f ' and b ', the distance that some f ' and b ' are ordered from c is respectively d ₁And d ₂, a given arithmetic number θ, wherein, 1.0＜θ≤10.0 are the center of circle with a c, respectively with θ d ₁With θ d ₂Long for radius, be two concentric circles C ₁And C ₂, be located at round C ₁Inner and have a few that be positioned on the prospect profile line is f ₁, f ₂..., f _k, these distances of ordering from c are d ₁₁, d ₁₂..., d _1k, at circle C ₂Inner and have a few that be positioned on the background outline line is b ₁, b ₂..., b _l, these distances of ordering from c are d ₂₁, d ₂₂..., d _2l, calculate f _i, wherein, i=1,2 ..., k, the weighted average of some color $\stackrel{\‾}{F}=\underset{i=1}{\overset{k}{\mathrm{\Σ}}}{w}_{1i}{f}_i,$ b _j, wherein, j=1,2 ..., l, the weighted average of some color $\stackrel{\‾}{B}=\underset{j=1}{\overset{l}{\mathrm{\Σ}}}{w}_{2j}{b}_j,$ Wherein $w_{1i}=\frac{\mathrm{\θ}}{\mathrm{\θ}-1}-\frac{1}{\mathrm{\θ}-1}\·\frac{d_{1i}}{d_1},$ In the formula, i=1,2 ..., k, $w_{2j}=\frac{\mathrm{\θ}}{\mathrm{\θ}-1}-\frac{1}{\mathrm{\θ}-1}\·\frac{d_{2j}}{d_2},$ In the formula, j=1,2 ..., l, F and B are exactly estimated initial prospect and background color component;

2) the initial prospect that goes out according to estimates and background color component F and B, take color adjustment measure to be adjusted into the value of F and B, estimate the alpha value that c is ordered at last, said alpha value is estimated: the steps include: 1) according to the colour brightness difference of foreground object in the image and background object, the value of decision aberration parameter ρ, 2) according to the relation of colour brightness ratio λ and the aberration parameter ρ of F and B, the value of adjusting F and B is F and B, 3) according to adjusted F and B, by formula $\mathrm{\α}=\frac{(C-B)\·(F-B)}{{\left|\right|F-B\left|\right|}^2}$ Calculate the alpha value α of 1 c in the zone of ignorance.

2. in a kind of Digital Image Processing according to claim 1 be with the arbitrary image background scratch drawing method fast, it is characterized in that said Region Segmentation is: two outline lines are drawn at manual prospect edge at image, article one, be the prospect profile line, the point that is in this outline line inside all is the point of foreground area, another outline line is the background outline line, the point that is in outside this outline line all is the point of background area, the point that is positioned within these two outline lines is the point of zone of ignorance, during craft outline line, make the point that does not comprise foreground area or background area in the zone of ignorance as far as possible.

3. in a kind of Digital Image Processing according to claim 1 be with the arbitrary image background scratch drawing method fast, it is characterized in that the value of aberration parameter ρ is: when the brightness ratio background object of current scenery body was big, the span of ρ was 1.0＜ρ≤2.0; The brightness ratio background object of current scenery body hour, the span of ρ is ρ＞2.0.

4. in a kind of Digital Image Processing according to claim 1 be with the arbitrary image background scratch drawing method fast, it is characterized in that the adjustment of F and B is: establish F=(F _r, F _g, F _b), B=(B _r, B _g, B _b), C=(C _r, C _g, C _b), A wherein _r, A _g, A _bBe the component of color A on three passages of RGB, A=F here, B, C calculates l _F=F _r+ F _g+ F _b, l _B=B _r+ B _g+ B _b, l _C=C _r+ C _g+ C _b, ratio λ=l _F/ l _B, be divided into three kinds of situation discussion this moment: if 1. λ＞1/ ρ and l _C＞l _F, perhaps λ≤1/ ρ and l _C＜l _F, then F is adjusted into F=(F _r* l _C/ l _F, F _g* l _C/ l _F, F _b* l _C/ l _F), B does not adjust, and is B=B yet, if 2. λ＞1/ ρ and l _C＜l _B, perhaps λ≤1/ ρ and l _C＞l _B, B is adjusted into B=(B _r* l _C/ l _B, B _g* l _C/ l _B, B _b* l _C/ l _B), F does not adjust, and is F=F yet, 3. under remaining situation, F and B all do not adjust, and are F=F yet, B=B.

Images