CN1852393A - Single-colour image area colour displacing method for virtual displaying system - Google Patents

Abstract

Based on dichromatic reflection model, the method includes steps: (1) estimating color unit vector of incident light; (2) estimating color unit vector of diffuse reflection component; resampling color signals, then using projection method to obtain reference color unit vector most near to actual color of object; (3) carrying out minimum mean-square value decomposion for all pixels between incident light vector and reference color vector so as to obtain linear coefficient of each pixel; (4) constructing new linear coefficient for each pixel, and then using object color value to calculate final resultant color. The invention realizes color replacement rapidly and effectively. The replaced result reflects effect of shade and shadow on surface of object truly. Features are: no need of users' interaction, small amount of calculation, and better use value.

Description

The single-colour image area colour displacing method that is used for virtual display system

Technical field

The present invention relates to a kind of single-colour image area colour displacing method that is used for virtual display system.

Background technology

The color displacement is a generic operation very important in image and the Video processing.In many Processing tasks, need make change to the color of target.The difficult point of color displacement is not only will guarantee that the zone of being replaced shows correct color, also needs to guarantee rational Luminance Distribution information in the replacement areas.In the virtual display system that matter material displacement customization function is provided, the color replacement algorithm has directly determined the sense of reality of replacement result especially.Present color replacement operator is divided into two classes substantially: change to the painted of black and white image with to the color of coloured image.

Can be divided into two class methods again to the painted of black and white image: a kind ofly provide a source figure close, use color on the figure of source automatically to the target graph coloring with image scene to be colored.In general, at first mark some significant rectangular areas, such as sky, trees, meadow, highway etc. from source figure and target figure.For each to doing color registration from the rectangular area of source figure and target figure respectively.The mode of registration is that the pixel among the figure of source is divided into some subclass, and each pixel on the target figure is found a subclass of mating most, uses the chromatic value of source figure and the brightness value of target figure to constitute final color then.After the color registration, utilize zone painted on the target figure painted for other pixels again as source region.Another kind is directly to give color to the partial-pixel in the painted zone of needs, spreads these new colors according to certain rule then.Diffusion principle is an adjacent pixels on time and space, if it has close brightness, then close color should be arranged.In realization, this hypothesis is described to the cost equation of a second order, finds the solution thereby coloring problem is converted into an optimization problem, and separating of cost minimum is final color rendering intent.Two kinds of above-mentioned methods all need to carry out certain interactive operation, and coloring effect is more intense to the dependence as a result of interactive operation.

Color displacement for coloured image also needs the more close image of two width of cloth scene content, and a width of cloth is source figure, and a width of cloth is target figure.This color displacement is the displacement of integral color, and target figure and source figure are being transformed to certain subjective colo(u)r space, behind l α β space, obtains average and mean square deviation on each, and these values have been considered to represent the integral color of image.For target image, obtain each pixel on each and corresponding average poor.Utilize each axial mean square deviation of source figure and target figure that this difference is carried out convergent-divergent, can try to achieve figure as a result.Automatically realize that this algorithm needs source figure and target figure scene content closely similar, otherwise just need on artificial assigned source figure and the target figure which zone to mate in terms of content.Therefore, the practicality of this method is not strong.

Aforesaid colouring algorithm in use all has certain limitation, is difficult to satisfy the needs of virtual display system.Such as, the amount of calculation of these algorithms is all very big, can't allow the user obtain the effect of matter material displacement in real time.Some method can only be handled some special scenes, such as natural land, is difficult to accomplish the accuracy on the details.Sixty-four dollar question is that these methods all are to use the mode that keeps original brightness to change colourity simply.According to colorimetry and physioptial principle, replacing the back Luminance Distribution in color can not remain unchanged.Thereby above-mentioned algorithm all can't guarantee the true effect of result on Luminance Distribution.

Fig. 6 has provided the contrast effect of the present invention and common HSV chroma replacement method.Shape is identical but plastic cup that color is different is placed on the position of sight line with two., shine from fixing height with a fluorescent lamp with the position that becomes miter angle with direction of visual lines.Capture apparatus is the digital slr camera of the EOS D30 of Canon, uses the Automatic white balance mode.Method of the present invention is handled this two width of cloth image, obtains the reference color of these two cups respectively.Reference color with the cyan cup is substituted on the red cup as new color, obtains the result as Fig. 6 (d).Simultaneously, all pixels of target figure and color conversion to be replaced be to the HSV space, then each pixel used the brightness and the new color of intensity value structure of chromatic value and the primitive color of waiting to replace color, obtains the result as Fig. 6 (c).Can find that by contrast the arithmetic result of this chapter is more approaching with true picture, and there is bigger error in the brightness excursion as a result of common HSV algorithm.

Summary of the invention

The purpose of this invention is to provide a kind of single-colour image area colour displacing method that is used for virtual display system.

The step of single-colour image area colour displacing method that is used for virtual display system is as follows:

1) the normalization colourity Λ of estimation diffuse reflection component, the steps include: that (1) resamples to the color signal in the target area, avoid The noise, (2) the color signal normalization after will resampling, (3) with the specific plane of the signal projection to after the normalization, (4) ask the outermost point that utilizes projection result, (5) put back projection with outermost, calculate the normalization chroma vector of diffuse reflection component;

2) the normalization colourity of estimation incident light component is reduced to white light with incident light, and applying unit vector Γ={ 1/ , 1/ , 1/  } is as the normalization colourity of incident light;

3) use the colourity of diffuse reflection component and the colourity of incident light component that all colours signal is carried out the least square decomposition, try to achieve two parameter sets S _αAnd S _β, these two parameters are that each color signal gets linear combination coefficient for diffuse reflection component and incident light component, promptly for color signal C _p, its parameter alpha _pAnd β _pSatisfy:

C _p=α _p* Λ+β _p* Γ formula 1

4) estimate the color value of reference color, obtain a reference color as the color that is replaced, the steps include: that (1) select all values β near 0 sampled point, therefrom choose 5% of α value maximum, establishing its average is α _m, (2) calculate parameter to (α _m, 0) and pairing color is as the reference color value;

5) color displacement the steps include: that (1) establishes a pixel is B, makes the projection X of B on the chromaticity unit vector of object color _BAs its object color component, it is as follows to calculate the new parameter formula of B point:

${\mathrm{\α}}^{\′}=\frac{\mathrm{\α}+\mathrm{\β}\×\cos \left(\mathrm{\θ}\right)}{n}$ Formula 2

β′＝β

Wherein θ is the angle of diffuse reflection color unit vector Λ and the color vector Γ of incident light unit, and n is the mould value of reference color, and (2) establish new color value is C _n, the color C after the displacement _{B '}Account form as follows:

C _{B '}=α ' * C _n+ β ' * Γ formula 3

Said color signal resampling step is: 1) all pixel color value are transformed into the HSV space from rgb space, 2) do the chroma histogram of all colours signal, its transverse axis is the colourity lattice, the longitudinal axis is the color signal number that belongs to each colourity lattice, use this histogram of Gaussian function match then, obtaining its intermediate value is x ₀, variance is δ, gets all chromatic values at [x ₀-δ, x ₀+ δ] color signal in the scope is as new sampled value.

To the signal projection after the normalization be: establishing the plane of being opened by diffuse reflection unit vector Λ and incident light unit vector Γ is Φ, if have a plane Ψ to cross initial point in addition and perpendicular to Φ, did certain color signal point and perpendicular to the straight line of Ψ, the intersection point of this straight line and Ψ i.e. the subpoint of this color signal point, concrete computational methods are: establish certain color signal p in the sampled point, its color value is C _p={ R _p, G _p, B _p, become C after the normalization _Pn={ R _Pn, G _Pn, Bp _n, R wherein _Pn ²+ G _Pn ²+ B _Pn ²=1, then the coordinate Calculation of p point on the Ψ plane is as follows:

$\left\{\begin{array}{c}{x}_p=\frac{2\×R_{\mathrm{pn}}-G_{\mathrm{pn}}-B_{\mathrm{pn}}}{3}\\ y_p=\frac{2\×G_{\mathrm{pn}}-R_{\mathrm{pn}}-B_{\mathrm{pn}}}{3}\end{array}\right.$ Formula 4

Ask the outermost point step of projection result to be: azimuthal average of 1) getting all subpoints and initial point line, do a ray that sends by initial point, 2) all subpoints are projected on the ray, get the subpoint of the mean place of 10% point farthest as outermost;

Back projection is calculated: the subpoint coordinate (x that tries to achieve outermost _o, y _o) after, association type 4 and formula 5 are found the solution Λ={ R _Pn, G _Pn, B _Pn;

R _Pn ²+ G _Pn ²+ B _Pn ²=1 formula 5

The present invention can fast and effeciently be that upward new color is replaced in the monochromatic target area on the image, waits that the effect of shadow of replacing the zone is well kept, and replaces the result and has the very strong sense of reality.Reflecting component estimating step in the method for the present invention only need be carried out once, and result of calculation can preserve, and use in follow-up replacement operator, so the computational speed of this method is exceedingly fast.With traditional method difference, color replacement process of the present invention is very easy to the user and uses without any need for man-machine interactively.

Description of drawings

Fig. 1 is the single-colour image area colour displacing method flow chart that is used for virtual display system;

Fig. 2 is the chroma histogram that uses in the resampling process of the present invention;

Fig. 3 is the schematic diagram that sciagraphy of the present invention is asked diffuse reflection chromaticity unit vector;

Fig. 4 is the schematic diagram that fits the subpoint process of the present invention;

Fig. 5 is the schematic diagram of the new linear coefficient of calculating of the present invention;

Fig. 6 (a) (b) (c) is that the present invention and common colourity are replaced the algorithm effects contrast experiment (d);

Fig. 7 is a color replacement result example 1 of the present invention;

Fig. 8 is a color replacement result example 2 of the present invention;

Fig. 9 is a color replacement result example 3 of the present invention.

Embodiment

The principle that is used for the single-colour image area colour displacing method of virtual display system is: for monochromatic target area, according to two look reflection models, the pixel color value in the zone should be the linear combination of body surface diffuse reflection component and specular components color value.This method is at first estimated the normalization chroma vector of light source and object intrinsic colour.Utilize least square method that each pixel is decomposed then, obtain the diffuse reflection component lines property coefficient and the border face reflecting component linear coefficient of this pixel.At last, determine final color value according to these coefficients and new color value.This method based on two look reflection models can simulate the physical process of surface reflection well, so the displacement result that this method obtains is very true.The steps include:

1) the color unit vector Λ of estimation diffuse reflection component the steps include: that (1) resamples to the color signal in the target area, avoids The noise.Before the colourity of estimating the diffuse reflection component,, need earlier the pixel in the target area to be carried out resampling for avoiding The noise.All pixel color value are transformed into the HSV space, can know by inference according to the definition of colourity H passage and white light unit vector Γ and formula 1, in the ideal case, all colours signal should have identical chromatic value, therefore with colourity as standard for manual sampling.Do the chroma histogram of all pixels, as shown in Figure 2.With this histogram of Gaussian function match, establishing its intermediate value is x ₀Variance is δ.Get all chromatic values at [x ₀-δ, x ₀+ δ] pixel value in the scope is as new sampled value.

(2) the color signal normalization after will resampling makes that promptly the quadratic sum of three components of this vector is 1.

(3) with the specific plane of the signal projection to after the normalization, according to the hypothesis of two look reflection models, ideally, all color signals all should drop on the plane Φ that is opened by unit vector Λ and Γ.If have a plane Ψ to cross initial point in addition and,,, then should drop on the line segment PQ on the Ψ if will project to after the color signal normalization on the Ψ of plane because Γ is the white light unit vector perpendicular to Φ.Wherein the pairing vector of P point should have same direction with Λ.Therefore,, counter throw in the color space and normalization it then if obtain the P point, just can be in the hope of Λ.Whole projection process is as shown in Figure 3: establish certain pixel p in the sampled point, its color value is C _p={ R _p, G _p, B _p, become C after the normalization _Pn={ R _Pn, G _Pn, B _Pn, R wherein _Pn ²+ G _Pn ²+ B _Pn ²=1.Then the coordinate Calculation of p point on the Ψ plane is as follows:

$\left\{\begin{array}{c}{x}_p=\frac{2\×R_{\mathrm{pn}}-G_{\mathrm{pn}}-B_{\mathrm{pn}}}{3}\\ y_p=\frac{2\×G_{\mathrm{pn}}-R_{\mathrm{pn}}-B_{\mathrm{pn}}}{3}\end{array}\right.$ Formula 4

(4) ask the outermost point that utilizes projection result: in real image, it not is that all points can both drop on the Φ of plane that the distortion that factors such as the nonlinear operation of camera, noise, compression losses cause makes, a lot of sampled points can be distributed in the both sides of Φ.Therefore, sampled point neither drop on the line segment PQ fully in the projection on the Ψ of plane.We use a simple approximating method to solve this problem.At first get azimuthal average of being had a few, do a ray that is sent by initial point, this ray can be represented the direction of PQ more exactly.All spot projections to ray, are got counter the throwing in the color space of coordinate apart from initial point point P ' farthest.The process of match in Practical Calculation, is got the subpoint of the mean place value diffuse reflection component color the most unit vector of 10% point farthest as shown in Figure 4.

(5) outermost is put back projection, calculate the normalization chroma vector of diffuse reflection component;

2) the color unit vector of estimation border face reflecting component: in two look reflection models, the color relation of supposing border face reflecting component is the color of incident light.Comparatively under the situation near white light, the white balance of camera is handled the influence that can remove light source color temperature effectively in the incident light color.The sensitivity function that this processing can be understood as each Color Channel adjusts to satisfy:

$\underset{w}{\∫}E\left(\mathrm{\λ}\right)R_R\left(\mathrm{\λ}\right)\mathrm{d\λ}=\underset{w}{\∫}E\left(\mathrm{\λ}\right)R_G\left(\mathrm{\λ}\right)\mathrm{d\λ}=\underset{w}{\∫}E\left(\mathrm{\λ}\right)R_B\left(\mathrm{\λ}\right)\mathrm{d\λ}$ Formula 6

Wherein E (λ) is the incident light spectral distribution, R _R(λ), R _G(λ), R _B(λ) be the sensitivity function of three passages respectively.This moment, the unit color vector of border face reflecting component was Γ={ 1/ , 1/ , 1/  }.Because all built-in white balance treatment mechanism of most image capture devices, can eliminate the influence of light source color temperature to color of image, so this method simply applying unit vector Γ={ 1/ , 1/ , 1/  } as the color unit vector of border face reflecting component;

3) use the color unit vector of diffuse reflection component and border face reflecting component that all colours signal is carried out the least square decomposition, try to achieve two parameter sets S _αAnd S _β, these two parameters are that each color signal gets linear combination coefficient for diffuse reflection component and border face reflecting component, promptly for color signal C _p, its parameter alpha _pAnd β _pSatisfy:

C _p=α _p* Λ+β _p* Γ formula 1

4) estimate the color value of reference color, consider and singly can't obtain the real diffuse reflectance of object that this method asks a special color value as the reference color, is that benchmark is realized comparatively real color displacement with this reference color from piece image.At first select all values β near 0 sampled point, choose 5% of α value maximum from these points, establishing its average is α _mWith parameter to (α _m, 0) and pairing color is as the reference color value.This reference color will be replaced by new color.

5) in order to embody the brightness of new color, we have proposed new linear coefficient.If a pixel is B, make the projection X of B on the chromaticity unit vector of object color _BAs the body color component, as shown in Figure 5.Be new object color linear coefficient with the object color component of each pixel and the likening to of mould value of reference color, computing formula is suc as formula 2, and wherein θ is the angle of object color direction vector Λ and incident light color direction vector Γ, and n is the mould value of reference color.When carrying out the color displacement, calculate and the new corresponding linear coefficient of color as the n value with the mould value of new color.And then with the direction vector of new color and new linear coefficient according to formula 3 result of calculation color values.The step of color displacement is: 1) establishing a pixel is B, makes the projection X of B on the chromaticity unit vector of object color _BAs its object color component, it is as follows to calculate the new parameter formula of B point:

${\mathrm{\α}}^{\′}=\frac{\mathrm{\α}+\mathrm{\β}\×\cos \left(\mathrm{\θ}\right)}{n}$ Formula 2

β′＝β

Wherein θ is the angle of diffuse reflection color unit vector Λ and border face reflection colour unit vector Γ, and n is the mould value of reference color, 2) establishing new color value is C _n, the color C after the displacement _{B '}Account form as follows:

C _{B '}=α ' * C _n+ β ' * Γ _nFormula 3

Here provide some carry out the color displacement to real image result.From example as can be seen: no matter be to almost all being irreflexive zone (Fig. 7), or the zone that certain transparent feel is arranged (Fig. 8) of similar yarn quality, or the bigger target area (Fig. 9) of brightness excursion, the replacement algorithm of this chapter all can obtain effect preferably.

Claims (5)

1. single-colour image area colour displacing method that is used for virtual display system, it is characterized in that: the step of method is as follows:

1) the normalization colourity Λ of estimation diffuse reflection component, the steps include: that (1) resamples to the color signal in the target area, avoid The noise, (2) the color signal normalization after will resampling, (3) with the specific plane of the signal projection to after the normalization, (4) ask the outermost point that utilizes projection result, (5) put back projection with outermost, calculate the normalization chroma vector of diffuse reflection component;

2) the normalization colourity of estimation incident light component is reduced to white light with incident light, the applying unit vector

$\mathrm{\Γ}=\{1/\sqrt{3},1/\sqrt{3},1/\sqrt{3}\}$ Normalization colourity as incident light;

3) use the colourity of diffuse reflection component and the colourity of incident light component that all colours signal is carried out the least square decomposition, try to achieve two parameter sets S _αAnd S _β, these two parameters are that each color signal gets linear combination coefficient for diffuse reflection component and incident light component, promptly for color signal C _p, its parameter alpha _pAnd β _pSatisfy:

C _p=α _p* Λ+β _p* Γ formula 1

4) estimate the color value of reference color, obtain a reference color as the color that is replaced, the steps include: that (1) select all values β near 0 sampled point, therefrom choose 5% of α value maximum, establishing its average is α _m, (2) calculate parameter to (α _m, 0) and pairing color is as the reference color value;

5) color displacement the steps include: that (1) establishes a pixel is B, makes the projection X of B on the chromaticity unit vector of object color _BAs its object color component, it is as follows to calculate the new parameter formula of B point:

${\mathrm{\α}}^{\′}=\frac{\mathrm{\α}+\mathrm{\β}\×\cos \left(\mathrm{\θ}\right)}{n}$

β '=β formula 2 wherein θ is the angle of diffuse reflection color unit vector Λ and the color vector Γ of incident light unit, and n is the mould value of reference color, and (2) establish new color value is C _n, the color C after the displacement _B' account form as follows:

C _B'=α ' * C _n+ β ' * Γ formula 3.

2. a kind of single-colour image area colour displacing method that is used for virtual display system according to claim 1, it is characterized in that said color signal resampling step is: 1) all pixel color value are transformed into the HSV space from rgb space, 2) do the chroma histogram of all colours signal, its transverse axis is the colourity lattice, the longitudinal axis is the color signal number that belongs to each colourity lattice, use this histogram of Gaussian function match then, obtaining its intermediate value is x ₀, variance is δ, gets all chromatic values at [x ₀-δ, x ₀+ δ] color signal in the scope is as new sampled value.

3. a kind of single-colour image area colour displacing method that is used for virtual display system according to claim 1, it is characterized in that saidly to the signal projection after the normalization being: establishing the plane of being opened by diffuse reflection unit vector Λ and incident light unit vector Γ is Φ, if have a plane ψ to cross initial point in addition and perpendicular to Φ, did certain color signal point and perpendicular to the straight line of ψ, the intersection point of this straight line and ψ i.e. the subpoint of this color signal point, concrete computational methods are: establish certain color signal p in the sampled point, its color value is C _p={ R _p, G _p, B _p, become C after the normalization _Pn={ R _Pn, G _Pn, B _Pn, R wherein _Pn ²+ G _Pn ²+ B _Pn ²=1, then the coordinate Calculation of p point on the ψ plane is as follows:

$\left\{\begin{array}{c}{x}_p=\frac{2\×R_{\mathrm{pn}}-G_{\mathrm{pn}}-B_{\mathrm{pn}}}{3}\\ y_p=\frac{2\×G_{\mathrm{pn}}-R_{\mathrm{pn}}-B_{\mathrm{pn}}}{3}\end{array}\right.$ Formula 4.

4. a kind of single-colour image area colour displacing method that is used for virtual display system according to claim 1, it is characterized in that the said outermost point step of projection result of asking is: azimuthal average of 1) getting all subpoints and initial point line, do a ray that sends by initial point, 2) all subpoints are projected on the ray, get the subpoint of the mean place of 10% point farthest as outermost.

5. a kind of single-colour image area colour displacing method that is used for virtual display system according to claim 1 is characterized in that the calculating of said back projection is: the subpoint coordinate (x that tries to achieve outermost _o, y _o) after, association type 4 and formula 5 are found the solution Λ={ R _Pn, G _Pn, B _Pn; R _Pn ²+ G _Pn ²+ B _Pn ²=1 formula 5.

Images