CN100542303C - A kind of method for correcting multi-viewpoint vedio color - Google Patents

Abstract

The invention discloses a kind of color correction method of videos of multiple viewpoints, comprise: by calculating target image and source images histogram, utilize dynamic programming algorithm to ask for the minimum cost path, thereby set up the color map relation between source images and target image, and by guaranteeing the level and smooth of path in level and vertical direction increase compensating factor, realize multi-viewpoint vedio color is proofreaied and correct by the video tracking technology at last, the present invention is under the prerequisite that guarantees multi-viewpoint vedio color correction accuracy, improved the precision of color map greatly, reduced the computation complexity that multi-viewpoint vedio color is proofreaied and correct, avoided that the discontinuous situation of color appears in the zone boundary in the color map process, eliminate edge distortion, improved speed and precision that multi-viewpoint vedio color is proofreaied and correct.

Description

A kind of method for correcting multi-viewpoint vedio color

Technical field

The present invention relates to a kind of processing method of multi-viewpoint video image, especially relate to a kind of method for correcting multi-viewpoint vedio color.

Background technology

In real world, the vision content that the observer saw depends on the position of observer with respect to observed object, and the observer can freely select each different angle to remove to observe and analyze things.In traditional video system, real scene is selected decision with respect to the picture of a viewpoint by cameraman or director, the sequence of video images that the user can only watch video camera to be produced on single viewpoint passively, and can not freely select other viewpoint to observe real scene.The video sequence that these folk prescriptions make progress can only reflect a side of real-world scene.The free viewpoint video system can make the user freely select viewpoint to go to watch any side in the certain limit in the real-world scene, by the MPEG of International Standards Organization (Moving Picture Experts Group: be called the developing direction of video system of future generation Motion Picture Experts Group).

The multi-viewpoint video image technology is a core link in the free viewpoint video technology, and it can provide the video image information of the different angles of captured scene.Fig. 1 is the parallel camera system imaging of a many viewpoints schematic diagram, and wherein n camera (or video camera) placed abreast to take multi-viewpoint video image.Utilize the information of a plurality of viewpoints in the multi-view point video signal can synthesize the image information of user-selected any viewpoint, reach the purpose of freely switching any visual point image.But because the key elements such as scene illumination, camera calibration, CCD noise, shutter speed and exposure of each viewpoint are inconsistent in gatherer process, can cause the color distinction between the image that diverse location gathers very big, bring great difficulty for follow-up multi-view point video three-dimensional display, video coding and virtual viewpoint rendering.Therefore must carry out color correction process to multi-view point video,, analyze then and compare, could guarantee the reliability of analysis result the color appearance of the different points of view image of the same target same color of reforming.

Each visual point image content is to the description of same target on diverse location in the multi-view point video signal, promptly has very high content similitude between each visual point image.Global statistics information, color region, color histogram etc. can be as carrying out the carrier that color is transmitted between viewpoint, by the color transmission, the color characteristic of a viewpoint put on another viewpoint, thereby reach the purpose of correction.Figure 2 shows that the schematic diagram of a typical color corrected system, color region in the source images that needs to proofread and correct passes through at the enterprising line search of reference view image, obtain the color region of optimum Match, the color characteristic that this is regional puts on current region, the image after just obtaining proofreading and correct.Therefore, the emphasis of color correction is exactly the process of color search, and search accuracy and speed have determined the efficient of correct operation.

Color correction is one of key technology during multi-view point video signal is handled, common operation as preliminary treatment or reprocessing.The zone coupling is the correction means that adopts usually, by target image and source images are carried out cluster segmentation, concerns in the most similar interregional color map of setting up, and with these mapping relations source images is proofreaied and correct.Histogram is a kind of important method of describing the color of image characteristic, needs loaded down with trivial details cutting operation to compare with the zone coupling, and histogram operation is very convenient.With the histogram coupling is example, by calculating the accumulative histogram of target image and source images, has identical histogram distribution as long as satisfy source images with target image, just the histogram territory of target image can be mapped to source images.

But for mapping curve from gray scale 0 to gray scale N, simple histogram coupling, the overall histogram that can not make correcting image and reference picture is the mapping of gap minimum, and its existence mapping precision is not high, color correction speed is slow, problems such as calculation of complex.

Summary of the invention

Technical problem to be solved by this invention provides a kind of method for correcting multi-viewpoint vedio color, when guaranteeing that multi-viewpoint vedio color is proofreaied and correct accuracy, has reduced the computation complexity of color correction.

The present invention solves the problems of the technologies described above the technical scheme that is adopted: a kind of method for correcting multi-viewpoint vedio color, and it may further comprise the steps:

(1) synchronization is defined as target image by a visual point image in the multi-view image of many view camera system shooting, be designated as T, and other visual point image is defined as source images, be designated as S, target image and source image data are converted into the HSI color space from the RGB color space, 3 color components are respectively tone H, saturation S and intensity I;

(2) calculate the hue histogram of target image respectively ^(T)h ^HHue histogram with source images ^(S)h ^H, calculate the cross-correlation matrix C of hue histogram again ^H, $C^H{=}^{\left(T\right)}{h}^H{\⊗}^{\left(S\right)}{h}^H=\left[\begin{array}{cccccc}{c}_{11}& c_{12}& .& .& .& c_{1N}\\ c_{21}& c_{22}& .& .& .& c_{2N}\\ .& .& & .& & .\\ .& .& & .& & .\\ .& .& & .& & .\\ c_{M1}& c_{M2}& .& .& .& c_{\mathrm{MN}}\end{array}\right],$ Wherein, M is the number of the hue histogram node of target image, and N is the number of the hue histogram node of source images, M=N, c _MnThe internodal distance of expression hue histogram;

(3) obtain cross-correlation matrix C in the step (2) by dynamic programming algorithm ^HIn from starting point c ₁₁To terminal point c _MNThe minimum cost path, the mapping relations of setting up from source images to the target image tone are f (H)=H ';

(4) on the basis of tone mapping relations, the saturation component of target image and source images is carried out continuous sampling, for tone in the target image is the histogram that the pixel of H ' is set up the saturation component ^(T)h _{H '} ^S, for tone in the source images is the histogram that the pixel of H is set up the saturation component ^(S)h _H ^S, calculate the histogrammic cross-correlation matrix C of saturation again ^S, $C^S{=}^{\left(T\right)}{h}_{H^{\′}}^S{\⊗}^{\left(S\right)}{h}_H^S=\left[\begin{array}{cccccc}{c}_{11}& c_{12}& .& .& .& c_{1Q}\\ c_{21}& c_{22}& .& .& .& c_{2Q}\\ .& .& & .& & .\\ .& .& & .& & .\\ .& .& & .& & .\\ c_{P1}& c_{P2}& .& .& .& c_{\mathrm{PQ}}\end{array}\right],$ Wherein, P is the number of the saturation histogram node of target image, and Q is the number of the saturation histogram node of source images, P=Q, c _PqThe internodal distance of expression saturation histogram;

(5) obtain cross-correlation matrix C in the step (4) by dynamic programming algorithm ^SIn from starting point c ₁₁To terminal point c _PQThe minimum cost path, set up from source images to the target image tone, the mapping relations of saturation be f (H, S)=(H ', S ');

(6) at tone, on the basis of saturation mapping relations, the strength component of target image and source images is carried out continuous sampling, set up the strength component histogram of target image ^(T)h _{H ', S '} ^IStrength component histogram with source images ^(S)h _{H, S} ^I, the histogrammic cross-correlation matrix C of calculating strength again ^I, $C^I{=}^{\left(T\right)}{h}_{H^{\′},S^{\′}}^I{\⊗}^{\left(S\right)}{h}_{H,S}^I=\left[\begin{array}{cccccc}{c}_{11}& c_{12}& .& .& .& c_{1V}\\ c_{21}& c_{22}& .& .& .& c_{2V}\\ .& .& & .& & .\\ .& .& & .& & .\\ .& .& & .& & .\\ c_{U1}& c_{U2}& .& .& .& c_{\mathrm{UV}}\end{array}\right],$ Wherein, U is the number of the intensity histogram node of target image, and V is the number of the intensity histogram node of source images, U=V, c _UvThe internodal distance of expression hue histogram;

(7) obtain cross-correlation matrix C in the step (6) by dynamic programming algorithm ^IIn from starting point c ₁₁To terminal point c _UVThe minimum cost path, set up from source images to the target image tone, the mapping relations of saturation and intensity be f (H, S, I)=(H ', S ', I ');

(8) with from source images to the target image tone, the mapping relations of saturation and intensity are carried out color correction to source images, and are transformed into the RGB color space;

(9) multi-view point video is carried out the video tracking operation, judge by the similitude of consecutive frame before and after the video and realize the vedio color correction.

In described minimum cost path computing process, on level and vertical edge, increase compensating factor δ, δ=α c _Max, c wherein _MaxBe the maximum in the cross-correlation matrix, $\mathrm{\α}=\frac{\underset{i=1}{\overset{R}{\mathrm{\Σ}}}{c}_{i,i}}{\max \{(\underset{i=1}{\overset{R}{\mathrm{\Σ}}}{c}_{1,i}+c_{i,R}),(\underset{i=1}{\overset{R}{\mathrm{\Σ}}}{c}_{i,1}+c_{R,i})\}},$ Be present node v _MnBy through v _{M-1, n}, v _{M, n-1}Or v _{M-1, n-1}The cost of 3 possible paths that node arrives is respectively Ω (v _{M-1, n})+e (v _{M, n}, v _{M-1, n})+δ, Ω (v _{M, n-1})+e (v _{M, n}, v _{M, n-1})+δ and Ω (v _{M-1, n-1})+e (v _{M, n}, v _{M-1, n-1}), e (v here _{M, n}, v _{M-1, n}) expression node v _{M-1, n}With node v _{M, n}Between distance, Ω (v _{M, n}) be by node v ₁₁Arrive node v _MnMinimum cost, its value is the minimum value of above-mentioned 3 paths costs.

The similitude determination methods of consecutive frame is before and after the described video: the similitude of definition consecutive frame $\mathrm{Sim}(h_t^H,h_{t+1}^H)=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}(1-\frac{\left|h_t^H\right[i]-h_{t+1}^H[i\left]\right|}{\max \left(h_t^H\right[i],h_{t+1}^H[i\left]\right)}),$ H wherein _t ^HHue histogram for the image of moment t, if the similitude of t and moment t+1 consecutive frame is less than the threshold value of regulation constantly, then enforcement of rights requires step (1)～(8) in 1 to upgrade the color map relation of current time multi-view point video, otherwise the color map relation of continuing to use previous moment is carried out color correction to multi-view point video.

Compared with prior art, the advantage of a kind of method for correcting multi-viewpoint vedio color provided by the present invention is:

1) by three dimensions is carried out continuous sampling, defined three-dimensional histogram, histogram is carried out the dynamic programming algorithm operation ask for the minimum cost path, set up the color map relation between source images and the target image, and then multi-view point video is carried out video tracking operate, realize the correction of multi-viewpoint vedio color simply and easily, its color appearance of the image after the correction is very close with target image;

2) based on cross-correlation matrix, ask for histogrammic minimum cost path by dynamic programming algorithm, improved the precision of mapping greatly;

3) the tone mapping curve that obtains by dynamic programming algorithm continuously and level and smooth has avoided that the discontinuous situation of color appears in the zone boundary in the color map process, has eliminated edge distortion effectively;

4) by the video tracking technology video data is proofreaied and correct, greatly reduced the computation complexity that multi-viewpoint vedio color is proofreaied and correct;

5) Yu to each passage ask for the one dimension histogram respectively, or ask for three-dimensional histogrammic traditional histogram building method according to the distribution of color information of three-dimensional and compare, the histogram of coloured image 3 passages of the present invention's definition, when keeping 3 interchannel correlations, keep the convenience of one dimension histogram operation, improved speed and precision that multi-viewpoint vedio color is proofreaied and correct.

Description of drawings

Fig. 1 is the parallel camera system imaging of a many viewpoints schematic diagram;

Fig. 2 is a color correction process schematic diagram of the present invention;

Fig. 3 is a minimum cost path definition schematic diagram;

Fig. 4 is the relationships between nodes schematic diagram;

Fig. 5 a is the target image of " golf2 " many viewpoints test set;

Fig. 5 b is the source images of " golf2 " many viewpoints test set;

Fig. 5 c is the correcting image of " golf2 " many viewpoints test set;

Fig. 6 a is the target image of " objects3 " many viewpoints test set;

Fig. 6 b is the source images of " objects3 " many viewpoints test set;

Fig. 6 c is the correcting image of " objects3 " many viewpoints test set;

Fig. 7 a is the target image of " Jungle " many viewpoints test set;

Fig. 7 b is the source images of " Jungle " many viewpoints test set;

Fig. 7 c is the correcting image of " Jungle " many viewpoints test set;

Fig. 8 a is the target image of " Uli " many viewpoints test set;

Fig. 8 b is the source images of " Uli " many viewpoints test set;

Fig. 8 c is the correcting image of " Uli " many viewpoints test set;

Fig. 9 is the image after proofreading and correct through video tracking of the present invention;

Figure 10 is " golf2 " tone mapping curve figure;

Figure 11 is " objects3 " tone mapping curve figure;

Figure 12 is " Jungle " tone mapping curve figure;

Figure 13 is " Uli " tone mapping curve figure.

Embodiment

Embodiment describes in further detail the present invention below in conjunction with accompanying drawing.

At first describe the notion in the defined minimum cost of the present invention path below and ask for the minimum cost path problems based on the Dynamic Programming technology.

The description in minimum cost path is as shown in Figure 3: the 1D histogram of supposing two width of cloth images is respectively h ₁={ h ₁[0], h ₁[1] ..., h ₁[M] } and h ₂={ h ₂[0], h ₂[1] ..., h ₂[N] }, definition cross-correlation matrix C is:

$C_{M\×N}=h_1\⊗h_2=\left[\begin{array}{cccccc}{c}_{11}& c_{12}& .& .& .& c_{1N}\\ c_{21}& c_{22}& .& .& .& c_{2N}\\ .& .& & .& & .\\ .& .& & .& & .\\ .& .& & .& & .\\ c_{M1}& c_{M2}& .& .& .& c_{\mathrm{MN}}\end{array}\right]$

Wherein M and N are the number of histogram node, M=N, c _MnThe internodal distance of histogram that expression is corresponding is usually with the criterion of absolute difference as distance, c _Mn=| h ₁[m]-h ₂[n] |.

From node c ₁₁To c _MNThere is mulitpath, from node c ₁₁To c _MNThe total cost of diagonal path can be expressed as $\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{c}_{\mathrm{mm}}=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\left|h_1\right[m]-h_2[m\left]\right|,$ Suppose that I is total node number of path process, $\sqrt{M^2+N^2}\≤I\≤M+N,$ { (m ₀, n ₀) ..., (m _i, n _i) ..., (m _I, n _I) be the label in path, then the notion in minimum cost path makes cost exactly Minimum path.

Adopt the Dijkstra dynamic programming algorithm to ask for the minimum cost path, suppose that v is a node, e is internodal distance, path p (v ₀, v _S)={ v ₀..., v _SCost be exactly the euclidean distance between node pair sum of all connections, be expressed as $\mathrm{\&Omega;}\left(p({\mathrm{<figure-callout\; id="0"\; label="v"\; filenames="C20071006770800171.png"\; state="\{\{state\}\}">v</figure-callout>}}_0,v_S)\right)=\underset{s=0}{\overset{S-1}{\mathrm{\&Sigma;}}}e(v_s,v_{s+1}).$ For the circulation in the overlapping and path of avoiding path node, suppose node v _Mn3 directivity edges are arranged, point to v respectively _{M+1, n}, v _{M, n+1}And v _{M+1, n+1}, that is to say present node v _MnExistence is from v _{M-1, n}, v _{M, n-1}And v _{M-1, n-1}3 possible paths of Zhi Xianging respectively, as shown in Figure 4.Suppose known from v ₁₁To v _{M-1, n}, v _{M, n-1}And v _{M-1, n-1}The minimum cost path, present node v so _MnMinimum cost can be expressed as:

Ω(v _m，n)＝min{Ω(v _i，j)+e(v _m，n，v _i，j)|v _i，j∈{v _m-1，n，v _m，n-1，v _m-1，n-1}}。

Record minimum cost Ω (v _{M, N}) all nodes of process, thereby establish the mapping relationship f (n of pixel between image _i)=m _i

Yet, in seeking the path process, in level or vertical direction the path can take place and gather situation with scattering, make that the path is not strict moves by diagonal positions, in order to reduce the influence to proofreading and correct in these paths, on level and vertical edge, increase compensating factor δ, δ=α c _Max, c wherein _MaxBe the maximum in the cross-correlation matrix, $\mathrm{\&alpha;}=\frac{\underset{i=1}{\overset{R}{\mathrm{\&Sigma;}}}{c}_{i,i}}{\max \{(\underset{i=1}{\overset{R}{\mathrm{\&Sigma;}}}{c}_{1,i}+c_{i,R}),(\underset{i=1}{\overset{R}{\mathrm{\&Sigma;}}}{c}_{i,1}+c_{R,i})\}},$ Promptly as shown in Figure 4, present node v _MnBy through v _{M-1, n}, v _{M, n-1}Or v _{M-1, n-1}The cost of 3 possible paths that node arrives is respectively Ω (v _{M-1, n})+e (v _{M, n}, v _{M-1, n})+δ, Ω (v _{M, n-1})+e (v _{M, n}, v _{M, n-1})+δ and Ω (v _{M-1, n-1})+e (v _{M, n}, v _{M-1, n-1}), and Ω (v _{M, n}) be the minimum value among the three.

In above-mentioned minimum cost path with ask for based on dynamic programming algorithm on the basis in minimum cost path, method for correcting multi-viewpoint vedio color step of the present invention is as follows:

At first a visual point image in the multi-view image that synchronization is taken by many view camera system is as target image, other visual point image is as source images, and view data is transformed into the HSI color space from the RGB color space, and using tone H respectively, saturation S and intensity I component are represented.RGB is shown to the map table of HSI $H=\arccos \left\{\frac{[(R-G)+(R-B)]/2}{{[{(R-G)}^2+(R-B)(G-B)]}^{1/2}}\right\},$ $S=1-\frac{3}{(R+G+B)}\left[\min (R,G,B)\right],$ $I=\frac{1}{3}(R+G+B).$ The dynamic range of H is [0,359], and the dynamic range of S is [0,1], and the dynamic range of I is [0,255].

Calculate the hue histogram of target image ^(T)h ^H= ^(T)h ^H[0], ^(T)h ^H[1] ..., ^(T)h ^H[M ^H] and the hue histogram of source images ^(S)h ^H= ^(S)h ^H[0], ^(S)h ^H[1] ..., ^(S)h ^H[N ^H], M wherein ^HAnd N ^HBe respectively the hue histogram node number of target image and source images, get M ^H=N ^H=360, ask for histogrammic cross-correlation matrix C by dynamic programming algorithm then ^HIn from starting point c ₁₁To terminal point c _MNThe minimum cost path, set up the tone mapping relations from the source images to the target image, be expressed as f (H)=H '.Wherein, cross-correlation matrix $C^H{=}^{\left(T\right)}{h}^H{\&CircleTimes;}^{\left(S\right)}{h}^H=\left[\begin{array}{cccccc}{c}_{11}& c_{12}& .& .& .& c_{1N}\\ c_{21}& c_{22}& .& .& .& c_{2N}\\ .& .& & .& & .\\ .& .& & .& & .\\ .& .& & .& & .\\ c_{M1}& c_{M2}& .& .& .& c_{\mathrm{MN}}\end{array}\right],$ M is the number of the hue histogram node of target image, and N is the number of the hue histogram node of source images, M=N, c _MnThe internodal distance of expression hue histogram is with the criterion of absolute difference as distance, c _Mn=| ^(T)h ^H[m]- ^(S)h ^H[n] |.

On the basis of tone mapping relations, the saturation component of target image and source images is carried out continuous sampling, the saturation histogram of component of definition source images is ${}^{\left(S\right)}h_H^S=\left\{{}^{\left(S\right)}h_H^S\right[0],{}^{\left(S\right)}h_H^S[1],...,{}^{\left(S\right)}h_H^S[N^S\left]\right\},$ The expression tone is the saturation distribution probability of H, and the saturation histogram of component of objective definition image is

The expression tone is the saturation distribution probability of H '=f (H), wherein M ^SAnd N ^SBe respectively the saturation histogram node number of target image and source images, get M ^S=N ^S=101, ask for histogrammic cross-correlation matrix C by dynamic programming algorithm ^SFrom starting point c ₁₁To terminal point c _PQThe minimum cost path, set up the tone from the source images to the target image, the saturation mapping relations, be expressed as f (H, S)=(H ', S ').Wherein, cross-correlation matrix $C^S{=}^{\left(T\right)}{h}_{H^{\&prime;}}^S{\&CircleTimes;}^{\left(S\right)}{h}_H^S=\left[\begin{array}{cccccc}{c}_{11}& c_{12}& .& .& .& c_{1Q}\\ c_{21}& c_{22}& .& .& .& c_{2Q}\\ .& .& & .& & .\\ .& .& & .& & .\\ .& .& & .& & .\\ c_{P1}& c_{P2}& .& .& .& c_{\mathrm{PQ}}\end{array}\right],$ P is the number of the saturation histogram node of target image, and Q is the number of the saturation histogram node of source images, P=Q, c _PqThe internodal distance of expression saturation histogram.

On the basis of color harmony saturation mapping relations, the strength component of target image and source images is carried out continuous sampling, the intensity histogram of definition source images is ${}^{\left(S\right)}h_{H,S}^I=\left\{{}^{\left(S\right)}h_{H,S}^I\right[0],{}^{\left(S\right)}h_{H,S}^I[1],...,{}^{\left(S\right)}h_{H,S}^I[N^I\left]\right\},$ The expression tone is that H and saturation are the intensity distributions probability of S, and the intensity histogram of target image is

The expression tone is that H '=f (H) and saturation are the intensity distributions probability of S '=f (S), wherein M ^IAnd N ^IBe respectively the intensity histogram node number of target image and source images, get M ^I=N ^I=256, ask for histogrammic cross-correlation matrix C by dynamic programming algorithm ^IIn from starting point c ₁₁To terminal point c _UVThe minimum cost path, set up its tone from the source images to the target image, the mapping relations of saturation and intensity, be expressed as f (H, S, I)=(H ', S ', I ').Wherein, cross-correlation matrix $C^I{=}^{\left(T\right)}{h}_{H^{\&prime;},S^{\&prime;}}^I{\&CircleTimes;}^{\left(S\right)}{h}_{H,S}^I=\left[\begin{array}{cccccc}{c}_{11}& c_{12}& .& .& .& c_{1V}\\ c_{21}& c_{22}& .& .& .& c_{2V}\\ .& .& & .& & .\\ .& .& & .& & .\\ .& .& & .& & .\\ c_{U1}& c_{U2}& .& .& .& c_{\mathrm{UV}}\end{array}\right],$ U is the number of the intensity histogram node of target image, and V is the number of the intensity histogram node of source images, U=V, c _UvThe internodal distance of expression hue histogram.

The color map of determining source images and target image through above-mentioned steps concerns, source images is proofreaied and correct.Because video exists extremely strong correlation in the time domain direction, multi-view point video is carried out the video tracking operation, promptly judge according to the similitude of consecutive frame before and after the video, adopt the color map relation of determining that follow-up frame is carried out identical correction.The similitude of consecutive frame is defined as:

$\mathrm{Sim}(h_t^H,h_{t+1}^H)=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}(1-\frac{\left|h_t^H\right[i]-h_{t+1}^H[i\left]\right|}{\max \left(h_t^H\right[i],h_{t+1}^H[i\left]\right)})$

H wherein _t ^HBe the hue histogram of the image of moment t, if constantly t and constantly the similitude of t+1 consecutive frame just recomputate and upgrade the color map relation less than certain threshold value T.In this embodiment, threshold value T is by to providing ' golf2 ' by KDDI company, ' Jungle ' that 620 frame videos of ' object3 ' multi-view point video test set and HHI research institute provide, 100 frame videos of ' Uli ' multi-view point video test set are added up, the result shows the value of similarity of every frame substantially more than 0.6, so selected threshold T=0.6.

Target image tone, saturation, intensity histogram node are counted M in theory ^H, M ^S, M ^ICount N with source images tone, saturation, intensity histogram node ^H, N ^S, N ^IBig more effect is good more, but because the increase of node number can cause the increase of computation complexity, therefore, should take all factors into consideration image rectification effect and computation complexity when choosing the value of node number.

Below carry out the subjective and objective performance that multi-viewpoint vedio color proofreaies and correct with regard to the present invention and compare.

To ' golf2 ' that provides by KDDI company, ' Jungle ' that ' object3 ' and HHI research institute provide, ' Uli ' four groups of multi-view point video test sets adopt method for correcting multi-viewpoint vedio color of the present invention to proofread and correct.Fig. 5 a, Fig. 6 a are ' golf2 ', the target image of ' objects3 ', and Fig. 5 b, Fig. 6 b are ' golf2 ', the source images of ' objects3 ', target image and source images size are 320 * 240; Fig. 7 a, Fig. 8 a are ' Jungle ', the target image of ' Uli ', and Fig. 7 b, Fig. 8 b are ' Jungle ', the source images of ' Uli ', target image and source images size are 1024 * 768.As can be seen from the figure, the color appearance of target image and source images is obviously inconsistent, it is carried out color correction just seem very necessary.Adopt the present invention to carry out image behind the color correction shown in Fig. 5 c, Fig. 6 c, Fig. 7 c and Fig. 8 c, from the subjective vision effect of image as can be seen, adopt its color appearance of image after the present invention proofreaies and correct very close with target image.

Adopt video tracking technology of the present invention that subsequent video images on the time domain is proofreaied and correct, as shown in Figure 9, as can be seen from the figure, compare with Fig. 5 b, Fig. 6 b, Fig. 7 b and Fig. 8 b source images, obvious variation has taken place in video content, but very close through its color appearance of image behind the tracking correction with target image, illustrate that video tracking technology of the present invention is effective.

The image after employing the inventive method is proofreaied and correct and absolute mean square error and Euler's distance of target image, compare with absolute mean square error and Euler's distance without the source images of overcorrect and target image, as shown in table 1, as can be seen from the table, adopt color calibration method of the present invention, absolute mean square error and Euler's distance all decrease drastically, and illustrate that the similitude that adopts the present invention to proofread and correct back image and target image is stronger.

Adopt video tracking technology of the present invention that video image is proofreaied and correct, with do not adopt the video tracking technology and promptly the method that every two field picture carries out color correction respectively compared, computation complexity obviously reduces, and the time of its saving is as shown in table 2, illustrates that video tracking technology of the present invention is effective.

The tone mapping curve that the present invention obtains by dynamic programming algorithm such as Figure 10, Figure 11, Figure 12 and shown in Figure 13, as can be seen from the figure mapping curve is relatively continuously with level and smooth, avoided that the discontinuous situation of color appears in the zone boundary in the color map process, eliminated edge distortion.

Table 1 proofreaies and correct/and the absolute mean square error of calibration source image and target image and Euler be not apart from comparison

Table 2 video tracking/the computation complexity of video tracking technology does not compare

Claims (3)

1. method for correcting multi-viewpoint vedio color, it is characterized in that: it may further comprise the steps:

(1) synchronization is defined as target image by a visual point image in the multi-view image of many view camera system shooting, be designated as T, and other visual point image is defined as source images, be designated as S, target image and source image data are converted into the HSI color space from the RGB color space, 3 color components are respectively tone H, saturation S and intensity I;

(2) calculate the hue histogram of target image respectively ^(T)h ^HHue histogram with source images ^(S)h ^H, calculate the cross-correlation matrix C of hue histogram again ^H, $C^H{=}^{\left(T\right)}{h}^H{\&CircleTimes;}^{\left(S\right)}{h}^H=\left[\begin{array}{cccc}{c}_{11}& c_{12}& ...& c_{1N}\\ c_{21}& c_{22}& ...& c_{2N}\\ .& .& .& .\\ .& .& .& .\\ .& .& .& .\\ c_{M1}& c_{M2}& ...& c_{\mathrm{MN}}\end{array}\right],$ Wherein, M is the number of the hue histogram node of target image, and N is the number of the hue histogram node of source images, M=N, c _MnThe internodal distance of expression hue histogram;

(3) obtain cross-correlation matrix C in the step (2) by dynamic programming algorithm ^HIn from starting point c ₁₁To terminal point c _MNThe minimum cost path, the mapping relations of setting up from source images to the target image tone are f (H)=H ';

(4) on the basis of tone mapping relations, the saturation component of target image and source images is carried out continuous sampling, for tone in the target image is the histogram that the pixel of H ' is set up the saturation component ^(T)h _{H '} ^S., for tone in the source images is the histogram that the pixel of H is set up the saturation component ^(S)h _H ^S, calculate the histogrammic cross-correlation matrix C of saturation again ^S, $C^S{=}^{\left(T\right)}{h}_{H^{\&prime;}}^S{\&CircleTimes;}^{\left(S\right)}{h}_H^S=\left[\begin{array}{cccc}{c}_{11}& c_{12}& \&CenterDot;\&CenterDot;\&CenterDot;& c_{1Q}\\ c_{21}& c_{22}& \&CenterDot;\&CenterDot;\&CenterDot;& c_{2Q}\\ \&CenterDot;& \&CenterDot;& \&CenterDot;& \&CenterDot;\\ \&CenterDot;& \&CenterDot;& \&CenterDot;& \&CenterDot;\\ \&CenterDot;& \&CenterDot;& \&CenterDot;& \&CenterDot;\\ c_{P1}& c_{P2}& \&CenterDot;\&CenterDot;\&CenterDot;& c_{\mathrm{PQ}}\end{array}\right],$ Wherein, P is the number of the saturation histogram node of target image, and Q is the number of the saturation histogram node of source images, P=Q, c _PqThe internodal distance of expression saturation histogram;

(5) obtain cross-correlation matrix C in the step (4) by dynamic programming algorithm ^SIn from starting point c ₁₁To terminal point c _PQThe minimum cost path, set up from source images to the target image tone, the mapping relations of saturation be f (H, S)=(H ', S ');

(6) at tone, on the basis of saturation mapping relations, the strength component of target image and source images is carried out continuous sampling, set up the strength component histogram of target image ^(T)h _{H ', S '} ^IStrength component histogram with source images ^(S)h _{H, S} ^I, the histogrammic cross-correlation matrix C of calculating strength again ^I, $C^I={}^{\left(T\right)}h_{H^{\&prime;},S^{\&prime;}}^I\&CircleTimes;{}^{\left(S\right)}h_{H,S}^I=\left[\begin{array}{cccc}{c}_{11}& c_{12}& ...& c_{1V}\\ c_{21}& c_{22}& ...& c_{2V}\\ .& .& .& .\\ .& .& .& .\\ .& .& .& .\\ c_{U1}& c_{U2}& ...& c_{\mathrm{UV}}\end{array}\right],$ Wherein, U is the number of the intensity histogram node of target image, and V is the number of the intensity histogram node of source images, U=V, c _UvThe internodal distance of expression hue histogram;

(7) obtain cross-correlation matrix C in the step (6) by dynamic programming algorithm ^IIn from starting point c ₁₁To terminal point c _UVThe minimum cost path, set up from source images to the target image tone, the mapping relations of saturation and intensity be f (H, S, I)=(H ', S ', I ');

(8) with from source images to the target image tone, the mapping relations of saturation and intensity are carried out color correction to source images, and are transformed into the RGB color space;

(9) multi-view point video is carried out the video tracking operation, judge by the similitude of consecutive frame before and after the video and realize the vedio color correction.

2. a kind of method for correcting multi-viewpoint vedio color as claimed in claim 1 is characterized in that: in described minimum cost path computing process, increase compensating factor δ, δ=α c on level and vertical edge _Max, c wherein _MaxBe the maximum in the cross-correlation matrix, $\mathrm{\&alpha;}=\frac{\underset{i=1}{\overset{R}{\mathrm{\&Sigma;}}}{c}_{i,i}}{\max \{(\underset{i=1}{\overset{R}{\mathrm{\&Sigma;}}}{c}_{1,i}+c_{i,R}),(\underset{i=1}{\overset{R}{\mathrm{\&Sigma;}}}{c}_{i,1}+c_{R,i})\}},$ Be present node v _MnBy through v _{M-1, n}, v _{M, n-1}Or v _{M-1, n-1}The cost of 3 possible paths that node arrives is respectively Ω (v _{M-1, n})+e (e _{M, n}, v _{M-1, n})+δ, Ω (v _{M, n-1})+e (v _{M, n}, v _{M, n-1})+δ and Ω (v _{M-1, n-1})+e (v _{M, n}, v _{M-1, n-1}), e (v here _{M, n}, v _{M-1, n}) expression node v _{M-1, n}With node v _{M, n}Between distance, Ω (v _{M, n}) be by node v ₁₁Arrive node v _MnMinimum cost, its value is the minimum value of above-mentioned 3 paths costs.

3. a kind of method for correcting multi-viewpoint vedio color as claimed in claim 1 is characterized in that: the similitude determination methods of consecutive frame is before and after the described video: the similitude of definition consecutive frame $\mathrm{Sim}(h_t^H,h_{t+1}^H)=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}(1-\frac{\left|h_t^H\right[i]-h_{t+1}^H[i]}{\max \left(h_t^H\right[i],h_{t+1}^H[i\left]\right)}),$ H wherein _t ^HHue histogram for the image of moment t, if the similitude of t and moment t+1 consecutive frame is less than the threshold value of regulation constantly, then enforcement of rights requires step (1)～(8) in 1 to upgrade the color map relation of current time multi-view point video, otherwise the color map relation of continuing to use previous moment is carried out color correction to multi-view point video.

Images