CN103295250A - Image colorization method based on L1 mixed norm solving - Google Patents

Abstract

The invention discloses an image colorization method based on L1 mixed norm solving. The image colorization method includes (1) inputting a gray image, (2) inputting initial color lines, namely painting the color lines on different areas of the input image to change the gray image into a color image with the color lines, (3) inducing U and V channel values for utilization of the Y channel value and converting the color image acquired before into the YUV space, and recording the initial color value of the image in the space as b0, (4) according to the fact that neighboring pixels have the similar color values, performing colorization by diffusing the U and V channel values of the initial color pixels to the surrounding, (5) normalizing the objective function for solving the U channel value to obtain the factor xu of the U channel value and the factor xv of the V channel value, and (6) solving the xu and xv according to the normalized objective function and converting the Y, U and V channel values into the RGB(red, green and blue) space. On the basis of an L1 norm algorithm, the final image is natural in terms of color saturation and has optimal visual effects.

Description

A kind of method of mixing the image colorization that norm finds the solution based on L1

Technical field

The present invention relates to technical field of image processing, be specifically related to a kind of method of mixing the image colorization that norm finds the solution based on L1.

Background technology

Color is being taken on important role in the cognitive process of the mankind to the world, for example remove to see black-and-white television after getting used to color TV again, will obviously experience the influence that color dropout causes the picture expressive force.Therefore, add that suitable color might greatly improve visual effect for black white image, movie or television program.Colorize was exactly such technology, just was used to handle the moon image that the Apollo moonfall is obtained as far back as 1970, had obtained very ten-strike again afterwards in the renovation of old film is handled.Even to this day, colorize remains image and handles interesting and challenging research topic of educational circles.

Being different from fields such as medical science, remote sensing is to highlight the pseudo-colours processing that picture material adopts, the colorize here require to black white image give nature, close to real color, belong to false color processing to reproduce its original appearance to greatest extent, strictly to say.The colorize of early stage film can be regarded as artistical making by hand, operator's level is required height, and efficient is lower.Aspect image colorization, occurred some semi-automatic disposal routes in recent years, roughly can be summed up as two classes, be i.e. color transfer and color expansion method.Color transfer is to seek as the colored reference picture of the one or more of getting the look source for pending black white image, utilizes the gray scale coupling to use the color of reference picture as painted result; The common processing mode of color expansion is at black white image local guiding look to be set earlier, manages local color is expanded to the full width scope again.Although various colorize methods are different on concrete coloring mode, all come from the observation of real image and the painted principle that draws based on same, if namely in the image neighbor gray scale close, then its color generally also is similar to; The local similar principle of color that Here it is.Colorize handles a key issue that relates to is how to select suitable color expression of space, and this article is set about from the local similar of investigating color, analyzes the applicability of different colours space in colorize is handled.

Yet a main difficult problem of colorize is that it is a costliness and time-consuming procedure.For example, for colorize one width of cloth rest image, artists' common practices is earlier it to be carried out the zone to divide, and then gives value of color to each zone.Unfortunately, partitioning algorithm often can not correctly be identified the zone of fuzzy or complex boundary automatically, such as the hair of dividing target and the border between face.Just because of this, artists those complicated borders between manual description region of having to.In addition, the film colorize also requires to follow the trail of each regional photographed frame.Existing track algorithm can not effectively be followed the tracks of nonrigid zone usually, and this just requires the more intervention of user to handling again.

In this article, we have described a new interactive dye technology, neither need manually to cut apart accurately, also do not need to follow the tracks of accurately.This technology is based on a kind of Unified frame that can apply to rest image and picture frame.What the user need do depicts the color that this zone need be given at each intra-zone exactly, and need not depict the exact boundary of this image.Based on the above-mentioned constraint that the user provides, our technology just can automatically be diffused into color the uncoloured part of image.But the prerequisite of this technology hypothesis is that the neighbor with similar gray level also has similar color-values.So, colorize problem in this paper just is converted into an optimization problem that can solve with ready-made algorithm.

Therefore, the contribution of this paper provides a kind of simply and efficiently interactive colorize technology, and this technology has greatly reduced user's input.Except can be to the colorize of black white image and film, present technique can also be applied to the painted aspect of weight of image.

A width of cloth mask color needs a width of cloth reference frame to come Freehandhand-drawing to come out at least in your original coloring process of mark.Motion detection is given birth to tracking thereupon, and color is distributed in other frame zones that take place to use automatically.And the color-values at mobile edge flows to assignment by optics, and this assign operation often requires manual control.

Though be used at present the correlated color system of industry and not too be known to the public, these systems remain by defined range and follow the tracks of that each frame realizes as can be seen.A business software BlackMagic for colorize has reacted these inside stories.Though software provides handy brush board and palette to the user, still all rely on the user to import for the work of image zoning.

Welsh has described a kind of semi-automatic technology to the black and white picture colorize, and this technology is to be transferred on this image by the color with reference picture.They find the similar pixel in the reference picture by checking the brightness value of neighbor in the Target Photo, then the value of color of this pixel are composed to the pixel in the Target Photo.This technology can be handled the picture that Luminance Distribution that those different colorize zones have significant difference or texture distribute well.For other situations, the user specifies corresponding reference picture piece must for each zone of Target Photo.Though this obvious technical effects, its requirement to input makes artists to give direct control to output.Because in processing procedure, artists must find the picture that has similar texture to Target Photo, and this picture also must have the color that his expectation obtains.And this processing also is unfavorable for problematic output is optionally adjusted.On the contrary, in our technology, artists can directly select required color, if needed, and can also the broken colour lines improve the result by adding more.In addition, the technology of Welsh et al. can not guarantee to export the continuity of the color of picture, and in some cases, the neighbor pixel with similar brightness may be endowed widely different color-values.

Colorize is a kind of computer assisted procedures of giving value of color for monochrome image or film, a kind of existing colorize method be exactly internal logic according to image with its zoning, respectively value of color is added in each zone then.

Though the obvious technical effects of Welsh et al., its requirement to input makes artists to give direct control to output.Because in processing procedure, artists must find the picture that has similar texture to Target Photo, and this picture also must have the color that his expectation obtains.And this processing also is unfavorable for problematic output is optionally adjusted.On the contrary, in our technology, artists can directly select required color, if needed, can also the broken colour lines improve the result by adding more, thus final the utilization found the solution optimization based on the mixing norm of L1 norm and obtained desired result.

Summary of the invention

The present invention has overcome the deficiency of image processing techniques in the prior art, provide a kind of and mixed the method that norm is found the solution image colorization based on L1, by keeping user's initial input based on L1 norm algorithm to greatest extent and adopt the gradient descent method to make final picture more natural aspect the color saturation, thereby reach best visual effect.

The invention provides a kind of method of mixing the image colorization that norm finds the solution based on L1, comprising:

Import a width of cloth gray scale picture;

The user imports the colored lines of preliminary examination, namely the lines of enameling is retouched in each zone of input picture, and this moment, the gray scale picture became the cromogram that has colored lines;

For the value release U that utilizes the Y passage, the value of V passage, the second step gained colour picture is converted to yuv space, the initial value of color of note picture under this space is b ₀

Also should have the fact of close color-values based on neighbor pixel, the colorize process is spread towards periphery by U, the V value of initial color pixel cell;

The target equation of the above-mentioned U of the asking channel value of now standardizing obtains the coefficient x of U channel value _uCoefficient x with the V channel value _v

Target equation according to after the above-mentioned standardization namely solves x _u, x _v, the value of Y, U, V passage is transformed into rgb space.

Described also should have the fact of close color-values based on neighbor pixel, and the colorize process is spread towards periphery by U, the V value of initial color pixel cell and comprises:

According to the target equation

$J\left(U\right)=\underset{r}{\mathrm{\Σ}}{(U\left(r\right)-\underset{s\∈N\left(r\right)}{\mathrm{\Σ}}{w}_{\mathrm{rs}}U\left(s\right))}^2,$ Wherein,

σ _rThe variance of pixel brightness value in the window r that expression pixel r and neighbor pixel constitute,, the V passage passes through

Find the solution minimum V value, optimize the color-values that equation can be obtained U, two passages of V by the Y channel value of picture by this;

The target equation of the described above-mentioned U of the asking channel value of now standardizing obtains the coefficient x of U channel value _uCoefficient x with the V channel value _vComprise:

Describe and find the solution with the standard mathematical optimization equation that is equivalent to above-mentioned thought and obtain:

$x_u=\underset{x_u}{\arg \min }\arg \min \{{\left|\right|\mathrm{Ax}-b_u\left|\right|}_2+\mathrm{\λ}\underset{k}{\mathrm{\Σ}}{\left|\right|U_k-\stackrel{\‾}{U_k}\left|\right|}_1\},$

Wherein A represents the symmetric matrix of weight between all pixels, b _uThe initial color value b that expression provides the family ₀Carry out the following part that represents the U passage in the vector of back of handling, the constraint of lamda back is constant in order to guarantee the color-values of handling on the initial colored lines in front and back, in like manner can obtain the coefficient x of V channel value _v

The present invention is in the processing of black and white video colorize and the application of black-and-white photograph colorize, provide a kind of and manually simply import the inside implementation algorithm that the preliminary examination color reaches overall colorize based on the user, by keeping user's initial input based on L1 norm algorithm to greatest extent and adopt the gradient descent method to make final picture more natural aspect the color saturation, thereby reach best visual effect.

Description of drawings

In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, to do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below, apparently, accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.

Fig. 1 is in the embodiment of the invention.

Embodiment

Below in conjunction with the accompanying drawing in the embodiment of the invention, the technical scheme in the embodiment of the invention is clearly and completely described, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills belong to the scope of protection of the invention not making all other embodiment that obtain under the creative work prerequisite.

The adjacent pixels point that hypothesis space has close brightness value has similar color-values, the present invention is based on this hypothesis and has proposed a kind of simple colorize method, and this method had not both required that accurate image is cut apart does not need area tracking accurately yet.Afterwards we this colorize problem is converted into one can be with the efficient optimization problem that solves of existing algorithm.In order to obtain better effect, we solve this optimization problem at forefathers with least square method, used the hybrid norm of L1 and L2 to find the solution (for above-mentioned thought is had better understanding, earlier L1 norm and L2 norm are explained as follows: what the L1 norm of vector was represented is the absolute value sum of nonzero element in this vector, and what of nonzero element in this vector are this value can reflect indirectly; Vector the L2 norm represents is the evolution of the quadratic sum of each element in this vector, represent namely that also this vectorial mould is long, reflected more indirectly should vector in nonzero element how much).We have showed that an amount of input also can produce high-quality picture in this article.The main design framework of system such as figure below are shown in Figure 1:

The optimization route of transmission:

Carry out independent processing for the ease of each passage to coloured image, we are transformed into yuv space with the coloured image of rgb space and handle (what wherein the Y passage was represented is the luminance signal of this image, and what U, V represented is two kinds of carrier chrominance signals).Have close chromatic value owing to have the pixel of close brightness value, so finally we can be released the value of U, V passage by the Y channel value, be expressed as follows with mathematical formulae summary:

$\underset{\mathrm{int\; ensity\; channel}}{Y}\⇒\underset{\mathrm{color\; channels}}{U,V}$

Fig. 1 shows and mixes the method flow diagram of the image colorization that norm finds the solution based on L1 in the embodiment of the invention, the present invention is based on a kind of YUV color space that is common in the video, wherein Y represents monochromatic brightness channel (back replaces with brightness), and U and V all represent chrominance channe.

This invention comprise brightness value Y of input (x, y, t), export two chromatic value U (x, y, t) and V (x, y, t).In order to represent conveniently, the brightness value of a pixel of Y (r) expression, wherein r represent (x, y, t).

According to hypothesis mentioned above, we wish to make the neighbor pixel with close brightness value also have close color-values by some constraints.Therefore we are by minimizing the weighted mean of certain picture element point chromatic value U (r) and its neighbor pixel, and available mathematic(al) representation summary is expressed as:

$J\left(U\right)=\underset{r}{\mathrm{\Σ}}{(U\left(r\right)-\underset{s\∈N\left(r\right)}{\mathrm{\Σ}}{w}_{\mathrm{rs}}U\left(s\right))}^2---\left(1\right)$

Wherein

(σ _rThe variance of pixel brightness value in the window r that expression pixel r and neighbor pixel constitute).

In sum, this technology can be represented with following mathematical model:

$x=\underset{x}{\arg \min }\arg \min \{{\left|\right|\mathrm{Ax}-b\left|\right|}_2+\mathrm{\λ}\underset{k}{\mathrm{\Σ}}{\left|\right|U_k-\stackrel{\‾}{U_k}\left|\right|}_1\}---\left(2\right)$

Wherein A represents the symmetric matrix of weight between all pixels, and b represents initial color value b that the user is provided ₀Vector after handling, the constraint of lamda back are constant in order to guarantee the color-values of handling on the initial colored lines in front and back, and then the x that obtains represents the coefficient that the value of U, V passage is represented with matrix A.

Because the characteristic of L1 norm is used the L1 norm wherein, find the solution complexity and increase to some extent though make, but can be more near desired optimal value.

Existing colorize derivation algorithm has been ignored the difference between colorize image chroma and saturation degree and has been done same processing, and so processing must depart from optimal result.At this problem, present technique has also increased a pre-service job on the basis of existing algorithm, and principle is exactly to have provided preliminary examination color-values b according to the gradient difference of brightness in the Target Photo and user ₀, provide the assignment weight of each point according to the gradient difference of initialization color point, provide initialization color-values b then again, can be expressed as with mathematical formulae:

$b=\underset{i}{\mathrm{\Σ}}\frac{w_i}{\underset{i}{\mathrm{\Σ}}{w}_i}{b}_0---\left(3\right),$ Wherein $w_i=e-\frac{1}{\left|\right|y_i\left|\right|}$

y _iThe brightness value of representing i pixel.

Find the solution according to above-mentioned optimization formula then and draw x, and then Ax is final design sketch.

The specific algorithm performing step is as follows:

The first step: import a width of cloth gray scale picture;

Second step: the user imports the colored lines of preliminary examination, namely the lines of enameling is retouched in each zone of input picture, and this moment, the gray scale picture became the cromogram that has colored lines;

The 3rd step: for the value release U that utilizes the Y passage, the value of V passage, the second step gained colour picture is converted to yuv space, the initial value of color of note picture under this space is b ₀(value that comprises U, two passages of V) now utilizes each passage to do following processing;

The 4th step: also should have the fact of close color-values based on neighbor pixel, the colorize process is spread towards periphery by U, the V value of initial color pixel cell.We draw the target equation (be and try to achieve the U value that makes following equation value minimum) of present technique

$J\left(U\right)=\underset{r}{\mathrm{\Σ}}{(U\left(r\right)-\underset{s\∈N\left(r\right)}{\mathrm{\Σ}}{w}_{\mathrm{rs}}U\left(s\right))}^2,$ Wherein,

(σ _rThe variance of pixel brightness value in the window r that expression pixel r and neighbor pixel constitute), similar, the V passage is handled too and (is namely found the solution and make $J\left(V\right)=\underset{r}{\mathrm{\Σ}}{(V\left(r\right)-\underset{s\∈N\left(r\right)}{\mathrm{\Σ}}{w}_{\mathrm{rs}}V\left(s\right))}^2$ The V value that value is minimum), we just can be obtained the color-values of U, two passages of V by the Y channel value of picture to optimize equation by this;

The 5th step: for the ease of finding the solution with software, the target equation of the above-mentioned U of the asking channel value of now standardizing (namely describe and find the solution with the standard mathematical optimization equation that is equivalent to above-mentioned thought) obtains $x_u=\underset{x_u}{\arg \min }\arg \min \{{\left|\right|\mathrm{Ax}-b_u\left|\right|}_2+\mathrm{\λ}\underset{k}{\mathrm{\Σ}}{\left|\right|U_k-\stackrel{\‾}{U_k}\left|\right|}_1\},$ Wherein A represents the symmetric matrix of weight between all pixels, b _uThe initial color value b that expression provides the family ₀Carry out the following part that represents the U passage in the vector of back of handling, the constraint of lamda back is constant in order to guarantee the color-values of handling on the initial colored lines in front and back, in like manner can obtain the coefficient x of V channel value _v

The 6th step: in order to make final colorize result more natural, even the local value of color of different brightness also has difference, so do following processing:

Wherein

(y _iThe brightness value of representing i pixel);

The 7th step: according to the target equation after the above-mentioned standardization, namely the equation solution in the 5th step goes out x _u, x _vSo Ax is U, the V value of final objective figure, at last the value of Y, U, V passage is transformed into rgb space and gets final product.

To sum up, the present invention is in the processing of black and white video colorize and the application of black-and-white photograph colorize, provide a kind of and manually simply import the inside implementation algorithm that the preliminary examination color reaches overall colorize based on the user, by keeping user's initial input based on L1 norm algorithm to greatest extent and adopt the gradient descent method to make final picture more natural aspect the color saturation, thereby reach best visual effect.

One of ordinary skill in the art will appreciate that all or part of step in the whole bag of tricks of above-described embodiment is to instruct relevant hardware to finish by program, this program can be stored in the computer-readable recording medium, storage medium can comprise: ROM (read-only memory) (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), disk or CD etc.

More than the method for mixing the image colorization that norm finds the solution based on L1 that the embodiment of the invention is provided be described in detail, used specific case herein principle of the present invention and embodiment are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, the part that all can change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.

Claims (3)

1. one kind is mixed the method for the image colorization that norm finds the solution based on L1, it is characterized in that, comprising:

Import a width of cloth gray scale picture;

The user imports the colored lines of preliminary examination, namely the lines of enameling is retouched in each zone of input picture, and this moment, the gray scale picture became the cromogram that has colored lines;

For the value release U that utilizes the Y passage, the value of V passage, the second step gained colour picture is converted to yuv space, the initial value of color of note picture under this space is b ₀

Also should have the fact of close color-values based on neighbor pixel, the colorize process is spread towards periphery by U, the V value of initial color pixel cell;

The target equation of the above-mentioned U of the asking channel value of now standardizing obtains the coefficient x of U channel value _uCoefficient x with the V channel value _v

Target equation according to after the above-mentioned standardization namely solves x _u, x _v, the value of Y, U, V passage is transformed into rgb space.

2. method of mixing the image colorization that norm finds the solution based on L1 as claimed in claim 1, it is characterized in that, described also should have the fact of close color-values based on neighbor pixel, and the colorize process is spread towards periphery by U, the V value of initial color pixel cell and comprises:

According to the target equation

Wherein,

σ _rThe variance of pixel brightness value in the window r that expression pixel r and neighbor pixel constitute,, the V passage passes through

Find the solution minimum V value, optimize the color-values that equation can be obtained U, two passages of V by the Y channel value of picture by this.

3. method of mixing the image colorization that norm finds the solution based on L1 as claimed in claim 2 is characterized in that the target equation of the described above-mentioned U of the asking channel value of now standardizing obtains the coefficient x of U channel value _uCoefficient x with the V channel value _vComprise:

Describe and find the solution with the standard mathematical optimization equation that is equivalent to above-mentioned thought and obtain:

Wherein A represents the symmetric matrix of weight between all pixels, b _uThe initial color value b that expression provides the family ₀Carry out the following part that represents the U passage in the vector of back of handling, the constraint of lamda back is constant in order to guarantee the color-values of handling on the initial colored lines in front and back, in like manner can obtain the coefficient x of V channel value _v

Images