WO2007077380A2

WO2007077380A2 - Colour image correction and obtaining of at least one colour invariant

Info

Publication number: WO2007077380A2
Application number: PCT/FR2006/051412
Authority: WO
Inventors: Bénédicte BASCLE; Olivier Bernier
Original assignee: France Telecom
Priority date: 2006-01-05
Filing date: 2006-12-21
Publication date: 2007-07-12
Also published as: WO2007077380A3; FR2895823A1

Abstract

The invention relates to a method of correcting a colour image so as to compensate for changes of illumination, using a neural net that is previously subjected to a learning phase, comprising cycles of propagation and back-propagation on pairs of pixels of different illuminations, during which when a pixel is applied to the input of the net, the associated pixel is applied to its output. According to the invention, - the output of a neuron (N1) of the median layer of the net has its output forced, during the propagation steps of the learning phase, to the luminance value (L) of the output pixel, - after the learning phase, said neuron (N1) has its output forced to a predetermined luminance value (L), while the pixels of the colour image to be corrected are applied to the input of the net so as to obtain corrected pixels at the output.

Description

Color image correction and obtaining at least one color invariant

The present invention is in the field of image processing. More specifically, the invention relates to a color image correction method for compensating for illumination changes, using a neural network. The use of cameras and artificial vision systems, in the field of image processing, makes it necessary to work on images of pixels whose colors do not correspond to those that a human being would perceive on the scenes corresponding to these images. images. Indeed, the appearance of the very diverse colors present in nature change according to the lighting. In order to more reliably determine the colors on the surface of objects, humans and some animals have developed the ability to ignore some of the color illumination, which translates into a certain constancy of colors. colors in human perception. The pixel images obtained by the image acquisition systems do not render this faculty, and therefore do not make it possible to render the color of the surface of an object in a realistic and constant manner. This disadvantage is particularly troublesome for certain applications, for example for controlling the quality of products at a distance.

Several color image correction methods exist in order to attempt to reproduce this color constancy on images obtained by an acquisition system such as a camera. Nevertheless, these methods are often complex and only allow the chromaticity of the colors to be coded by means of a single color invariant.

Indeed, it is possible to define a color according to its chromaticity, coded generally by one or two color invariants, not depending on its iumination, and by its luminance, which depends on it. The use of two Color invariants instead of a single color invariant to define the chromaticity of a color allows for more accurate indexing of colors, and therefore to better represent them. The use of more than two color invariants to encode this chromaticity is also conceivable, but would add a potentially annoying noise to the corrected images by the image correction methods that would use this option.

It should be noted that conventional color coding systems, such as Red / Green / Blue RGB according to English "Red Green Blue" or Hue / Saturation / HSV Value according to English " Hue Saturation Value ", for example, does not effectively separate the chromaticity of a color from its luminance. For example the hue, called "hue" in English, has a certain sensitivity to changes in illumination.

Current image correction methods for correcting images to compensate for illumination changes include the method described in IEEE, "Institute of Electrical and Electronic Engineer". , by SM Courtney, LH Finkel and G. Buchsbaum, published in July 1995 and entitled "A multistage neural network for color constancy and color induction". This process uses a biology-inspired neural network, that is, the behavior of neurons in the human brain, to obtain a corrected color that is invariant to illumination from a color. The neural network used is therefore complex. In addition it allows to obtain only one invariant color by color.

It is an object of the present invention to overcome the disadvantages of the prior art by providing a method and a color image correction device for compensating for illumination changes and using a neural network. This method makes it possible in particular to obtain, from a color of the image to be corrected, one or more color invariants corresponding to this color. Moreover this process is fast, because it makes it possible to correct each pixel independently, whereas other methods of Color image correction requires the use of a larger or smaller neighborhood of each pixel or even the use of the entire image to be corrected.

To this end, the invention proposes a method of correcting a color image of pixels to compensate for its sensitivity to changes in illumination, using a multilayer neural network having an input and an output, submitted before a learning phase. comprising cycles of propagation and back propagation steps, said method being characterized in that:

said neuron network used comprises a middle layer,

during said learning phase, using pairs of learning pixels having undergone illumination changes between them, when a learning pixel of one of said pairs is applied to said input of the neural network, the pixel associated learning is applied to said output of the neural network,

during said propagation steps of said learning phase, a neuron of said middle layer has its forced output at the value of the luminance of said learning pixel applied to the output of the neural network,

and after the learning phase, said neuron has its forced output at a predetermined luminance value, while the pixels of the color image to be corrected are successively applied to said input of the neural network in order to obtain at output the pixels of a corrected color image.

Thanks to the invention, we obtain images for which the lighting changes are compensated so as to make these images closer to our visual perception, by a simple, effective and rapid implementation. The invention also relates to a method for obtaining, for a given color, at least one color invariant, using a network of multilayer neurons having an input and an output, said neural network being subjected beforehand to a learning phase comprising cycles of propagation and back propagation stages, said method being characterized in that: said neural network used has a middle layer,

during said learning phase, using pairs of learning colors having undergone illumination changes between them, when a training color of one of said pairs is applied to said input of the neural network, the color associated learning is applied to said output of the neural network,

during said propagation steps of said learning phase, a neuron of said middle layer has its forced output at the value of the luminance of said learning color applied to the output of the neural network, and after the learning, said given color is applied to said input of the neural network and said at least one color invariant is obtained by at least one output value from the other neurons of said middle layer.

This method of obtaining color invariants has the advantage of a simple implementation, fast, and to obtain more than one invariant color by color.

According to a preferred characteristic of the method for obtaining at least one color invariant according to the invention, for a given color applied at the input of the neuron network, two color invariants are obtained, the middle layer of the neuron network comprising three neurons.

As indicated above, obtaining two color invariants to characterize the chromaticity of a color is interesting to then obtain a good representation of the colors.

According to a preferred feature of these methods according to the invention, the neural network used comprises five layers of neurons. The limitation of the number of layers of the neural network to five, which is a minimal number given the particular structure of the neural network according to the invention, makes it possible to reduce the risks of "over-learning", or "learning by heart", of the neural network. According to another preferred characteristic of these methods according to the invention, the second and fourth layers of the neuron network used each comprise eight neurons.

The value eight for the choice of the number of neurons of the intermediate layers of the neural network seems to be an optimal value which allows a low sensitivity to noise, that is to say, further limits the risks of

"over-learning", and gives good performance of the network in the implementation of the methods according to the invention.

The invention further relates to a color image correction device implementing the method of correcting a color image according to the invention.

The invention also relates to a device for obtaining at least one color invariant implementing the method for obtaining at least one color invariant according to the invention.

The invention also relates to a computer program comprising instructions for implementing the methods according to the invention previously presented.

The devices for correcting a color image and obtaining at least one color invariant, as well as the computer program, have advantages similar to those of the methods according to the invention.

Other features and advantages will appear on reading a preferred embodiment described with reference to the figures in which: FIG. 1 represents a neural network used by the methods according to the invention, FIG. to which this network of neurons is subjected. FIG. 3 represents equipment making it possible to implement the methods according to the invention,

FIG. 4 represents the different steps of a learning phase to which the neural network used by the methods according to the invention is subjected.

According to a preferred embodiment of the invention, the methods for correcting color images and obtaining color invariants use an array of RES neurons shown in FIG. 1. This neural network is for example a multilayer perceptron also called MLP, according to the English "Multi Layer Perceptron", comprising in this embodiment five layers of neurons, including a first input layer, a final output layer, and three hidden layers. It is possible to use more hidden layers, if their number remains odd, but a large number of hidden layers makes the model of color correction implemented by the neural network too complex: the neural network risk in this case of learn noise, a problem called "over-learning".

The second and fourth layers of neurons comprise a number of arbitrary neurons, typically between three and ten neurons, eight neurons, bias excluded, appearing to be an optimal number for the implementation of the methods according to the invention.

The median layer of the neural network includes, excluding bias, as many neurons as color invariants that it is desired to obtain from a given color, plus a neuron. In this embodiment, two color invariants are obtained for a given color, and the middle layer is the third layer. This one therefore comprises three neurons.

In addition, in this embodiment, we work on images of pixels. The colors are coded according to the RGB color coding system. This is why the first layer comprises three neurons, bias excluded, whose input values R _x , G _x and B _x allow to receive the coding of a pixel according to the RGB color coding system. The last layer also includes three neurons, bias excluded, whose output values R _y , G _y and By allow to receive the coding of a pixel according to the RGB system.

Alternatively, we work on images whose colors are coded according to other color coding systems, for example HSV, or chrominance systems of the International Commission on Illumination (CIE) * b ^* and LuV, or still the systems used in television standards such as YUV, YIQ, and YCbCr. The number of neurons in the input and output layers is then equal to the number of dimensions used by the chosen color coding system. Alternatively, it is also possible to work on images whose colors are encoded according to a combination of different color coding systems. For example, HSV and RGB systems can be combined to produce a hybrid HSVRGB color representation. It should be noted that such a representation of color has redundancies. The number of neurons in the input and output layers is then equal to the number of dimensions used by the chosen color coding system.

In this multilayer perceptron, the neurons have a sigmoid activation function, the links between the neurons are associated with weights and the network also includes terms of bias and moments, a learning rate, and a gain associated with the sigmoid function. The notions of bias, moment, learning rate and gain are known to those skilled in the art who work on neural networks. The values of the biases of the MLP neurons are set at the value 1, a value generally used by the person skilled in the art.The gain also has the value 1. The values of the moments are for example fixed at an experimentally chosen constant 0.01. The learning rate is also experimentally set to the value 0.001 The initial values of the weights are randomly chosen between zero and one, but different from zero. The neural network RES is subjected, prior to its use by the methods according to the invention during the use phase φ2 shown in FIG. 2, to a learning phase φ1.

The neural network RES is typically implemented in a software manner in an ORD computer, shown in FIG. 3. The ORD computer implemented the learning phase φλ in a learning module MAP, and the use phase φ2 in a MCO correction module. Each of these MAP and MCO modules implements the RES neural network. The learning phase φ \ requires previously the constitution of a base of BDD image pairs, taken with different CAM cameras, connected for example to the ORD computer in turn. A MAC acquisition module in the computer makes it possible to acquire these images and to record them in the BDD image database. Each pair of images corresponds to images of identical scenes taken before and after a change of lighting. Different lighting is used to create this base of BDD images, for example fluorescent lighting, various interior lighting and natural outdoor illumination.

This image database serves, during the learning phase φ \ of the neural network RES, to cause it to obtain the color color invariants subjected to illumination changes. The weights Pj of the neural network, after this learning phase, converged on local mintma. The neural network RES is then ready to be used during the use phase φ2 to quickly correct an image so as to make its colors less sensitive to illumination.

The learning phase φt of the neural network RES is detailed in FIG. In a first step e1, a subset of learning pixels is selected in the BDD image database in order to be used for the learning phase. For this purpose, a random number of pixels of the image database BDD is selected for example from a constant number of pixels in one of the images of this pair of images. To each pixel thus randomly matched, to the same coordinates as this pixel, a pixel of the other image in the image pair, a selection of pairs of learning pixels is thus obtained. Each pair of pixels corresponds to the same color before and after change of illumination. In a second step e2, a pair of pixels of a pair of images Q shown in FIG. 3 is selected randomly from among the subset selected in step e1. The color of the first pixel is then applied to the input of the RES neural network. In this embodiment of the method according to the invention, its color is coded by the values R _x , G _x and B _x , represented in FIG. 1. The color of the second pixel is applied to the output of the neural network RES. Its color is therefore coded by the values R _y , G _y and B _y .

In a third step e3, the neural network RES is propagated, updating the weights P, by gradient descent. This propagation step nevertheless differs from a conventional propagation step in that one of the neurons of the middle layer has a forced output at a given value. Indeed, during this step e3, the output value of the neuron N1 of the median layer of the neural network RES is forced to a luminance value L equal to the luminance of the second pixel, according to the equation;

3

In a fourth step e4, a backpropagation of the neural network RES is performed, the only difference with respect to a ciassic backpropagation of multilayer perceptron being that the error is not retro-propagated on the output of the neuron. nl Steps e2 to e4 are then cyclically repeated until the weights Pj of the RES neural network converge to local minima. At the end of each cycle or learning epoch, the new weights are validated on a set of test pixel pairs. These pairs of test pixels are randomly selected in the BDD database in the same manner as the learning pixel pairs, but so that the subset of test pixels is distinct from the subset of pixels. learning. For example, the test pixels and the learning pixels are randomly drawn on different image pairs. This combined use mechanism during learning of training data and test data, the data here being pixels, is known to those skilled in the art. This validation of the weights Pj by test pixels makes it possible to verify that the neural network has not performed "over-learning". At the end of this learning phase 01, the output values λ and μ of the two other neurons of the middle layer, shown in FIG. 1, define the chromaticity of the color of a first pixel applied to the input of the RES network, and a second pixel applied to the output of the network RES. Indeed, the network has been trained during this learning phase, to reconstruct at best the color R _y G _y B _y of the output pixel from its luminance L and intermediate output values λ and //. Since the luminance L is directly a function of the illumination, the output values λ and μ are therefore related to the invariant part of the illumination of the color of the output pixel.

During the utilization phase <ρ2, the RES neural network with the weights P, learned is used on an IM image, shown in Figure 3, pixel by pixel. For this, each pixel of the image IM is applied to the input of the neural network RES and propagated to obtain the color invariants of this pixel, given by the output values λ and μ of the two neurons other than Nl of the layer median network RES, This gives an image of color invariants corresponding to IM image. If instead of obtaining an image of color invariants, it is desired to correct the IM image to obtain a corrected color image IM ₀ , each pixel of the IM image is also successively applied to the input of the neural network RES, and propagating each pixel in the network while forcing the output of the neuron N1 to a predetermined luminance value L. This luminance L is arbitrary, it is for example chosen such that the corrected image is illuminated by a uniform white light. In this case the value of the luminance L is the same for all the pixels of the IM image. The pixels obtained successively at the output make it possible to reconstruct, pixel by pixel, the corrected image IM _C corresponding to the IM image in which the colors have been made closer to our visual perception.

As a variant, the value of the luminance L applied at the output of the neuron N1 for each pixel of the image IM is chosen proportional to the luminance value of this pixel. In this case the value of the luminance L is different according to the pixels of the image IM. The pixels obtained successively at the output make it possible to reconstruct, pixel by pixel, a corrected image IM _C having illumination, for example less important compared to the original image IM.

Claims

1. Method for correcting a color image (IM) of pixels to compensate for its sensitivity to changes in illumination, using a multi-layer neural network (RES) having an input and an output, previously subjected to a learning phase (01) having cycles of propagation steps (e3) and back propagation steps (e4), said method being characterized in that: said used neural network (RES) comprises a middle layer,

during said learning phase (01), using pairs of learning pixels having undergone illumination changes between them, when a learning pixel of one of said pairs is applied to said input of the neural network (RES), the associated learning pixel is applied to said output of the neural network (RES),

during said propagation steps (e3) of said learning phase (01), a neuron (N1) of said middle layer has its forced output at the value of the luminance (L) of said learning pixel applied to the output of the neural network (RES), and after the learning phase (# 1), said neuron (N1) has its forced output at a predetermined luminance value (L), whereas the pixels of the color image at correct (IM) are successively applied to said input of the neural network to output the pixels of a corrected color image (IM _C ).

2. Method for obtaining, for a given color, at least one color invariant, using a multilayer neural network (RES) having an input and an output, said neural network (RES) being subjected beforehand to a phase of training (<ρi) comprising cycles of propagation (e3) and backpropagation (e4) steps, said method being characterized in that: said neural network used (RES) comprises a middle layer,

during said learning phase ($ 1), using pairs of learning colors having undergone illumination changes between them, when a learning color of one of said pairs is applied to said input of the neural network (RES), the associated learning color is applied to said output of the neural network (RES),

during said propagation steps (e3) of said learning phase (01), a neuron (N1) of said middle layer has its forced output at the value of the luminance (L) of said training color applied to the output of the neural network (RES),

and after the learning phase (φ 1), said given color is applied to said input of the neural network (RES) and said at least one color invariant is obtained by at least one output value (λ, μ) among the other neurons of said middle layer.

3. Method for obtaining color invariants according to claim 2, characterized in that for a given color applied at the input of the neural network (RES), two color invariants are obtained, the middle layer of the neural network (RES). having three neurons.

4. Method according to any one of claims 1 to 3, characterized in that the neural network (RES) comprises five layers of neurons.

5. Method according to claim 4, characterized in that the second and fourth layers of the neural network (RES) each comprise eight neurons.

6. Device for correcting a color image implementing the method according to claim 1,

7. Device for obtaining at least one color invariant implementing the method according to claim 2 or 3.

8. Computer program comprising instructions for implementing the method according to any one of claims 1 to 5, when executed on a computer.