EP2803039A1

EP2803039A1 - Hair colouring device and method

Info

Publication number: EP2803039A1
Application number: EP13711449.2A
Authority: EP
Inventors: Christophe Blanc
Original assignee: Wisimage
Current assignee: Wisimage
Priority date: 2012-01-13
Filing date: 2013-01-11
Publication date: 2014-11-19
Also published as: WO2013104848A1; US20140354676A1; JP2015511339A

Abstract

The invention relates to a hair colouring device and method. The method can be used to generate a hair colouring simulation using a target image of a user, said method comprising the following steps: an image module receives data from a target image to be processed; an interface module generates a user interface, allowing reception of the selection of at least one colouring zone identification point; after reception of the selection data, the interface module sends the data to a hair mask segmentation module; and the hair mask segmentation module extracts the zone corresponding to the user's scalp.

Description

Method and device for coloring hair

Technical field of the invention

The invention relates to a method for the virtual extraction and coloration of a target person's capillary mask in a substantially real time by using information on the texture of the hair mask provided by the user using a support digital device with or attached to a touch screen.

State of the art

We now know several types of digital technologies that allow a user to simulate makeup, or to virtually test makeup using digital facial photography and color selection and application tools. Despite the multitude of solutions available to users, solutions to achieve realistic results are few. Several reasons explain the imperfections of the proposed solutions. Most often, the difficulty of accurately detecting the various areas of the face to make up is a source of multiple problems.

For applications that also take into account the hair color simulation, in addition to the difficulty of defining the target area to be colored, other difficulties make the color simulation particularly delicate to achieve accurately, reliably and realistically. . Thus, the very high gradient of color variation in very small areas due to the large number of hair, the glare problems, and the fact that a single hair often has different color characteristics, are only a few problems to overcome to perform such simulations.

Several approaches to virtual hair coloration on a digital photograph exist. They can be classified into four groups.

INCORPORATED (E) BY REFERENCE (RULE 20.6) The first set is based on an automatic detection (no information or constraint is made to the detection algorithm) or semi-automatic (a mask with the position of certain elements of the face is used to constrain the position of the person's face target on digital photography) of the target person's face. The objective is to detect and model the face by different characteristic points and a contour of the face. Once this information is determined, a virtual wig is "posed" on the head of the target person. This method allows the laying of a predefined hair and not a virtual hair coloration of the target person.

The second set is based on the use of a capillary mask that can be either predefined or defined by the user as shown in Figure 3. This capillary mask is defined / drawn by several points connected to each other by lines. In the case where this capillary mask is predefined, the user will have to move the points defining the capillary mask, with the aid of a cursor, to adapt this generic mask to the hair of the target person. In the second case, the user must, by means of a cursor, manually define all the points defining his capillary mask. As soon as the capillary mask is well adapted to that of the target person, a hair coloring is applied in the zone defined by these different points. This method is binding for the user because he must define all or part of the capillary mask. In addition, this definition must be as precise as possible so that the hair coloring is located on all the hair of the target person only and not on part of the background or face.

The third set is based on a capillary segmentation approach. For existing methods, the goal is to semi-automatically extract the capillary mask. However, as is, the user must perform several binding actions. First, he must make a photograph of the body of his body centered with a uniform background preferably. Secondly, he must use a mouse to place a rectangle within which his face is located (see Figure 4). Third, always with the mouse and without any user-friendly interface, he must place dots on the image on his hair and on the bottom.

INCORPORATED (E) BY REFERENCE (RULE 20.6) Even if this process is semi-automatic, its use is therefore very restrictive for the user because he must perform many actions under pain of getting poor results in hair coloring.

The last type of approach is based on touchscreen technology. After making a digital photograph and recording it, the user must "color" the area where the hair is located by using his finger with the touch screen. This solution is minimalist because no automatic or semi-automatic detection of the capillary mask is performed. The coloring of the hair will be applied only in the areas defined by the passage of the finger of the user on the touch interface. This solution therefore has two major disadvantages: the user must color all the hair manually and the coloring is uniformly applied to the colored parts without any artificial intelligence.

None of the existing solutions to date makes it possible to take into account in a simple, fast, reliable and efficient way all the technical difficulties previously exposed. Today, there is a great need to find an effective solution to these many problems.

Presentation of the invention

In order to avoid the disadvantages of the approaches described in the preceding paragraph, the invention provides:

A step of providing the user with a useful hair color zone selection interface: during this step, as shown in FIG. 5, the user uses the interface and the selection tool made available to select the hair area. Several selection modes can be proposed, depending on the case. In the illustrated example, the user selects a plurality of points by drawing at least one line in the area of the hair. This variant is particularly advantageous because it allows the taking into account of several points. Since in

INCORPORATED (E) BY REFERENCE (RULE 20.6) Many hair color varies significantly from one point to another, this approach can indicate a series of color data to take into account. In a simplified variant, the user selects at least one point of the area subject to coloration. Also alternatively, the method provides an interface allowing the user to indicate the area surrounding the hair in the background of the subject.

• The method allows for the segmentation of the capillary mask from information previously obtained on the one hand and provided by the user on the other hand. The method therefore provides for the reception of coloring data corresponding to the points and / or zones selected by the user. This input of information makes it possible to initialize in an optimal manner the segmentation algorithm of the capillary mask.

Advantageously, a step is provided for searching the face of the target person in the image. From this data locating the face of the target person, the image is cropped and resized so that it is centered on it. This method therefore makes it possible to reduce the stress on the position of the face during the photographing step.

Thanks to this automatic method, it is no longer necessary to manually define the capillary mask or to force the user to make a photograph with strong constraints on the position of his face in the image. The only action he must perform is to define the points describing the hair texture of the target person using a "friendly" interface.

In an exemplary embodiment illustrated in the figures, the automatic hair dyeing method is based on a tactile interface is illustrated in Figure 6. It is based on the use of a touch interface (slab, mobile phone or digital tablet) associated with a server performing the calculations necessary for the

INCORPORATED (E) BY REFERENCE (RULE 20.6) location of the target person's face, segmentation and staining of the hair mask. The communication between the two entities is based on known means such as a Wifi or Wimax protocol or a cable when a touch screen is used with a PC or server, or any other means for the exchange of data.

To illustrate the process according to the invention, the process is divided into the different steps shown in FIG. These stages are described by relying on the modules necessary for the realization of these. a) Step 0: Obtain an image of the target person

Upstream of the method, a shooting of the head of the target person can be carried out according to two different approaches. Either the user uses a remote digital camera, or he uses the device integrated in the touch support like a digital tablet (laptop without keyboard with touch screen and intuitive user interface such as products marketed under the name "iPad") or a smartphone ("iPhone" for example). A previously existing image can also be used.

In the case where the shooting is based on the use of a tablet or a smartphone, the user can use the camera taking module. It is thus guided by the presence of a target (circle or shaded area) on the screen serving as an interface that allows it to make an optimal shooting ensuring the smooth operation of the application and a simplified use of its point of view. b) Step 1: questionnaire (optional)

Thanks to an interactive graphical interface provided by the tactile support, it is possible to ask questions to the user about his type of hair and the color he wants to apply.

In step 5, this information allows a determination of the color to be applied to the hair corresponding more precisely to the wishes of the user. This step makes it possible to personalize the hair coloring and to respond more easily to the expectations of the user.

INCORPORATED (E) BY REFERENCE (RULE 20.6) c) Step 2: manual selection of the hair

This step corresponds to the determination of the texture of the hair by the user of the invention. It is preferably based on the use of a touch interface. Passing a finger on the hair of the target person as shown in Figure No. 5, the user selects one to several points describing the texture of the hair, area described as the first plane in Figure No. 1. This operation can be repeated identically to describe the texture of the areas forming the background described in Figure No. 1. Once the operation is complete, validation enables the information (points of interest and image of the target person) to be transmitted to the server that will perform steps 3 to 5.

In the end, this step is based on three modules (see Figure 6):

• The module selection points of interest to receive and save the points described by the passage of the finger of the user on the touch support.

• The result display module displays on the touch screen in real time the points determined by the user superimposed on the image of the target person.

• The module sends data allows, once the points selected and validated, to send to the server the image accompanied by the points of interest.

d) Step 3: Face detection and resizing of the image

This step corresponds to a search of the face in the image using an image processing algorithm such as that based on a learning method "AdaBoost" with Haar wavelet descriptor. This algorithm traverses the image in its entirety to search for pixel images, described by the wavelets, identical to the information obtained with the learning provided a priori. Once the comparison is positive, the face of the target person is located in the image.

INCORPORATED (E) BY REFERENCE (RULE 20.6) From this detection, the method performs two operations. It automatically chooses points belonging to the detected face that are identified as belonging to the background. It resizes and reframes the image on the face of the target person.

All of these actions are advantageously implemented by the face detection module (see FIG.

The use of this methodology has the advantage of not constraining the position of the face of the target person in the center of the image during the production of the photograph. e) Step 4: Segmentation of the capillary mask

This automatic segmentation of the capillary mask is advantageously performed by an image processing algorithm named "Grabcut". This algorithm traverses the entire image by searching for the pixels having the same intensity as the pixels associated with the background or the capillary mask provided as input. The goal is to label all pixels in the image as either a background or a hair mask. Then, the algorithm seeks to optimize a boundary between the two classes of pixels always obtained by relying on the strong data input constraints. In other words, the algorithm looks for the best compromise between the two zones by using strong binding information in input.

Thus, the texture of the hair described by the points of interest positioned by the user (hair mask in Figure No. 1) and the texture of the skin of the face obtained with the points detected in step 4 (part from the background in FIG. 1) are therefore used as input to the algorithm (segmentation module of the capillary mask). Since this initialization is correct (strong constraint), the algorithm detects more quickly and more precisely the parts of the image of the target person corresponding to this texture.

Finally, the reception of the colorimetric data of the area of the background makes it possible to exclude from the area to be colored the points whose colorimetric data

INCORPORATED (E) BY REFERENCE (RULE 20.6) are identical or substantially identical to those received for this zone. Then, the reception of the colorimetric data of the face area also makes it possible to exclude from the zone to be colored the points whose colorimetric data are identical or substantially identical to those received for this zone. At the output of this module, we will obtain an area corresponding to the capillary mask with the texture identified by the user of the invention. f) Step 5: Selection of the color

This step is broken down into two distinct parts. First, the color of the hair contained in the hair mask is determined. With this value and the possible information obtained in step 1, the method automatically determines several colors of capillary coloration that can be envisaged corresponding to both the current capillary mask and the user's expectations (color determination module in FIG. ).

These colors are represented by patches as the example shown in Figure 7. These patches represent a palette of the same color with different luminosities, hues and contrasts. This information makes it possible, during step 6, to preserve the different reflections observable on the capillary mask of the target person on the original image. g) Step 6: Creating a layer of the colored capillary mask

This step corresponds to the creation of a capillary mask using one of the patches determined in step 5. This function is implemented by the mask creation module. The objective of this step is to obtain a uniform and realistic coloration of the hair mask. To do this, it is necessary to perform several consecutive treatments applied on the one hand to the color patch determined in step 5, on the other hand to the capillary mask extracted from the image of the target person during the treatment. step 4.

This step breaks down into several distinct parts. In a first part, the color patch and the capillary mask are translated into histograms defined from a colorimetric criterion. For an optimal rendering of the coloring,

INCORPORATED (E) BY REFERENCE (RULE 20.6) these histograms must be in the same area of values according to the colorimetric criterion. If this is not the case, the colors of the target person's capillary mask are transposed into the color patch's definition area. Then, in a second part, the colors of the chosen patch are all extracted by eliminating the duplicates (pixels with identical characteristics). Then, these pixels are classified according to color criteria. In a third part, the same process of sorting pixels is applied to the capillary mask. In a last operation, the sorted pixels of each graphic entity are mapped using the colorimetric criterion previously used.

Starting from the capillary mask extracted during step 4, all the pixels are colored using the chosen patch. However, this coloration is not uniform. It respects the existing reflections (contrast, hue and intensity of the different pixels) on the original hair mask. Thanks to this process, the final rendering of the hair coloring is realistic.

In the example shown, this step is the last performed by the server. Once the layer is made, the server sends the information to the touch support. h) Step 7: Display on a screen

Following the reception of the data, the tactile support displays the result of the hair coloring by superimposing the colored layer with the original image of the target person (see figure n ° 8).

The user can, after observing the first result, select a different color. The tactile support redefines a pattern and displays the result by superimposing it with the image of the target person (see figure 8). i) Step 8: Making a beauty prescription

This step carries out, through a graphical interface, a summary of the previous operations performed by the invention:

INCORPORATED (E) BY REFERENCE (RULE 20.6) ^• A display of the image of the target person with the hair coloring chosen;

• A list of the different products needed for hair coloring;

• Explanations, based on diagrams, to correctly perform hair coloring.

These orders are printed in fine in a pdf format.

The method and the device according to the invention are illustrated and described in the foregoing by a system with two separate entities. Several variants are also possible without departing from the scope of the invention. For example, an all-in-one pc-type computer (with display, microprocessor and other integral elements in one block) can also be used for the implementation of the invention. In another variant, a touch pad with sufficient computing capacity can also be used.

In another variant, the touch interface may be replaced by a known type of mobile cursor remotely operated by a mouse, a keypad, or other known type of moving means. In such an example, the selection made by the user's finger in the previous description is replaced by a selection made by moving the cursor in the relevant area to be selected.

Finally, with both a remotely operable movable cursor and a touch interface, the selection of the area to be colored can be performed by one or preferably a plurality of discrete points included in the area to be colored.

The implementation of the various modules of the coloring device previously described is advantageously carried out by means of instructions or commands, enabling the modules to perform the operation or operations specifically provided for the module concerned. The instructions may be in the form of one or more software or software modules implemented by one or more microprocessors. The module (s) and / or the software (s) are advantageously provided in a computer program product comprising a recording medium or recording medium that can be used by a computer.

INCORPORATED (E) BY REFERENCE (RULE 20.6) and having programmed code readable by a computer integrated in said medium or medium, allowing application software to run on a computer or other device having a microprocessor such as a touch screen tablet.

INCORPORATED (E) BY REFERENCE (RULE 20.6)

Claims

1) A hair dyeing method allowing, from a target image of a user, to generate a hair coloring simulation, comprising the steps according to which:

an image module receives the data of a target image to be processed;

an interface module generates a user interface for receiving a selection of at least one color zone identification point;

after receiving the selection data, the interface module sends the data to a capillary mask segmentation module;

the capillary mask segmentation module extracts the zone corresponding to the user's hair.

2) A hair dyeing method according to claim 1, wherein an interface module provides a coloring choice for the user and receives a color selection.

3) A hair dyeing method according to claim 2, wherein a mask creation module generates a mask for application on the target image and displays the colored image.

4) Hair dyeing device for the implementation of the method according to one of claims 1 to 3, comprising:

an image module capable of receiving the data of a target image to be processed;

INCORPORATED (E) BY REFERENCE (RULE 20.6) an interface module capable of generating a user interface making it possible to receive a selection of at least one color zone identification point;

the capillary mask segmentation module capable of extracting the zone corresponding to the user's hair.

5) Hair dyeing device according to claim 4, further comprising a mask creation module capable of generating a mask for application on the target image and displays the colored image.

6) Hair dyeing device according to one of claims 4 or 5, wherein the interface module is also adapted to provide a coloring choice for the user and receive a color selection.

INCORPORATED (E) BY REFERENCE (RULE 20.6)