FR2920560A1 - Three-dimensional synthetic actor i.e. avatar, constructing and immersing method, involves constructing psychic profile from characteristic points and features, and fabricating animated scene from head of profile and animation base - Google Patents

Abstract

The method involves morphing a three dimensional head (26) for matching characteristic points of a head with characteristic points (2) of a face (19) by considering characteristic features (23). A texture of an extracted face is corrected in brightness and chrominance, and a sample of the extracted texture of the face is joined on hidden parts of the deformed head. Fabricated accessories are added in synthesized images taken in a base. A psychic profile is constructed from the points and features, and an animated scene is fabricated from the deformed, mapped head of profile and animation base.

Description

The present invention relates to a method for automatically manufacturing and immersing 3-dimensional synthetic characters capable of animation from 2-dimensional face shots obtained under random shooting conditions; this process is intended to produce manipulable synthetic characters resembling the original photo, and to immerse them in scenes whose animations reflect the psychological characters close to those of the person who served as subject to the photo.

The development of multimedia communication tools (especially virtual worlds, whether gaming like World of Warcraft (registered trademark), 8 million subscribers, or simply exchanges, such as Second Life (registered trademark), 3 millions of registrants) creates the need to find in a virtual world an image that can represent the user. Otherwise, the development of community video sharing sites encourages users to stage themselves in increasingly risky scenarios (eg e-suicide published on Scoopeo (registered trademark) in early 2007).

Today we use as representation of the user mainly avatars.

These are selected and configured manually by the users according to their tastes. Each user can try to distort the avatar so that it looks like him, with more or less sophisticated systems, ranging from the deformation of the head to the plating of a photo. In general, these systems are interfaces with pre-configured professional software, which allows the synthesis of faces. Indeed, professional software is most often intended for experts (MAYA, 3DSMax, Softimage - trademarks), and are not directly exploitable by the general public.

However, a very public use of such a function can only develop if the creation of the avatar resembling the user is completely automatic.

The disparity of the human face shapes, associated with the disparity of the conditions and the qualities of shooting inherent in the project (conditions general public) are two conditions which do not make it possible to use only solutions based on the resemblance with an example , or solutions based on the deformation of a model, to set up a 100% automatic process of recognition and detection of the relevant parameters of a face.

The particular case of the hair, very important in the process of identification to a face is in itself a complex problem: an American study of July 2006 shows that the detection of hair has a success rate of the order of 50 % on a series of photos taken under homogeneous lighting conditions. Moreover, it is not known to create real hair animations, the invention therefore proposes a mechanism for superimposing an artificial hair on a head resembling a real character.

All by itself, the problem of the erasure of glasses (which must be solved in order to animate a spectacle-wearing face by making them an independent object), which consists in identifying the possible presence of glasses on a photograph , then to erase the face by recreating the missing parts, then to add in 3D a pair of artificial glasses resembling the original pair has no solutions to date.

The problem of rebuilding a 3D volume from a single 2D image (necessary if the avatar must rotate with respect to the camera) is also complex. It is theoretically soluble to the extent that the object is of a known form. However, in this case, there is a great disparity in human profiles that does not allow for an approach based exclusively on a model.

Downstream, other types of problems must be solved, around the automatic creation of scenes when the characters are not known in their definitive forms at the time of creation. For example, collisions between forms that interpenetrate each other. This happens especially when it comes to preparing scenes in which the avatars will immerse themselves and will have movements defined directly by users (as in games for example).

The addition of a layer of artificial intelligence to give a more natural behavior to avatars according to situations encountered is also a problem still largely fallow, on the borders of artificial intelligence and behavioral sciences, which finds First, applications in the psychological problem solving of group dynamics or group management of people in crisis. Placing a real psychological profile on an artificial psychological contour is also a new subject.

The state of the art tells us that there are systems for creating avatars based on the physical appearance of the user, built from photos and parameters entered by the user. In the majority of cases, we go through the representation of N characteristic points of the face, which serve as a bridge between the real photo and its virtual representation. N can vary from thirty to nearly 200 points.

Thus, the company Gizmoz (registered trademark), for example, creates similar avatars by asking in advance a certain amount of information about the user (so the process is not entirely automatic), and by limiting the possibilities of animation of the avatar not to reveal its limits (the hair in particular are not animated).

More specifically, the detection of faces in images is a fairly well-known process (the company LTU Technologies for example markets Image Seeker (registered trademark), an image indexing system). Many books describe various techniques adapted to face recognition for security purposes, identity verification (up to product marketing, for example the company Bioscrypt (registered trademark) in Canada which announced a product in March 2007, or the company PolarRose (registered trademark), in Sweden, whose current beta version offers to recognize faces in photos), for the security of PCs in enterprise, management of multimedia databases or creation entertaining content. Recent advances in form recognition and learning are helping to develop more and more robust techniques for these uses.

Osuna's work (Osuna and Girosi, 1997), in which the authors propose an original detector based on a support vector machine to classify faces into two classes: faces and non-faces.

Support vector machines (SVM) have often been used as shape detection and thus face detection tools, even in real-time applications, or tolerant of small rotations of the face, with for example an application to remote monitoring [Carminati , 2003]. 25 Detecting a face, then inside a face of the characteristic places (eyes, mouth), is the subject of works among which we can mention those of Rachid Belaroussi (University Pierre and Marie Curie, 2005), based on the fusion of three detectors (ellipse, appearance, flesh) for the face, and the detection of local maxima of gradients associated with a Chinese transformation. The detection success rate is of the order of 90% in this case.

In another example, patent WO0163560 3D game avatar using physical characteristics describes a system that includes a user interface for receiving data, a method for creating 3D representations, and an interface that allows the creation of 3D environment to integrate the data. created representations. This patent explicitly mentions various means for the user to inform the system so that its image can be modeled in 3D, and specifies that it could incorporate a feature recognition process without indicating how the latter method would work. All the methods described in this patent are based on one or more actions of the user beyond the simple sending of a photo.

EP 1510973 Method and apparatus for image-based photorealistic 3D face modeling describes a method composed of the following phases: detection of characteristics in front and profile images; generating a 3D head by adapting a generic head; generation of a realistic texture from face and profile images; display of the texture on the 3D head. The methods described refer to generic models for each element of the face to be reconstructed. It is also necessary to take two photos to make a reconstruction in 3 dimensions.

The patent 6,044,168 of March 28, 2000 entitled Model based faced coding and decoding using feature detection and eigenface coding is based on a model of face in 3 dimensions to analyze unknown images in input and to reconstruct at the exit faces resembling, by mapping of texture in using eigenfaces technology. In this example the search is done by comparison with a theoretical model of face that can recognize where are the key elements of the face: eyes, nose, mouth.

The article "Hierarchical wavelet networks for facial feature localization" describes a wavelet technique for locating the characteristic features of a face. This technique is done in two stages and by comparison with a database of faces.

The article Primitive-based image coding technique for still pictures describes a compression technique by projection of an unknown image on a space of primitives obtained by coding an already known database.

Component-based LDA method focuses on the problem of face recognition in the absence of a large database of faces, by deformation of a single reference face. View-based and Modular Eigenspaces for Face Recognition presents a solution for recognizing a face in a base of thousands of faces, looking for a unique signature independent of the exposure modes of the photo.

Patent WO 2004/040502 describes the method for erasing the glasses of a person's photo. This method is based on the difference between the texture of the eyeglass frame and the surrounding texture30 to erase the frame. It should be noted that this method does not make it possible to erase sunglasses for example.

Patent No. WO 99/27486 of June 3, 1999, describes a method of image recognition with respect to a model by eliminating different parts (wearing occasional glasses for example).

The patent entitled 3D object recognition No. US 2006/0039600 of 23 February 2006 indicates a 3D object recognition method which is known a 2D image by extension of this image in 3D, comparison with a 3D database, and classification. The learning phase is based on a set of 2D projections of a 3D object.

Many other examples exist, but are even more laborious in the way of analyzing the face.

Other studies attempt to automatically create expressions for pre-existing avatars (Marco Paleari and Christine Lisetti, Humaine summerschool, September 2006, Italy) Many studies propose to detect expressions of faces to reproduce them on avatars ( see acquisition and animation of realistic clones for telecommunications, AC Andres del Valle, JL Dugelay, Garcia E, S Valente, Eurecom Institute, 2000).

This state of the art shows a lot of work, but relatively mixed results.

In particular, the detection of elements such as the eyes, the nose or the mouth to take these 3 examples especially treated, sufficiently precise to be able to place there characteristic points which will be the support of the animations, in a sufficiently robust way to be able to hold account of general shooting conditions, and sufficiently simple to be satisfied with a single 2D photo as given, is an unresolved problem. Similarly, the automatic construction of an animation-capable avatar physically and psychologically similar to a person with only a 2D photo, and its automatic immersion in 3D scenes where it plays a role resembling its character was not made.

Compared to this state of the art, the object of the present invention is a particularly robust and fully automatic method for constructing a 3D avatar resembling a 2D model picture taken in random view conditions, to associate psychological characteristics with it. close to those of the model, and immerse in scenes shot in computer graphics whose scenarios are adapted to obtain a realistic rendering both physically and psychologically.

In synthesis, it is a method for automatically manufacturing and immersing in 3D scenes a 3-dimensional synthetic character made from a 2-dimensional photograph characterized in that it comprises: a reception step of photo (16) through a transmission channel (17), a step (18) for extracting the face or faces (19) present in the photograph, a step (20) for determining the characteristic points (2) of said face by comparing the area surrounding each feature point with the equivalent area in the faces of a predefined database (21) and by consistency testing the relative positionings of the points with respect to a model (39), a step (22) determining the characteristic features (23) of the extracted face (s), by calculation from the found characteristic points (2), by comparison with the equivalent parts in the faces of the predefined database (21) or with pre-calculated ccessories (24), a step (25) of morphing a 3D neutral head (26) to match the characteristic points (2) of the head with those from step (20) taking into account the features features (23), a step (27) for mapping the texture of the extracted face (19) and corrected in luminance and chrominance (28) by a process (29) and for gluing a sample of uniform texture extracted from the face ( 28) in a position determined by (22) on the hidden parts of the deformed head (30), a step (31) of adding accessories manufactured in computer images taken in the base (24) and corresponding to the face ( 19) as detected by step (22) of features (23) on the deformed and mapped head (32), a step (33) of psychological profile construction (34) defined from the characteristic points (2) and features (23), which is associated with the deformed, mapped and accessorized head (35), and a step (36) for producing an animated scene (38) from the deformed, mapped and accessorized head (35), the psychological profile (34), and an animation database. (37).

More precisely, to represent by a spiral vector the area around each characteristic point makes it possible to overweight the close neighborhood of the characteristic point.

The identification of the characteristic points (2) as such is a mixture of statistical analysis solutions that is the reduction of the representation space by means of a principal component analysis, followed by a classification based on the algorithm of the k nearest neighbors, and of solutions based on a model, that are the coherence tests taking into account the particular topography of each zone of the face, the coherence of the zones of the face between them, points of each zone between them and the proximity of the barycentre of the statistical cloud to the points considered as good candidates in the database.

As for the characteristic features, they include theoretical models of beards, haircuts and pairs of glasses, which are used for recognition and are then used in the synthesis of the final head in 3D. For the recognition of the hair is used the piecewise integral of a characteristic curve of the thickness of the hair according to cuts passing through a central point of the face to classify the unknown haircut among a finite number of typical haircuts prepared in computer graphics. The detection and the erasure of glasses on the image to be analyzed, is done by identification with respect to standard glasses by means of an edge detection, then by a reconstruction of the pixels (5) hidden by the glasses, by proceeding the extension of the immediately peripheral and visible zones around the glasses of the image to be analyzed by analogy with the equivalent areas on images of a known database of faces without glasses. The third dimension of the detected points on the 2D image to be analyzed is calculated by associating all or part of the 2D projections of the characteristic points found all or part of those of the 3D head taken in a database of 3D heads having a 2D projection. its closest characteristic points, and assigning to the characteristic point being computed the third dimension of the point of the corresponding model, or an average of the third dimensions of the nearest points. The invention is enhanced by applying an emotion detector to the reconstructed face, and moving the feature points (2) to obtain a neutral emotion prior to the morphing step. The psychological profile is determined by a morphopsychology technique that uses the coordinates of the characteristic points to evaluate distances, angles, areas and volumes specific to each face, so as to classify each face in a pre-determined psychological profile. whose interactions with other profiles or in key situations is defined and is used in the choice of the evolution of a scene divided into several tables, with several pre-written possibilities for each table, among which possibilities one is chosen in profile function.

This process is broken down into three main steps as described in Figure 18.

A first step ((20), (22)) of extraction of characteristic points and features of a face has the originality of relying both on a statistical analysis of resemblance to a database real faces (21), and an analysis of relevance to theoretical models of faces. This coupling offers several advantages, such as the robustness compared to shooting in difficult lighting and position conditions, or the extraction of points or hidden lines on the photo, or the 3D reconstruction from the image. a single photo in 2D (16), or the recognition and classification of accessories such as beard, hair, or glasses. It also allows a constant improvement of the detection process by increasing the learning database, while avoiding the drifts inherent in statistical methods (especially computation time) by including a modeling dose.

A second modeling step ((25), (27), (31)) has the originality of mixing on the same 3D object textures from real data and objects (such as glasses, eyes, interior of the mouth or the hair) selected from a base of 3D objects (24). This coupling offers the advantage of a better quality of animation. This step can be fully automatic also thanks to the relevance of the characteristic features determined in the first step. Indeed, it is important to note that this second step is closely related to the processes implemented during the first step.

The third step (36) is that of the animation of the object constructed after the steps 1 and 2. It consists in creating animated scenes from a generic object (26), which will be replaced at the moment manufacturing by the object (35) from steps 1 and 2. These scenes (37) may include a fine animation of the 3D model. This animation will be automatically transposed to the manufactured object, giving the illusion of directly animating the avatar character. In addition, several animations can be planned in parallel in the same scene, an algorithm choosing at each moment the good animation function of the psychological parameters determined in the first step, to lead to a final scene (38) representative of the expected behavior of the character regard to his psychological profile.

This invention thus offers the greatest ease of use since it is not necessary to take several photos, or to inform the program with additional information, or to have a photograph correctly taken, to extract the characteristic features. . This invention is particularly personalized, since it deduces directly from a single photo a 3D avatar resembling the character present in the photo, and staged in animations with behavior similar to that of said character.

Animations are pre-defined, without user intervention. All movements are known, the design of the animations will take into account possible deformations of the model head to avoid collisions during the calculation with the final 3D avatar. Several animations are possible for the same scene. The choice of animations selected depends on the psychological profile found for the avatar.

This invention is particularly useful in the case of shooting with mobile phones equipped with medium quality cameras, whose user interface must remain very simple. But these cameras today represent the largest volume of cameras sold in the world, and are potentially the first source of images. The arrival of television reception or other sources of animated images on the same phones makes sense of this invention in a remote mode: the phone sends the photo, and receives a personalized animated scene.

According to the present invention, and as explained in Figure 18, a picture (16) is first transmitted to a central computer by any means of communication (17) (eg Internet transmission or MMS transmission). The photo is scanned (18) for areas that are likely to be faces. Then, each area corresponding to a face is isolated and reformatted (19). Each face thus isolated is compared to a previously constituted base of faces (21), whose characteristic points (to the number between 30 and 200) have been indicated, as well as a number of accessory features such as the beard, the hair, glasses or azimuth (angle of rotation relative to the vertical axis of the face), or lighting. The comparison methods concern the comparable elements of the face between the base and the face to be analyzed. They make it possible to evaluate the most probable position for the present characteristic points (visible) on the face to be analyzed, by statistical analysis of the faces of the base. The missing elements in the face to be analyzed are finally reconstructed from the average of the elements present in the base, weighted by the coefficients found on the elements present. This method makes it possible to find and place with great precision the characteristic points of an unknown face without resorting to the slightest manipulation on the part of the user, nor to sophisticated models of face modeling. In addition, it allows to reconstitute hidden parts of the face to be analyzed, for example behind glasses or a beard. It also makes it possible to print a rotation along a vertical axis to a face that has not been taken from the front during the shooting. It finally allows a 3D reconstruction and correction of expressions.

The first step of face detection in an image can use different methods: Hough Transform, to detect the ellipses corresponding to the presumed faces. Gabor wavelet lattice representation, Gaussian estimation and maximum likelihood, particularly adapted in this case since there is a base of faces on which we can apply a principal component analysis and apply the research on the principal components of the image by comparing them to Eigenfaces (a method that uses a representation of the characteristic elements of a face image from grayscale model images). Variants of Eigenface are frequently used as a basis for other recognition methods. The maximum likelihood method then determines the probability that an image belongs to a given class based on the Kullback-Leibler divergence.

Each of these methods is applied as a function of a strategy of moving the image through smaller and smaller areas centered on each pixel. A classifier is then interrogated on each zone, for example the Fisher classifier, or a vector support machine (called SVM or vector support machine).

Once the face is detected, it is straightened (relative to an axis perpendicular to the plane of the photo) and standardized to offer a number of pixels and a constant height / width ratio. This format must correspond to the format of the faces of the base, themselves already indexed of the characteristic points, and listed according to all the cases (type of beard, glasses, hair and azimuth).

A classifier determines the accessory elements. The body of the method explained below makes it possible to find the characteristic points not hidden by one of the accessories. In the case of a rotation, this operation can be done after straightening the unknown face once its determined angle, or on a subset of the base corresponding to faces having the same rotation. Then we explain how to reconstruct the hidden characteristic points, as well as the third coordinate of all the characteristic points.

To identify the visible characteristic points, we use a method based on two algorithms: one for the reduction of the representation space and the other for the classification. For reduction, a principal component analysis is used. For the classification, is used the algorithm of the k nearest neighbors. To do this, we represent each point by the area around it in the image. The pixels of this area are then spread out to form a first large vector. This area should be large enough to accommodate enough information, but small enough that the difference between two nearby points is substantial. We can possibly replace the points of this area that overflow the image with gray dots. This makes it possible to always have the same initial dimension, and thus to drive the reduction algorithm once. Using a representation of the area around each point a spiral vector avoids too many coordinates, and at the same time has many close coordinates and some distant coordinates.

The reduction algorithm is driven on a subset of homogeneous characteristic points (or separately for each point), as well as for points deemed to be invalid.

The following operations are then carried out for each of the characteristic points sought, preferably in the order of the easiest point to be detected at the least difficult point (In this way the points already placed with certainty can be used to assist in placing the new more problematic points): - Selection of the relevant images of the base: We identify the area of the sought point (nose, forehead, left eye, right, mouth, chin, left side, right, left eyebrow, right, cheek left, right. ) The relevant images are those for which this area is visible. Indeed the base is indexed according to these criteria: visible front, visible ears, beard or not. It is also possible to perform filters on various conditions of the image so that the selected images correspond to conditions similar to the conditions of the new image. The base can therefore be indexed according to discrete values of illumination, presence or absence of glasses, a beard, degree of rotation of the face, expression. A base containing a sufficient number of cases corresponding to each association of these conditions makes it possible to solve all the cases. - cutting of the image: on each image, we detect the rectangular zone in which the probability is greatest to find the eyes or the mouth. This detection is done by calculating the vertical gradients on the image transformed into gray levels, then looking for the maxima of the sum of these gradients calculated on a rectangular zone of approximate size identical to that of the eyes or the mouth. The rectangular areas corresponding to the nose, the ears, the jaw, or the forehead are deduced from the first.

First positioning test: Depending on the area of the desired point, a mask of relevant points is determined (the position of which is likely to give an idea of the position of the desired point). We make the intersection of this mask with the points already placed with certainty.

If this intersection is empty, place the new point at its average position relative to the corresponding rectangular area in the relevant images of the base. If it contains an element, the new point is placed at its average position in the base relative to the first one. If it contains more than one element we take the mask. The mask is a vector representing the distances of these points to their center of gravity. We take in the base the most resembling the mask of the image, we take the position of the sought point relative to the centroid in the corresponding image of the base, and place the new point relative to the centroid of the mask in the new image, possibly correcting the homothety.

- Training of the detection algorithm: In each image of the base, one draws a hundred points located in the immediate vicinity of the sought point, whose position is known for each image of the base. We take for each of these points the centered area, which we transform into a vector whose dimensionality is reduced by using the reduction algorithm. The detection algorithm is then trained by specifying that these points are out-of-class and that the searched points are in-class. So we have about 99% out-ofclass and 1% in-class points in the training. Figure 1 (Distribution of the points recorded in each image of the database to drive the detection algorithm for the first point sought) gives an idea of the distribution (1) of these points.

- Search around the first positioning test: We go around an area around the first test and we call the detection algorithm on each point of this area. Among the points for which the classification algorithm gives a maximum confidence index, the center of gravity is calculated, and the nearest point of this centroid is chosen as the most probable.

- Inter-point coherence test: The algorithm searches for points in a predetermined order that favors points that are deemed easy to find. For each zone (each eye, nose, mouth, etc.), the most difficult points to find are tested once found to verify that their position is consistent with that of the first points easy to find. If it is not, the algorithm drops the found point and looks for a new candidate.

Global Likelihood Test: For each zone, the confidence indices found for each point (number of nearest neighbors known as in-class) are summed. If the global confidence index is not good, the algorithm is restarted for the zone, widening the search area of the first point.

Figure 2 (An unknown face indexed by its characteristic points) shows an unknown face on which were placed the characteristic points (2) found by this method before optimization.

Several accessory elements, or characteristic features are useful for a good definition of the face and a correct deformation of the synthesis head provided in step 2 of the invention. It is, among other things, the hair, the color of the eyes, the inclination of the face relative to a vertical plane orthogonal to the plane of the photo, a forehead zone corresponding to the skin texture of the subject, the beard, the glasses.

The goal of haircut detection is to be able to associate with the generated 3D avatar an artificial haircut as close as possible to its actual cut. The artificial haircuts are listed according to 5 criteria: height of hair on the forehead, length of hair on the sides, volume of the hair on the sides, percentage of covered front, symmetry of the hairstyle on the front. It is therefore a question of measuring these 5 parameters on the image of the face to associate a model of haircut which will be renewed on the 3D avatar.

The haircut is characterized by a curve. As indicated in FIGS. 3 and 4, this curve is representative of the thickness of hair taken on the half-line (3) starting from the center of the characteristic points above the eyebrows and forming an angle 1 with the ordinate axis. .

cl) moves in a clockwise direction by taking (1) = 0 when the line is vertical.

FIG. 4 shows an example of a thickness curve (5) associated with the detection of the interface points (4) between forehead and hair shown in FIG. 3.

The curves calculated for the different types of hair thus have characteristics that are simply differentiable for the different types of haircuts. An additional advantage of this method is a low computation time compared to the calculation times involved by image processing algorithms. Indeed, we will perform the calculations on 1D signals and not 2D signals.

The thickness curve has properties that are directly related to the haircut considered.

So its integral gives the volume of the hair zone. Its integral limited to values of (1) between 0 and 45 ° on the one hand, and 0 and -45 ° on the other hand gives the symmetry of the hairstyle 25 on the front. The other values of the parameters necessary for the identification of the hairstyle with respect to the reference sections are deduced in the same way, starting from sorting of the values of certain points or by extracting the angles corresponding to maxima or minima.

To define the curve, one begins by reducing the information contained in the image, by suppressing the information contained below the eyebrows, as shown in FIGS. 5, 6 and 7, which show the automatic elimination of the non-visible areas. relevant (6) of the face.

Then we define the half-lines starting from the pixel closest to the coordinate of the center of the segment delimited by the characteristic points of the upper eyebrows. Noting A20 and A16 these two points we have the center point coordinate: O which is defined as follows: O (x) = integer ((A20 (x) + A16 (x)) / 2) 0 (y) = integer ((A20 (y) + A16 (y)) / 2) With

P (x) the abscissa of the point PP (y) the ordinate of the point P Integer (a): the function which returns the higher integer of the real to the set of lines is defined by a set of discrete values of the o angle. This discrete set of values will be in the range: [-125: 125]. In order to obtain lines which regularly cross the pixels the values of 6p are defined as follows: n = sin 1 (~ n2 + P2) For n and p belonging to [0.3] the lines (7 ) are defined in the North-East frame are shown in Figure 8. Figure 10 shows the values of the points (9) belonging to 7 lines (8) described in Figure 9 for cp taking values spaced 45 °, either for n and p belonging to {0,1}.

The detection of the hair consists in carrying out the second derivative of the signal. Practically, this amounts to filtering by a Laplacian band pass filter, the result of which is shown in FIG. 11. The points where the signal is zero (10) are the amplitude jumps that correspond to the most marked outlines, so the pixels for which an outline is detected.

To define the most relevant points, the hue and the average brightness are first extracted on a set of pixels representing hair. The area where this information can be considered is that located at the first zero of the signal obtained in the previous step on the vertical half-line. The only case where this area does not contain hair is the case where the person represented is bald. In order to overcome this particular case, the hue of this area is compared with the hue 14 of the pixels at the top left of the image. If this case is met, the person will be considered bald.

The second step of this treatment is to take the two most relevant 'zeros'. The previous treatment can indeed give, with a priori a low probability, a result in which there are more than two contours detected by half-right (passage of the half-right by an ear, folds very marked in the hair .. .). In order to take into account only the points where the detected contour is due to the transition between face / hair and hair / background, on both sides of the point the information of hue and average brightness are measured. If these hue and brightness information are close to the hue and the average brightness of the hair determined in the previous step, the transition is classified in one of the following cases: hair transition / background or transition face / hair.

In case of a bad measurement of the thickness of the hair for a given angle which results in a peak, a median filter (statistical filtering) on a window of length 3 makes it possible to preserve the transitions while eliminating the peaks.

The color of the eyes follows the following process: The hue and average saturation of the pixels are measured immediately around the central characteristic points of the pupils. On a histogram, we find any white peaks (white eyes or reflections on the pupil) or red (flash effect), and we eliminate the corresponding pixels. If there are two peaks (eyelid margins and pupil color), the two values are compared. This hue (or these two hues), which can be averaged over both eyes to reduce noise, should be compared to the reference hues given by predefined images. The closest average gives the color of the eyes of the unknown photo.

The inclination of the face is measured by taking the tangent of the angle passing through the two characteristic points of the outer corners of the eyes and a horizontal. This measurement can be refined by averaging it by the tangent of the angle passing through the middle of the two characteristic points of the outer corners of the eyes and the characteristic point of the tip of the nose or the middle of the chin, and a vertical.

It is useful to locate a relatively neutral skin area to map the 3D head representing the avatar in areas where the texture of the photo does not apply (ears hidden by the hair, forehead hidden by the hair, wings of the nose , etc ...). The forehead area necessary for the texture of the skin is defined as a zone of approximately square shape, which falls between the line passing through the characteristic points of the eyebrow tops, and the characteristic points of the junction between the forehead. and the hair.

The recognition of the beard is based on the method of similarity with a template (called template matching). In both cases, the method makes it possible to bring the beard or the hair close to a list of beards and hair so-called representative, to determine the one of which it approaches the most. The information that will be transmitted by the method is a code that defines the type of hair or beard recognized.

As for the glasses, an example of a method is indicated below and illustrated by FIGS. 13 to 17 which show the results of the various steps from an original image in FIG. 12: A contour detection pretreatment makes it possible to highlight the traced glasses in the rest of the image.

This pretreatment consists of several steps: - Prewitt filtering illustrated in Figure 13: it is a strong gradient detector using a 3x3 neighborhood for each pixel. It makes it possible both to detect contours (11) and to determine their orientation. - Thresholding, cleaning and binary expansion, illustrated in Figure 14: in order to obtain a binary image, threshold the image of Prewitt so as to keep only the strongest gradients, then we apply a cleaning and a binary dilation, operations mathematical morphology to remove the isolated points and improve the connectivity of the outline of the glasses (12). The structuring element is a disk of radius 3. At the end of these operations we have identified the pixels belonging to the glasses and the strongest light reflections on the glasses. To identify the shape of the glasses, two approaches are possible: on the one hand, predefined types of glasses (for example elliptical, rectangular ...), identify the type of glasses then estimate the characteristic parameters (length of the axes and center of the glasses). ellipses, etc.); on the other hand, to allow more free forms by determining a closed closed end approaching the shape of the glasses. This last option is feasible by the technique of active contours (called snakes). This technique is iterative; the contour is initialized by a circle around the center of the eye (already calculated), then at each iteration, the contour is deformed to approach the zones of strong gradient detected by filtering of Prewitt and thresholding. Finally, to reconstruct the pixels hidden by the glasses, we use the technique called "Eigenfaces". The different steps are as follows: - Construction of the base of clean faces: The database being labeled, we know the position of the center of each eye. We select a square centered on the center of the left eye and 151 pixels side. So the eyes are all aligned for all faces. Each eye becomes the column of a matrix on which we perform a decomposition into singular values. The result of this decomposition is a basis of "left-eye space" (after re-sizing the columns into 151x151 pixel matrices). We do the same for the right eye. - Locating the eyes on the new face: Using the search scripts of the characteristic points, the centers of the eyes of the new faces are determined. - Projection: The 151x151 square around the marked center is projected on the basis of "clean eyes". This is the closest image to the eye of departure in the eye space without glasses. -Eclacement of the glasses shown in Figures 15, 16 and 17. Each previously detected point as belonging to the glasses is replaced by its counterpart in the reconstructed eye. We force the non-replacement of a small neighborhood around the counter of the eye (which contains strong gradients but where can not find the glasses). FIG. 15 shows the result on the left eye (13) before restoring the pixels recognized as not belonging to the glasses, FIG. 16 shows the restoration of the pixels (14) not belonging to the glasses, FIG. result on both eyes with the area (15) previously occupied by the glasses.

The glasses are then added to the avatar when compared with a base of typical glasses (for example elliptical, rectangular ...), from an estimation of the characteristic parameters (length of the axes and center of the ellipses, etc.) . With regard to the azimuth (the azimuth of the face is the angle of the vertical plane of the face, plane containing the two ears, with the plane of projection of the photograph, reference plane), one proceeds as follows: For the 5 positions selected: face (0 °), quarter-profile (22 °), half-profile (45 °), three-quarter-profile (67 °) and profile (90 °), an average face is determined. which is simply the average of the gray levels of all the learning images of the corresponding position. To classify a new face, we calculate its correlation coefficient with the 5 reference mean images: corr (x, y) = [1 / n] [1 (xi - x) (yi - -y)] / ax oy

where x is a column vector containing the pixels xi of the image, -x its mean and 6x its variance. This coefficient is even closer to 1 as the images are correlated. Of the 5 correlation coefficients calculated, the highest determines the angle of the new face. A rotation function will then be applied to the characteristic points found to give them their face coordinates.

With regard to the 3-dimensional reconstruction of the coordinates of the characteristic points, it is necessary that the base of reference faces was previously filled in the 3 dimensions, that is to say that each face of this base was taken. in photo according to two angles, one of face, the other of profile. Each of these faces will have been informed, that is to say that its characteristic points will have been defined, for example by a manual operation, in the 3 dimensions. From a 2D photo, the previous method allows to reconstruct the projection of the characteristic points in two dimensions. Each characteristic point reconstructed in two dimensions is either an average of points actually filled in the face base, or directly the point corresponding to one of the faces. Since these points have their third coordinates entered in the face database, reconstructing the third coordinate of the searched point according to the same method (average or direct association) will finally give for the whole face a third dimension calculated from the third dimension of the images. of the weighted base 25 of the same coefficients as for the two-dimensional reconstruction, or directly the third coordinate of the point found in the base if there is only one. A first global approximation of this method consists in finding, for the 2D projection of all the characteristic points of the image to be analyzed, the closest projection among a set of N 3D models (N being of the order of 300) representing the human heads in their diversity. The relevance of this method is based on the fact that the information projected in a two-dimensional plane of a face has information as to the third dimension of the face.

The invention therefore makes it possible to construct from an imperfectly taken or incomplete face photograph all the characteristic points and elements making it possible to create a virtual image as close as possible to the initial face, including in the three dimensions and taking into account possible rotation. It also makes it possible to characterize the accessory elements of the face that are the hair, the beard and the glasses. All of this information allows you to create a 3D synthesis head that looks like the person in the picture.

To create the synthesis head, we start from a neutral and bald head on which all the characteristic points are represented. This head is defined in a computer-generated image software context (eg Autodesk's MAYA software), and has animation (it has the points, parameters, and texture needed for a professional can animate it). This neutral head is then deformed by morphing until the characteristic points of the neutral head have the same frontal projection as the characteristic points of the photo. If the base is filled in 3D, the characteristic points found on the photo will also have a third dimension filled, and one can also deform the head so that the projection of profile corresponds. Otherwise, we keep a so-called average profile. We then map the texture of the face of the photo on the deformed head, and we map the texture identified on the forehead on the non-visible areas of the head. Finally, the hair nearest to the real hair is placed on the head among a set of synthetic hair, and the same is done for the beards and glasses.

To take into account differences in shooting conditions, an illumination correction module (29) is added, which consists of measuring luminance and chrominance on a central zone of the face, and modifying the entire image RGB settings for luminance and chrominance to fit within pre-defined intervals to ensure good color temperature and good brightness.

An improvement of the invention consists, once the characteristic points have been reconstructed, of analyzing the emotion expressed by the face by measuring the deviation of parameters derived from the positions of the characteristic points with respect to a neutral expression, and then straightening said face up to to give him a neutral expression. Thus the characteristic points will be those of the neutral face, the best able to be then manipulated by any type of process.

The avatar thus defined is completed by a psychological profile intended to refine its behavior in future scenes. This psychological profile is defined from the analysis of the characteristic points of the face found during the first step of the invention, according to a method borrowed from morphopsychology, as described for example in Faces and Characters, by Louis Corman, 1985. A classifier is applied to the measurements of the distances, angles and integrals in surface and volume between the different characteristic points, the lines which join them and the surfaces and volumes delimited for the eyes, the nose, the mouth, the forehead. , chin, jaws. Each face is thus classified according to N axes describing the personality, N being able to vary between 5 and 20.

The classes of a face along these axes are then projected onto a matrix describing the interaction potentials, in the face of typical situations or profiles. The results of this projection give the character's typical interactions with typical profiles or typical situations. They are used in the choice of the most relevant animations among the animations prepared to calculate the scene most representative of the character and his psychology. In a way, the scenario of the final scene is constructed by successive choices of animations, each choice being driven by the relevance with respect to the specific types of interaction.

The sources in the appendix can be consulted for more information on the proposed methods.

Appendix: Hierarchical Wavelet Networks for Facial Feature Localization (2001) Rogerio Schmidt Feris, Jim Gemmell, Kentaro Toyama, Volker Krueger 2 Face Reconnection by Elastic Bunch Matching Graph (1996) Laurenz Wiskott, Jean-Marc Fellous, Norbert Kruger, Christoph von der Malsburg proc. 7th Intern. Conf. on Computer Analysis of Images and Patterns, CA1P'97, Kiel Face recognition: A literature survey, ACM Computing Surveys (CSUR) archive Volume 35, Issue 4 (December 2003) Hough Transform Face Detection: http: // www. rennes.supelec.fr/ren/persQ/rseguier/pages/detection.html Ellipse detection, general case: http: //www.tsi. enst.fr/tsi/enseigi iement / ressources / mti / ell ipses / Houghol. Htm Human face segmentation and identification - Saad Ahmed Sirohey - University of Maryland A subspace method for a maximum likelihood of detection, Proceeding of the 1995 International Conference on Image Processing (Vol.3) - Training Support Vector Machines: an Application to Face Detection, CVPR Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97) Facial Segmentation: Human Face Segmentation and Identification, Sirohey, Saad Ahmed, UM Computer Science Department; CS-TR-3176 CAR-TR-695 Comparison to the database: ACP: General presentation: httpi / zoonek2.free.fr / UNIX / 483 / 05.htrnl Methods EigenFace: Eigenface-based facial recognition Dimitri PISSARENKO February 13, 2003 openbio.sourceforge.net/resources/eigenfaces/eigenfaces-html/facesOptions. htm l For radial-based function networks, see for example: www.scico.u-bordeaux2.fr/--corsini/Pedagogie/ANN/main/node38.html - 6k - 3D face processing: modeling, analysis & synthesis , (Int. Series in video computing, Vol 8) Authors: WEN Zhen, HUANG Thomas S .. Release date: 07-2004 On Morphopsychology: ABC of Morphopsychology, Carleen Binet, 2005.

On Psychology and Interactions: The Enneagram, René de Lassus, 1997.

Claims (4)

claims 1 - A method for automatically manufacturing and immersing in 3D scenes a 3-dimensional synthetic character made from a 2-dimensional photograph characterized in that it comprises: a step of receiving a photo (16) through a channel transmission device (17), a step (18) for extracting the face or faces (19) present in the photograph, a step (20) for determining the characteristic points (2) of said face by comparing the zone surrounding each characteristic point with the equivalent area in the faces of a predefined database (21) and by consistency test on the relative positions of the points with respect to a model (39), a step (22) for determining the characteristic features (23) the face or faces extracted, by calculation from the found characteristic points (2), by comparison with the equivalent parts in the faces of the predefined database (21) or with pre-calculated accessories s (24), a step (25) of morphing a 3D neutral head (26) to match the characteristic points (2) of the head with those from step (20) taking into account the characteristic features ( 23), a step (27) for mapping the texture of the extracted face (19) and corrected in luminance and chrominance (28) by a process (29) and for gluing a sample of uniform texture extracted from the face (28) in a position determined by (22) on the hidden parts of the deformed head (30), a step (31) of adding accessories made in computer images taken in the base (24) and corresponding to the face (19) as detected by step (22) of features (23) on the deformed and mapped head (32), a step (33) of psychological profile construction (34) defined from the characteristic points (2) and features (23), which is associated with the deformed, mapped and accessorized head (35), and a step (36) manufacturing an animated scene (38) from the deformed, mapped and accessorized head (35), the psychological profile (34), and an animation base (37). 2 - method according to claim 1 characterized in that the area around each characteristic point is represented by a spiral vector. 3 - method according to claims 1 or 2 characterized in that the identification of the characteristic points (2) is a mixture of statistical analysis solutions that are the reduction of the representation space by means of a principal component analysis , followed by a classification based on the k nearest neighbor algorithm, and model-based solutions, which are the coherence tests taking into account the particular topography of each zone of the face, the coherence of the zones of the face between they, points of each zone between them and the proximity of the barycentre of the statistical cloud of the points considered good candidates taken in the database. 4 - A method according to any one of claims 1 to 3 characterized in that the characteristic features include theoretical models of beards, haircuts and pairs of fins, which are used for recognition and are then used in the synthesis of the final head In 3D. 5. Method according to claim 4, characterized in that the recognition of the hair uses the piecewise integral of a curve characteristic of the thickness of the hair in sections passing through a central point of the face to classify the haircut unknown among a finite number of typical haircuts prepared in computer graphics. 6 - Process according to any one of claims 1 to 5, characterized in that it comprises a step of detecting and erasing glasses on the image to be analyzed, by identification with respect to typical glasses by means of a contour detection, then by a reconstruction of the pixels (5) hidden by the glasses, by extending the immediately peripheral and visible zones around the glasses of the image to be analyzed by analogy with the equivalent zones on the images of a known database of faces without glasses. 7 - method according to any one of claims 1 to 6 characterized in that the third dimension of the detected points on the 2D image to be analyzed is calculated by associating all or part of the 2D projections of the characteristic points found all or part of those the 3D head taken from a database of 3D heads having a 2D projection of its closest characteristic points, and assigning to the characteristic point being computed the third dimension of the point of the corresponding model, or an average of the third dimensions closest points. 8 - method according to any one of claims 1 to 7 characterized in that an emotion detector is applied to the reconstructed face, and the characteristic points (2) are moved to obtain a neutral emotion before the morphing step . 9 - method according to any one of claims 1 to 8 characterized in that the psychological profile is determined by a morphopsychology technique that uses the coordinates of the characteristic points to evaluate distances, angles, areas and volumes specific to each face, in order to classify each face in a pre-determined psychological profile whose interactions with other profiles or in key situations is defined and is used in the choice of the evolution of a scene cut into several tables, with several pre-written possibilities for each table, among which possibilities one is chosen according to the profile.