FR2843472A1 - 3D representation method, especially for representation of a human face, whereby a plurality of characteristic points is selected on a 3D-face representation and then matched with image points from corresponding images - Google Patents

Abstract

Method for 3D representation of a human face has the following steps: provision of a 3D face representation; selection of a plurality of characteristic points; identification of image points in two images corresponding to selected characteristic points; displacement of the characteristic points such that projections of first and second images match the displacement and suppression of each peak point that that is not a characteristic point. An Independent claim is made for a device for 3D representation of a human face.

Description

i The present invention relates to a method and a device for

  generate a three-dimensional representation of an object belonging to a kind of predetermined object, in particular human faces, from at least two two-dimensional images of said object. Such a three-dimensional representation, for example of a human face, can be used in applications for creating virtual environments, for video games, video telecommunications, such as videoconferencing, and graphical control interfaces. Document WO 99/63490 A describes a method for generating a three-dimensional representation of a human face from at least two two-dimensional images of said face. This known method comprises the steps of: providing a generic three-dimensional representation of said face, said generic representation comprising a plurality of vertices defining a mesh of said face, providing a first two-dimensional image of said face seen in a first direction of projection and a second two-dimensional image of said face seen in a second direction of projection, distorting the generic three-dimensional representation to obtain a

  specific three-dimensional representation of the face.

  This known method has drawbacks. Indeed, the three-dimensional geometry of the face to be represented is determined by correlating the points of the two images, which is extremely costly in terms of calculations to be made. In addition, we can only correlate the points appearing in the two images, which makes each image usable only in the measurement of the portion of

the object common to the two images.

  Furthermore, this known method uses a region growth method. Such a method is not well suited to recognizing the geometry of a human face because it is made up of heterogeneous elements not having the same physical and therefore photometric characteristics, that is to say by example eyes, eyebrows, hair, lips and skin. The growth methods of

  region do not allow to recognize the whole picture in such cases.

  They generally lead to the recognition of gaps or discontinuous areas. Therefore, this process generally does not recognize a face in its entirety, that is to say without forgetting

  some of its constituent elements.

  The object of the invention is to provide a method and a device which solve at least some of the abovementioned drawbacks, in order to obtain a better resemblance between the face to be represented and the three-dimensional representation generated and to obtain this resemblance

  with as little computation as possible.

  For this, the invention provides a method of the aforementioned type, characterized in that it comprises the steps consisting in: selecting a plurality of characteristic points from the vertices of said generic representation, identifying a first set composed of those of said characteristic points which are visible in said first direction of projection, for each characteristic point belonging to said first set, identify a similar point in said first image, identify a second set formed of those of said characteristic points which are visible in said second direction of projection, for each characteristic point belonging to said second set, identifying a similar point in said second image, said deformation step comprising a displacement of said characteristic points so as to: for each characteristic point belonging to said first set, have a pr ojection in the first direction of said characteristic point with said analogous point, for each characteristic point belonging to said second set, having a projection in the second direction of said characteristic point with said analogous point, said deformation step comprising a displacement of each vertex of said generic representation which is not a characteristic point as a function of the displacements of the characteristic points

which are neighbors of said summit.

  By immediately selecting certain characteristic points

  of the generic representation, one uses only these characteristic points to pilot and control the deformation to be carried out. Therefore, the number of parameters of the deformation, and consequently the 5 cost of the process in terms of calculations, can be modulated according to the number of characteristic points used.

  In addition, the characteristic points which are visible on the two images of the object are positioned in space so as to coincide with their analogous points in the two images, while any characteristic point which would appear only on one image will be positioned in space so as to coincide with its analogous point in this image. We thus use all the information concerning the

  characteristic points available in the images.

  The displacement of the vertices which are not characteristic points as a function of the displacements of the characteristic points which are adjacent to them makes it possible to preserve the neutral aspect of the forms of the generic representation which are not selected as characteristic point, in order to obtain overall a good level of resemblance between the generated representation and the object, even with a relatively small number of characteristic points, for example one to

a few dozen.

  Preferably, the method according to the invention comprises the steps consisting in: defining a reduced mesh of said object, said characteristic points 25 forming the vertices of said reduced mesh, for each of the characteristic points, defining an area of influence comprising all of the facets of said reduced mesh of which said characteristic point is a vertex, any vertex of said generic representation which is not a characteristic point being displaced, in said deformation step, in proportion to the displacements of any characteristic point in the zone

  of influence from which said summit belongs.

  Advantageously, the membership of a vertex in the zone of influence of a characteristic point is determined, for each of the 35 directions of projection, by projection of said vertex on triangular facets of said reduced mesh according to the corresponding projection direction, or by projection on circles circumscribed to said facets. Preferably, to reconcile a characteristic point

  with a similar point in one of the images, said characteristic point 5 is displaced parallel to the projection direction corresponding to the other image.

  Advantageously, the points of said first image and / or of said second image which are analogous to the characteristic points of the first and / or of the second set are identified by detection of the contours, that is to say by detection of the points of the image for

  which a color property has a maximum gradient.

  Preferably, said color property is chosen

  from a gray level, a red level, a blue level, a green level of the image pixels and a combination of at least some of said levels.

  Advantageously, the step of identifying the points of said images which are analogous to the characteristic points of the first and / or of the second set is carried out iteratively by moving approximation points in said images from 20 initial positions determined as a function of shapes and proportions

known generics of said object.

  According to a particular embodiment of the invention, said approximation points form at least one broken line which deforms during the displacements of said approximation points under prescribed conditions of geometric regularity, for example a

  minimum distance between two successive points of said broken line.

  Advantageously, said broken line is deformed by tending to minimize an internal energy Eint of said broken line defined by: Eint = (Il Vi Vi-, | I + Il | Vi + l - 2Vi + Vi- II), oo and / denote predetermined real parameters, Vi designates the vector position of a point of said broken line, Il. It designates a standard

  and the sum is extended to all the points of said broken line.

  Advantageously, the method according to the invention comprises a step consisting in putting the two images on the same scale, the deformation step comprising a global deformation step consisting in applying a scale factor to said generic representation in order to adjust at least one dimension of said generic representation to a corresponding dimension of the object represented in the images.

  According to another particular embodiment of the invention, said kind of object includes human faces, said characteristic points comprising points around the eyes, points around the mouth, points around the nose and points of the contour of the face, said images comprising a planar projection in front view of a face and a planar projection in profile view of said face.

  Preferably in this case, said generic three-dimensional representation is deformed symmetrically with respect to a plane

vertical median of said face.

  Advantageously, the method according to the invention comprises a step of texturing said specific three-dimensional representation of the object, consisting in expanding in a rectangle a part of the image in front view which represents the face and in associating by projection the texture of said part of the image expanded to all of the

  facets of the three-dimensional representation of the face.

  According to yet another embodiment, the method according to the invention comprises a step of texturing said specific three-dimensional representation of the object, consisting in associating at least the texture of a central part of the image in front view with a central part of the three-dimensional representation of the face and associating at least the texture of a central part of the image in profile view with at least one lateral part of the three-dimensional representation of the face. Advantageously in this case, said texturing step comprises the operations consisting in: deforming the image in front view and the image in profile view so as to obtain the images which would result if the face had a cylindrical exterior surface which the one unwound, superimposing said deformed images so as to make corresponding points in said images merge, merging said superimposed images into a merged image in which at least some pixels have a color resulting from a weighted average of the colors of the pixels of said images which are overlap,

  apply the merged image to the specific three-dimensional representation. For example, the corresponding points used to position the images relative to one another are chosen at the level of the ears.

  Preferably, said weighted average is calculated by weighting each of said pixels which are superimposed using a respective weighting coefficient, said coefficient being all the more large as the deformation undergone by the image to which said pixel belongs. during said deformation operation is low at the level of said pixel. Advantageously, the method according to the invention comprises

  a step of colorimetric calibration of the two images to harmonize the textures of the parts of the face appearing in the two images.

  The invention also provides a device for generating a three-dimensional representation of an object belonging to a kind of predetermined object from at least two two-dimensional images of said object, comprising: means for providing a generic three-dimensional representation of said object said generic representation comprising a plurality of vertices defining a mesh of said object, means for providing a first two-dimensional image of said object seen in a first direction of projection and means for providing a second two-dimensional image of said object seen in a second projection direction, deformation means for deforming said generic three-dimensional representation in order to obtain said three-dimensional representation of the object, characterized in that it comprises: means for selecting a plurality of characteristic points from among said s vertices, means for identifying a first set composed of those of said characteristic points which are visible in said first direction of projection, means for identifying an analogous point of each characteristic point belonging to said first set in said first image, means for identifying a second set formed from those of said characteristic points which are visible in said second direction of projection, means for identifying a similar point of each characteristic point belonging to said second set in said second image, said deformation means comprising means for moving the 10 characteristic points to move said characteristic points so as to: for each characteristic point belonging to said first set, have a projection in the first direction of said characteristic point reconcile by with said analogous point, for each characterized point tick belonging to said second set, making a projection in the second direction of said characteristic point coincide with with said analogous point, said deformation means comprising means for moving the other vertices to move each vertex of said generic representation which is not a characteristic point according to

  displacements of the characteristic points which are neighbors of said vertex.

  Within the meaning of the invention, the provision of the images and of the three-dimensional representation can be carried out in several ways. Within the meaning of the invention, making an entity available includes producing the entity, or receiving the entity from an external source, for example from a removable data medium such as a magnetic or optical disk or from a transport network. data, or store the entity, for example in a storage means such as a mass memory or a central computer memory, or else access the entity, for example by reading in a storage means or

on a data carrier.

  The device according to the invention can be produced as an apparatus whose hardware design is specific for this purpose, or as an apparatus of conventional hardware design, for example a generic microcomputer, programmed by means of a program.

  specific computer for this purpose, or as a combination of the two.

  The device according to the invention can also be produced as a computer program. Within the meaning of the invention, a computer program comprises instruction codes capable of being read or stored

  on a medium and executable by a computer or similar device.

  The invention will be better understood, and other aims, details,

  characteristics and advantages thereof will appear more clearly during the following description of several particular embodiments of the invention, given solely by way of illustration and not

  limiting, with reference to the accompanying drawing. In this drawing: - Figure 1 is a schematic representation of the main steps of the methods according to the invention, - Figure 2 shows a mask used to calculate the gradient of the properties of the pixels of an image, - Figure 3 shows schematically a active contour used to identify points of the images analogous to the characteristic points, FIG. 4 represents an image of a face in front view with an active contour at the start and at the end of a step of trimming the process of FIG. 1, FIG. 5 represents proportions of the human faces known a priori, - FIG. 6 represents the image of FIG. 4 with active contours such that at the end of a step of identifying the process of the Figure 1, - Figure 7 shows a generic representation of a face used in the process of Figure 1, - Figure 8 shows schematically and partially a reduced mesh of the representation generic of figure 7, - figure 9 schematically represents the zone of influence of a point characteristic of the generic representation of figure 7, 35 - figures 10 and 11 represent a facet of the reduced mesh of figure 8 before and after an elementary displacement of a characteristic point, - FIGS. 12A and 12B represent a stage of dilation of an image part for a stage of texturing according to a first embodiment, - FIG. 13 represents a result of the step of FIGS. 12A and 12B, - FIG. 14 represents a texturing step according to a second embodiment, 10 - FIG. 15 schematically represents the zones of influence of the characteristic points of the generic representation of FIG. 7, according to a variant embodiment of the invention, - Figure 16 shows a profile view image 15 as deformed during a texturing step according to a third mode of r realization, - Figure 17 represents a merged image obtained in the texturing step according to the third embodiment, - Figure 18 represents an image

  textured three-dimensional obtained at the end of the texturing step according to the third embodiment.

  Referring to FIG. 1, a method 25 according to the invention will now be described for generating a three-dimensional digital image.

of a face.

  In step 5, a generic representation of the human face, also called a neutral mask, is stored in the memory of a microcomputer. FIG. 1 shows this generic representation 1, in the form of a triangular faceted mesh comprising 256 vertices. Such a generic representation is known from the document "Rapid Modeling of Animated Faces from Video", technical report MSR-TR-2000-11 by Z. Liu et al., Microsoft Research

Corp. (2000).

  On the generic representation, certain vertices are selected as characteristic points. For example, we select as characteristic points the internal, external, upper and lower vertices of the edges of the eyes (4 vertices per eye), the tip of the nose and the entrance of the nostrils (3 vertices), the vertices of edge in width and height of the mouth (4 vertices), the apex of the tip of the 5 chin, the apex of the top of the forehead, and the two vertices defining the maximum width of the mask, the ears can be included in the

  neutral mask 1 or be excluded from it.

  In step 10, two digitized images of a

  face. This step is carried out for example by means of a digital camera.

  To simplify the image processing operations, these two photos are taken in front of a white background and the person photographed has no glasses, no extravagant hairstyle, or any other accessory that could hinder facial recognition. The two photos are a 15-sided photo and a profile photo. Between the two shots, the subject performs, as a single movement, a rotation of the head or of the whole body by 90 around a vertical axis. Equivalently, the subject can remain stationary and the camera can move 90 on a horizontal arc around the subject's head. The focal length of the camera is not changed between the two shots. We thus obtain two photos on the same scale. We carry out a centered framing of the

  face in each of the shots.

  Alternatively, both photos can also be taken

  simultaneously by two identical cameras positioned 90 25 apart from each other on a horizontal arc centered on the subject's head.

  The two digital photos can also be acquired using a

  scan from slides or photo paper prints.

  The two digital photos, expressed in any suitable format, are transferred to the memory of the microcomputer by a data transfer link between the camera.

  digital or scanner and microcomputer. Such an image contains data that defines a two-dimensional array of pixels and the color properties of each pixel on three RGB channels. The two directions of the images are denoted x and y thereafter. The pixels of the 35 images are identified by two corresponding coordinates.

  For each of the images, there is then carried out a step 20 of

  face clipping. There are three main parts in the images: the background, the face, and other parts of the body, such as hair and shoulders. The purpose of step 20 is to delimit the part of the images 5 corresponding to the face itself. This step is very important to generate the specific three-dimensional representation because both the geometry of the face and its texture depend on the result of this step.

  For this, in step 20, the contour is detected in each image

outside of the face.

  Step 20 is carried out by a method of detecting contours by iterative deformation of an active contour. Such a method is known from the article "Snakes: Active contour models" by Michael Kass et al. published in International Journal of Computer Vision,

Flight. 1, pages 321-331 (1987).

  An active contour is a set of approximation points which define a broken line constituting an approximation of the contour that one wishes to detect. Figure 3 shows an outline

  active with four approximation points Vi_1, Vi, Vi + 1, Vi + 2.

  Firstly, the active contour is positioned at a predetermined initial position. FIG. 4 represents an example of a face photo in front view. Line 21a is the active contour at its initial position. At this position, the active contour forms a polygon, for example a hexagon, positioned so as to ensure that the object whose contour must be recognized, i.e. the face, is inside the initial outline. In this way, the background being white, it is ensured with better chances that the active contour will effectively converge towards the edges of the face. This initial positioning can be carried out manually by an operator, using the microcomputer graphic interface, or automatically, for example by positioning the active contour centrally in the image and at a short distance from the

peripheral edges of the image.

  Preferably, the active contour is given an initial shape adapted to the a priori known shape of the face, for example a shape

substantially elliptical.

  Secondly, it deforms iteratively

  active contour by moving the approximation points from pixel to pixel.

  This deformation is controlled by a total energy And of the active contour which one seeks to minimize. The total energy Et of the active contour is defined as the sum of a potential energy Ep, of an energy

  internal Eint and a constraint energy Eo.

  The potential energy of the active contour is defined in such a way as to attract the points of approximation towards the areas of the image where the gradient of a colorimetric intensity C is maximum, that is to say the areas of the image. which actually represent contours. For a black and white image, the gray level of the pixels of the image is used as color intensity C. For a color image, we can use as color intensity C, for example, the level on one of the red, green and blue channels, or a combination of these levels as the luminance Y of the pixels, defined for the pixel (j , k) by: Y (j, k) = 0.299 R (j, k) + 0.587 V (j, k) + 0.114 B (j, k), o R (j, k) V (j, k) B (j, k) denote respectively the level of red, of

pixel green and blue (j, k).

  The potential energy Ep of the active contour is the sum of the potential energies Epi of the approximation points Vi belonging to the active contour:

Ep = Ei Epi.

  For an approximation point Vi positioned on the pixel (j, k), the potential energy Epi of this point is: Epi (, k) = - | I Grad C (j, k) 11, o .11 denotes a vector norm and Grad designates the gradient vector. The gradient of the function C can be calculated by any known method. Preferably, we use the approximate calculation method based

  on the Sobel mask, which is shown in Figure 2.

  In this method, to calculate the horizontal component GradX of the gradient at the pixel (j, k), this pixel is placed in the box in the center of the mask, each box representing a neighboring pixel. Then, for each pixel covered by the mask, the product of the value of the function C in this pixel is produced by the coefficient in the corresponding box of the mask. These 35 products are then added up. We thus obtain: Gradx C (j, k) = C (j-l, k-l1) + 2C (j, k-l1) + C (j + l, k-l) -C (j-l,

k + l) -2C (j, k + l) -C (j + l, k + l).

  To calculate the Grady vertical component of the pixel gradient (i, j), we turn the mask 90 in the forward direction and we perform the same calculations. We then obtain: Grad C (j, k) Il = {[Gradx C (j, k)] 2 + [Grady C (j, k)] 2} /. We can also use as color intensity C, for a color image, the level on the red, green or blue channel

  level for which the gradient has a maximum modulus.

  The internal energy Eint of the active contour is a clean energy depending only on the shape of the active contour. It is defined by: Eint = Ei Einti and Eint i = (a V1 v I 1 -Vi Vj + - 2Vi + Vi-1), where Vi denotes the position vector of the approximation point Vi and the sum is extended to all the points of the active contour, being a closed contour, or at all points except those of the two ends,

being an open contour.

  The coefficients ot and / are two positive real numbers chosen according to the influence that one wishes to give to each of the 20 components of internal energy, that is to say on the one hand the norm of the derivative first of the position vector of the approximation point Vi as a function of the curvilinear abscissa and on the other hand the norm of the tangential component of the second derivative of the position vector of the approximation point Vi as a function of the curvilinear abscissa. The larger the coefficient oa, the more flexible the active contour, ie able to follow acute angles. The larger the coefficient 3, the more regular the active contour, i.e. able to follow an average curve

  without making brackets towards small irregularities in the image.

  The stress energy Ec is also an energy depending on the geometry of the active contour. It is chosen so as to impose additional geometric constraints on the active contour. For example, this energy is defined by: Ec = Li Eci (Vi - Vil), where Eci denotes, for each point Vi of the contour, a potential interaction energy of hard spheres type, worth 0 35 when the previous point Vi- 1 of the contour is at a distance greater than a predetermined minimum distance and being equal to + oo when the preceding point of the contour is at a distance less than or equal to said predetermined minimum distance. It is thus ensured that the consecutive points of the active contour remain at a distance greater than said

  predetermined minimum distance from each other.

  Before we start the iterative deformation of the contour

  active, its total energy has a certain initial value Et = Ep + Eint + Ec.

  We then move the approximation points so as to minimize the total energy function, the variables being the coordinates of each approximation point Vi belonging to the active contour. There are many mathematical methods, for example variational methods, capable of solving this problem of optimizing a

multivariate function.

  Preferably, a dynamic programming method is used, which is particularly suited to the form of the present problem. 15 Such a method is known from the article "Using dynamic programming for solving variational problems in vision", by Amini

  Amir, in PAMI, 12 (9), pages 855-867 (1990).

  Because of the definitions above, the potential energy of a point of the contour depends only on the position of this point; its internal energy depends on the positions of this point and the two neighboring points, and its stress energy depends only on the position of this point and that of the previous point. If the active contour comprising N approximation points is an open contour, the terms forming the total energy Et of the active contour are grouped in the form: Et = El (V1, V2, V3) + E2 (V2, V3, V4 ) + ... + EN-2 (VN-2, VN1, VN), with El = Epl + Eint 2+ Ec2, E2 = Ep2 + Eint 3+ Ec3, etc. If the active contour with N approximation points is a closed contour, the following terms must be added: EN-i (VN1, VN, Vl) + EN (VN, Vl, V2)

  with E N- = EPN-1 + EintN + EN and EpN = EN + Eint 1+ Ec.

  The principle of the dynamic programming method is to first minimize the function E1 with respect to the variable V1, then minimize with respect to the variable V2 the function: S2 = minvI [EI (VI, V2, V3)] + E2 (V2, V3, V4), which does not depend on the variable VI. We continue so on by following

the order of the points of the contour.

  In practice, the active contour being at a given position,

  each approximation point coincides with a certain pixel of the image.

  We start by evaluating the total energy And of the active contour for any triplet of positions (VI, V2, V3), when each of these points successively takes its present position and all the positions adjacent to the pixel where it is located, positions which are represented by the frame 22 in FIG. 3 for the point Vi + 1. We retain that of these triplets (V1, V2, V3) which minimizes Et. We then fix point VI. Then the above operation is repeated with the triplet (V2, V3, V4), and so on 10 until the triplet (VN-2, VN-1, VN).

  For an open contour, when all the points V1 to VN. have thus been fixed, the calculation is iterated by resuming from the position triplet (VI, V2, V3). For a closed contour, we continue with the triplets (VN-1, VN, V1) and (VN, Vl, V2), which makes it possible to iterate calculation 15 on a new lap. Iterations are done this way until a

  maximum number of iterations is reached or a condition of convergence is fulfilled, for example the absence of significant variation of the total energy And from one iteration to the next. The active contour is then in its final position, represented by line 21b in phantom in FIG. 20 4.

  Among the points of the converged active contours, we then have

  detected, on each of the images, the position of some of the points analogous to the characteristic points which have been selected on the neutral mask. These analogous points are identified by their coordinates 25 in the image.

  For the facial trimming step, the result is all the better since the subject does not have a beard and has a different skin color from that of his hair. Otherwise, there is a risk of detecting a part of the contour of the beard or of the hair instead of the contour of the face itself. If necessary, an additional characteristic point is used located at the top of the hairstyle, the analog point of which is easier to detect than the analog point of the top of the forehead if the

  contrast between hair and skin is low.

  When the clipping step 20 has been carried out for the two images, a step 30 for scaling the two images is carried out, in case the two images have not been taken under the prescribed conditions. For example, we apply a homothety to one of the images to obtain the same height of the contour of the face in the

two images.

  A step 40 is then carried out of detecting the points 5 of the images which are analogous to the characteristic points situated inside the contour of the face on one or the other of the images. For this, the eye contour, the lower nose contour and the mouth contour are detected on the front view image. We also detect

  the outline of the eyes and mouth on the image in profile view.

  For step 40, the calculation method used is a contour detection method analogous to that of step 20. The only difference with step 20 lies in the initial placement of the active contours. To improve the accuracy of the detection of the abovementioned contours, the active contours are initially positioned using the a priori knowledge that human faces are present. This knowledge consists of average proportions which are

  shown in Figure 5 and known average shapes.

  The active contour must thus be placed before converging around an eye in the form of a rhombus with its lower point substantially on a horizontal line cutting the face into two equal parts and with its points of longitudinal ends spaced about 1 / 5 of the width of the face. For the image in front view, the active contour is represented by the

  line 41 of figure 6 after convergence.

  The active outline is placed before converging around the nose 25 with its point less than about 1/3 of the height of the face starting from the bottom, on a vertical median line. For the front view image, this active contour is represented by line 42 of FIG. 6 after convergence. The active contour which is to converge around the mouth is placed in the form of a diamond centered around a horizontal line extending to about 2/9 of the height of the face starting from the bottom. For the image in front view, this active contour is represented by line 43 of

Figure 6 after convergence.

  At the end of step 40, all the points analogous to the characteristic points selected in step 5 are identified in the two images. All the analogous points appear among the points of the contours

  converged assets. Their coordinates are calculated and stored.

  We go to a step 50 of deformation of the neutral mask

  to generate the desired three-dimensional representation of the face 5 represented in the images. Step 50 includes a global deformation step followed by a local deformation step.

  In the overall deformation step, a homothetic transformation is applied to the neutral mask 1 to bring at least one external dimension of the three-dimensional representation (height 10 and / or width of the face) into agreement with the dimension (s)

  corresponding exterior (s) identified on the images.

  In the local deformation step, for each of the images, the projected image of the three-dimensional representation 1 is calculated according to the direction of shooting of the corresponding image. We identify the characteristic points that are visible

  in each projected image. The characteristic points visible in the projection in front view constitute a first set, which contains all the characteristic points in the present example. The characteristic points visible in the projection in profile view constitute a second set, which contains all the characteristic points of a demivisage in the present example.

  In a first local deformation operation, for the image projected in front view of the three-dimensional representation 1, each characteristic point of the first set is moved until it coincides, in the projected image, with the analogous point of the image

in front view.

  In a second local deformation operation, for the image projected in view of the profile of the three-dimensional representation 1, each characteristic point of the second set 30 is displaced until it makes it coincide, in the projected image, with the analogous point

  of the image in profile view. In the second operation, the characteristic points of the second set are moved parallel to the direction of projection of the image in front view, so as not to modify the projection of these characteristic points according to the front view.

  For the characteristic points not belonging to the second set, the second deformation operation is carried out so as to preserve the symmetry of the three-dimensional representation

  of the face with respect to a median vertical plane.

  In a preferred variant, the two operations are carried out

  deformation in a symmetrical mode, that is to say, the characteristic points of a single half-face are made to coincide with the corresponding analogous points in the two images, and the other characteristic points are displaced by symmetry by with respect to a median vertical plane.

  In step 50, we also move the vertices of the mask

  neutral which are not characteristic points, using a free form deformation method known from the article "Free form deformation of solid geometric models" by T.W. Sederberg et al. 15 in Proceedings of SIGGRAPH'86, Vol. 20, pages 151-160, ed.

  Computer Graphics (1986). This method amounts to segmenting the space into tetrahedral regions delimited by the characteristic points and to considering that each characteristic point acts on tetrahedral regions, which constitute its zone of influence, as well as on all the vertices which belong to this zone.

  For this, a reduced mesh 52 of the neutral mask is defined. This reduced mesh is a triangular faceted mesh which contains only the characteristic points as vertices. It is partially shown in FIG. 8, in which the solid points P are characteristic points and the empty points S are vertices of the neutral mask other than the characteristic points. The reduced mesh

  52 is used to define the zone of influence of each characteristic point.

  For each of the directions of projection, the images projected from the vertices S are calculated on the facets of the reduced mesh. For each of the directions of projection, each vertex S is thus considered in the zone of influence of three characteristic points forming the vertices of the facet of the reduced mesh on which said vertex is projected. In other words, the zone of influence of each characteristic point P extends to all the facets of the reduced mesh 35 whose said characteristic point P forms a vertex and includes all the vertices S of the three-dimensional representation which project onto

  said facets. FIG. 9 represents a part of the reduced mesh with, hatched, the zone of influence 51 of a central characteristic point.

  During the local deformation step, for each

  displacement of a characteristic point during the first operation, all the vertices S belonging to the zone of influence of said characteristic point are displaced according to the projection direction of the front view. Figures 10 and 1 1 illustrate how these movements are made.

  FIG. 10 represents, in projection in the direction of the front view, a facet of the reduced mesh whose vertices are A, B 10 and C. S denotes a vertex of the neutral mask which belongs to the zone of influence of the points A , B and C according to the projection direction of the front view. When the characteristic point A is moved to the position A ", represented in FIG. 11, the vertex S is displaced so as to preserve the proportions of the initial mesh. The new position S 'of the vertex S is thus calculated from so as to keep the following distance ratios: AC '/ AB = A "C" / A "B and,

AB '/ AC = A "B" / A "C.

  In the same way, during the second deformation operation, each vertex S which is not a characteristic point is displaced in proportion to the displacements of the characteristic points in the zone of influence of which said vertex S is located according to the

second projection direction.

  According to another alternative embodiment of step 50, for the displacement of the vertices of the neutral mask 1 other than the characteristic points, a location of each vertex S which is not a characteristic point is first performed using a Sibson coordinate system, as described in Sibson R., "A vector identity for the Dirichlet Tesselation", Math. Proc. pp 151-155, Cambridge 30 Philos. soc. (1980). For this, we consider the reduced mesh described above, in association with a segmentation of the space defined by the

  spheres circumscribed to the tetrahedrons delimited by this reduced mesh.

  The zone of influence of each characteristic point is then defined as the meeting of the circumscribed spheres to which said point 35 belongs. For each vertex S other than the characteristic points, the characteristic points which are its natural neighbors are defined as being the characteristic points of all the circumscribed spheres in which said vertex S is located. In practice, the membership of a vertex S is determined in the zone of influence of a characteristic point by projection along each of the directions defined by the images. In FIG. 15, two triangular facets {P1, P2, P4} and {P3, P2, P4} of the reduced mesh and their respective circumscribed circles 53 and 54 have been shown in projection in a plane. inside the two circles 53 and 54, which shows that the points Pl to P4 10 are the natural neighbors of the vertex p, at least for this direction of projection. For each neighboring point P1 to P4, a respective Sibson coordinate / & to gu4 is assigned to the vertex p according to the formulas: p =] i (/ i Pi), dgi (= [(X-_X) 2+ (yy) 2] I /] j [(X-Xj) 2+ (y-yj) 2] -l, O x and y are Cartesian coordinates of the vertex p in the plane, xi and yi being the

  Cartesian coordinates of the characteristic point Pi.

  Thereafter, the displacements of the vertices other than the characteristic points are carried out by keeping constant all their coordinates of Sibson. These coordinates are natural in that the influence of a characteristic point on a vertex is all the greater the closer this vertex is to this characteristic point. In so doing, when a natural neighbor Pi of the vertex p is displaced by a vector quantity APi, the vertex p is displaced by / i APi. Therefore, the final position p 'of the vertex p, with respect to its original position Po in the neutral mask 1, is:

P '= pO + Ei = 1 to 4 (/ 1i APi).

  This formula is valid both for the three-dimensional representation and for the plane representation obtained in each projection. As a variant, the membership of a vertex in an area of influence could be determined directly by a three-dimensional calculation. At the end of step 50, a specific three-dimensional representation of the face is obtained which has the same number of 35 facets as the neutral mask 1 supplied in step 5 and whose geometry

  three-dimensional has been reconciled with the two images.

  The invention is not limited to a particular choice of

  characteristic points. It is important to select as characteristic points the most relevant vertices to describe the geometry of a face, that is to say those where the inflection of the face surface is the most pronounced. The choice mentioned in step 5 meets this requirement.

  It is of course possible to choose more characteristic points, to obtain a better resemblance between the specific final representation and the face represented on the images. Conversely, depending on the subsequent applications of the specific three-dimensional representation obtained, a smaller number of characteristic points may be sufficient if one can be satisfied with a more approximate final representation. In step 60, we apply a texture to the representation

  specific three-dimensional obtained. Three embodiments of this step are proposed.

  In the first embodiment of step 60, the part of the image in front view delimited by the contour 21b is considered. This image part 61 is shown diagrammatically in FIG. 12A. It represents the face itself. This image portion 61 is virtually expanded, as indicated by the arrows 62, until a square image 63, shown schematically in Figure 12B, is obtained. Figure 13 shows an example of the result of this operation. We then associate a texture with the facets of the three-dimensional representation

  specific by projecting image 63 onto it.

  In the second embodiment of step 60, a colorimetric calibration step of the images is carried out, so that the areas of the face appearing in the two images have a similar texture (color, brightness) in the two images. Then, as visible in FIG. 14, the front view image 66 is separated into three parts 30 delimited by two lines 64 passing through a corner of the mouth and an external edge of the eye. The central part 65 of the image in front view is associated by projection with the corresponding facets of the specific three-dimensional representation. The image in profile view is separated in the same way by a line passing through the corner of the mouth and the outer edge of the eye and the rear part of the image in profile view is associated with the facets of the two sides of the representation

  three-dimensional face, by symmetry.

  According to a preferred variant, step 60 is entirely carried out in a symmetrical fashion with respect to a median vertical plane of the face. In the third embodiment of step 60, we

  proceeds in four sub-steps consisting in deforming the two two-dimensional images, superimposing the images thus distorted, merging the images thus superimposed and applying the image resulting from the fusion onto the specific three-dimensional representation.

  For the first sub-step, the face represented on the two-dimensional images is roughly assimilated to a cylinder, the diameter 2R of which is known from the apparent width of the face on the corresponding image. Then the image in front view and the image in side view are distorted so as to obtain the resulting images.

  if we rolled out the cylindrical outer surface of the face. This operation amounts to laterally expanding the images by a factor of 7r / 2 4.57.

  In this deformation, the lateral dilation is amplified on the lateral edges of the image and attenuated in the middle of the image. In fact, the lateral x coordinate of the pixels, measured relative to the middle of the image, is modified according to the transformation:

  x '= R Arcsin (x / R), where x' is the lateral coordinates in the distorted image.

  FIG. 16 represents a profile view image thus deformed.

  In the second substep, the profile view image is symmetrized to form a view according to the other profile, unless actually using a third image representing the other profile of the face. Then the images respectively representing the face and the two profiles of the face are partially superimposed. For this, the point or points of each image corresponding to the lower lobe of each ear are identified. For example, the identification of this point was carried out during step 20 on the front view and during step 40 on the profile view. The three images are superimposed by making the points of the front view corresponding respectively to the two ears coincide with the points

  corresponding on the two respective profile views.

  In the third sub-step, a local weighting coefficient is assigned to each pixel of each of the superimposed images. This

  weighting coefficient is chosen equal to 0 when the pixel presents the color of the background, that is to say that it does not represent a point of the face or of the head. When the pixel does not have the background color, it is assigned a weighting coefficient which is all the smaller when the image distortion rate at this pixel is high, this distortion rate being defined by x '/ x = (x' / R) / Sin (x '/ R).

  In other words, the weighting coefficient of a pixel is all the higher when the image has been slightly deformed at the level of this pixel during the first substep. For example, the pixel weighting coefficient is chosen proportional to Cos (x '/ R) or

proportional to Sin (x '/ R) / (x' / R).

  Then the superimposed images are merged into a new image representing the whole face in the form of a planisphere, that is to say representing in fact the whole head seen on substantially 360. In the merge image, each pixel has a color which is an average of the colors of the pixels of the images which overlap at that location, said average being weighted using the aforementioned weighting coefficients. Figure 17 shows an example image

thus obtained.

  Finally, in the fourth sub-step, the fusion image is applied to the facets of the specific three-dimensional representation, using the tip of the nose as a reference point to ensure the correct positioning of the merged image with respect to to three-dimensional representation. To carry out this sub-step, a conventional texturing method is used, as described for example in Paul S. Heckbert, "Survey of texture mapping", IEEE Computer Graphies and Applications, pp 56-57 (November 1986). Figure 18

  represents an example of the result thus obtained.

  At the end of step 60, a textured three-dimensional image of the face is obtained, which can be used in various applications such as video games and video telecommunications. In the preferred embodiment, the process described above is implemented by a microcomputer provided with a graphics card and programmed by means of a computer program in one or more modules, the writing of which is human skill

of career.

  The embodiments of the invention having been described in the particular context of the three-dimensional representation of a human face, the invention however also relates to the three-dimensional representation of any type of object for which a generic representation can be provided, for example human creations,

  animals, plants and other objects.

  Although the invention has been described in connection with several particular embodiments, it is obvious that it is in no way limited thereto and that it includes all the technical equivalents of the means described as well as their combinations if these these enter the

part of the invention.

Claims (14)

  1. Method for generating a three-dimensional representation of an object. belonging to a kind of predetermined object, in particular human faces, from at least two two-dimensional images of said object, comprising the steps consisting in: making available (5) a generic three-dimensional representation (1) of said object, said generic representation comprising a plurality of vertices defining a mesh of said object, making available (10) a first two-dimensional image of said object seen in a first projection direction and a second two-dimensional image of said object seen in a second projection direction, deforming said representation generic three-dimensional to obtain a specific three-dimensional representation of the object, characterized in that it comprises the steps consisting in: selecting a plurality of characteristic points (P) from said vertices, identifying a first set composed of those of said points 20 characteristics which are screw ibles according to said first direction of projection, for each characteristic point belonging to said first set, identify (20, 40) an analogous point (Vi) in said first image, identify a second set formed of those of said characteristic points which are visible according to said second direction of projection, for each characteristic point belonging to said second set, identifying (20, 40) an analogous point (Vi) in said second image, said deformation step (50) comprising a displacement of said 30 characteristic points so as to: each characteristic point (P) belonging to said first set, reconciling a projection in the first direction of said characteristic point with said analogous point (Vi), for each characteristic point (P) belonging to said second set, reconciling a projection according to the second direction of said characteristic point with said analog point (Vi), said deformation step (50) comprising a displacement of each vertex (S) of said generic representation which is not a characteristic point as a function of the displacements of the characteristic points   (A, B, C) which are neighbors of said vertex.   2. Method according to claim 1, characterized in that it comprises the steps consisting in: defining a reduced mesh (52) of said object, said characteristic points (P) forming the vertices of said reduced mesh, for each of the characteristic points, defining an area of influence 10 (51) comprising all of the facets of said reduced mesh whose said characteristic point is a vertex,   any vertex (S) of said generic representation which is not a characteristic point being displaced, in said deformation step, in proportion to the displacements of any characteristic point in the zone of influence to which said vertex belongs.   3. Method according to claim 2, characterized in that the belonging of a vertex (S) to the zone of influence of a characteristic point (A) is determined, for each of the directions of projection, by projection of said vertex on triangular facets 20 of said reduced mesh in the corresponding projection direction, or   by projection on circles circumscribed to said facets.   4. Method according to one of claims 1 to 3,   characterized by the fact that the points (Vi) of said first image and / or of said second image are identified which are analogous to the characteristic points of the first and / or of the second set by detection of the points of the image for which a color property present a maximum gradient.   5. Method according to claim 4, characterized in that the said colorimetric property is chosen from a gray level, a red level, a blue level, a pixel green level   of the image and a combination of at least some of said levels.   6. Method according to one of claims 1 to 5,   characterized in that the step of identifying the points of said images which are analogous to the characteristic points of the first and / or 35 of the second set is carried out iteratively by moving approximation points (Vi) in said images from initial positions (21a), determined according to shapes and proportions known generics of said object.   7. Method according to claim 6, characterized in that the said approximation points form at least one broken line 5 (21a, 21b, 41, 42, 43) which deforms during the displacements of the said approximation points under prescribed geometric regularity conditions, for example a minimum distance between two points successive of said broken line.   8. Method according to claim 7, characterized in that one deforms said broken line by tending to minimize an internal energy Eint of said broken line defined by: Eint =, i (o Il Vi - Vi- l + i | Vi + i -2Vi + V-, II), oa and / denote predetermined real parameters, Vi denotes the vector position of a point of said broken line, Il. I denotes a vector norm and the sum is extended to all the points of said broken line.   9. Method according to one of claims 1 to 8,   characterized by the fact that said kind of object includes human faces, said characteristic points comprising points (41) around the eyes, points (43) around the mouth, points (42) around the nose and points (2 lb) of the contour of the face, said images comprising a planar projection in front view of a face and a   flat projection in profile view of said face.   10. Method according to claim 9, characterized in that it comprises a step (60) of texturing said specific three-dimensional representation of the object, consisting in expanding (62) a part (61) of the image in front view which represents the face and to associate by projection the texture of said expanded image part (63) with all the facets of the three-dimensional representation of the face. 30 11. Method according to claim 9, characterized in that it comprises a step (60) of texturing said specific three-dimensional representation of the object, consisting in associating at least the texture of a central part (65) of the image in front view with a central part of the three-dimensional representation of the face and associating at least the texture of a central part of the image in profile view with at least one lateral part of the three-dimensional representation of the face. 12. Method according to claim 11, characterized in that said texturing step (60) comprises the operations consisting in: deforming the image in front view and the image in profile view so as to obtain the images which would result if the face had a cylindrical outer surface which was unrolled, superimpose said distorted images so as to make the corresponding 10 points coincide in said images, merge said superimposed images into a merged image in which at least some pixels have a color resulting from 'a weighted average of the colors of the pixels of said overlapping images, applying the merged image to the specific three-dimensional representation. 13. The method according to claim 12, characterized in that said weighted average is calculated by weighting each of said pixels which are superimposed using a respective weighting coefficient, said coefficient being all the greater as the deformation undergone by the image to which said pixel belongs during said operation   deformation is low at said pixel.   14. Device for generating a three-dimensional representation of an object belonging to a kind of predetermined object from at least two two-dimensional images of said object, comprising: means for making available (5) a generic three-dimensional representation (1 ) of said object, said generic representation comprising a plurality of vertices defining a mesh 30 of said object, means for making available (10) a first two-dimensional image of said object seen in a first direction of projection and means for making available a second two-dimensional image of said object seen in a second direction of projection, deformation means for deforming (50) said generic three-dimensional representation in order to obtain said three-dimensional representation of the object, characterized in that it comprises: means for select a plurality of characteristic points (P) from the said s vertices, means for identifying a first set composed of those of said characteristic points which are visible in said first direction of projection, means for identifying (20, 40) an analogous point (Vi) of each characteristic point belonging to said first set in said first image, means for identifying a second set formed of those of said characteristic points which are visible in said second direction of projection, means for identifying (20, 40) an analogous point (Vi) of each characteristic point belonging to said second set in said second image, said deformation means comprising means for moving the characteristic points to move said characteristic points so as to: for each characteristic point (P) belonging to said first set, make a projection in the first direction of said point coincide characteristic with said analog point (V y), for each characteristic point (P) belonging to said second set, making a projection in the second direction 25 of said characteristic point reconcile with said analogous point (Vi),   said deformation means comprising means for moving the other vertices to move each vertex (S) of said generic representation which is not a characteristic point as a function of the displacements of the characteristic points (A, B, C) which are neighbors of said Mountain peak.