FR2838846A1 - Real time 3D image generation system uses detailed projected two dimensional image on solid model with moving parts having erased relief - Google Patents

Abstract

A real time 3D image generation system obtains a two dimensional synthesised or photographic video image of a subject from an observation point and projects (26) it from the viewpoint (23) with a simplified texture onto a three dimensional model (20) of the subject shape with erased relief around the lips (25) which can move in the 3D model. Includes an Independent claim for a video game using the technique.

Description

audit at least one remote control device.

  The present invention relates to a method for real-time generation of three-dimensional images in an interactive video game, as well as an interactive video game using

this process works.

  More specifically, the invention relates to the field of interactive trot-dimensional video games, that is to say games in which objects or characters

  animated three-dimensional evolve in a three-dimensional decor.

  It should first of all be recalled that in the field of interactive video games,

  a technical difficulty arises from the need to perform calculations in real time.

  This constraint results from the fact that the user of a game must have a certain freedom of choice as for the progress of the game. It is therefore not possible, except in exceptional cases, to record precalculated sequences in advance for which significant resources

  calculations would have been used differently.

  In the following description, we will call quiet any object or character before

to be viewed in a game.

  Video games have always evolved in the direction of an increase in their complexity. The first improvements had consisted in representing in trots

  dimensions of the subjects which, at the beginning, had only a two-dimensional representation.

  This had required the implementation of significant computing resources and was only possible when the computers accessible to the general public, in the form of computers

  personal or game consoles, had reached a sufficient level of power.

  But even today, the evolution of computing capacities is insufficient for complex and finely detailed three-dimensional shapes to be processed in

  real time as part of an interactive video game.

  In particular, the texture of the pieces, that is to say their visual appearance, which depends on the material of which they are made, their color, the lighting conditions and

  of observation, is often extremely simplified.

  The reason for such simplification is as follows.

  The calculations which must be executed to display a three-dimensional image of a subject belong to two categories, namely on the one hand the so-called "geometric" calculations, which treat the three-dimensional shape of the subject and determine its visible parts relative to the point d observation from which it is viewed, taking into account any other surrounding three-dimensional quiets which may mask it partially or completely, and on the other hand the so-called "texture integration" calculations which, starting from a texture of base applied to each facet of the three-dimensional model, determining the nuances to be applied to each visible point of this facet, to give it its appearance

as realistic as possible.

  -2 To approach an acceptable realism, these calculations must be very thorough and

  consequently turn out to be enormously consumers of computer resources.

  As a result, for most video games, the prospects for improving this realism are closely linked to the evolution of computers and game consoles on which they run. The present invention aims to provide a simple and economical solution for obtaining a visual rendering of subjects very close to a photographic or cinematographic rendering, and without resorting to particularly powerful means of calculation. In other words, the invention makes it possible, on computers currently available to the general public, to execute interactive three-dimensional video games in which the characters, animated objects and decors are represented in three-dimensional and

  have a very realistic visual aspect.

  The subject of the present invention is a method of real-time generation of three-dimensional images of a subject in an interactive video game, characterized in that it comprises the following stages: - obtaining a two-dimensional representation of the subject seen from an initial observation point, - creation of a three-dimensional model of the subject's shape, projection of the two-dimensional representation of the subject on the three-dimensional model from a projection point substantially coinciding with

  ['location of the initial observation point.

  In other words, the method according to the invention consists, in contrast to the known methods, in replacing the texture integration calculations by placing a picture prepared in advance of the subject, directly on the three-dimensional model. of shape

quiet.

  The inventors on the basis of the present invention have found, against all expectations, that the realism of the images used prevails over the detail of the volumes. Therefore, by plating very defined and very detailed images on simplex volumes, we give

  The impression that the player evolves in complex and detailed volumes.

  The effect obtained can in particular be used to reduce the three-dimensional definition of quiets, which paradoxically amounts to requiring less power

  of calculation on the part of the computers and consoles used for the execution of the game.

  Thus, the method according to the invention not only has the effect of providing a very realistic visual rendering but also of enabling the rehabilitation of

average power.

  -3 Thanks to the invention, one can visualize the object in three dimensions, vary the orientation and the width of the visualization field, add all the volume effects desired, all while benefiting from the depth of the image. provided by the three-dimensional model. For example, you can crop the subject or part of the subject, or even move the field of view to the entire image. Camera movements such as panning, tilting, zooming or slight translational movements can also be performed. In each case, the image obtained results from the projection of the two-dimensional representation

  on the three-dimensional model calculated.

  In a preferred embodiment of the invention, a simplified texture integration calculation is applied to the projected two-dimensional image which aims only to redefine the lighting and possibly to add special effects to the texture.

plate on the model.

  We then use the fact that the subject has a volume, for example to calculate projected shadows or reflections, and we obtain a visual aspect which cannot be

  obtained with a two-dimensional representation of the subject.

  In the hypothesis where the subject is visualized only from a point coinciding with the initial observation point, or located in the immediate vicinity of this point, no further definition of texture can be found. is necessary and the projection of the two-dimensional image on the three-dimensional model of the subject is sufficient to fully define

['image to display in real time.

  With this technique, one can for example carry out the decor of a scene of which all the plans will have been previously determined. To this end, we first identify the subjects making up the scene, that is to say the elements which must be treated separately during the geometric calculation, and we model each of these subjects in three-dimensional volume. We then capture one or more images of the scene according to all the visualization plans determined in advance, that is to say from one or more predetermined initial viewing points, we reconstitute the global three-dimensional model of the scene by combining all the three-dimensional models of all the elements making up the scene, pcis, during the generation of the image of the scene seen from one of the viewing points, the image previously captured from this point is generally projected onto the part of the global three-dimensional model visible from this same point, which provides

  the texture of each of the elements present on the scene.

  For a subject to be viewed during the game from different observation points distant from each other, ie the method according to the invention consists, in a particular implementation mode, in combining the technique described above of projection -4 of the two-dimensional representation of the subject with a more traditional technique of texture integration calculation. More precisely, part of the subject is textured according to the invention by placing a two-dimensional representation of the subject seen from the initial observation point and the rest of the subject is textured by a traditional three-dimensional synthesis technique, preferably with a reduced definition. For example, for a piece of furniture with inlaid in fa, cade, one can model the piece of furniture by a parallelepiped with six faces, of which five sides without of inlay wind simulated by calculation of texture integration and whose sixth face re

  a projection of a photograph of the inlaid side of the real furniture.

  This piece of furniture can then be viewed from any observation point,

  even away from the original observation point.

  We understand that the method according to the invention allows to save, tent during the phase of creation of the game that during its progress, many hours of modeling and calculation necessary for ia reconstruction of the face rich in details of the

1 5 furniture.

  It should be noted that it is not necessary, according to the invention, that the photograph of the detailed face of the piece of furniture be captured individually. Wile can come from a

  global shot incorporating different elements of the decor.

  One of the original features of the method according to the invention is that only the three-dimensional images actually viewed during the game can actually be generated to build the game, unlike traditional synthesis solutions which allow generation of all the possible images, viewed from any observation point. This unusual approach to building a video game goes against the common practices of a person skilled in the art, a design specialist and

of the realization of video games.

  In a particular embodiment of the invention, the two-dimensional representation of the subject consists not of a single fixed image but of a

  succession of images, also designated video sequence.

  In this case, the projection of the two-dimensional representation of the subject on his three-dimensional model serves not only to provide the many visual details of the subject

  but also to animate it with a very strong realism.

  In a particularly advantageous variant, the subject's three-dimensional model includes a mobile part whose wind movements are calculated during the geometric calculation phase. The setting in motion of this mobile part does not require very important calculations because, as has just been explained, the realism of the projected images prevails over the detail of the volumes. Consequently, by means of a simple animation of the three-dimensional volume on which the image is placed, the visual rendering is further improved,

  without overloading the computer.

  This variant is particularly interesting to animate the face of a character. In this case, the head of the character modeled in three-dimensional form includes a part corresponding to the chin which is made movable between an open mouth position and a closed mouth position. The face, mobile chin included, is the part of the three-dimensional model on which we project a video sequence showing the animated face of the character and the rest of the head, costing, depending on the case, a texturing

  synthetic or projected texturing taken from a 360 degree still image.

  The attached video sequence provides the texture of the character's face, chin included, which gives the player the impression that the entire face, i.e. the eyelids, the lips, the eyes, the different muscles of the face, is mobile while in

  reality only the chin moves in rhythm with the jaw.

  We can thus restore the realism and subtlety of the game of an actor whose face has been filmed, with all the palette of emotions that can be expressed by a human face, and this on game consoles of the type currently available on the market or

  on computers whose shield is already installed.

  In the particular case of a character's face, it is advantageous, according to the invention, to model the volume of the face by erasing the lips and the commissures so as to eliminate all the reliefs around the lips. In this way, the video sequence can recreate impressions of volume on a smooth surface and take precedence over real volume, giving the impression that there are moving reliefs around the lips. In addition, it is thus possible to recalculate the light of the face as a function of variable lighting sources without generating shadows which would interfere with the projected images and

  would reveal the fixed details of the facial geometric.

  The invention finds a particular interest when it is a question of making a character speak, because even in renouncement, ant to animate the facial features, simply the synchronization of its lips with a recorded voice would require in three-dimensional synthesis of

  calculations inaccessible to standard computers for an equivalent level of realism.

  Preferably, to integrate the projected video sequence with the texture of the rest of the model, that is to say to the fixed texture parts of the model, we fade in images at

  the periphery of the images in the video sequence.

  In a particular embodiment of the invention, several distinct video sequences are simultaneously projected onto adjacent parts of the three-dimensional model. -6 For example, to create new facial expressions, we combine together an upper half and a lower half of a face, which can be useful especially if the video sequence projected on the three-dimensional model of the face was created by filming the face of an actor and if only certain features of the face of the actor wind retained for a given situation of the play. So, more generally, this mode of implementation provides greater freedom in the creation of facial expressions during the making of the game, because expressions

  news can be obtained by mixing halves of the face.

  According to the invention, the two-dimensional representation of the subject, which is placed on the three-dimensional model, can be obtained by synthesis or by capture.

  photographic or cinematographic.

  If, in summary, important calculations are required, this does not constitute a difficulty because it is a question of preparing in advance the two-dimensional representation which will be registered and then be projected in real time on the model. . This preparation can therefore

  result from different calculations without disturbing the progress of the game.

  The present invention also relates to a video game obtained by

covers of the process described above.

  In order to facilitate understanding of the invention, we will now describe some

  embodiments, given by way of nonlimiting examples, with reference to the drawing

  appendix in which: - figure 1 is a shooting of a part constituting a decor of a video game, - figure 2 represents a three-dimensional model of the shape of the elements composing the decor of figure 1, - figure 3 is a view obtained by image mapping on the model of the figure - FIG. 4 represents a three-dimensional model of the shape of a head of a video game character, - FIG. 5 is an image of the same model, certain facets of which have re, cu texture, - Figure 6 is another image of the same model on which we have an image, - Figure 7 is an image similar to that of Figure 6, the model being seen from another point observation, - Figure 8 is an image similar to that of Figure 6, the model being seen from another observation point, 7 - Figure 9 is an image of the model of Figures 4 to 8 combined with a model of

character body.

  In FIG. 1, we decorate a scene comprising a corner of a room, the shape of two walls 1, 2, a floor 3 and a ceiling 4. The left wall 1 has a window 5 overlooking the exterior and the right wall 2 supports a frame 6. The walls 1, 2 are covered with a striped fabric and the floor is paved with black and white tiles.

Ceiling 4 is white.

  Two objects rest on the ground 3. One is a ball 8, the other a cube 9.

  As we fly the scene from the observation point used to take this

  seen, each object appears in perspective. The scene image is referenced 7.

  There are three subjects in the sense of the invention: the ball 8, the cube 9 and ['together constitutes the rest of the decor. Note that frame 6 and window 5 belong

  we assimilate them to the corresponding walls. In the following description,

  we designate this set by "background".

  In accordance with the invention, a three-dimensional model of the shape of

  each of the elements, or subjects, of the decor.

  Thus, as shown in Figure 2, the ball 8 is modeled by an object 8 'to 256

  planar facets, the cube 9 by its six faces 9 ', and the bottom by an object 10' has 58 faces.

  Each of the three-dimensional models is relatively simple and can be used as a basis for geometric calculations making it possible to visualize the subjects modeled from other

  observation points, if the scenario of the game allows it.

  The realistic visual rendering of the three subjects resulting from the operation consists in tackling

  overall [image 7 of figure 1 on three-dimensional model trots.

  Indeed, by restoring the image 7 of FIG. 1, which is global but rich in details, for all the subjects, we obtain in a simple way, for each of them, a realism close to that of a photographic or a movie. Thus, although this has not been shown on the drawing for reasons of simplification, if details of shape or ornamentation are visible on one of the subjects of the scene, they are restored during the placing of the image on three-dimensional model trots. These visual details create the illusion that the same geometric details have been taken into account in the modeling of subject shapes, while these shapes are actually relatively simple and make geometric calculations affordable for medium power calculators. -8 Given that the image plate on the three-dimensional models was taken from a given point of observation, only the faces of the subjects visible from this point

  observation can be restored by the projection operation.

  Thus, if we observe the same scene from another observation point, taking into account the volumes defined by the three-dimensional model trots 8 ', 9', 10 'of the ball, the cube and the background, the texture of the background contains the parts 8a of the ball 8 and 9a of the cube 9 projecting onto the ground 3 and the walls 1,2, because these parts are considered to form the texture of the background at these locations. This is what we observe in Figure 3, which represents the scene from an observation point located to the left of the scene seen according to the

figure 1.

  This apparent limitation of the method according to the invention, which limits the possibilities of viewing subjects under other planes than those initially planned, is in reality an advantage if we consider the computer resources which are freed from the fact that all the plans viewable from the scene, but ultimately not used in the game, are not processed. These freed-up resources can in particular be used to increase

  of the definition of the images actually visible in the course of the game.

  However, even in a game having only one image of the scene taken from a second observation point, for example [the image of figure 1, this image plate on the three-dimensional models of figure 2 is enough to obtain as much different views as desired, with a different viewing angle and framing than that of Figure 1, the only constraint being to keep an identical or very close observation point

  near the initial observation point.

  If changes of angle of view are planned, on the other hand, we can repeat the same work of image capture and projection of this image on the three-dimensional model trots seen from the observation point, which remains less resource-consuming. than providing all possible views from anywhere

what an observation point.

  Finally, for camera movements involving parallax faults between the first and last planes of the scene, it may be sufficient to locally reconstitute the texture of the background at the locations of the background discovered by the objects in the foreground during the

  displacement, without redoing all the work previously described.

  Thanks to the three-dimensional model of the scene, special effects such as projection of shadows and reflections can be calculated taking into account the shape of the subjects, and this in a relatively simple way given the limited definition of

  three-dimensional forms of the models of each subject.

  The scene modeled in three-dimensional in Figures 1 to 3 can also integrate a character, taking into account the relative positions of the character and the subjects constitute the decor. In other words, by calculating the interactions of this character with

its environment.

  The invention also applies to the representation of animated objects or characters. Figures 4 to 9 illustrate a character who can integrate into the scene of Figures 1 to 3. The tate of this character can be viewed by setting covers of the process

according to the invention.

  This head of character constitutes a subject in tent as such. within the meaning of the invention.

  The three-dimensional model 20 of this head is represented in FIG. 4. From the mail lage represented, it appears that the facets 2 1 of the three-dimensional form

  modeled relatively coarse wind.

  However, as previously explained, the precision of the image plate on this

  mesh more than makes up for this relatively low definition of volume.

  In addition, in this example, the subject is animated in its part 22 corresponding to the face. In Figure 5, we fly the three-dimensional model of Figure 4 whose facets, a

  [With the exception of those of the face, have received a texture from a 360 degree still image.

  As shown in Figure 6, in part 22 corresponds to the face, we project a mobile representation 26, that is to say a video sequence, from a virtual projection point which coincides with the observation point from which the projected image was acquired. For understanding, this projection point has been materialized by a virtual projector 23 oriented along an axis 24 passing substantially at the center of the face. Preferably, but not limited to, this observation point is located in front of the character's face, at a sufficient distance to have a field of vision covering the entire face of the person swimming. The acquisition is the result of a video recording of the

  face of an actor playing his role in the scene considered.

  Thus, the images 26 projected on the part 22 correspond to the face of the three-dimensional model 20 providing the details of the eyes, nose and mouth of the character. The projection of the images on the three-dimensional model provides a very realistic visual rendering despite the relative simplicity of the model. By this simple but effective process, one can visualize the facial expressions of the actor on the character of the game, while such expressions would require calculations impossible to perform -10 in real time on current standard computers and game consoles if he graduated from

model them integrally.

  How the volt in figures 7 and 8, the face thus restored can be visualized

  from any observation point other than the projection point.

  A fade-in of images is performed at the periphery of the face, that is to say at the border between part 22 and the rest of the party model, in order to connect continuously the images of the video sequence projected on the face with the still image make up the rest

of the character’s head.

  We understand that the video sequence makes it possible to animate the character's face only by modifying the facial expressions, without modifying the shape of the latter. In an advantageous variant, the m menton 25 of the personage is modeled in tent that mobile element can take a low position and a high position which

  correspond to the movements of the character's mouth when he speaks.

  This mobile element 25 does not require a lot of geometric account calculation

  given its relatively low definition, as can be seen in Figure 4.

  Facial chin movements synchronize movements

  from the mouth with a text that is spoken by the character.

  We also see, in Figure 4, that the character's mouth has been erased in the three-dimensional model and has been replaced by a slightly hemispherical zone, more conducive to image projection, so that the projected images visually reconstruct the reliefs of mouth only through shaded areas

marking the lips.

  In this way, it is possible, by processing the lights, to modify the lighting of the

  face without highlighting undesirable details of the three-dimensional model.

  Furthermore, this eliminates the risk that the position of the mouth of the three-dimensional model does not exactly coincide with the mouth of the projected video sequence. As shown in figure 9, model 20 of the head can be combined with a

  model 30 of character body, to then be integrated into the decor of Figures 1 to 3.

  It is understood that the embodiments which have just been described do not

  have no limiting nature and that they may receive any desirable modification

  without departing from the scope of the invention.

-1 1

Claims (11)

  1. Method for real-time generation of three-dimensional images of a subject in an interactive video game, characterized in that it comprises the following steps: - obtaining a two-dimensional representation (7, 26) of the quiet seen from a point initial observation, - creation of a three-dimensional model (8 ', 9', 10 ', 20) of the subject's shape, - projection of the two-dimensional representation (7, 26) of the subject on the three-dimensional model (8' , 9 ', 10', 20) from a projection point (23)   substantially coinciding with the location of the initial observation point.   2. Method according to claim 1, in which a simplified texture calculation is applied to the projected two-dimensional image (7, 26) which is intended solely to redefine the lighting and possibly to add special effects to the texture.   plate on the three-dimensional model (8 ', 9', 10 ', 20).   3. Method according to any one of claims 1 and 2, applied to the   realization of the decor of a scene of which all the shots have been previously determined, in which we first identify the subjects (8, 9, 1, 2, 3, 4, 5, 6) composing the scene, we model each these quiets in three-dimensional volume, one or more images (7) of the scene are captured according to all the viewing plans determined in advance, that is to say from one or more predetermined initial viewing points, the image is reconstructed. global three-dimensional model of the scene by combining all the three-dimensional models (8 ', 9', 10 ') of all the elements making up the scene, puts, when generating the image of the scene seen from one of the viewing points , we project globally the image (7) previously captured from this point on the part of the   global three-dimensional model visible from this same point.   4. Method according to any one of claims 1 to 3, wherein   combines the projection of the two-dimensional representation (26) of the subject with a texture calculation technique, a part (22) of the subject being textured by projection of a two-dimensional representation (26) of the subject seen from the initial observation point and the rest of the subject being textured by a three-dimensional synthesis technique,   preferably with a reduced definition.   5. Method according to any one of claims 1 to 4, wherein the   two-dimensional representation (26) of the subject consists of a succession of images, also designated video sequence.   6. Method according to claim 5, in which the three-dimensional model (20) of the   subject has a movable part (25) whose calculated wind movements. -12   7. Method according to any one of claims 5 and 6, applies to the   generation of the image of a character's face, in which the volume of the face is modeled by erasing the lips and the commissures so as to eliminate all of the reliefs around the lips.   8. Method according to any one of claims 5 to 7, in which one realizes   a blend of images on the periphery of the images in the video sequence.   9. Method according to any one of claims 5 to 8, in which one projects   simultaneously several distinct video sequences on adjacent parts of the three-dimensional model.   10. Method according to any one of claims 1 to 9, in which the   two-dimensional representation (7, 26) of the subject which is placed on the three-dimensional model (8 ', 9', 10 ', 20) is obtained by synthesis or by photographic capture or cinematographic.   11. Three-dimensional interactive video game obtained by implementing the method according to   Any of claims 1 to 10, wherein the representation of at least one   element results from the projection of a two-dimensional representation of the element on a