JPH08305894A - Three-dimensional image generating device capable of representing wrinkle - Google Patents

Abstract

PURPOSE: To provide a three-dimensional image generating device which can represent wrinkles of an object in an actual space on a three-dimensional image in a virtual space. CONSTITUTION: When a wrinkled texture can be obtained, it is acquired and its edge is detected to generate a luminance map IP showing the displacement of the wrinkes. When the wrinkled texture can not be obtained, on the other hand, wrinkle elements WP are generated by using a shade function and on the basis of them, the illuminance map IP and wrinkled texture W are generated inside. On the basis of the luminance map IM, a wire frame model is deformed and according to the model, no-wrinkle texture is changed into the wrinkled texture.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional image generating apparatus capable of expressing wrinkles, and more specifically, it photographs an object in a real space and produces a three-dimensional image corresponding to the photographed object in a virtual space. The present invention relates to a three-dimensional image generation device that generates in real time.

[0002]

2. Description of the Related Art In recent years, various proposals have been made for the realization of an immersive communication conference in which a plurality of people in different places can hold a meeting as if they were gathering together. . The purpose of the immersive communication conference is to place an image of conference participants in different locations in the created virtual space, and provide an environment where all participants can have meetings and collaborate as if they are sharing the same space. Especially.

In order to realize such a realistic communication conference, a person on the transmitting side is modeled in advance using a wire frame model, and the wire frame model is deformed according to the movement of the image of the person, and the model is deformed. A system has been proposed in which a person image is stereoscopically displayed on the receiving side by mapping the texture of the person on a wire frame model. An example of such a system is, for example, “Otani et al.,“ Real-time display of three-dimensional face image in realistic communication conference ”, Jikken Giho, HC92-61 (19
93-1) ", pages 23 to 28.

[0004]

However, the system described above has a problem in that it is not realistic because it is not possible to represent wrinkles on the clothes and skin of a person.

[T. Kunii et al., "Singularity theo
retical modeling and animation of garment wrinkle
formation processes ", Visual Computer, Vol-6, No.
6, pp.326-336 (1990) "and" M. Carignan et al.
“Dressing animated sinthetic actors with complex
deformable clothes ", Computer Graphics, 26, 2, pp.
99-104 (1992.7) ”describes a method of representing clothes using a physical model and thereby generating wrinkles. According to this method, since a physical model is used, physically accurate wrinkles can be generated, but since the amount of calculation is large and the calculation is complicated, it takes a long time to generate wrinkles. Therefore, this method is not suitable for a realistic teleconferencing system. This is because it is necessary to generate wrinkles in real time according to the movement of the person on the transmitting side in the realistic communication conference.

Therefore, an object of the present invention is to provide a three-dimensional image generation apparatus capable of expressing wrinkles of an object on a three-dimensional image.

Another object of the present invention is to provide a three-dimensional image generating apparatus capable of expressing wrinkles in real time as accurately as possible.

[0008]

A three-dimensional image generation apparatus capable of expressing wrinkles according to claim 1 photographs an object in a real space and produces a three-dimensional image corresponding to the photographed object in a virtual space. A three-dimensional model generation means, a wrinkle-free texture acquisition means, a wrinkle-free texture mapping means, a wrinkled texture acquisition means, a wrinkled texture mapping means, and a displacement map generation means, which are generated in real time. , Three-dimensional model transformation means and morphing means. The three-dimensional model generation means generates a three-dimensional model by modeling an object. The wrinkle-free texture acquiring means acquires a wrinkle-free texture from the surface of the object when there is no wrinkle on the surface. The wrinkle-free texture mapping means maps the texture on the three-dimensional model.
The wrinkled texture acquisition means acquires the wrinkled texture from the surface of the object when the surface of the object has wrinkles. The wrinkled texture mapping means maps the wrinkled texture to the three-dimensional model. The displacement map generating means generates a displacement map indicating the displacement of the surface of the object when the wrinkled texture is acquired. The three-dimensional model deforming means deforms the three-dimensional model according to the movement of the object. The morphing means morphs from a wrinkle-free texture to a wrinkled texture.

A three-dimensional image generating device according to a second aspect is for photographing an object in a real space and generating a three-dimensional image corresponding to the imaged object in a virtual space in real time. An original model generation means, a wrinkle-free texture acquisition means, a wrinkle-free texture mapping means, a wrinkle element generation means, a displacement map generation means, a wrinkled texture generation means, a three-dimensional model deformation means, and a morphing means. Prepare The three-dimensional model generation means generates a three-dimensional model by modeling the object as a whole. The wrinkle-free texture mapping means maps the wrinkle-free texture on the three-dimensional model. The wrinkle element generation means generates a wrinkle element using an implicit function. The displacement map generation means generates a displacement map using the wrinkle element. The wrinkled texture generating means generates a wrinkled texture from the wrinkled texture according to the displacement map. The wrinkled texture mapping means maps the wrinkled texture to the three-dimensional model. The three-dimensional model deforming means deforms the three-dimensional model according to the displacement map. The morphing means morphs from a wrinkle-free texture to a wrinkled texture.

[0010]

In the three-dimensional image generating apparatus according to the first aspect,
A wrinkled texture is acquired from the surface of the object in advance, and a displacement map indicating the displacement of the surface is generated. Then, the surface of the three-dimensional model is deformed according to the displacement map, and is further morphed from a wrinkle-free texture to a wrinkled texture. Therefore, the wrinkles of the object in the real space can be generated in real time on the three-dimensional image in the virtual space.

In the three-dimensional image generating apparatus according to the second aspect, the wrinkle element is generated in advance using the implicit function, and the displacement map is generated using the wrinkle element. A wrinkled texture is generated according to the displacement map, and the wrinkled texture is mapped to a three-dimensional model. Then, the three-dimensional model is deformed according to the displacement map, and is further morphed from a wrinkle-free texture to a wrinkled texture. Therefore, the wrinkles of the object in the real space can be generated in real time on the three-dimensional image in the virtual space.

[0012]

Embodiments of the present invention will be described in detail below with reference to the drawings. In the drawings, the same reference numerals indicate the same or corresponding parts.

FIG. 1 is a block diagram showing the overall construction of a three-dimensional image generating apparatus according to an embodiment of the present invention. With reference to FIG. 1, this three-dimensional image generation device includes a wireframe model generation unit 1, a color texture acquisition unit 2, a wrinkle element generation unit 3, an edge detection unit 4, and a brightness map generation unit 5.
And a wrinkled texture generation unit 6, a color texture mapping unit 7, and a texture morphing / wireframe model transformation unit 8. The wireframe model generation unit 1 is a part that creates in advance a wireframe model of a person who participates in the presence communication conference before the conference. The color texture acquisition unit 2 is a unit that acquires in advance the color texture of clothes or skin to be mapped to the generated wireframe model. The wrinkle element generator 3 uses a predetermined implicit function (Implicit Fun
ction) to generate wrinkle elements. The wrinkle element represents the amount of offset (displacement) from the generated wire frame model surface. The edge detecting section 4 is a section that photographs the wrinkled clothes and detects the edges thereof. According to this, the brightness of the ridge portion of the wrinkle is high and the brightness of the valley portion is low. The brightness map generation unit 5 generates a brightness map based on the edge intensity I obtained by the edge detection unit 4, or a part that generates a brightness map based on the wrinkle element WP obtained by the wrinkle element generation unit 3. Is. The wrinkled texture generation unit 6 supplies the wrinkled texture W obtained by the color texture acquisition unit 2 to the color texture mapping unit 7 as it is, or uses the brightness map IM obtained by the brightness map generation unit 5 by itself. This is a part for generating a wrinkled texture and supplying the generated wrinkled texture W to the color texture mapping unit 7. The color texture mapping unit 7 maps the wrinkle-free texture N obtained by the color texture acquisition unit 2 onto the wireframe model WFM obtained by the wireframe model generation unit 1 and also obtained by the wrinkle texture generation unit 6. This is a portion for mapping the wrinkled texture W. In the above-mentioned parts 1 to 7, respective processing is performed in advance before the presence communication conference.

The texture morphing / wireframe model transformation unit 8 uses the luminance map IM obtained by the luminance map generation unit 5 during the realistic communication conference to obtain the wireframe model WFM obtained by the wireframe model generation unit 1. It is the part that changes the texture from wrinkle-free texture to the texture with wrinkle by morphing. The wire frame model on which the wrinkle-free and wrinkled textures are mapped is supplied from the color texture mapping unit 7 to the transformation unit 8.

Next, the operation of this three-dimensional image generating apparatus will be described. This three-dimensional image generation device has a mode for acquiring a wrinkled texture from an actual person and a mode for internally generating a wrinkled texture.

(1) Wrinkled texture acquisition mode (when wrinkled texture can be acquired) Wrinkled texture may not be directly acquired from an actual person depending on its shape and bias, but here the wrinkled texture is actually obtained. Described below is a case where the person can be acquired from.

First, in the wire frame model generator 1,
By modeling the person who participates in the realistic communication conference and tries to generate the three-dimensional image in the virtual space,
A predetermined wireframe model is generated. Such modeling is performed for each human body part such as a person's head, upper body, arms, hands, and feet. Human parts such as the head, upper body, and arms are modeled from cyberware using a three-dimensional digitizer. This digitizer irradiates a laser beam while rotating around a person and inputs the shape of each human body part of the person.

The color texture acquisition unit 2 acquires the color texture of the clothes or skin of the actual person described above. Here, a wrinkle-free texture is acquired from the surface of a person when the surface is not wrinkled, and a wrinkled texture is acquired from the surface when the surface of a person is wrinkled. Such a color texture is created based on an image of a person being input at the same time when the shape described above is input.

The edge detecting section 4 detects an edge by processing the image photographed for acquiring the above-mentioned color texture. When the edge detection is performed, the height of the wrinkles is represented by the shade of the image.

Next, the brightness map generator 5 generates a brightness map IM based on the edge strength I supplied from the edge detector 4. This edge strength I is the output of the edge filter and is proportional to the wrinkle height of the acquired wrinkled texture. Therefore, in the brightness map IM, the higher the wrinkle height, the whiter it is, and the lower the wrinkle height, the blacker it is. Generated luminance map IM
Are supplied to the texture morphing / wireframe model transformation unit 8 during the presence communication conference.

On the other hand, the color texture mapping section 7
Then, the wrinkle-free texture N supplied from the color texture acquisition unit 2 is mapped to the wire frame model WFM supplied from the wire frame model generation unit 1. At the same time, the wrinkled texture W supplied from the color texture acquisition unit 2 through the wrinkled texture generation unit 6 is mapped to the wire frame model WFM supplied from the wire frame model generation unit 1. As a result, all the preprocessing that should be performed in advance prior to the presence communication conference is completed.

Next, when the realistic communication conference is started, the texture morphing / wireframe model transformation unit 8 supplies the wireframe from the wireframe model generation unit 1 according to the actual movement of the person in the real space. The model WFM is deformed as a whole, and the wire frame model WFM is locally deformed according to the brightness map IM supplied from the brightness map generator 5.
At the same time, texture morphing is performed, whereby the texture N without wrinkles is gradually switched to the texture W with wrinkles. Therefore, when a person in the real space moves and wrinkles occur on clothes, wrinkles corresponding to the wrinkles are also generated in the three-dimensional image.

(2) Wrinkled texture internal generation mode (when the wrinkled texture cannot be acquired) Next, the case where the wrinkled texture cannot be acquired from an actual person in the real space will be described. In this case, the wireframe model generation unit 1 generates the wireframe model and the three-dimensional mesh for generating wrinkles. This 3D mesh is a local wireframe model. When there is no wrinkle on the surface of the person, the color texture acquisition unit 2 acquires only the wrinkle-free texture N from the surface.

The wrinkle element generator 3 uses the implicit function f (X) shown in FIG. 2A to generate the wrinkle element WP as shown in FIG. 2B. As shown in FIG. 2A, the implicit function f (X) becomes “1” when the variable X is “0” and becomes “0” when the variable X is “1”. The differential function f '(X) becomes "0" when the variable X is "0" or "1". Therefore, as shown in FIG. 7B, the wrinkle element WP represented by using the implicit function f (X)
The displacement increases as it approaches the center of the wrinkle, and the displacement decreases as it moves away from the center of the wrinkle. By using such an implicit function f (x), it is possible to generate wrinkle elements corresponding to wrinkles having various shapes as shown in FIGS.

The wrinkle element WP will be described in detail with reference to FIGS. 2 (a) and 2 (b). The implicit surface is the formula F
It is defined as a set of P satisfying (P) = 0. A valid implicit function can be generated as an algebraic combination of polynomial functions, each polynomial function being defined over a finite amount.
For example, what is produced by the addition is the smooth blending of the implicit function. Here, F (P) is represented by the following equation.

[0026]

[Equation 1]

Where i is the polynomial function f _i of the wrinkle element and T is the threshold value E [0,1]. Subsets of these implicit surface are distance surface and convolution surface. These are assembled by combining several implicit function elements. Each element is a finite field V (typically a sphere) and a skeleton within the volume S (typically the center of the sphere)
And a function f (typically a polynomial with certain properties). The element only affects the surface in V. The surface is then created by the combination of influences from the elements. Also called the density function, F with desirable properties:
An example of [0,1] → [0,1] is shown in FIG. For a point P in V, the function value is determined as follows. First, a value between [0,1] is calculated, which is the ratio between the closest Euclidean distance from P to point Q on S and the value defined by the shape of V (generally a constant radius value R). Often used. This value given to f gives the function value of point P. The above description produces what is called an offset surface. A range surface class is created when the radius R does not have to be constant. The convolution surface requires the integral of the function value obtained with Q varying over each point on S, not just the closest point of S. The difference between the two surfaces lies in the function value calculated for P. The convoluted surface provides a surface having properties such as seamlessly mixed recesses and no unwanted bulges in the mix.

To generate the intensity map IM, an offset surface density map with transformed characteristic shapes, such as the skeleton S of FIG. 2 (b), is constructed around each wrinkle. The wrinkle offset element is constructed as follows.

The normalized characteristic shape is first transformed based on the factor of the wrinkle degree. This is performed by appropriately finely moving the control point 9 having a skeletal shape with joints. This skeleton shape S is spatially transformed into a common space in which the luminance maps are combined.

The thickness parameter controls the radius R of the offset surface. The displacement parameter appropriately scales the value of the displacement (implicit function).

The secondary intensity map is then calculated by evaluating the effect of the added displacement due to wrinkles on each pixel of the map. For all pixels P in the map, the effect of wrinkle displacement is calculated as:

The point Q on S that has the closest Euclidean distance to P is calculated. The wrinkle affects the displacement amount at P only when | PQ | <R within the wrinkle radius R.

The attribute value of the displacement amount of the characteristic shape in Q is
It is obtained as an interpolation of the attribute values of the two closest control points i1 and i2 existing therebetween. This value is the i _Q.

The wrinkle's displacement contribution is displacement x i _Q ×
f (| PQ | / R). Here, the displacement amount is a parameter controlled by the user of the wrinkle example, and f is a density function (implicit function) as shown in FIG.
The brightness map generator 5 uses the wrinkle element WP supplied from the wrinkle element generator 3 to generate the brightness map IM.

The wrinkled texture generation unit 6 uses the wrinkle-free texture N supplied from the color texture acquisition unit 2 and the brightness map IM supplied from the brightness map generation unit 5 to generate a wrinkled texture W. . At this time, the lighting condition of the place where the realistic communication conference is held is appropriately given. Further, here, the three-dimensional mesh, which is a local wire frame model generated for the wrinkle expression, is deformed.

Next, the color texture mapping unit 7
Then, the three-dimensional mesh supplied from the wrinkled texture generation unit 6 is fitted into the wire frame model WFM supplied from the wire frame model generation unit 1. In the color texture mapping unit 7, the wrinkle-free texture N supplied from the color texture acquisition unit 2 is mapped to the wireframe model WFM supplied from the wireframe model generation unit 1 and the wrinkle texture generation unit 6 also The supplied wrinkled texture W is similarly mapped to the wire frame model WFM supplied from the wire frame model generation unit 1. This provides a three-dimensional wireframe model with wrinkles on the garment and a corresponding color texture. By the above, all the preprocessing that should be performed in advance prior to the presence communication conference is completed.

Next, when the realistic communication conference is started, the texture morphing / wireframe model transformation unit 8
Then, the wireframe model is deformed and the color texture is morphed as in the wrinkle texture acquisition mode described above. That is, the wireframe model is deformed using the brightness map IM supplied from the brightness map generator 5, and texture morphing is performed accordingly, whereby the wrinkle-free texture is gradually switched to the wrinkled texture.

As described above, according to this embodiment, since wrinkles are expressed by the deformation of the wire frame model and the texture morphing without using a complicated physical model, there may be a case where faithful reproducibility is lacking. , Real wrinkles can be generated at high speed. Therefore,
It is suitable for a system that requires real-time processing such as a realistic communication conference. In addition, since wireframe deformation and texture morphing are often installed in a normal workstation, it is possible to display at sufficiently high speed with existing hardware.

When a wrinkled texture can be obtained, the wrinkled texture is positively used to represent wrinkles, so that complicated calculations are reduced and the scale of necessary hardware can be reduced. On the other hand, even when the wrinkled texture cannot be obtained, since the wrinkled texture is internally generated using a simple implicit function, it is possible to generate a real wrinkle in real time.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to the embodiments described above. For example, the present invention can be applied not only to wrinkles on clothes and skin but also to wrinkles formed on the surface of a general plastic object. The present invention has various improvements based on the knowledge of those skilled in the art without departing from the spirit of the invention. The present invention can be implemented in a mode in which modifications, modifications, and the like are added.

[0041]

According to the three-dimensional image generation apparatus of the first aspect, a wrinkled texture is obtained and a displacement map corresponding to the wrinkled texture is generated, whereby the wireframe model is deformed and the wrinkle-free texture is obtained by morphing. Since the texture is deformed with wrinkles, the wrinkles formed on the surface of the object can be reproduced in real time on a three-dimensional image. Moreover, since the wrinkled texture is obtained directly from the object, it is not necessary to perform a large amount of complicated calculation for expressing wrinkles, and the size of this device can be kept small.

According to the three-dimensional image generating apparatus of the second aspect, the wrinkled texture is internally generated, and
A displacement map corresponding to that is generated, and the 3D model is deformed by this, and since the wrinkle-free texture is transformed to the wrinkled texture by morphing, the wrinkles occurring on the surface of the object are displayed on the 3D image in real time. Can be reproduced at. Moreover, since the wrinkled texture is generated internally, wrinkles can be expressed even if the wrinkled texture cannot be obtained directly from the object.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a three-dimensional image generation device according to an embodiment of the present invention.

2A and 2B are views for explaining the method of generating the wrinkle element shown in FIG. 1, FIG. 2A shows an example of an implicit function used to generate the wrinkle element, and FIG. An example of the wrinkle element generated using the function is shown.

3 (a) to 3 (c) are schematic diagrams showing various forms of wrinkles occurring in clothes and the like.

[Explanation of symbols]

1 wireframe model generation unit 2 color texture acquisition unit 3 wrinkle element generation unit 4 edge detection unit 5 luminance map generation unit 6 wrinkle texture generation unit 7 color texture mapping unit 8 texture morphing / wireframe model transformation unit WFM wireframe model N Wrinkle-free texture W Wrinkled texture WP Wrinkle element IM Luminance map

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication location H04N 13/02 9365-5H G06F 15/72 450A (72) Inventor Jun Otani Seika-cho, Soraku-gun, Kyoto Prefecture Daiji Intani Shoji 5 Hiratani No.5 AT R Communication System Research Institute Co., Ltd. (72) Inventor Fumiro Kishino 5 Seiji-cho, Soraku-gun Kyoto Pref. In-house

Claims (2)

[Claims] 1. A three-dimensional image generation apparatus for photographing an object in a real space and generating a three-dimensional image corresponding to the photographed object in a virtual space in real time, wherein the object is modeled. A three-dimensional model generation means for generating a three-dimensional model by the above, wrinkle-free texture acquisition means for acquiring a wrinkle-free texture from the surface of the object when there is no wrinkle, and the wrinkle-free texture in the three-dimensional model. Wrinkle-free texture mapping means for mapping, wrinkled texture acquisition means for acquiring wrinkled texture from the surface of the object when there are wrinkles, and wrinkled texture mapping for mapping the wrinkled texture in the three-dimensional model Means and when the wrinkled texture is acquired Displacement map generation means for generating a displacement map indicating the displacement of the surface of the object, three-dimensional model deformation means for deforming the three-dimensional model according to the displacement map, from the wrinkle-free texture to the wrinkled texture A three-dimensional image generation device capable of expressing wrinkles, comprising a morphing means for morphing. 2. A three-dimensional image generation apparatus for photographing an object in a real space and generating a three-dimensional image corresponding to the photographed object in a virtual space in real time, wherein the object is modeled. A three-dimensional model generation means for generating a three-dimensional model by the above, wrinkle-free texture acquisition means for acquiring a wrinkle-free texture from the surface of the object when there is no wrinkle, and the wrinkle-free texture in the three-dimensional model. Wrinkle-free texture mapping means for mapping, wrinkle element generation means for generating wrinkle elements using an implicit function, displacement map generation means for generating a displacement map using the wrinkle elements, and the wrinkle according to the displacement maps. A wrinkled texture generating means for generating a wrinkled texture from an untextured texture; A wrinkled texture mapping means for mapping the wrinkled texture, a three-dimensional model deforming means for deforming the three-dimensional model according to the displacement map, and a morphing means for morphing the wrinkled texture to the wrinkled texture. A three-dimensional image generator capable of expressing wrinkles.