JP2784097B2 - Image synthesizer - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizer for deforming and synthesizing an image in a three-dimensional direction and providing a synthesized image output.

[0002]

2. Description of the Related Art FIG.
FIG. 1 is a conceptual diagram of a conventional image transmission system disclosed in Japanese Patent Application Laid-Open Publication No. HEI 10-125, 1988.
In FIG. 6, 11 is a code book, 12 is a memory, 1
Reference numeral 3 denotes a synthesis unit, 116 denotes phoneme codes in audio information, 117 denotes mouth shape parameter information, 118 denotes mouth image information, 119 denotes face image information other than the mouth, and 120 denotes reproduced face image information.

Next, the operation will be described. First, for the phoneme code 116 in the voice information from the transmitting side, the corresponding mouth shape parameter 117 is selected from the codebook 11 in which the parameter values of the mouth shape model image are stored for all phoneme codes as shown in FIG. . Based on the mouth shape parameter 117, the mouth shape model image in the memory 12 storing the mouth shape model image as shown in FIG. 8 defined by a set of mouth shape parameters indicating the geometric shape of the mouth portion is modified. The mouth shape model image 118 and the face moving image 119 other than the mouth from the transmitting side are synthesized by the synthesizing unit 13 and the reproduced face image information 120 is output.

[0004]

Since the conventional image transmission system is configured as described above, the face image information other than the mouth inputted by the image pickup system such as a camera is limited to the original image, so that it is composed. The reproduced image output from the unit is limited to the same image as the input image except for the mouth. Therefore, there has been a problem that a plurality of shade information cannot be selected (except for the mouth shape model image) for the face image to be the target of the composite image.

SUMMARY OF THE INVENTION The present invention has been made to solve such a problem, and arbitrarily automatically generates a composite image different from an original image by performing texture mapping processing of a plurality of grayscale information on a plurality of three-dimensional wireframes. It is intended to be.

[0006]

According to the present invention, there is provided a three-dimensional wire frame deforming unit for deforming a basic three-dimensional wire frame and outputting wire frame information in accordance with three-dimensional configuration information of a subject image input from an image input unit. A combined image generation processing unit for matching images based on the wire frame information and the grayscale information; a wire frame information storage unit storing a plurality of wire frame information; a wire frame information of the storage unit and the three-dimensional wire frame First switching means for switching and outputting the output of the deforming section to the composite image generation processing section, a density information storage section storing various density information, density information from this storage section, and density obtained from the subject image Second switching means for switching and outputting information to the composite image generation processing unit,
The composite image generation processing unit includes at least two three-dimensional wires
Each composite distribution for frame and at least two shade information
The value is arbitrarily determined.

Further, according to the present invention, there is provided a three-dimensional digitizer for extracting a feature point from a subject image obtained by projecting slit light on an object and calculating three-dimensional structure information of the subject based on the feature point. Is provided.

[0008]

In the three-dimensional wire frame deforming unit according to the present invention, the basic three-dimensional wire frame is deformed by the three-dimensional coordinates of the subject measured by the above-described method, and a three-dimensional wire frame matched with the subject image is obtained. The input of the composite image generation processing unit is provided with first switching means so that three-dimensional wire frame information matched with the subject image and other three-dimensional wire frame information in the wire frame information storage unit can be freely selected.

On the other hand, the second information is provided so that the density information of the subject image and other density information in the density information storage section can be freely selected.
A switching means is provided, each input information is determined by switching between the two switching means, and a texture mapping process is performed on the selected shade information to the selected three-dimensional wire frame information, so that at least two three-dimensional wires are obtained. Fret
And each of the composite distribution values for the
Generate an infinite number of composite images by arbitrarily determining
be able to.

The three-dimensional digitizer in the image synthesizer configured as described above becomes effective information for expressing the three-dimensional structure of the object with respect to the object image obtained by projecting the slit light on the object. The feature points are extracted, the three-dimensional coordinates of each obtained feature point are calculated, and the three-dimensional structure information of the object is extracted.

[0011]

Embodiment 1 FIG. 1 is a block diagram showing an embodiment of the present invention.
In FIG. 1, 1 is a slit light irradiating unit, 2 is an image input unit, 3 is a noise filter, 4 is a feature point extracting unit, and 5 is 3
Dimensional coordinate calculator, 6 is a basic three-dimensional wireframe database, 7 is a three-dimensional wireframe deformer, 8 is 3
A dimensional wire frame database, 9 is a shade information database, and 10 is a composite image generation processing unit. Also, 1
01 is an input image, 102 is an input image obtained by projecting slit light, 103 is an input image after noise removal, 104 is feature point information, 105 is three-dimensional coordinate information, 106 is basic three-dimensional wireframe information, and 107 is a subject image 108 is registered three-dimensional wire frame information, 109 is shading information of the subject image,
Reference numeral 10 denotes registered grayscale information, 111 denotes a composite image output, 112 denotes input wire frame information, and 113 denotes input grayscale information. Further, in FIG. 1, reference numerals 1 to 5 constitute a known three-dimensional digitizer.

In the composite image generator configured as described above, the slit light irradiating unit 1
Thus, the input image 102 (FIG. 2) on which the slit light is projected is obtained. In FIG. 2, reference numeral 114 denotes a subject image in the input image, and 115 denotes a set of irradiated slit light points. The input image 102 irradiated with the slit light is passed through the noise filter 3, and the feature point extracting unit 4 extracts a feature point from the noise-removed image 103. With respect to the feature point information 104 extracted by the feature point extraction unit 4, the three-dimensional coordinate calculation unit 5 calculates three-dimensional coordinates of each feature point. Based on the calculated three-dimensional coordinate data 105, the basic three-dimensional wire frame (FIG. 3) registered in the basic three-dimensional wire frame database 6 matches the subject image in the three-dimensional wire frame deforming unit 7. Be transformed.

Then, the three-dimensional wire frame information 107 after the matching is output. Further, one of the wire frame information 107 of the subject image and the other wire frame information 108 in the three-dimensional wire frame database 8 is input to the composite image generation processing unit 10 by switching the first switch sw1. Either the grayscale information 109 of the subject image or another grayscale information 110 registered in the grayscale information database 9 is input to the composite image generation processing unit 10 by switching the second switch sw2, and the input wireframe information is input. The texture mapping process is performed by using the input density information 113 for each triangular patch which is a component of the
Is output.

Here, the wireframe data format is the x, y, z coordinates of three vertices of a triangular patch which is a component of the wireframe database, and connection information of the triangular patch. FIG. 4 explains these, but as described above, (a) and (b) are stored in the three-dimensional wireframe database as data files. The basic three-dimensional wireframe information 106 and the three-dimensional wireframe information 108 of the subject image include both FIGS.

FIG. 5 shows an outline of the texture mapping process. Pixel values in a triangular patch, which is a component of a three-dimensional wire frame, can be obtained by oblique coordinates, and the position of a point inside the triangular shape can be obtained by a ratio of length to two sides. In the figure, an arbitrary point P in a triangular patch DEF of a three-dimensional wire frame of a subject image can be represented by the following equation using vectors on each side. A = sP1 + tP2

However, 0 ≦ s ≦ 1, 0 ≦ t ≦ 1

Here, vectors A, P1, and P2 are represented by A = (a1, a2), P1 = (p11, p12), and P2, respectively.
= (P21, p22) From the above equation, a1 = sP11 + tP12 a2 = sP21 + tP22 From the above two equations, s and t are obtained and applied to the corresponding triangular patch ABC of the basic three-dimensional wire frame to correspond to the point P. Get point Q. Therefore, the triangular patch AB
The shading value of the point Q in C is set as the shading value of the point P in the triangular patch DEF, and the above operation may be similarly performed for all the points in the triangular DEF name.

As a result, one input density information corresponds to one
In addition to generating an image using one piece of input wire frame information, an arbitrary composite distribution value can be selected for a plurality of pieces of shade information or a plurality of pieces of wire frame information, and improvement in versatility can be expected. The input wire frame information from the n pieces of three-dimensional wire frame information can be obtained by linearly interpolating the vertices of the corresponding triangles. Therefore, if the x coordinate of the j-th vertex of the updated input wireframe information is Xj,

(Equation 1) It is.

Here, xij is the x coordinate of the j-th vertex of the i-th three-dimensional wire frame, and rxi is the composite distribution value of each three-dimensional wire frame. A similar method is used for the y and x coordinates, and a new three-dimensional wire frame composed of the obtained three-dimensional coordinate information becomes input wire frame information. Also, the above method is applied to the grayscale information to obtain new input grayscale information.

Second Embodiment In the first embodiment, two databases, a basic three-dimensional wireframe database and another three-dimensional wireframe database, are provided as the three-dimensional wireframe database. The same effect can be expected even if the database is used instead. However, when there is a read command from the three-dimensional wire frame deformation unit, basic three-dimensional wire frame information is output from the database.

Third Embodiment In the first embodiment, the input image to the image input unit is an image captured by an image pickup system such as a single camera. If it is used, it is expected that a wider image of the subject can be obtained for the target object.

[0022]

Since the present invention is configured as described above, it has the following effects.

The grayscale information and three-dimensional wireframe information obtained from the subject image and a plurality of grayscale information and three-dimensional wireframe information registered in each information storage unit are
By switching the first and second switching means, it is possible to freely select a combination of various types of information for generating a composite image. Therefore, texture mapping is performed on M pieces of three-dimensional wire frame information in the wire frame information storage unit. When there are N pieces of density information to be processed, M × N combined images can be generated.

Also, a composite distribution for a plurality of input density information
Ri and the sum of the input wireframe information.
At least two wires by changing the distribution rw arbitrarily
-From the frame information and at least two corresponding shade information,
An infinite number of synthesized images can be generated.

Further, by using a three-dimensional digitizer using slit light, it is possible to extract three-dimensional structure information of a target object by a stable method.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a diagram showing an input image projected with slit light according to the present invention.

FIG. 3 is a diagram showing a basic three-dimensional wire frame according to the present invention.

FIG. 4 is a diagram showing a format of a three-dimensional wire frame database according to the present invention.

FIG. 5 is a conceptual diagram of a texture mapping process according to the present invention.

FIG. 6 is a conceptual diagram of a conventional image transmission system.

FIG. 7 is a codebook configuration example of a conventional image transmission method.

FIG. 8 is an example of a mouth image for each phoneme in the conventional image transmission method.

[Explanation of symbols]

 Reference Signs List 1 slit light irradiation unit 2 image input unit 3 noise filter 4 feature point extraction unit 5 three-dimensional coordinate calculation unit 6 basic three-dimensional wireframe database 7 three-dimensional wireframe deformation unit 8 three-dimensional wireframe database 9 grayscale information database 10 synthesized image Generation processing unit 101 Input image 102 Input image projected with slit light 103 Input image after noise removal 104 Feature point information 105 3D coordinate information 106 Basic 3D wireframe information 107 3D wireframe information of subject image 108 3D wire Frame information 109 Shading information of subject image 110 Shading information 111 Synthetic image output 112 Input wireframe information 113 Input shading information 114 Subject image sw1 First switch sw2 Second switch

Claims (2)

(57) [Claims] 1. An object image input from an image input unit.
A three-dimensional wire frame deforming unit that deforms a basic three-dimensional wire frame according to the three-dimensional configuration information and outputs wire frame information; a composite image generation processing unit that matches an image based on the wire frame information and gray level information; A wire frame information storage unit that stores the wire frame information, and first switching means for switching and outputting the wire frame information of the storage unit and the output of the three-dimensional wire frame deforming unit to the composite image generation processing unit;
A gray-scale information storage unit storing various gray-scale information, and second switching means for switching and outputting gray-scale information from the storage unit and gray-scale information obtained from the subject image to the composite image generation processing unit; The composite image generation processing unit includes:
At least two 3D wireframes and at least two shades
An image synthesizer characterized by arbitrarily determining each composite distribution value for a report . 2. A three-dimensional digitizer for extracting a feature point from a subject image obtained by projecting slit light on an object and calculating three-dimensional structure information of the subject based on the feature point. An image synthesizer according to claim 1.