KR100292238B1 - Method for matching of components in facial texture and model images - Google Patents

Abstract

PURPOSE: A matching method for texture mapping images and durability components of standard shape models is provided to reduce complicated preprocessing by carrying out standard shape models with a face outline by using scale factors of texture face images. CONSTITUTION: A matching method for texture mapping images and durability components of standard shape models includes the steps of obtaining texture face images of a transmitter by a texture image acquiring part(10), providing 3D standard shape models of a wire frame by a standard shape model providing part(20), matching the texture images with outlines of standard shape models by an outline matching part(30), extracting positions of each face component of the face of the texture images in a component position providing part(40) according to characteristic values obtained by a characteristic extracting algorithm based on the knowledge by a component matching part(50), matching the outline matched texture images with components of standard shape models entirely according to the position values, and carrying out texture mapping by projecting texture data of the texture images to the entirely matched standard shape models by a texture mapping part(60).

Description

METHODS FOR MATCHING OF COMPONENTS IN FACIAL TEXTURE AND MODEL IMAGES}

The present invention relates to texture mapping of a 3D shape model-based coding system using a human shape model, and more particularly, to a method of matching face components of a texture and a standard face image for texture mapping.

Application techniques using three-dimensional object models, especially human shape models, have been applied in the fields of video coding, virtual reality, and computer graphics. In the field of video coding that is currently in progress, research to transmit a satisfactory image quality with a smaller amount of data continues. Looking at the research projects according to the amount of data to be transmitted, there is MPEG4, which starts from HDTV and is currently being actively studied. In MPEG4, a video signal is transmitted using a very small amount of data. Therefore, not only a transformation using information theory, for example, a cosine transform coefficient method, but also other computer vision, The techniques used in computer graphics are also applied.

In general, the shape of a person seen in a videophone or video conference only shows the chest above the waist. In particular, the speaker's face takes up most of the video information transmission. If only non-human face information, ie background information, can be ignored and only regular, characteristic information about the face shape can be sent, the transmission rate can be greatly reduced. In other words, it finds the rules of the general face, face, expression, and emotional expressions of the face and mouth when speaking, and sends them to the other party through one informational code. By deforming the face shape according to the received code, the expression shape of the sender may be recognized with a small amount of data.

More specifically, the three-dimensional model-based coding system targets coding in which two people talk in real time, such as a video phone, and transmit not only sound but also a changing face or body in real time. However, unlike general video coding, the transmission data does not handle an image composed of pixels that change every moment. Instead, it extracts and transmits a certain motion variable, and then the data possessed at the receiving side, that is, the image of the sender's face. It is combined with a dimensional head shape model and a body shape model to form a face or body image of a final changer after reconstruction.

Of course, such a system configuration can be used when the amount of data to be transmitted is very limited by the data transmission bandwidth but not a large amount, but requires sufficient image quality for accurate recognition between the two parties. If the bandwidth is large enough to send enough data, the advantages of this system will be lower. This is because regular code lacks a natural look.

Such a 3D model-based coding system matches a standard shape model consisting of a three-dimensional wire frame to a sender's own face image, and performs texture mapping to project texture data of the sender's face image onto a standard shape model. The texture mapping must obtain the coordinate values of all components such as eyes, nose, mouth, etc. in the texture image, and superimpose the components of the corresponding standard shape model with these coordinate values.

However, since we already know when constructing the coordinate value wireframe of the standard shape model, but we do not know a few feature points of the coordinate value of the texture image, the exact location of the sender's own components in the given texture image for text mapping Should know. However, there is a problem that the preprocessing step is complicated and time-consuming to extract coordinate values of the sender's own components from the texture image.

Therefore, an object of the present invention is to provide a method for matching with a standard shape model without extracting the coordinate values of all contours and components in a texture image of a sender for texture mapping in a 3D model-based coding system.

According to the present invention for achieving the above object, in the three-dimensional model-based system using a human shape model, a texture image for texture mapping and a component matching method in a standard shape model include: acquiring a texture face image of a sender ; Providing a standard shape model of the wire frame; Matching face contours of the texture image with a standard shape model; Extracting a position of each component in the face of the texture image according to a feature value obtained by using a knowledge-based feature extraction algorithm; Performing matching of the components between the face contour-matched texture image and standard shape models as a whole using the position values of the face components of the texture image; Projecting texture data of the texture image onto the globally matched standard shape model to perform texture mapping;

The facial limbal matching step is:

Obtaining a difference between a center coordinate of the texture image and a center coordinate of the standard shape model;

Moving the center coordinates of the standard shape model to the center coordinates of the texture image by the difference to match the center of the standard shape model to the center of the texture image;

And matching the contour of the center-matched standard shape model to the contour of the texture image using the scale factor of the texture image.

1 is a schematic block diagram of a face component matching device for texture mapping in a 3D model based coding system constructed according to a preferred embodiment of the present invention;

2A and 2B are diagrams illustrating texture face images and standard shape models, respectively, provided for texture mapping;

3A, 3B and 3C are diagrams exemplarily illustrating a process of contour matching performed in the contour matching unit shown in FIG. 1;

4 is a diagram illustrating a knowledge representation for extracting features from an edge face image;

5 is a diagram illustrating five features of a extracted texture face image based on knowledge;

6 is a diagram illustrating a result of registration of facial components performed in the component matching unit shown in FIG. 1;

FIG. 7 is a diagram illustrating a texture mapping result performed by the texture mapping unit shown in FIG. 1. FIG.

<Explanation of symbols for main parts of drawing>

10: texture image acquisition unit 20: standard shape model providing unit

30: contour matching unit 40: component position providing unit

50: component matching part

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Referring to FIG. 1, a schematic block diagram of a face image texture mapping apparatus used in a 3D shape model based coding system according to the present invention is shown. The facial image texture mapping apparatus of the present invention includes a texture image acquisition unit 10, a standard shape model providing unit 20, an outline matching unit 30, a component position providing unit 40, a component matching unit 50, and The texture mapping unit 60 is included.

The texture image acquisition unit 10 acquires a two-dimensional texture face image 100 having a size of 512 x 512 pixels as shown in FIG. 2A using a CCD digital camera (not shown). The texture image acquiring unit 10 calculates coordinate values of several feature points, for example, forehead, right and left ears, and nose, from the texture face image 100, respectively (P ₁ ), (P ₂ ), (P ₄ ) and It is provided to the contour matching unit 30 together with the texture value of the texture image 100 extracted and extracted as P _c .

The standard shape model providing unit 20 has a three-dimensional standard shape model 200 as illustrated in FIG. 2B. The standard shape model 200 is a wire frame having a form of a network connecting a plurality of polygons constructed using a tool called 3D Wavefront available from Silicon Graphics Interface (SGI). It is stored in a 3D computer graphics form called a frame. The vertices of the standard shape model 200 have three-dimensional coordinate values (x, y, z), and the coordinate values of each vertex are already known when constructing the standard shape model of the entire face. The coordinate values of the standard shape model 200 are provided by setting the coordinate values on all z-axis to ″ 0 ″ for matching with the two-dimensional texture face image 100 in the contour matching unit 30 of the next stage. .

The contour matching unit 30 registers the contours of the texture face image 100 provided by the texture image acquisition unit 10 and the standard shape model 200 received from the standard shape model providing unit 20. The contour matching process performed by the contour matching unit 30 will be described in detail with reference to FIG. 3.

The contour matching unit 30 first superimposes the standard shape model 200 on the texture image 100 in the same pixel coordinate system as illustrated in FIG. 3A. When the texture image and the standard shape model 200 overlap, the centers of the two images 100 and 200 do not coincide. Therefore, the contour matching unit 30 uses a ratio of ± x and y directions from the center point of each of the texture image and the standard shape model to the face contour to match the centers of the two images 100 and 200. In more detail, x and the center of the texture image 100, for example, the coordinates of the nose P _c (see FIG. 2A) and the corresponding center coordinates Q _c (see FIG. 2B) of the standard shape model 200. The distance differences Δx and Δy in the y direction are calculated by using Equations 1 and 2 below.

In Equation 1 described above, the distance differences Δx and Δy are summarized as in Equation 2 below.

Next, the contour matching unit 30 applies the distance differences Δx and Δy calculated from the above Equations 1 and 2 to the standard shape model 200, and as shown in FIG. 3B, the standard shape model 200. ) then the center point (Q _c) moved by the center of the texture image (100) (P _c) against the distance difference (Δx, Δy) of the.

In the state where the center points of the texture image 100 and the standard shape model 200 coincide with each other, the contour matching unit 30 applies the scale factor of the texture image 100 to the standard shape model 200 to apply the standard shape model 200. The contour of the shape model 200 is matched to the contour of the texture image 100. The scale factor according to the present invention is an aspect ratio of the aspect ratio between the texture image 100 and the standard shape model 200. The scale factor is used to adjust the outline of the standard shape model 200 to the outline of the texture image 100 using the scale factor. Contour matching is performed while expanding in the form of a circle. 3C illustrates a process in which the outline of the standard shape model 200 is expanded for the above-described contour matching. The image matched by the contour matching unit 30 is provided to the component matching unit 50 of the next stage.

The component matching unit 50 performs matching between the texture image 100, which is contour-matched by the contour matching unit 30, and components such as eyebrows, eyes, nose, and mouth in the standard shape model 200. In this case, as described above, the coordinate values of the vertices of the standard shape model 200 are already known, but since the texture image 100 knows only the coordinate values of the forehead, the right and left ears, and the nose, Matching is performed using the position values of the components, such as eyebrows, eyes, nose, and mouth, which are provided by the component position providing unit 40.

The component position providing unit 40 according to the present invention acquires about 20 texture face images having a size of 256 x 256 pixels by using a CCD camera (not shown), and rapidly increases the brightness of the image. Edge portions are extracted from the face image using an edge detection method using a brightness distribution of a changing original image. The extracted edge image is illustrated in FIG. 4, and the positions of the five feature values are extracted by applying a ″ knowledge based feature extraction algorithm ″ to the edge image. ″ Knowledge based feature extraction algorithm ″ can be defined as follows.

Knowledge 1: As a result of projecting the edge image shown in FIG. 4 in the horizontal direction, the portion of the projection value having the largest value is generally the portion of the eye or the eyebrow.

Knowledge 2:

Knowledge 3:

Knowledge 4:

Knowledge 5:

Knowledge 6:

Knowledge 1 above means that the y-axis value of the eye can be determined by the largest horizontal projection value, which is verified by knowledge 2, knowledge 3 and knowledge 4. Knowledge 2 means that the distance from the eyes to the nose is greater than the distance from the nose to the mouth. Knowledge 3 means that the face is symmetrical around the nose. Knowledge 4 represents the left eye, nose and right eye sequence. Knowledge 5 represents the order of the eyebrows, nose, mouth and chin. Knowledge 6 means that the ratio of face to face is approximately 1: 1.5.

As shown in FIG. 5, the feature values based on the above-described knowledge include the diagonal distances a and b between the left and right eyes and the nose, the distance between the nose and the incident c, and the distance between the eyes and the nose in the face image. A straight line distance d and a straight line distance e from the mouth edge in the x-axis direction. The five feature values acquired by the component position providing unit 40 are provided to the component matching unit 50 as position values.

Accordingly, the component matching unit 50 may be configured such as an eyebrow, an eye, a nose, and a mouth in the image shown in FIG. 3C of the standard shape model 200 and the texture matched image 100 at the contour matching unit 30. Match the components. In this case, as described above, since the coordinate values of the components in the texture image 100 are not known, the components such as eyebrows, eyes, nose, and mouth provided by the component position providing unit 50 described below may be Match using position value. Matching of the components executed in the component matching unit 50 is performed in consideration of the scale factors executed in the above-described contour matching unit 30. 6 illustrates a component matching result performed by the component matching unit 50.

The texture mapping unit 60 performs texture mapping by projecting texture values of polygons of the texture image 100 corresponding to each vertex coordinate of the polygons of the standard shape model 200 as in the matched image illustrated in FIG. 6. do. 7 illustrates a result of texture mapping by the texture mapping unit 60.

As described above, the complex preprocessing process is shortened by performing contour matching with the standard shape model using the scale factor of the texture face image for face contour matching for texture mapping in a 3D model-based coding system using a human shape model. By using a knowledge-based feature extraction algorithm, the positions of components such as eyes, nose, and lips of the texture image may be extracted, and texture mapping may be accurately performed on corresponding component positions of the standard shape model.

Claims (4)

In the texture mapping method of a 3D model based system using a human shape model, Acquiring a texture face image of the sender; Providing a three-dimensional standard shape model of a wire frame; Matching face contours of the texture image with a standard shape model; Extracting a position of each component in the face of the texture image according to a feature value obtained by using a knowledge-based feature extraction algorithm; Performing matching of the components between the face contour-matched texture image and standard shape models as a whole using the position values of the face components of the texture image; Projecting texture data of the texture image onto the globally matched standard shape model to perform texture mapping; The facial limbal matching step is: Obtaining a difference between a center coordinate of the texture image and a center coordinate of the standard shape model; Moving the center coordinates of the standard shape model to the center coordinates of the texture image by the difference to match the center of the standard shape model to the center of the texture image; And matching the contour of the center-matched standard shape model to the contour of the texture image using a scale factor between the texture image and the standard shape model. The method of claim 1, wherein obtaining a difference between the center coordinates of the texture image and the center coordinates of the standard shape model is as follows: The aspect ratio of the three-dimensional shape model based system, characterized in that calculated using the aspect ratio of ± x, y direction from the center point of each of the texture image and the standard shape model to the face contour of each of the texture image and the standard shape model Texture mapping method. The texture mapping method of claim 2, wherein the scale factor is a face aspect ratio of the standard shape model and the texture image. The method of claim 1, wherein the extracting of the position of each component in the face of the texture image comprises: Obtaining an edge image with respect to the texture image; By applying the knowledge-based feature extraction algorithm, the diagonal distances (a and b) between the left and right eyes and the nose, the distance between the nose and the incidence (c), and the linear distance between the eyes and the nose (d) in the texture face image And extracting feature values representing a straight line distance (e) from the edge of the mouth to the x-axis direction, and the like.