KR20000043591A - Method for segmenting facial image in model-basis coding system - Google Patents

Abstract

PURPOSE: A method for segmenting a facial image is provided to reduce calculating complexity and to maintain a gentle facial contour, by segmenting facial components such as eyes, a nose, ears, a mouth to search the components in the facial image, when a texture facial image is interfaced with a three-dimensional face model in a three-dimensional model-basis coding system. CONSTITUTION: A method for segmenting facial components of a texture facial image, in a three-dimensional model-basis coding system, comprises the steps of: supplying a three-dimensional facial model composed of many polygonal wire frames; interfacing a facial contour of the three-dimensional facial model with the texture facial image; combining vertex regions, extended with each polygonal vertex interfaced on each facial component on the center, according to groups having similar pixel dispersion values, and generating the combined regions; absorbing external pixels having similarities with internal pixels of the combined regions, in the combined regions as pixel units, and segmenting each facial component in the texture facial image.

Description

Face Image Segmentation Method in Model-Based Coding System

The present invention relates to a face image mapping system. More specifically, the present invention relates to a face image mapping system. In detail, the present invention relates to a face image mapping system. It relates to a method of segmenting these areas to find a location.

In digital television systems such as video telephony, video conferencing, and high-definition television systems, because a video signal within a video frame signal consists of a series of digital data called pixel values, a large amount of digital is required to define each video frame signal. You need data. However, because the available frequency bandwidth of conventional transmission channels is limited, in order to transmit large amounts of digital data through them, low bit rate video signals such as video telephony and video conferencing systems employed to transmit human shapes In the case of an encoder, it is necessary to reduce or compress the amount of data using various data compression techniques.

On the other hand, in video telephony and video conferencing systems, video is mainly composed of head and shoulder portions, such as the human torso. The main concern of the viewer is the human face, and the viewer focuses on the moving parts, not the background scenes or other elements, especially around the mouth, including the lips, chin, head, etc., which are moved when the human speaks in the video scene. do. Thus, if only general information about the shape of the face is transmitted, the amount of digital data is significantly reduced.

Therefore, in the 3D model-based coding system, instead of transmitting all the constantly changing pixel values, a specific motion variable is extracted from the face image and transmitted to the receiving end. In order to reconstruct the face image at the receiver, the received motion variable is merged with the basic face image of the person previously transmitted to the receiver. The basic face image of the person is obtained by matching the texture face image of the person with a three-dimensional face model.

According to a conventional 3D model-based encoding system, a process of matching a texture face image of a person to a 3D face model firstly overlaps two face images. Then, the aspect ratio of the texture face image is calculated. Based on the calculated ratio, the three-dimensional face model is expanded or contracted to match the texture face image.

Then, the positions of the components, such as eyes, nose and mouth, are detected in the texture face image and the expanded or reduced three-dimensional face model, and each component in the texture face image is a corresponding component in the expanded or reduced three-dimensional face model. Matches an element. The texture data of the texture face image is projected onto the 3D face model.

In the matching process of the texture face image and the 3D face model, the positions of the components of the 3D face model are known in advance, but the positions of the components of the texture face image are not known except for some feature points. Therefore, in order to accurately match the texture face image with the 3D face model, it is necessary to find out which part of the texture face image is matched with the 3D face model, and all eyes, noses, ears, mouths, etc. constituting the face. It should be extracted in this texture face image. Next, it must be comprehensively analyzed to determine the face.

A typical decision method is to use a segmentation algorithm, which finds the most similar area by comparing any partial image of all components in the texture face image, namely, eyes, nose, ears, mouth, etc., with the three-dimensional face model. Next, a rough face component is extracted by determining the relative position of each component of the texture face image compared to the three-dimensional face model. Finally, the face image is segmented by segmenting the extracted face component area by component.

However, the above-described segmentation technique has a large amount of calculation that cannot be ignored depending on the accuracy of the extraction result of the component, and the image data of each component used for matching must also have many forms in order to ensure the accuracy of extraction. . In addition, natural shape contour extraction requires complex algorithms separate from the extraction of facial components.

The present invention has been made to solve the above-described problems, and an object thereof is to provide a method for effectively segmenting each component of a texture face image used in a 3D model-based encoding system.

In order to achieve the object as described above, the segmentation method of the present invention comprises: providing a three-dimensional face model consisting of a plurality of polygonal wireframe; Registering contours of the three-dimensional face model and the texture face image; Generating a merged area by merging vertex areas that are extended around vertices of respective polygons matched on each component in the texture face image by groups having similar pixel dispersion values; And segmenting each component in the texture face image by absorbing pixels outside the merged area, which are similar to pixels in the merged area, in the merged area on a pixel-by-pixel basis.

1 is a schematic block diagram of a face image matching device used in a model-based encoding system according to the present invention;

FIG. 2A illustrates a texture face image of a user, FIG. 2B illustrates a three-dimensional face model, and FIG. 2C illustrates a matched face image, respectively.

3A and 3B illustrate a process of generating an extended text area performed by the first segmentation block illustrated in FIG. 1;

FIG. 4 is a diagram illustrating a texture face image segmentation process performed by the second segmentation block illustrated in FIG. 2.

<Description of the symbols for the main parts of the drawings>

10: texture face image acquisition unit 20: area setting unit

30: 3D face model provider 40: face matching unit

50: first segmentation block 52: vertex region forming unit

54: variance value calculator 56: vertex region merger

60: second segment block 100: texture face image

200: three-dimensional face model 400: merged area

Hereinafter, the component segmentation method in the texture face image according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic block diagram of a model-based encoding system according to an exemplary embodiment of the present invention, wherein the texture face image acquisition unit 10, the region setting unit 20, the matching unit 30, and the first segmentation block ( 40 and a second segmentation block 50.

The texture face image acquisition unit 10 generates a texture face image of the sender using, for example, a charge coupled device (CCD) digital camera. The texture face image captured by the texture image acquisition unit 10 is displayed on the display (not shown) of the texture image acquisition unit 10 as shown in FIG. 2A, and the texture face image 100 is an example. For example it has a size of 512 × 512 pixels.

The region setting unit 20 sets an area for removing a surrounding background and extracting only a face part in order to perform a minimum image processing on the texture face image 100 generated by the texture image acquisition unit 10. . As described with reference to FIG. 2B, the facial region area 110 set according to the present invention includes a left vertical line 112, a right side face, and a right side separated into a left side face and a left ear of the texture face image 100. A right vertical segment 114 separated by the ear, an upper horizontal segment 116 across the front forehead above the eyebrows, and a rectangular region formed by the lower horizontal segment 118 across the lower jaw. In addition, the quadrangular face region 110 may be divided into a central segment 120 that bisects the upper and lower horizontal horizontal segments 116 and 118 in the symmetrical direction of the face. The face region area setting is performed by a user viewing and setting the texture face image 100 on the display. The texture face image 100 set by the region setting unit 20 to the face region is provided to the face matching unit 40.

Meanwhile, the 3D standard model providing unit 30 has a 3D head model, and the 3D face model 200 uses a silicon graphics interface (SGI) workstation as illustrated in FIG. 2B. It is stored in the form of a three-dimensional computer graphics called a wire frame (wire frame) in the form of a network connecting a plurality of polygons configured. The 3D face model 200 is provided to the face matching unit 40.

The face matching unit 40 matches the contours of the texture face image 100 provided from the texture image obtaining unit 10 and the 3D face model 200 provided from the 3D model providing unit 30. The face matching process performed by the face matching unit 40 superimposes the three-dimensional face model 200 on the texture face image 100 in the same two-dimensional pixel coordinate system, and textures the left and right aspect ratios of the three-dimensional face model 200. It expands or contracts to fit the left and right aspect ratios of the image 100. Next, the three-dimensional face such that the uppermost part of the left and right symmetry points of the extended three-dimensional face model 200 and the intersection point of the center vertical segment 120 and the image horizontal segment 116 of the texture face image 100 coincide with each other. The y coordinates of the vertices of the 3D face model 200 are moved in parallel, and the bottom horizontal segment 118 of the texture face image 100 coincides with the bottom part of the 3D face model 200. Readjust the value, and adjust the x coordinate value of each vertex of the 3D face model 200 so that the left and right vertical segments 112 and 114 of the texture face image 100 and the left and right portions of the 3D face model 200 coincide with each other. Readjust. The result of the face registration by the face matching unit 40 is illustrated as the face image 300 in FIG. 2C, where the exact matching between the texture face image 100 and the three-dimensional face model 200 is not achieved, but is approximately. Many parts overlap, and many of the vertices of the wire frames constituting the 3D face model 200 in the texture face image 100 coincide. Accordingly, the face matching unit 40 may grasp components of the texture face image 200 from the matched face image 300, that is, the rough positions of eyes, nose, mouth, and the like.

The first and second segmentation blocks 50 and 60 according to the present invention set the elements such as eyes, nose, and mouth of the matched face image 300 formed by the face matching unit 40 as areas distinguished from each other. The segmentation process is performed, and the segmentation of the present invention is performed based on vertices of polygons constituting the wireframe.

The segmentation process according to the present invention will be described with reference to FIGS. 3 and 4, and FIG. 3 illustrates a polygon of the three-dimensional face model 200 matched on a partial region of the matched face image 300.

As shown in FIG. 1, the first segmentation block 50 includes a vertex region forming unit 52, a dispersion value calculating unit 54, and a vertex region merging unit 56. The vertex area forming unit 52 forms an extended vertex area around each of the vertices 310, 320, 330, 340, 350, 360, and 370 matched on each component of the matched face image 300. do. In FIG. 3, only the expanded vertex regions 315, 325, 365 corresponding to the vertices 310, 320, 360 are shown for simplicity of the drawing. Information related to the extended vertex areas 315, 325, and 365 formed in the vertex area forming unit 52 is provided to the dispersion value calculator 54 and the vertex area merger 56.

The variance value calculator 54 calculates a variance value of the pixels included in the radius R of the vertex areas 315, 325, 365,... That extend around the corresponding vertex, and calculates the calculated variance value. Compare with the preset reference variance value. At this time, if the calculated variance value is less than or equal to the reference variance value, all pixels included in the extended vertex areas 315, 325, 365,... May be determined as elements defining one specific component. There will be. However, if the calculated variance is greater than the reference variance, then the extended vertex region 315, 325, 365,... Contains a pattern of components other than the specific component, or an edge of the specific component. It can be determined that there is. The information about the expanded vertex area compared in the variance value calculator 54 is provided to the vertex area merger 56.

According to the present invention, the reference variance value is set to a different value for each segmentation region of each component in the face. In addition, each extended vertex region 315, 325, 365 has a radius of 5 to 10 pixels around each corresponding vertex, the radius of which is greater than half the line length L connecting the adjacent vertices. It is desirable to. This is because the dispersion value calculator 54 is useful for evaluating the dispersion value of the pixels included in the extended vertex areas 315, 325, 365,...

The vertex area merging unit 56 merges the expanded vertex area according to the dispersion information of the expanded vertex area 54 provided from the dispersion value calculator 54. More specifically, the vertex region merging unit 56 examines similarities between adjacent extended vertex regions, for example, regions 315, 325, and 365, classifies groups of expanded vertex regions with similarities, and classifies the vertices. Merges zone groups into one zone. 3B illustrates a merge region 400 in which similar expanded vertex region groups are merged into one region, and the unmerged region represents a vertex region group having no similarity to the classified vertex region group, and includes edges. Or other facial component. Information about the area 400 merged by the vertex area merging unit 56 is provided to the second segmentation block 60.

The second segmentation block 60 merges the pixels 410 outside the merged area 400 having the similarity with pixels in the merged area 400 formed by the first segmentation block 50 in a pixel unit. Absorption in 400 forms a segmentation area that distinguishes each element in the texture face image 100. In this case, in the second segmentation block 60, the similarity determination between the pixels in the merged region 400 and the pixels outside the region is determined by the difference between the average luminance of the pixels in the merged region 400 and the pixel luminance outside the region. It is assumed to be equal to or more than the preset reference luminance value. In addition, the pixel absorption by the second segmentation block 60 is further merged region, eg, region 500, having a variance value different from the merged region 400 composed of vertex region groups as shown in FIG. 4. Until it is found or until the edge of the texture face image 100 is found. As a result, it is possible to segment each component in the texture face image 100.

Therefore, according to the present invention, since the components such as eyes, nose, mouth, etc. constituting the texture face image are segmented using the three-dimensional face model, computational complexity can be reduced, and smoother face contours can be reduced. I can keep it.

Claims (4)

A method of segmenting components of a texture face image in a 3D model-based encoding system, Providing a three-dimensional face model composed of a plurality of polygonal wireframes; Registering contours of the three-dimensional face model and the texture face image; Generating a merged area by merging vertex areas that are extended around vertices of respective polygons matched on each component in the texture face image by groups having similar pixel dispersion values; And segmenting each component in the texture face image by absorbing pixels outside the merged area similarly to the pixels in the merged area into the merged area in units of pixels. Face image segmentation method of the system. The method of claim 1, wherein the merging region generation step: Forming an extended vertex area around a vertex of each polygon matched on each component in the texture face image area; Calculating a variance value of the pixels in each of the extended vertex regions; Comparing the variance value with a preset reference variance value of each component; Grouping extended vertex areas for each component that satisfy the preset reference dispersion value; And merging the expanded vertex region group to form the merged region for each of the components. The method of claim 1, wherein the segmentation step is: Setting each point on the boundary of the merged area as an extension start point; Selecting a pixel adjacent the extension start point having a value similar to a pixel value on the extension start point; And segmenting each of the components by absorbing the selected adjacent pixels into the merged region until a pixel adjacent to the expansion start point that is not similar to the pixel value on the expansion start point is found. Face Image Segmentation Method of Coding System. The method of claim 1, further comprising: forming an area defining a face portion of the texture face image to match the three-dimensional face model within the face region. Facial image segmentation method.