KR20020085669A - The Apparatus and Method for Abstracting Peculiarity of Two-Dimensional Image & The Apparatus and Method for Creating Three-Dimensional Image Using Them - Google Patents

Abstract

PURPOSE: A device and a method for extracting 2D image characteristics for producing 3D images and a device and a method for producing 3D images thereby are provided to achieve the generation of 3D image by a single image from a 2D image from which the characteristics are able to be extracted, thereby obtaining the 3D images in a short time. CONSTITUTION: A device for extracting 2D image characteristics for producing 3D images includes an area detecting element(101) for detecting required areas by receiving 2D images, noise removing elements(103,104) for removing noises from the detected 2D images, a block binary element(105) for carrying out the binary of the 2D images from which the noises are removed, a line segment element(106) for carrying out the line segment for the binary 2D images, a labeling and size verifying element(107) for assigning labels to the images and carrying out the size verification of the respective labels, and a characteristic extracting element(108) for extracting characteristics of the images by geometrical position information.

Description

Apparatus and Method for Abstracting Peculiarity of Two-Dimensional Image & The Apparatus and Method for Creating Three -Dimensional Image Using Them}

The present invention relates to an apparatus and method for extracting a feature of a 2D image for generating a 3D image, and an apparatus and method for generating a 3D image using the same, and more particularly, to an image processing technique. A method of extracting feature points of a 2D image (eg, a front face image of a human face) for generating a 3D image, an apparatus and method for automatically generating a 3D image using the same, and a program for realizing the methods The present invention relates to a computer-readable recording medium having recorded thereon.

Hereinafter, in the related art and embodiments for explaining the present invention, only a human face image has been described as an example, but all two-dimensional images which the present invention intends to transform into a three-dimensional image as well as a human face image are generated. It will be obvious that it can be applied to.

Although the definition of virtual reality varies from scholar to scholar, it can be said that human beings feel real in computer-generated cyberspace. In order to realize such a virtual reality, various environments must be prepared, one of which is a three-dimensional representation of an object.

In other words, the real world must be simulated in order to have a similar effect to reality. For this purpose, a process of three-dimensionally modeling objects and people using three-dimensional graphics technology is necessary.

Currently, three-dimensional face modeling can be divided into three-dimensional scanner method, manual method using three-dimensional authoring software, modeling method using statistical data of face, and method using two-dimensional image feature. Can be.

First, a method of using a 3D scanner is a method of directly extracting a model from an object, but it requires a skilled technique for manipulating the extracted data with high cost, although it accurately represents the shape of the face. .

Next, the manual method using the 3D authoring software is as if the designer draws the drawing manually using 3D authoring software such as "3D MAX" and "Maya". As a way of creating shapes, this also requires a lot of skilled designers, and the modeling takes a lot of time.

In addition, the method of modeling by using statistical data of a face requires that a database for calculating statistics be established in advance, and that an expert for that model is required separately in order to model like an individual using the database. There was this.

Finally, in the method of using the features of the 2D image, there has been a problem that many of the individual features cannot be saved because only a simplified method for applying facial expression animation has been attempted.

The present invention proposed to solve the above problems, an apparatus for extracting a feature point of a two-dimensional image (for example, a front image of a human face) for generating a three-dimensional image using an image processing technique and It is an object of the present invention to provide an apparatus for automatically generating a three-dimensional image using the method, the method, and a computer-readable recording medium recording a program for implementing the methods.

1A is a block diagram of a feature extraction apparatus of a 2D image for generating a 3D image according to the present invention.

1B is a flowchart illustrating a feature extraction method of a 2D image for generating a 3D image according to the present invention.

1C is a diagram illustrating an embodiment of a feature extraction method of a 2D image for generating a 3D image according to the present invention.

Figure 2a is a block diagram of an embodiment of a three-dimensional image generating apparatus according to the present invention.

2B is a detailed flowchart of an embodiment of a method of generating a 3D image according to the present invention;

* Explanation of symbols for the main parts of the drawings

101: area detection unit 102: black and white

103: Gaussian smoothing filter 104: Contrast adjustment unit

105: block binarization unit 106: line segment portion

107: labeling and size verification unit 108: feature point extraction unit

An apparatus of the present invention for achieving the above object, the feature extraction apparatus of a two-dimensional image, region detection means for detecting a necessary area by receiving the two-dimensional image; Noise removing means for removing noise with respect to the two-dimensional image in which the region is detected; Binarization means for binarizing the noise-free two-dimensional image; Line segment means for performing a line segment on the binarized image; Labeling and size verification means for assigning a label to the image on which the line segment has been performed and for verifying the size of each label; And feature point extracting means for extracting feature points using geometric position information on the image of which the verification of the label size is completed.

In addition, the method of the present invention, the feature extraction method of the two-dimensional image applied to the image generation system, comprising: a first step of receiving a two-dimensional image to detect the required area; A second step of removing noise with respect to the 2D image from which the area is detected; A third step of binarizing the noise-free two-dimensional image; A fourth step of performing a line segment on the binarized image; A fifth step of attaching a label to the image on which the line segment is performed and verifying the size of each label; And extracting a feature point using geometric position information on the image of which the verification of the label size is completed.

The present invention also provides a video generation system having a processor, comprising: a first function of receiving a two-dimensional image and detecting a required area; A second function of removing noise with respect to the 2D image from which the area is detected; A third function of binarizing the noise-free two-dimensional image; A fourth function of performing a line segment on the binarized image; A fifth function of giving a label to the image on which the line segment is performed and verifying the size of each label; And a computer readable recording medium having recorded thereon a program for realizing a sixth function of extracting a feature point using geometric position information, on the image of which the verification of the label size is completed.

On the other hand, the apparatus of the present invention for achieving the above another object, in the three-dimensional image generating apparatus, by selecting a basic model, and set a parameter straight line to each important feature of the selected basic model, Basic model feature group setting means for setting a feature group connected with a parameter straight line; Feature extraction means for receiving a two-dimensional image and extracting the feature; Model deforming means for deforming a basic model by moving the set parameter straight line according to the extracted feature points; And texture mapping means for generating a 3D model by performing texture mapping on the modified basic model.

In addition, the method of the present invention, in the three-dimensional image generation method applied to the image generation system, by selecting a base model, to set a parameter straight line to each important feature of the selected base model, the set parameter Setting a feature group connected with a straight line; A second step of receiving a 2D image and extracting feature points thereof; A third step of deforming a basic model by moving the set parameter straight line according to the extracted feature points; And a fourth step of generating a 3D model by performing texture mapping on the modified basic model.

In addition, the present invention, in the image generation system having a processor, by selecting a base model, to set a parameter straight line to each of the important feature areas of the selected base model, and the feature group connected to the set parameter straight line A first function to set; A second function of receiving a 2D image and extracting feature points thereof; A third function of deforming a basic model by moving the set parameter straight line according to the extracted feature points; And a computer-readable recording medium having recorded thereon a program for realizing a fourth function of generating a three-dimensional model by performing texture mapping on the modified basic model.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

As described above, in the following description of the present invention, only a face image of a person has been described as an example. However, the present invention is not only a face image of a person, but also a two-dimensional image which is intended to be transformed into a three-dimensional image. It will be obvious that it can be applied to images.

FIG. 1A is a configuration diagram of an apparatus for extracting a feature of a 2D image for generating a 3D image according to the present invention. FIG. 1B is a flowchart according to the present invention, and FIG. 1C is an image generated as a result of each process. An illustration of the.

The features to be extracted will vary depending on what the 2D image is to be transformed into a 3D image, but in one embodiment of the present invention, a human face image is taken as an example. Features that should be limited to the outline of the face, the size and position of the eyes, the size and position of the nose, the size and position of the mouth.

In addition to the above features, various factors such as the protrusion of the cheekbones and the shape of the eyebrows will act as factors in determining the impression of the person, but the biggest characteristic is limited to the above four elements. This is because the image processing technique has difficulty in accurate detection, and the eyebrows can be similarly processed by simple texture mapping because of the small protrusion size.

However, it is possible to add and remove not only the above four elements but also other characteristic elements.

Once the feature point is determined, as shown in FIGS. 1A to 1C, in order to extract the feature of the 2D image for generating the 3D image according to the present invention, the face area must be detected through the area detector 101. De (110), the present invention used a blue screen for this purpose.

Of course, even if the background is complex or simple, it is possible to detect the feature points using a variety of conventional methods, but even in recent studies, the recognition rate is often lowered and thus a limited condition is used. Rather than focusing on detection, we focused on three-dimensional modeling similar to the input image using a single frontal image, so we chose a face region detection method using a blue screen.

To use a blue screen, you need to do color image processing.

Color Space, which is widely used in the related art, is an RGB coordinate system, but since the RGB coordinate system is a mixture of color and brightness components, a change in brightness is a major cause of misperception when performing image processing. . Therefore, it is more effective to separate and use the brightness component, YIQ, YUV, HSI is mainly used as the coordinate system in which the brightness component and the color component are separated, in one embodiment of the present invention used the HSI coordinate system.

Next, the monochromeization unit 102 performs monochromeization on the 2D image 111 in which the face region is detected (120).

This is because, although the input image used in the present invention receives a color image for 3D texture mapping in the future, the feature point detection method may be a black and white image.

In operation 130, a Gaussian smoothing process is performed on the 2D image 121 in which the black and whitening process is completed using a Gaussian smoothing filter 103.

The Gaussian smoothing filter 103 is a very efficient filter for removing scattered noise. The Gaussian smoothing filter 103 multiplies each pixel with a different weight, and then divides the total of the weight values. To smooth out the splashes, typically in black and white.

At this time, in the present invention, by using a filter of the size 5 * 5, to solve both problems of the computation time and the efficiency of noise cancellation at the same time.

In the image 131 from which the primary noise is removed through the Gaussian smoothing process, the secondary noise is removed through the contrast adjusting process by the contrast adjusting unit 104 (140).

Contrast is the ratio between the light and dark areas of the screen. Contrast is a high-contrast image that is easy to see because the outline of the subject is clear. Low-contrast images do not become easy to see because the outline is not clear. . That is, an image with low contrast can be converted into an image that is easier to process by emphasizing contrast by converting the bright portion brighter and the dark portion darker.

In other words, the characteristic and non-characteristic parts of the face have a clear contrast in the black and white image, so that the emphasis can be further emphasized to facilitate recognition.

General contrast is expressed as Equation 1 below.

g (x, y) = nf (x, y)

The density f (x, y) of the pixel is n times the position (x, y) of the original image, and the density g (x, y) is set at (x, y) of the processed image.

Since image data is treated as one pixel and 8 bits in size, the range of density that can be expressed is from 0 to 255. Thus, if the result of a calculation exceeds 255, it is limited to 255 regardless of the result of the calculation.

In the present invention, by slightly modifying it, the following equation (2) is used.

(x, y) = f (x, y) + (f (x, y) -128) * Cv / 128

This is a method for emphasizing the boundary value in binarization. In the basic processing, the low contrast image is darker than the contrast and the bright image is brighter.

That is, since the maximum contrast value of each pixel is 255, the median value 128 is subtracted from this and added to the original value, so that the original value is 128 and the weight value is positive (+). If the value is less than 128, the weight is made negative to play this role.

In addition, although a fluid treatment was considered by placing a parameter called Cv here, in the present invention, this value was experimentally set to 90.

As described above, in order to reduce the noise of the black-and-white image 121, two processes are performed in the present invention, the first of which is the Gaussian Smoothing process 130 and the second is the contrast adjustment process 140. to be.

In the image of the human face as in the present embodiment, there is a big difference in terms of the surrounding skin and contrast, and since the area occupies a fairly large area in the face part, when the smoothing process is performed as a whole, small noises disappear. Performing a contrast process on the image emphasizes each feature point with smooth boundaries.

Next, the block binarization unit 105 performs a block binarization process on the 2D image 141 on which the contrast process is performed (150).

Binary is a technique that distinguishes black and white based on a threshold value from a black and white image having 256 levels of brightness. Generally, if the image is evenly distributed and the image quality is good, there is no problem. If the image has a lot of damage and an irregular distribution, the optimal binarization image for the image cannot be obtained with one threshold.

That is, the basic binarization technique uses one threshold for the whole image, so if the contrast value of the shaded part (for example, the lower jaw) is different from the contrast value of the entire image, the desired result image can be obtained. There will be no.

Therefore, a threshold value for each part needs to be set for an image having strong partial features. For this, an embodiment of the present invention uses a block binarization technique among various conventional improved algorithms.

The block binarization technique is a method of setting a block having a desired size and setting a threshold value in consideration of a contrast value of each block, so that each block may have a specific threshold value. If you do, you can effectively handle the shadows that appear on your chin or under your nose. In other words, the shadow part is processed by its own block, so the overall contrast is less affected.

However, since the feature point extraction process of the human face image, such as the embodiment of the present invention, recognizes and processes a mass of an object, only the block binarized image may have an error. This is because, due to the nature of image processing, even if only one pixel is separated, the processor recognizes different parts.

To this end, a line segment process is performed on the block binarized image 151 through a line segment unit 106 (160).

The line segment process is a process of connecting the two lines when the distance between the two lines is less than a predetermined segment value for the lines A and B horizontally separated from each other. In the present invention, the segment value was set to 5 in an experiment. That is, straight lines having a distance difference of less than 5 pixels are connected to each other.

This is because of the experimental result that the maximum distance of the main feature points such as eyes, nose and mouth, which are easy to be separated because the input image is about 640 * 480 and limited the shooting distance, does not exceed 5 pixels. According to various conditions such as the input image and the shooting distance, the segment value may be appropriately adjusted.

With respect to the image 161 on which the line segment process is performed, the labeling and verifying size unit 107 labels and orders areas separated from each other up, down, left, and right, and then verifies the size of each label. Verify Size) (170).

In order to perform the above process, it is necessary to determine the maximum, minimum, maximum and minimum values of height for each label. In the embodiment of the present invention, the following conditions were used by experiment.

Max Width of Label: 20 pixels

Min Height of Label: 5 pixels

Max Height of Label: 100 pixels

Minimum width of label (length multiplied by height): 50

Maximum width of label (multiplied by length and height): 800

The above figures were obtained by quantifying the average of 100 test subjects when the front still image of 640 * 480 was recognized with error within 10cm from the distance of 80cm from the camera. The obtained image is 171 of FIG. 1C.

After completing the Verify Size process 170 for the image, the feature point extractor 108 detects an eye, a nose, and a mouth using a geometric position feature (180).

In general, in the detection of eyes, nose and mouth, the eyes are first searched, and then a number of methods are used to find other parts by using geometric position features, which is the most detectable part in a special environment of a human face. This is because the eyes are symmetrical from side to side, and after finding them, you can use the positional information that the nose and mouth are below them.

Taking advantage of the fact that the eyes are always paired on the face, first select one label, then check whether there are pairs of labels whose size does not vary greatly in the horizontal direction, and adopt them as candidates for the eyes.

Since the eyebrow and the eye are not distinguished when using this method, in one embodiment of the present invention, the best of these paired parts is the eyebrow, the lower pair is the eye, and only one pair exists It was detected by eye.

The above method is possible because the part which has been paired from above among the images already removed after the verify size processing has the highest probability of being an eyebrow and an eye.

Based on the detected eyes, the nose located at the center of both eyes and located at the bottom of the eyes was recognized as a nose, and the area appearing below was detected by the mouth. This is because, with the current treatment, there are rarely any other characteristic parts between the eyes, nose and mouth.

The images obtained through the above process are 181 and 182 of FIG. 1C.

Next, finally, the outside of the face is detected (190), the information of the outside of the face used in the embodiment of the present invention is only information on the part where the outside of the face, such as a jaw line that largely determines the impression of the person, is rapidly changed. Since it is sufficient, the point of contact with the end point of the face area is drawn horizontally at the position of the detected eye area 191, and then the total length of the left and right faces of the person can be known. Draw a straight line 191 2 horizontally at the position portion of the point to meet the end point of the face area (b) of 191, and draw a straight line 191 3 vertically at the center of the lip, and then Using the point 191 c to obtain the features of the outside of the face.

In the 191 image of FIG. 1C, it is not difficult to find the points (a) and (b). Because the above process is performed in the state that only the face region is extracted in the preprocessing process using the blue screen, it is easy to find the intersection of the horizontal coordinate and the end point of the face region if only the position of the eyes and the mouth is detected. Because.

However, in the process of finding the jaw line, the separation between the jaw and the neck is often difficult, so the embodiment of the present invention went through the following process.

First, a rectangular candidate area is determined based on the middle point of the mouth area by 5 pixels left and right, that is, 10 pixels horizontally, and a point where the face area ends vertically with the lower part of the mouth area. .

Next, the sum of the horizontal axes from the top to the bottom for the candidate area is estimated when the following two conditions are satisfied.

1) When there is a horizontal axis where the difference in sum of the connected horizontal axes is more than twice

2) When the area of this horizontal axis continues more than 5 pixels

The top of the head is pre-processed after filling the blue screen with black after removing the blue screen. Therefore, the part where the forehead and the hair meet is designated as the tip of the head.

To do this, in the same way, add up the horizontal axis of 10 pixels horizontally from the midpoint above the eye area, so that when an area where this value is more than three times larger than the previous value continues for more than 5 pixels, The first horizontal axis was assumed to be the top of the head.

The image 192 of FIG. 1C is represented by a rectangle composed of two vertical lines representing points (a) and (b) and one horizontal axis representing point (c) using the feature point detection method used in the embodiment of the present invention. Face area.

FIG. 2A is a block diagram of an embodiment of an apparatus for generating a 3D image according to the present invention, and FIG. 2B is a detailed flowchart thereof.

As shown in Figs. 2a and 2b, in the present invention, a method of changing the shape of the basic face by the feature point parameter using a deformation algorithm of the model based on the parameters used in the facial expression animation system proposed by "Parke". 3D modeling was performed.

That is, the basic model feature group setting unit 201 first selects a basic model (211), sets a parameter straight line to each of the important feature areas of the selected basic model (212), and the feature group connected to the parameter straight line By setting (213) to determine the deformation region independent of the model (210).

When the two-dimensional image is input through the input unit 202 (221), and the feature point is extracted by the feature point extractor 203 according to the method described with reference to FIGS. 1A to 1C (222), the model deformer 204 The original feature parameter line is moved using the extracted feature points (230). At this time, the feature parameter is first transformed (231) and the group is transformed again (232). The feature groups are transformed into features with similar characteristics to the input person.

Next, the texture mapping unit 205 performs texture mapping to increase the realism (240), and directly deforms the mesh by the feature points of the input two-dimensional image to surround the eyes, nose, mouth, and face of the input image. After generating a similar three-dimensional model is reflected, the output through the output unit 206.

As described above, the method of the present invention may be implemented as a program and stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.) in a computer-readable form.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the spirit of the present invention for those skilled in the art to which the present invention pertains, and the above-described embodiments and accompanying It is not limited to the drawing.

The present invention as described above, in generating a three-dimensional image from the two-dimensional image, if the two-dimensional image that can extract the necessary feature information, by making it possible to extract and generate the three-dimensional image with only one image In addition, it has the effect of generating three-dimensional images at a lower cost and a faster speed.

Claims (16)

In the feature extraction apparatus of a two-dimensional image, Area detection means for receiving a two-dimensional image and detecting a required area; Noise removing means for removing noise with respect to the two-dimensional image in which the region is detected; Binarization means for binarizing the noise-free two-dimensional image; Line segment means for performing a line segment on the binarized image; Labeling and size verification means for assigning a label to the image on which the line segment has been performed and for verifying the size of each label; And Feature point extracting means for extracting feature points using geometric position information for the image of which label size verification is completed Feature extraction apparatus for a two-dimensional image comprising a. The method of claim 1, Black and whiteization means for performing black and whiteization on the received 2D image when the received 2D image is a color image Feature extraction apparatus of a two-dimensional image further comprising. In the feature extraction method of a two-dimensional image applied to an image generating system, A first step of receiving a 2D image and detecting a required region; A second step of removing noise with respect to the 2D image from which the area is detected; A third step of binarizing the noise-free two-dimensional image; A fourth step of performing a line segment on the binarized image; A fifth step of attaching a label to the image on which the line segment is performed and verifying the size of each label; And A sixth step of extracting feature points using geometric position information of the label size verified image; Feature extraction method of a two-dimensional image comprising a. The method of claim 3, wherein A seventh step of performing black and white on the received two-dimensional image when the received two-dimensional image is a color image; Feature extraction method of a two-dimensional image further comprising. The method according to claim 3 or 4, The first step is, A method of extracting a feature of a two-dimensional image, characterized by using a blue screen to detect a required area. The method according to claim 3 or 4, The second step, A method for extracting features of a 2D image using Gaussian smoothing and contrast control to remove noise. The method according to claim 3 or 4, The third step, In order to binarize a 2D image, a block binarization technique is used. In an image generation system having a processor, A first function of receiving a 2D image and detecting a required area; A second function of removing noise with respect to the 2D image from which the area is detected; A third function of binarizing the noise-free two-dimensional image; A fourth function of performing a line segment on the binarized image; A fifth function of giving a label to the image on which the line segment is performed and verifying the size of each label; And A sixth function of extracting a feature point using geometric position information on the image of which the label size is verified; A computer-readable recording medium having recorded thereon a program for realizing this. The method of claim 8, A seventh function of performing monochromeization on the received 2D image when the received 2D image is a color image; A computer-readable recording medium that records a program for further realization. In the 3D image generating device, Basic model feature group setting means for selecting a basic model, setting a parameter straight line at each important feature part of the selected basic model, and setting a feature group connected to the set parameter straight line; Feature extraction means for receiving a two-dimensional image and extracting the feature; Model deforming means for deforming a basic model by moving the set parameter straight line according to the extracted feature points; And Texture mapping means for generating a 3D model by performing texture mapping on the deformed base model 3D image generating apparatus comprising a. The method of claim 10, The feature extraction means, Area detection means for receiving a two-dimensional image and detecting a required area; Noise removing means for removing noise with respect to the two-dimensional image in which the region is detected; Binarization means for binarizing the noise-free two-dimensional image; Line segment means for performing a line segment on the binarized image; Labeling and size verification means for assigning a label to the image on which the line segment has been performed and for verifying the size of each label; And And feature point extracting means for extracting feature points using geometric position information on the image of which the verification of the label size is completed. The method of claim 11, Black and whiteization means for performing black and whiteization on the received 2D image when the received 2D image is a color image 3D image generating device further comprising. In the 3D image generating method applied to the image generating system, A first step of selecting a basic model, setting a parameter straight line at each important feature part of the selected basic model, and setting a feature group connected to the set parameter straight line; Receiving a 2D image and extracting a feature thereof; A third step of deforming a basic model by moving the set parameter straight line according to the extracted feature points; And A fourth step of generating a 3D model by performing texture mapping on the deformed base model 3D image generation method comprising a. The method of claim 13, The second step, A fifth step of receiving a 2D image and detecting a required area; A sixth step of removing noise with respect to the two-dimensional image from which the region is detected; A seventh step of binarizing the two-dimensional image from which the noise is removed; An eighth step of performing a line segment on the binarized image; A ninth step of attaching a label to the image on which the line segment is performed and verifying the size of each label; And And a tenth step of extracting a feature point using geometric position information of the image of which the verification of the label size is completed. The method of claim 14, An eleventh step of performing monochromeization on the received 2D image when the received 2D image is a color image; 3D image generating method further comprising. In an image generation system having a processor, A first function of selecting a basic model, setting a parameter straight line to each important feature part of the selected basic model, and setting a feature group connected to the set parameter straight line; A second function of receiving a 2D image and extracting a feature thereof; A third function of deforming a basic model by moving the set parameter straight line according to the extracted feature points; And A fourth function of generating a 3D model by performing texture mapping on the deformed base model A computer-readable recording medium having recorded thereon a program for realizing this.