KR100618493B1 - The Apparatus and Method for Creating Three-Dimensional Image Automatically - Google Patents

Abstract

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates to an apparatus and method for automatically generating 3D images.

2. The technical problem to be solved by the invention

The present invention provides an apparatus and method for automatically generating a three-dimensional image for automatically generating a three-dimensional image from the input two-dimensional image information and providing it to a user in various forms, or applying it to an additional service through an internet network. To provide a computer-readable recording medium that records a program for realizing this.

3. Summary of Solution to Invention

According to an aspect of the present invention, there is provided an apparatus for automatically generating a 3D image, the apparatus comprising: image input means for receiving a 2D image; Feature extraction means for extracting a feature of the 2D image received from the image input means; Storage means for storing a basic three-dimensional image model and additional information; 3D deforming means for deforming a basic three-dimensional image model stored in the storage means by using features of the input image extracted by the feature extracting means; User input means for receiving a necessary item from a user; Synthesizing means for synthesizing the additional information stored in the storage means and the image deformed by the 3D deforming means according to the content input through the user input means to generate a three-dimensional image; And output means for outputting the 3D image generated by the synthesizing means.

4. Important uses of the invention                 

The present invention is used for image generation system.

3D image, image generation, automatic generation, feature extraction, virtual reality

Description

Apparatus and Method for Creating Three-Dimensional Image Automatically}             

1A is a block diagram of an embodiment of an apparatus for automatically generating 3D images according to the present invention.

Figure 1b is a configuration diagram of an embodiment of a three-dimensional automatic image generation system using the Internet network according to the present invention.

2 is a flowchart illustrating an embodiment of a method for automatically generating a 3D image according to the present invention.

3A is a block diagram of an embodiment of a 2D image feature extraction unit for automatically generating a 3D image according to the present invention.

3B is a flowchart illustrating a method of extracting a 2D image feature for automatically generating a 3D image according to the present invention.

Figure 3c is an embodiment explanatory diagram of a method for extracting a two-dimensional image feature for automatically generating a three-dimensional image according to the present invention.

* Explanation of symbols for the main parts of the drawings

101: image input unit 102: feature extraction unit

103: user input unit 104: correction unit                 

105: 3D deformation unit 106: database

107: synthesis unit 108: output unit

The present invention relates to an apparatus and method for automatically generating a 3D image, and more particularly, to automatically generate a 3D image from input 2D image information, and to provide it to a user in various forms, or to add it through an internet network. The present invention relates to an apparatus for automatically generating a three-dimensional image for application to a service, and a method and a computer-readable recording medium recording a program for realizing the method.

 Although the definition of virtual reality varies from scholar to scholar, it can be said that human beings feel real in computer-generated cyberspace. In order to realize such a virtual reality, various environments must be prepared, one of which is a three-dimensional representation of an object.

However, the conventional 3D image generating technology has not been widely used because it requires a lot of cost and time, and because of this, there has been a problem that it has been limited in the application field despite various applications.

The present invention proposed to solve the problems described above, 3 to automatically generate a three-dimensional image from the input two-dimensional image information to provide to the user in various forms, or to apply to additional services over the Internet network 3 An object of the present invention is to provide an apparatus for automatically generating a dimensional image, a method thereof, and a computer-readable recording medium recording a program for realizing the method.

According to an aspect of the present invention, there is provided an apparatus for automatically generating a 3D image, the apparatus comprising: image input means for receiving a 2D image; Feature extraction means for extracting a feature of the 2D image received from the image input means; Storage means for storing a basic three-dimensional image model and additional information; 3D deforming means for deforming a basic three-dimensional image model stored in the storage means by using features of the input image extracted by the feature extracting means; User input means for receiving a necessary item from a user; Synthesizing means for synthesizing the additional information stored in the storage means and the image deformed by the 3D deforming means according to the content input through the user input means to generate a three-dimensional image; And output means for outputting the 3D image generated by the synthesizing means.

In addition, the method of the present invention, a method for automatically generating a three-dimensional image applied to the image generating system, the first step of receiving a two-dimensional image, and extracting the feature of the received two-dimensional image; A second step of modifying a previously stored basic three-dimensional image model by using the extracted feature information of the input image; And a third step of synthesizing the stored additional information and the modified image according to a requirement input from a user to generate and output a 3D image.

In addition, the present invention, the image generation system having a processor, the image generation system having a processor, receiving a two-dimensional image, the first function of extracting the features of the received two-dimensional image; A second function of modifying a pre-stored basic 3D image model by using feature information of the extracted input image; And a computer-readable recording medium having recorded thereon a program for realizing a third function of generating and outputting a three-dimensional image by synthesizing the stored additional information and the modified image according to a requirement input from a user. .

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Hereinafter, in the exemplary embodiment for describing the present invention, only the human face image has been described as an example, but the present invention may be applied to all two-dimensional images to be transformed and generated into a three-dimensional image as well as a human face image. Will be self explanatory.

1A is a block diagram of an embodiment of an apparatus for automatically generating 3D images according to the present invention.

As shown in FIG. 1A, the apparatus for automatically generating a 3D image according to the present invention includes extracting a feature of a 2D image received from the image input unit 101 and the image input unit 101 for receiving a 2D image. The feature extractor 102 for correcting the feature information extracted by the feature extractor 102 according to the contents input through the user input unit 103 and the user input unit 103 for inputting necessary items from the user. The correction unit 104, a basic three-dimensional image model and a database (DB, 106) for storing additional information, and a feature of the input image corrected by the correction unit 104 to the database 106. 3D deformer 105 for deforming the stored basic three-dimensional image model, by the 3D deformer 105 and the additional information stored in the database 106 according to the content input through the user input unit 103 Hyeongdoen synthesized image to an output unit 108 for outputting the 3D image generated by the three-dimensional composite unit 107 and the combining unit 107 for generating an image.

Looking at the detailed configuration and operation of each component as follows.

First, the image input unit 101 is a component for receiving a 2D image that is desired to be transformed into a 3D image. Generally, a digital camera is used. It can be input after converting to digital image using scanner.

Next, the feature extractor 102 is a component that extracts a feature of the input 2D image to convert the 2D image input through the image input unit 101 into a 3D image. It is possible to generate a three-dimensional image only by the feature information obtained from the. That is, if the two-dimensional image that can extract the necessary feature information, by generating a three-dimensional image with only one image, it is possible to greatly reduce the time and cost required for the conventional three-dimensional image generation .

A detailed process of feature extraction performed by the feature extraction unit 102 will be described with reference to FIGS. 3A to 3C below.

The feature of the input 2D image extracted by the feature extractor 102 is corrected by the corrector 104. That is, the user may determine whether the location of the displayed feature point is appropriate and, if not appropriate, directly modify the location of the extracted feature point through the user input unit 103.

Next, the feature information corrected by the correction unit 104 is input to the 3D deformer 105, in which the 3D deformer 105 is input to the database 106 using the corrected feature information. A 3D image is obtained by transforming a previously stored basic 3D image model.

The 3D image thus obtained is further synthesized with the additional information stored in the database 106 to generate a complete 3D image. In this case, the additional information may include background music and other additional image information such as a background screen. The same information is included, and in this case, if there are a plurality of additional information that can be selected, the user's selection may be directly input through the user input unit 103.

That is, when a human face image is input, a complete 3D character is generated by synthesizing the selected additional information and the generated 3D face image by allowing a user to directly select a hairstyle, dress, body type, etc. in addition to the face information. .

Finally, the 3D image generated through the above process is output through the output unit 108, wherein the output unit 108, as well as the basic output means such as a computer monitor or a printer, the generated It is a concept including a recording medium manufacturing means such as a disk drive or a CD-ROM writer for storing three-dimensional image information, or a communication relay means that can be transmitted through a communication network.

That is, as in the embodiment, when a human face image is input and a three-dimensional image of a person is generated, the user can check the information through a monitor or a printer, and store the information on a recording medium such as a diskette or a CD-ROM. It may be provided, or may be used in various forms on the Web by transmitting through the Internet.

Therefore, the apparatus may be manufactured in the form of a vending machine, and may be used in the form of selling the produced diskette or CD-ROM, or in the form of Internet content using a character.

1B is a block diagram of an embodiment of a 3D image automatic generation system using an internet network according to the present invention.

As shown in FIG. 1B, in the 3D image automatic generation system using the Internet network according to the present invention, a server 120 in which a plurality of image generating apparatuses 110 is connected to a database 130 through the Internet network 140 is provided. It is configured in connection with.

Therefore, such a system can be utilized in various forms on the web by transmitting image data generated through the image generating apparatus 110 through the internet network 140, as described briefly with reference to FIG. 1A. Make sure

2 is a flowchart illustrating a method of automatically generating a 3D image according to the present invention.                     

As shown in FIG. 2, in the method for automatically generating a 3D image according to the present invention, when a 2D image is input (210), after extracting a feature of the input image (220), it is determined whether the extracted feature is appropriate. If it is not appropriate (230) and corrects it (240), if the extracted feature is appropriate, it performs a three-dimensional deformation using it (250).

In this case, the process of extracting the feature of the input image 220 will be described in detail with reference to FIGS. 3A to 3C below, and the method 250 for performing the 3D transformation is the extracted 2D Using the feature of the image, it is performed by transforming the basic three-dimensional image model already stored.

Next, after generating the complete three-dimensional image by synthesizing additional information to the three-dimensional deformed 250 base image (260), and outputs it (270).

Here, the synthesized additional information 260 includes information such as background music in addition to other additional video information such as a background screen, and in this case, if there are a plurality of additional information that can be selected, You can also choose.

In addition, the output process 270, as well as output using a basic output means such as a computer monitor or a printer, a disk drive or a CD-ROM writer (CD-ROM writer) for storing the generated three-dimensional image information, etc. This is a concept that includes both recording media production and transmission through a communication network.

3A is a block diagram of an embodiment of a 2D image feature extraction unit for automatically generating a 3D image according to the present invention, FIG. 3B is a flowchart according to the present invention, and FIG. 3C is an example of an image generated as a result of each process. It is also.                     

As described above, the feature extraction unit 102 of FIG. 1A and the feature extraction process 220 of FIG. 2 will be described in detail with reference to FIGS. 3A to 3C.

The features to be extracted will vary depending on what the 2D image is to be transformed into a 3D image, but in one embodiment of the present invention, a human face image is taken as an example. Features that should be limited to the outline of the face, the size and position of the eyes, the size and position of the nose, the size and position of the mouth.

In addition to the above features, various factors such as the protrusion of the cheekbones and the shape of the eyebrows will act as factors in determining the impression of the person, but the biggest characteristic is limited to the above four elements. This is because the image processing technique has difficulty in accurate detection, and the eyebrows can be similarly processed by simple texture mapping because of the small protrusion size.

However, it is possible to add and remove not only the above four elements but also other characteristic elements.

Once the feature point is determined, as shown in FIGS. 3A to 3C, in order to extract the feature of the 2D image for generating the 3D image according to the present invention, the face area must first be detected by the area detector 301. (310), the present invention used a blue screen for this purpose.

Of course, even if the background is complex or simple, it is possible to detect the feature points using a variety of conventional methods, but even in recent studies, the recognition rate is often lowered and thus a limited condition is used. Rather than focusing on the detection, we focused on three-dimensional modeling similar to the input image using a single frontal image.

To use a blue screen, you need to do color image processing.

Color Space, which is widely used in the related art, is an RGB coordinate system, but since the RGB coordinate system is a mixture of color and brightness components, a change in brightness is a major cause of misperception when performing image processing. . Therefore, it is more effective to separate and use the brightness component, YIQ, YUV, HSI is mainly used as the coordinate system in which the brightness component and the color component are separated, in one embodiment of the present invention used the HSI coordinate system.

Next, the monochromeization unit 302 performs monochromeization on the 2D image 311 in which the face region is detected (320).

This is because, although the input image used in the present invention receives a color image for 3D texture mapping in the future, the feature point detection method may be a black and white image.

In operation 330, the Gaussian smoothing process is performed on the 2D image 321 in which the black and whitening process is completed using a Gaussian smoothing filter 303.

The Gaussian smoothing filter 303 is a very efficient filter for removing distributed noise. The Gaussian smoothing filter 303 multiplies each pixel with a different weight, and then divides the total of the weight values. To smooth out the splashes, typically in black and white.

At this time, in the present invention, by using a filter of the size 5 * 5, to solve both problems of the computation time and the efficiency of noise cancellation at the same time.

In the image 331 in which the primary noise is removed through the Gaussian smoothing process, the secondary noise is removed through the contrast adjusting process by the contrast adjusting unit 304 (340).

Contrast is the ratio between the light and dark areas of the screen. Contrast is a high-contrast image that is easy to see because the outline of the subject is clear. Low-contrast images do not become easy to see because the outline is not clear. . That is, an image with low contrast can be converted into an image that is easier to process by emphasizing contrast by converting the bright part to be brighter and the dark part to be darker.

In other words, the characteristic and non-characteristic parts of the face have a clear contrast in the black and white image, so that the emphasis can be further emphasized to facilitate recognition.

General contrast is expressed as Equation 1 below.

g (x, y) = nf (x, y)

The density f (x, y) of the pixel is n times the position (x, y) of the original image, and the density g (x, y) is set at (x, y) of the processed image.

Since image data is treated as one pixel and 8 bits in size, the range of density that can be expressed is from 0 to 255. Thus, if the result of a calculation exceeds 255, it is limited to 255 regardless of the result of the calculation.

In the present invention, by slightly modifying it, the following equation (2) is used.

(x, y) = f (x, y) + (f (x, y) -128) * Cv / 128

This is a method for emphasizing the boundary value in binarization. In the basic processing, the low contrast image is darker than the contrast and the bright image is brighter.

That is, since the maximum contrast value of each pixel is 255, the median value 128 is subtracted from this and added to the original value, so that the original value is 128 and the weight value is positive (+). If the value is less than 128, the weight is made negative to play this role.

In addition, a parameter called Cv was placed here to allow fluid treatment, but in the present invention, this value was experimentally set to 90.

As described above, in order to reduce the noise of the black-and-white image 321, two processes are performed in the present invention. The first is the Gaussian Smoothing process 330, and the second is the contrast adjustment process 340. to be.

In the image of the human face as in the present embodiment, there is a big difference in terms of the surrounding skin and contrast, and since the area occupies a fairly large area in the face part, when the smoothing process is performed as a whole, small noises disappear. Performing a contrast process on the image emphasizes each feature point with smooth boundaries.

Next, the block binarization unit 305 performs the block binarization process on the 2D image 341 on which the contrast process is performed (350).

Binary is a method of dividing black and white based on a threshold value from a black and white image having 256 levels of brightness. Generally, if the image is evenly distributed and the image quality is good, there is no problem. If the image has a lot of damage and an irregular distribution, the optimal binarization image for the image cannot be obtained with one threshold.

That is, the basic binarization technique uses one threshold for the whole image, so if the contrast value of the shaded part (for example, the lower jaw) is different from the contrast value of the entire image, the desired result image can be obtained. There will be no.

Therefore, the threshold value for each part should be set for an image having a strong partial feature. In this embodiment of the present invention, a block binarization technique is used. .

The block binarization technique is a method of setting a block having a desired size and setting a threshold value in consideration of a contrast value of each block, so that each block may have a specific threshold value. If you do, you can effectively handle the shadows that appear on your chin or under your nose. In other words, the shadow part is processed by its own block, so the overall contrast is less affected.

However, since the feature point extraction process of the human face image, such as the embodiment of the present invention, recognizes and processes a mass of an object, only the block binarized image may have an error. This is because, due to the nature of image processing, even if only one pixel is separated, the processor recognizes different parts.

To this end, a line segment process is performed on the block binarized image 351 through a line segment unit 306 (360).

The line segment process is a process of connecting the two lines when the distance between the two lines is less than a predetermined segment value for the lines A and B horizontally separated from each other. In the present invention, the segment value was set to 5 in an experiment. That is, straight lines having a distance difference of less than 5 pixels are connected to each other.

This is because of the experimental result that the maximum distance of the main feature points such as eyes, nose and mouth, which are easy to be separated because the input image is about 640 * 480 and limited the shooting distance, does not exceed 5 pixels. According to various conditions such as the input image and the shooting distance, the segment value may be appropriately adjusted.

For the image 361 in which the line segment process is performed, the labeling and verifying size unit 307 labels and orders the areas spaced apart from each other, and then verifies the size of each label. Verify Size) (370).

In order to perform the above process, it is necessary to determine the maximum, minimum, maximum and minimum values of height for each label. In the embodiment of the present invention, the following conditions were used by experiment.

Max Width of Label: 20 pixels

Min Height of Label: 5 pixels

Max Height of Label: 100 pixels

Minimum width of label (length multiplied by height): 50

Maximum width of label (multiplied by length and height): 800

The above figures were obtained by quantifying the average of 100 test subjects when the front still image of 640 * 480 was recognized with error within 10cm from the distance of 80cm from the camera. The obtained image is 371 of FIG. 3C.

After the verification of the label size for the image (Verify Size) process 370, the feature extraction unit 308 performs a process of detecting the eyes, nose and mouth using the geometric position feature (380) .

In general, in the detection of eyes, nose and mouth, the eyes are first searched, and then a number of methods are used to find other parts by using geometric position features, which is the most detectable part in a special environment of a human face. This is because the eyes are symmetrical from side to side, and after finding them, you can use the positional information that the nose and mouth are below them.

Taking advantage of the fact that the eyes are always paired on the face, first select one label, then check whether there are pairs of labels whose size does not vary greatly in the horizontal direction, and adopt them as candidates for the eyes.

Since the eyebrow and the eye are not distinguished when using this method, in one embodiment of the present invention, the best of these paired parts is the eyebrow, the lower pair is the eye, and only one pair exists It was detected by eye.

The above method is possible because the part which has been paired from above among the images already removed after the verify size processing has the highest probability of being an eyebrow and an eye.

Based on the detected eyes, the nose located at the center of both eyes and located at the bottom of the eyes was recognized as a nose, and the area appearing below was detected by the mouth. This is because, with the current treatment, there are few other distinctive features between the eyes, nose and mouth.

The images obtained through the above process are 381 and 382 of FIG. 3C.

Next, finally, the outside of the face is detected (390), the information of the outside of the face used in an embodiment of the present invention is only information on the part where the outside of the face is rapidly changed, such as a jaw line that greatly determines the impression of the person. Since it is sufficient, the straight line 391 1 is drawn horizontally at the position of the detected eye, and the point 339 a that meets the end point of the face area shows the overall left and right length of the face of the person. Draw a straight line (2 of 391) horizontally at the position of to meet the end point of the face area (b of 391), and a straight line (3 of 391) vertically from the center of the lips, and then Using the point 391c, the facial outline may be obtained.

In the 391 image of FIG. 3C, it is not difficult to find the points (a) and (b). Because the above process is performed in the state that only the face area is extracted in the preprocessing process using the blue screen, it is easy to find the intersection point of the horizontal coordinate and the end point of the face area if only the position of the eye and mouth is detected. Because.

However, in the process of finding the jaw line, the separation between the jaw and the neck is often difficult, so the embodiment of the present invention went through the following process.

First, a rectangular candidate area is determined based on the middle point of the mouth area by 5 pixels left and right, that is, 10 pixels horizontally, and a point where the face area ends vertically with the lower part of the mouth area. .

Next, the sum of the horizontal axes from the top to the bottom for the candidate area is estimated when the following two conditions are satisfied.

1) When there is a horizontal axis where the difference in sum of the connected horizontal axes is more than twice

2) When the area of this horizontal axis continues more than 5 pixels

The top of the head is pre-processed after filling the blue screen with black after removing the blue screen. Therefore, the part where the forehead and the hair meet is designated as the tip of the head.

To do this, in the same way, add up the horizontal axis of 10 pixels horizontally from the midpoint above the eye area, so that when an area where this value is more than three times larger than the previous value continues for more than 5 pixels, The first horizontal axis was assumed to be the top of the head.

The 392 image of FIG. 3C is represented by a rectangle composed of two vertical lines representing points (a) and (b) and one horizontal axis representing point (c) using the feature point detection method used in the embodiment of the present invention. Face area.

As described above, the method of the present invention may be implemented as a program and stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.) in a computer-readable form.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the spirit of the present invention for those skilled in the art to which the present invention pertains, and the above-described embodiments and accompanying It is not limited to the drawing.

The present invention as described above, in generating a three-dimensional image from a two-dimensional image, by making it possible to automatically generate a three-dimensional image with only one image, if the two-dimensional image that can extract the necessary feature information, There is an effect that can generate a three-dimensional image at a lower cost and faster speed.

In addition, the present invention, by allowing the generated three-dimensional image to be transmitted through the Internet network, it can be widely used in the field of TV and virtual reality, as well as the media industry including games and e-mail (E-mail). It has an effect.

Claims (12)

In the 3D image automatic generation device, Image input means for receiving a two-dimensional image; Feature extraction means for extracting a feature of the 2D image received from the image input means; Storage means for storing a basic three-dimensional image model and additional information; 3D deforming means for deforming a basic three-dimensional image model stored in the storage means by using features of the input image extracted by the feature extracting means; User input means for receiving a necessary item from a user; Synthesizing means for synthesizing the additional information stored in the storage means and the image deformed by the 3D deforming means according to the content input through the user input means to generate a three-dimensional image; And Output means for outputting the 3D image generated by the synthesizing means 3D image automatic generation device comprising a. The method of claim 1, Correction means for correcting the feature information extracted by the feature extraction means according to the content input through the user input means 3D image automatic generation device further comprising. The method according to claim 1 or 2, The feature extraction means, Area detection means for receiving a two-dimensional image and detecting a required area; Noise removing means for removing noise with respect to the two-dimensional image in which the region is detected; Binarization means for binarizing the noise-free two-dimensional image; Line segment means for performing a line segment on the binarized image; Labeling and size verification means for assigning a label to the image on which the line segment has been performed and for verifying the size of each label; And And feature point extracting means for extracting feature points by using geometric position information of the image of which the label size verification is completed. The method of claim 3, wherein The feature extraction means, If the input 2D image is a color image, the apparatus for automatically generating 3D images, characterized in that it further comprises a black and whiteization means for performing the black and white for the input 2D image. The method according to claim 1 or 2, The storage means, An apparatus for automatically generating 3D image, characterized in that it stores not only additional image information selectable according to a user's preference but also additional information such as music information, in addition to the basic 3D image model. The method according to claim 1 or 2, The output means, In addition to basic output means such as a computer monitor or a printer, a recording medium manufacturing means such as a disk drive or a CD-ROM writer for storing the generated 3D image information, or a communication relay that can be transmitted through a communication network. And a means for automatically generating a three-dimensional image. In the 3D image automatic generation method applied to the image generation system, A first step of receiving a 2D image and extracting a feature of the received 2D image; A second step of modifying a previously stored basic 3D image model by using feature information of the extracted input image; And A third step of synthesizing the stored additional information and the deformed image according to a requirement input from a user to generate and output a 3D image; 3D image automatic generation method comprising a. The method of claim 7, wherein A fourth step of correcting the extracted feature information according to a requirement input from a user 3D image automatic generation method further comprising. The method according to claim 7 or 8, The first step is, A fifth step of receiving a 2D image and detecting a required area; A sixth step of removing noise with respect to the two-dimensional image from which the region is detected; A seventh step of binarizing the two-dimensional image from which the noise is removed; An eighth step of performing a line segment on the binarized image; A ninth step of attaching a label to the image on which the line segment is performed and verifying the size of each label; And And a tenth step of extracting a feature point using geometric position information on the image of which the verification of the label size is completed. The method of claim 9, The first step is, If the input 2D image is a color image, the method further comprises the eleventh step of performing a black and white image for the received 2D image. In an image generation system having a processor, A first function of receiving a 2D image and extracting a feature of the received 2D image; A second function of modifying a pre-stored basic 3D image model by using feature information of the extracted input image; And A third function of synthesizing the stored additional information and the deformed image according to a requirement input from a user to generate and output a 3D image; A computer-readable recording medium having recorded thereon a program for realizing this. The method of claim 11, A fourth function of correcting the extracted feature information according to a requirement input from a user A computer-readable recording medium that records a program for further realization.

Images