KR100948369B1 - Apparatus and method for complexing image in visual communication terminal - Google Patents

Abstract

The present invention synthesizes a variety of characters, such as a mustache, glasses or tears images to the face image obtained from the camera, to express the emotional state of the user or the face of the user decorated with a variety of characters with the video communication terminal on the other side of the call An image synthesizing apparatus and method for a communication terminal.

In addition, the image synthesizing apparatus of the video communication terminal of the present invention is based on the center of gravity search unit for finding the center of gravity of the skin region from the camera image acquired by the camera and the center of gravity found by the center of gravity found by the center of gravity search unit Find the change area and replace the change area with a character image prepared in advance to generate a first composite image, find a return area in the first synthesized image and replace the return area with the camera image to create a second synthesized image. It comprises a video synthesizer to generate.

Videophone, Synthetic, Skin, Core, Binarization, Grey, Character

Description

Apparatus and method for complexing image in visual communication terminal

The present invention relates to an image synthesizing apparatus and method of a video communication terminal, and in particular, by synthesizing various characters, such as a mustache, glasses or tear images, to a face image obtained from a camera, the emotional state of the user to the video communication terminal on the other side of the call The present invention relates to an image synthesizing apparatus and method for a video communication terminal capable of expressing a user's face decorated with various characters.

In recent years, thanks to the rapid development of information and communication fields, mobile communication terminals have become so popular that they are essential portable devices for the whole nation. Furthermore, voice calls and text messages have been the main functions. With the advent of the new models with various functions are released one after another to replace the old models quickly.

In particular, a third generation communication network supporting video telephony or a video conferencing system, such as a W-CDMA network or a next generation (fourth generation) mobile communication network, which is a development model thereof, such as a wireless broadband (WiBro) network is established. In the state of being stopped or moving, a video telephone service is introduced to exchange voice and video with a call counterpart through a mobile communication terminal.

As the video phone service became commercially available, on the other hand, there was a problem that one's self-hidden appearance could be shown to the other party on the phone without filtering. For example, sleeping eyes, messed up hair, or bare face without a toilet can be passed on to people who are not acquaintances, which can interfere with the privacy of users.

Accordingly, the present inventors have led to the present invention by studying an image synthesis method that can protect the user's face and further enhance the user's face in a fun or beautiful way.

The present invention has been made to solve the above-mentioned problem, finds the center of gravity of the skin region in the image obtained from the camera and synthesizes the character image prepared in advance to the image obtained from the camera to vary the user's face An object of the present invention is to provide an apparatus and method for synthesizing an image communication terminal.

In order to achieve the above object, the present invention is based on the center of gravity search unit for finding the center of gravity of the skin region in the camera image obtained by the camera and the change region in the camera image based on the center of gravity found by the center of gravity search unit Find and replace the changed area with a character image prepared in advance to generate a first composite image, find a return region in the first composite image, and replace the return region with the camera image to generate a second composite image Provided is an image synthesizing apparatus of a video communication terminal including a synthesizing unit.

In the above-described configuration, the character image is composed of a character portion and a black or white portion which is distinguished from the character portion, and the black or white portion is preferably the return region.

Preferably, the image synthesizing unit finds a defective region having no original pixel value of the camera image in the secondary composite image, and corrects the defective region with the camera image.

An image synthesizing apparatus of the image communication terminal includes: a background processing unit for recognizing a skin region in the camera image, and processing a background region other than that in black; A gray imager which grayates the skin region in the image input from the background processor; During the binary morphology calculation, the noise removal unit for removing noise components from the image inputted from the gray imager by performing erosion and expansion operations, and the gray image inputted from the noise remover 130 are binarized into a black and white image to weigh the weight. It is preferable to further comprise a binarization unit for outputting to the center search unit.

The center of gravity search unit performs a distance measurement operation on each point of the binarization image input from the binarization unit using a matrix type distance measuring mask, and sets a point representing the maximum value as the center of gravity of the skin region. It is desirable to.

In addition, the present invention is carried out in a video communication terminal equipped with an image synthesizing apparatus for synthesizing a camera image obtained from the camera and a character image prepared in advance, step (a) of finding the center of gravity of the skin region in the camera image ; On the basis of the center of gravity found in the step (a) to find a change region in the camera image to replace the change region with the character image to generate a first composite image (b) and (b) And (c) generating a second composite image by finding a return region in the first composite image and replacing the returned region with the camera image.

The character image is composed of a character part and a black or white part that is distinguished from the character part, and the searching of the return area in the step (c) is preferably a process of finding the black or white part.

The image synthesizing method of the video communication terminal detects a defective region having no original pixel value of the camera image in the secondary composite image generated in step (c), and corrects the defective region with the camera image (d). It is preferable that the step further comprises a).

The image synthesizing method of the video communication terminal comprises the steps of: recognizing a skin region in the camera image, and processing other background regions in black (ba1); (Ba2) graying the skin region in the camera image processed by the step (ba1); During the binary morphology calculation, a gray image is obtained by performing the erosion and expansion operations to remove the noise component from the gray image resulting from the execution of step (ba3) and the gray image resulting from the execution of the step (ba3) and (ba3). It is preferable to further comprise the step of binarizing (ba4), wherein step (a) is preferably performed by finding the center of gravity in the binarization image by the result of the step (ba4).

The step (a) is performed by performing a distance measurement operation on each point of the binarized image with a distance measurement mask in the form of a matrix, and setting the point representing the maximum value as the result of the distance measurement operation as the center of gravity. desirable.

According to the image synthesizing apparatus and method of the video communication terminal of the present invention, by finding the center of gravity of the skin region in the image obtained from the camera by combining the character image prepared in advance to the image obtained from the camera, When calling, it is possible to hide some of one's own face that he does not want to show to the other party or to decorate the user's face funnyly or beautifully.

Hereinafter, an image synthesizing apparatus and method of a video communication terminal according to a preferred embodiment of the present invention with reference to the accompanying drawings will be described in detail.

1 is a diagram illustrating a network in which a video phone service may be implemented.

As shown in FIG. 1, a network configuration of a system for implementing a video telephony service includes a computer, a wired telephone, and a mobile communication terminal (hereinafter, collectively referred to as a video communication terminal) each having a video telephony function (100). ) Is connected by the video communication network (200). In addition, the video communication network 200 may be connected to the Internet 400 to the gateway 300, whereby the video communication terminal 100 may download various characters necessary for image synthesis from the character providing server 500. .

Here, the video communication network 200 may be implemented as a W-CDMA network, the specific configuration of which is a radio signal transmission and reception, wireless channel encoding and decoding between the mobile station, that is, the mobile communication terminal, signal strength and image quality measurement, A base station that performs baseband signal processing, diversity, radio resource management, and self-maintenance functions, and a wireless network controller that controls a plurality of base stations, performs matching with base stations, inter-cell handover processing, and call control. It provides a video call and various additional functions between a radio network controller, a video telephone line switching network to which a wired telephone or a webcam-equipped computer is connected, and a video telephone line switching network and a wireless network controller. It may comprise a Mobile Switching Center (MSC) that provides connectivity to various other equipment and external networks.

Next, the Internet 400, as is well known, is a network of networks that connect a computer network based on a communication protocol called TCP / IP (Transmission Control Protocol / Internet Protocol) scattered around the world through wired / wireless networks. Should be interpreted as Here, the method of wirelessly accessing the Internet 400 is based on a platform such as WAP (Wireless Application Protocol) or WIPI (Wireless Internet Platform for Interoperability), or accesses through a mobile communication network or uses a public wireless LAN and an access point. Wi-Fi, a wireless Internet service that can provide high-speed Internet access even when stationary or slow, at ADSL quality and cost, and also known as a portable Internet.

2 is a block diagram of a video communication terminal having an image synthesizing apparatus according to an embodiment of the present invention, and FIGS. 5 to 10 are screen views of images processed by the image synthesizing apparatus shown in FIG. 2.

As shown in FIG. 2, the video communication terminal 100 of the present invention recognizes the skin region 10 from the camera 115 and an image obtained from the camera 115 (hereinafter referred to as a 'camera image'). 5, the background area 20 is input from the background processor 120 and the background processor 120 to be processed in black, as shown in FIG. 5. The gray imager 125 grays 10) and removes noise components from the image input from the gray imager 125, that is, to clearly distinguish the skin region 10 from the background region 20. As illustrated in FIG. 7, the image is input from the noise remover 130 and the noise remover 130, and the skin is divided into binarized images 135 and binarized from the binarized image 135. Center of gravity search unit 140 to find the center of gravity of the area 10, center of gravity search As shown in FIG. 10, various video images are downloaded from the image synthesis unit 145 and the character providing server 500 to synthesize and output the camera image and the character image based on the center of gravity found at 140. Character image storage unit 150, the RF transmitting and receiving unit 155 for transmitting and receiving radio frequency (RF) signal through the antenna, the key input unit 165, the display unit 160 and the image synthesis unit 145 And a control unit 110 that collectively controls the video communication terminal 100, such as controlling the RF transmitter / receiver 155 to transmit the synthesized image.

In the above-described configuration, specifically, the background processor 120 converts the RGB component of the camera image input from the camera 115 into the YCbCr format, and recognizes the skin region 10 using Cb and Cr, which are color components. The background area 20 other than the recognized skin area 10 is to adjust the RGB value to '0' so as to be completely black as shown in FIG.

Next, the gray imaging unit 125 performs a preprocessing process so that noise removal and binarization of the image can be easily performed. As shown in FIG. 6, the gray imaging unit 125 adjusts the RGB component of the skin region 10. The skin region 10 is to be grayed out.

Next, the noise removing unit 130 performs an erosion operation and a dilation operation during a binary morphology operation, which is well known as an image processing technique, and performs a skin region on a gray image input from the gray imager 125. (10) and the background area 20 are clearly distinguished. In more detail, the noise removing unit 130 first expands the dark portion of the gray image input from the gray imager 125 by using an erosion mask having a 3 * 3 or 5 * 5 matrix. As a result, the blemish in the background region 20 as shown in FIG. 5, that is, the part that is incorrectly recognized as the skin region 10 is removed. On the other hand, when this erosion operation is performed, the dark part of the actual skin region 10 is further expanded. Thus, in order to correct this, the noise removing unit 130 extends the bright part of the gray image in which the erosion operation is performed using the expansion mask, and the result thereof is as shown in FIG. 6.

Next, the binarization unit 135 receives the gray image from which the blemish is removed from the noise removing unit 130 and converts the gray image to a binarization image having only 0 (white) and 1 (black) values as shown in FIG. 7. It is. At this time, the binarization unit 135 performs an operation called threshold calculation. Specifically, data of a predetermined reference value among the values of 0 to 255 of the gray image are reset to 255 and the following is reset to 0 to correct the black and white. It is to obtain a binarized image expressed only in white.

Next, the center of gravity search unit 140 measures the distance between the black and white boundary point and each white point in the binarized image, and sets the point representing the maximum value as the center of gravity of the skin region 10. In other words, the center of gravity search unit 140 performs a distance measurement operation on each point of the binarized image using a distance measuring mask having a 3 * 3 or 5 * 5 matrix form, and as a result, the point indicating the maximum value is defined as a skin area ( It is set as the center of gravity of 10). Here, the center of gravity thus obtained will generally be the weight, the nose or the brow.

Next, the image synthesis unit 145 first finds a change area 31 to be replaced with the character image 40 in the camera image 30 based on the center of gravity searched by the center of gravity search unit 140. Is replaced with the character image 40 to generate a primary composite image as shown in FIG. 8. For example, the camera image 30 has a size of 640 * 480, the coordinates of the center of gravity found by the center of gravity search unit 140 are (320, 240), and the character image 40 has a size of 320 * 240. In the image synthesis unit 145, the corner points of the change area 31 are to be (160, 120), (160, 360), (480, 120) and (480, 360), respectively. Invar replaces this change area 31 with the character image 40.

Next, the image synthesizing unit 145 finds a return area to be returned to the original camera image 30 from the change area 31 replaced with the character image 40. That is, as shown in FIG. 8, the character image 40 is composed of a character part that is substantially necessary, such as glasses and a mustache, and a black part distinctly distinguished from the character part. Find the return area. Here, the image synthesis unit 145 may find the return area using, for example, conditional sentence 1 below. In addition, looking at the character image 40, if the character part is a dark color system, the unnecessary part may be white to contrast with this.

Conditional statement 1

if (image-> imageData [(640 * y + x) * 3] == 0)

Next, the image synthesizer 145 may replace the return region with the original camera image 30 to generate a secondary composite image as shown in FIG. 9 and output the second composite image to the controller 110. have. For example, the image synthesis unit 145 may replace the return region with the original camera image 30 by using the following execution statement for the return region that satisfies the conditional statement 1 described above.

Execution statement

image-> imageData [(640 * y + x)] = image_copy-> imageData [(640 * y + x)];

On the other hand, when viewing the secondary composite image generated in this way, there may be a problem in that a pixel of the camera image 30 proximate to the character image 40 does not have an original value and a defect occurs in the return process.

Therefore, once again, the image synthesis unit 145 finds the defect area 32 and corrects it to the camera image 30 to generate the final composite image as shown in FIG. 10, and outputs it to the controller 100. It is desirable to. Here, the image synthesis unit 145 may find the defect area 32 using, for example, conditional sentence 2 below.

Conditional statement 2

if ((unsigned char) image-> imageData [(640 * y + x) * 3] <10 && (unsigned char) image-> imageData [(640 * y + x) * 3 + 1] <10 && (unsigned char) image-> imageData [(640 * y + x) * 3 + 2] <10)

3 and 4 are flowcharts illustrating an image synthesizing method of a video communication terminal according to an embodiment of the present invention.

First, as shown in FIG. 3, an image is acquired by the camera 115 in step S11. Next, in step S13, the background processing unit 120 recognizes the skin region in the camera image, and then proceeds to step S15 to black process the background region.

Next, in step S17, the gray imager 125 adjusts the brightness of the skin area to gray the skin area. In step S21, the noise removal unit 130 performs erosion and expansion operations on the gray image. To remove blemishes.

Next, in step S23, the gray image is converted into a binarized image by the binarization unit 135, and then in step S25, the weight center search unit 140 finds the weight center of the skin region.

Next, as shown in FIG. 4, in step S27, the image synthesis unit 145 finds the changed region in the camera image based on the center of gravity, and then proceeds to step S29 to replace the found changed region with the character image. Create a primary composite image. Then, in step S31, after finding the return region in the primary composite image, the flow proceeds to step S33 to replace the return region with a camera image to generate a secondary composite image.

Next, in step S35, after finding the defective region in the secondary composite image by the image combining unit 145, the secondary composite image is corrected by replacing the defective region with the camera image in step S37. Then, the process proceeds to step S39 to output the corrected final composite image data to the controller 110.

The video synthesizing apparatus and method of the video communication terminal of the present invention are not limited to the above-described embodiments, and various modifications can be made within the scope of the technical idea of the present invention.

1 is a diagram illustrating a network in which a video phone service may be implemented.

2 is a block diagram of a video communication terminal having an image synthesizing apparatus according to an embodiment of the present invention.

3 and 4 are flowcharts for explaining a video synthesis method of a video communication terminal according to an embodiment of the present invention;

5 to 10 are screen examples of an image processed by the image synthesizing apparatus shown in FIG. 2.

*** Explanation of symbols for the main parts of the drawing ***

100: video communication terminal 110: control unit

115: camera 120: background processing unit

125: gray imaging unit 130: noise removing unit

135: binarization unit 140: center of gravity search unit

145: image synthesis unit 150: character image storage unit

155: RF transmitting and receiving unit 160: display unit

165: key input unit

200: video communication network 300: gateway

400: Internet 500: Character server

Claims (10)

delete Center of gravity search unit for finding the center of gravity of the skin area in the camera image acquired by the camera and Based on the center of gravity found by the center of gravity search unit, a change region is found in the camera image, and the change region is replaced with a character image prepared in advance to generate a first composite image, and a return region in the first composite image. And an image synthesizing unit generating a second synthetic image by replacing the return region with the camera image. The character image is composed of a character portion and a black or white portion that is distinguished from the character portion, wherein the black or white portion is an image combining device of the video communication terminal, characterized in that the return area. The method of claim 2, And the image synthesizing unit finds a defective region having no original pixel value of the camera image in the secondary composite image, and corrects the defective region with the camera image. The method of claim 3, wherein A background processor for recognizing a skin region in the camera image, and processing a background region other than that in black; A gray imager which grayates the skin region in the image input from the background processor; A noise removal unit for removing noise components from an image input from the gray imager by performing erosion and expansion operations during a binary morphology calculation; And a binarization unit for binarizing the gray image input from the noise removing unit into a black and white image and outputting the gray image to the center of gravity search unit. The method according to any one of claims 2 to 4, The center of gravity search unit performs a distance measurement operation on each point of the binarization image input from the binarization unit using a matrix type distance measuring mask, and sets a point representing the maximum value as the center of gravity of the skin region. And a video synthesizing apparatus for a video communication terminal. delete It is performed in a video communication terminal equipped with a video synthesizing apparatus for synthesizing a camera image obtained from the camera and a character image prepared in advance, (A) finding a center of gravity of the skin region in the camera image; (B) generating a first composite image by finding a change region in the camera image and replacing the change region with the character image based on the center of gravity found in step (a); and And (c) generating a second composite image by finding a return region in the first composite image generated in step (b) and replacing the returned region with the camera image. The character image is composed of a character portion and a black or white portion that is distinguished from the character portion, and the search of the return area in the step (c) is an image of the video communication terminal, characterized in that the process of finding the black or white portion. Synthetic method. The method of claim 7, wherein And (d) finding a defective area having no original pixel value of the camera image in the secondary composite image generated in step (c) and correcting the defective area with the camera image. A video synthesis method of a video communication terminal. The method of claim 8, Recognizing a skin region in the camera image, and processing other background regions in black (ba1); (Ba2) graying the skin region in the camera image processed by the step (ba1); Performing a erosion and expansion operation during the binary morphology operation, removing the noise component from the gray image by the result of performing the step (ba3), and The method further includes binarizing the gray image by the result of performing the step (ba3) into a black and white image. Wherein step (a) is performed by finding the center of gravity in the binarized image by the result of performing the step (ba4) video synthesis method of a video communication terminal. The method according to any one of claims 7 to 9, The step (a) is performed by performing a distance measurement operation on each point of the binarized image with a distance measurement mask in the form of a matrix, and setting the point indicating the maximum value as the result of the distance measurement operation as the center of gravity. A video synthesizing method of a video communication terminal.

Images