WO2009066910A2 - Apparatus and method for complexing image in visual communication terminal - Google Patents

Apparatus and method for complexing image in visual communication terminal Download PDF

Info

Publication number
WO2009066910A2
WO2009066910A2 PCT/KR2008/006754 KR2008006754W WO2009066910A2 WO 2009066910 A2 WO2009066910 A2 WO 2009066910A2 KR 2008006754 W KR2008006754 W KR 2008006754W WO 2009066910 A2 WO2009066910 A2 WO 2009066910A2
Authority
WO
WIPO (PCT)
Prior art keywords
image
region
camera
unit
searching
Prior art date
Application number
PCT/KR2008/006754
Other languages
French (fr)
Other versions
WO2009066910A3 (en
Inventor
Hyun-Joo Lee
Original Assignee
Hyun-Joo Lee
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to KR10-2007-0119382 priority Critical
Priority to KR1020070119382A priority patent/KR100948369B1/en
Priority to KR1020080015409A priority patent/KR100948370B1/en
Priority to KR10-2008-0015409 priority
Application filed by Hyun-Joo Lee filed Critical Hyun-Joo Lee
Publication of WO2009066910A2 publication Critical patent/WO2009066910A2/en
Publication of WO2009066910A3 publication Critical patent/WO2009066910A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00221Acquiring or recognising human faces, facial parts, facial sketches, facial expressions
    • G06K9/00228Detection; Localisation; Normalisation
    • G06K9/00234Detection; Localisation; Normalisation using pixel segmentation or colour matching
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/60Editing figures and text; Combining figures or text
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/262Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/272Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

The present invention relates to an image composition apparatus and method for video communication terminals, which combine various characters, with a face image acquired by a camera, thus allowing a user' s face, which expresses an emotional state or which is adorned with various characters, to be displayed on the video communication terminal of the other party of a call. The image composition apparatus includes a gravity center location unit (140) for locating a center of gravity of a skin region in a camera image. An image composition unit (145) generates a primary composite image by searching the camera image for a change region on a basis of the center of gravity and replacing the change region with a prepared character image, and generates a secondary composite image by searching the primary composite image for a return region and replacing the return region with the camera image.

Description

[DESCRIPTION]
[Invention Title]
APPARATUS AND METHOD FOR COMPLEXING IMAGE IN VISUAL COMMUNICATION TERMINAL
[Technical Field]
The present invention relates, in general, to an image composition apparatus and method for video communication terminals, and, more particularly, to an image composition apparatus and method for video communication terminals, which combine various characters, such as images of a mustache, glasses or tears, with a face image acquired by a camera, thus allowing a user' face, which expresses his or her emotional state or which is adorned with various characters, to be displayed on the video communication terminal of the other party of a call.
[Background Art]
Recently, with the rapid development of information/communication fields, mobile communication terminals have been universalized to such an extent that they are regarded as essential personal belongings by the population of the entire nation. Moreover, voice calls or the transmission/reception of text messages were the main functions of mobile communication terminals long ago. With the appearance of new technologies, new models having a variety of functions have successively come onto the market and have rapidly replaced older models.
In particular, a third-generation communication network which supports video calls or video conference systems, for example, a Wireless Code Division Multiple Access (W-CDMA) network, or the next-generation (fourth- generation) mobile communication network which is a model developed from the third-generation communication network, for example, a Wireless Broadband (WiBro) network, is constructed, so that a video call service, allowing a user to exchange audio and video with the other party of a call through mobile communication terminals even in the non-motion state or the movement state of a user, has been introduced. As video call services have been commercialized in this way, there has occurred a problem in that a user' s own appearance desired to be hidden may be shown to the other party of a call as it is. For example, appearances such as swollen eyes, messed up hair or a face without make-up appearing when the user is just waking from sleep, may also be transferred to parties who are not acquaintances, thus causing inconveniences to the user' s private life.
Accordingly, the present applicant did research concerning an image composition method capable of protecting a user' s face or additionally adorning the user' s face in interesting or beautiful ways, thereby culminating in the present invention.
[Disclosure]
[Technical Problem]
Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and an object of the present invention is to provide an image composition apparatus and method for video communication terminals, which search an image acquired by a camera for a skin region or additionally an eye region, locate the center of gravity in the found skin or eye region, and combine a prepared character image with the image acquired by the camera on the basis of the located center of gravity, thus allowing a user' s face to be variously adorned.
[Technical Solution]
In accordance with an aspect of the present invention to accomplish the above object, there is provided an image composition apparatus for video communication terminals, comprising a gravity center location unit for locating a center of gravity of a skin region in a camera image acquired by a camera; and an image composition unit for generating a primary composite image by searching the camera image for a change region on a basis of the center of gravity located by the gravity center location unit and by replacing the change region with a prepared character image, and for generating a secondary composite image by searching the primary composite image for a return region and by replacing the return region with the camera image.
In the above construction, preferably, the character image may include a character part and a black or white part distinguished from the character part, and the black or white part may be the return region.
Preferably, the image composition unit may correct a defective region in the secondary composite image.
Preferably, the image composition apparatus may further comprise a background processing unit for detecting a skin region from the camera image, and processing a remaining background region as a black color; a gray image conversion unit for converting the skin region in an image received from the background processing unit into a gray image; a noise elimination unit for performing erosion and dilation operations, among binary morphology operations, thus eliminating a noise component from the gray image received from the gray image conversion unit; and a binarization unit for binarizing the gray image received from the noise elimination unit into a binary image, and outputting the binary image to the gravity center location unit.
Preferably, the gravity center location unit may perform a distance measurement operation on respective locations of the binary image, received from the binarization unit, using a distance measurement mask having a matrix form, and set a location having a maximum value as a result of the distance measurement operation as a center of gravity of the skin region.
In accordance with an aspect of the present invention to accomplish the above object, there is provided an image composition method for video communication terminals, the image composition method being performed by a video communication terminal equipped with an image composition apparatus for combining a camera image, acquired by a camera, with a prepared character image, comprising the steps of (a) locating a center of gravity of a skin region in the camera image; (b) searching the camera image for a change region on a basis of the center of gravity located at step (a) and replacing the change region with the character image, thus generating a primary composite image! and (c) searching the primary composite image generated at step (b) for a return region and replacing the return region with the camera image, thus generating a secondary composite image.
In the above construction, the character image may include a character part and a black or white part distinguished from the character part, and at step (c), the searching for the return region may be a procedure for searching for the black or white part.
Preferably, the image composition method may further comprise the step of (d) correcting a defective region in the secondary composite image generated at step (c) .
Preferably, the image composition method may further comprise the steps of (bal) detecting the skin region in the camera image and processing a remaining background region as a black color; (ba2) converting the skin region in the camera image processed at step (bal) into a region of a gray image; (ba3) performing erosion and dilation operations among binary morphology operations, thus eliminating a noise component from the gray image obtained as a result of performance of step (ba2); and (ba4) binarizing the gray image obtained as a result of performance of step (ba3) into a binary image, wherein step (a) is performed by locating the center of gravity in the binary image obtained as a result of performance of step (ba4).
Preferably, step (a) may be performed by performing a distance measurement operation on respective locations of the binary image using a distance measurement mask having a matrix form, and setting a location having a maximum value as a result of the distance measurement operation as the center of gravity.
In accordance with another aspect of the present invention to accomplish the above object, there is provided an image composition apparatus for video communication terminals, comprising a skin region searching unit for searching a camera image acquired by a camera for a skin region; an eye region searching unit for searching the camera image for an eye region by performing template matching between the skin region found by the skin region searching unit and pre-stored eye region templates; a gravity center location unit for locating a center of gravity of the eye region found by the eye region searching unit; and an image composition unit for generating a primary composite image by searching the camera image for a change region on a basis of the center of gravity located by the gravity center location unit and by replacing the change region with a prepared character image, and for generating a secondary composite image by searching the primary composite image for a return region and by replacing the return region with the camera image.
In the above construction, the character image may include a character part and a black or white part distinguished from the character part, and the black or white part may be the return region.
Preferably, the image composition unit may correct a defective region in the secondary composite image.
Preferably, the image composition unit may replace the defective region with the camera image.
Preferably, the camera may acquire a personal image which includes a target person to be combined with the character image, and a background image which does not include the person, the image composition apparatus further comprising a gray image conversion unit for converting the camera images acquired by the camera into gray images; a subtraction unit for performing a first subtraction operation of receiving the personal image and the background image from the gray image conversion unit and subtracting gray scale values of the background image from gray scale values of the personal image, obtaining a first subtraction image by adding a maximum value of a gray scale range to a negative value generated from the first subtraction operation, performing a second subtraction operation of subtracting the gray scale values of the personal image from the gray scale values of the background image, and obtaining a second subtraction image by adding the maximum value of the gray scale range to a negative value generated from the second subtraction operation; and an addition unit for performing an addition operation on the first subtraction image and the second subtraction image received from the subtraction unit, thus obtaining a binary image, wherein the skin region searching unit searches for the skin region using the binary image.
In accordance with another aspect of the present invention to accomplish the above object, there is provided an image composition method for video communication terminals, the image composition method being performed by a video communication terminal equipped with an image composition apparatus for combining a camera image, acquired by a camera, with a prepared character image, comprising the steps of (a) searching the camera image acquired by the camera for a skin region! (b) searching the camera image for an eye region by performing template matching between the skin region found at step (a) and pre-stored eye region templates; (c) locating a center of gravity of the eye region found at step (b); (d) searching the camera image for a change region on a basis of the center of gravity located at step (c) and replacing the change region with the character image, thus generating a primary composite image; and (e) searching the primary composite image generated at step (d) for a return region and replacing the return region with the camera image, thus generating a secondary composite image.
In the above construction, preferably, the character image may include a character part and a black or white part distinguished from the character part, and, at step (e), the searching for the return region may be a procedure for searching for the black or white part.
Preferably, the image composition method may further comprise the step of (f) correcting a defective region in the secondary composite image generated at step (e).
Preferably, step (f) may be performed by replacing the defective region with the camera image.
Preferably, the camera acquires a personal image which includes a target person to be combined with the character image, and a background image which does not include the person, the image composition method further comprising the steps of (bal) converting the camera images acquired by the camera into gray images; (ba2) performing a first subtraction operation of subtracting gray scale values of the background image from gray scale values of the personal image, and obtaining a first subtraction image by adding a maximum value of a gray scale range to a negative value generated from the first subtraction operation; (ba3) performing a second subtraction operation of subtracting the gray scale values of the personal image from the gray scale values of the background image, and obtaining a second subtraction image by adding the maximum value of the gray scale range to a negative value generated from the second subtraction operation; and (ba4) performing an addition on the first subtraction image and the second subtraction image, thus obtaining a binary image, wherein step (a) is performed by searching for the skin region using the binary image obtained at step (ba4).
[Advantageous Effects]
In accordance with the image composition apparatus and method for video communication terminals according to the present invention, there is an advantage in that a skin region or additionally an eye region are found in an image acquired by a camera, the center of gravity is located in the detected skin or eye region, and a prepared character image is combined with the image acquired by the camera on the basis of the located center of gravity, thus allowing a user to hide part of his or her face which is desired to be hidden from the other party or enabling the user' s face to be adorned in interesting or beautiful ways during a video call.
[Description of Drawings]
Fig. 1 is a diagram showing the configuration of a network in which a video call service can be implemented;
Fig. 2 is a block diagram showing a video communication terminal equipped with an image composition apparatus according to an embodiment of the present invention;
Figs. 3 and 4 are flowcharts showing an image composition method for video communication terminals according to an embodiment of the present invention;
Figs. 5 to 10 are views illustrating the screen of images processed by the image composition apparatus of Fig. 2;
Fig. 11 is a block diagram showing a video communication terminal equipped with an image composition apparatus according to another embodiment of the present invention;
Figs. 12 and 13 are flowcharts showing an image composition method for video communication terminals according to another embodiment of the present invention; and
Figs. 14 to 19 are views illustrating the screen of images processed by the image composition apparatus of Fig. 11.
*** Description of reference numerals of principal elements in the drawings ***
100: video communication terminal
110: control unit 115: camera
120: background processing unit 125: gray image conversion unit
130: noise elimination unit 135: binarization unit
140: gravity center location unit 145: image composition unit
150: character image storage unit 155: RF transmission / reception unit
160: display unit 165: key input unit
200: video communication network 300: gateway
400: Internet 500: character provision server
600: video communication terminal
615: camera 620: gray image conversion unit
625: subtraction unit 630: addition unit
635: skin region searching unit 640: eye region searching unit
645: gravity center location unit 650: image composition unit
655: character image storage unit
660: eye region template storage unit
665: background image storage unit 670: control unit 675: RF transmission / reception unit 680: display unit
685: key input unit [Mode for Invention]
Hereinafter, with reference to the attached drawings, an image composition apparatus and method for video communication terminals according to an embodiment of the present invention will be described in detail. Before the description, the configuration of a network in which a video call service can be implemented is described with reference to Fig. 1.
Fig. 1 is a diagram showing the configuration of a network in which a video call service can be implemented.
As shown in Fig. 1, the network configuration of a system for implementing a video call service is implemented such that a computer, a wired phone and a mobile communication terminal, each having a video call function (they are collectively called 'video communication terminals' 100 or 600), are connected to each other through a video communication network 200. Further, the video communication network 200 may also be connected to the Internet 400 through a gateway 300. Through this connection, the video communication terminals 100 and 600 can download various types of characters required for image composition from a character provision server 500.
Here, the video communication network 200 can be implemented as a W- CDMA network, and in detail includes base stations for performing functions such as the transmission/reception of Radio Frequency (RF) signals, radio channel encoding and decoding, the measurement of signal intensity and video quality, baseband signal processing, diversity processing, radio resource management and their own maintenance, between mobile stations, that is, mobile communication terminals, a radio network controller for controlling a plurality of base stations and performing functions such as interfacing with the base stations, processing handovers between cells, and processing calls, a circuit-switched network for video calls to which a wired phone or a computer equipped with a web camera is connected, and a Mobile Switching Center (MSC) for providing a video call function and various additional functions between the circuit-switched network for video calls and the radio network controller, setting up a channel, and providing a connection to other types of equipment and external networks.
Next, as is well known to those skilled in the art, the Internet 400 should be interpreted as a network of networks for connecting in a wired/wireless manner computer networks which are based on a communication protocol called Transmission Control Protocol / Internet Protocol (TCP/IP) and are distributed all over the world. Here, a method of accessing the Internet 400 in a wireless manner may include a method using the wireless Internet which is accessed over a mobile communication network on the basis of platforms, such as Wireless Application Protocol (WAP) and Wireless Internet Platform for Interoperability (WIPI), or is accessed over a public Wireless LAN and an Access Point (AP), and a method using a Wireless Broadband (WiBro) network, so-called portable Internet, which enables high¬ speed Internet access in the non-motion state or low-speed movement state of a user at communication quality and costs similar to those of Asymmetric Digital Subscriber Line (ADSL).
(Embodiment)
Fig. 2 is a block diagram showing a video communication terminal equipped with an image composition apparatus according to an embodiment of the present invention, and Figs. 5 to 10 are views illustrating the screen of images processed by the image composition apparatus of Fig. 2.
As shown in Fig. 2, the video communication terminal 100 according to the present invention includes a camera 115, a background processing unit 120 for detecting a skin region 10 in an image acquired by the camera 115 (hereinafter referred to as a 'camera image' and processing the remaining background region 20 as a black color, as shown in Fig. 5, a gray image conversion unit 125 for converting the skin region 10 in the image of Fig. 5 which is received from the background processing unit 120 into a region of a gray image, a noise elimination unit 130 for eliminating a noise component from the image received from the gray image conversion unit 125, in other words, for definitely distinguishing the skin region 10 from the background region 20, a binarization unit 135 for binarizing the image received from the noise elimination unit 130 into a binary image, as shown in Fig. 7, a gravity center location unit 140 for locating the center of gravity of the skin region 10 in the binary image received from the binarization unit 135, an image composition unit 145 for combining the camera image with a character image on the basis of the center of gravity located by the gravity center location unit 140, as shown in Fig. 10, and outputting a composite image, a character image storage unit 150 for storing various types of character images downloaded from the character provision server 500, an RF transmission/reception unit 155 for transmitting/receiving Radio Frequency (RF) signals to/from a base station through an antenna, a key input unit 165, a display unit 160, and a control unit 110 for controlling the entire operation of the video communication terminal 100, such as by controlling the RF transmission/reception unit 155 so that the composite image generated by the image composition unit 145 is transmitted.
In the above construction, in detail, the background processing unit 120 converts the RGB components of the camera image received from the camera 115 into a YCbCr format, detects the skin region 10 using chromatic components Cb and Cr, and adjusts the RGB values of the background region 20, other than the detected skin region 10, to '0' so that the color of the background region 20 completely becomes black, as shown in Fig. 5.
The gray image conversion unit 125 performs a preprocessing operation of allowing the noise elimination and the binarization of images to be easily performed at a subsequent stage. As shown in Fig. 6, the RGB components of the skin region 10 are adjusted, and thus the skin region 10 is converted into a region of a gray image.
The noise elimination unit 130 is intended to definitely distinguish the skin region 10 from the background region 20 in the gray image received from the gray image conversion unit 125 by performing an erosion operation and a dilation operation, among binary morphology operations which are well- known image processing techniques. In more detail, the noise elimination unit 130 dilates a dark portion of the gray image received from the gray image conversion unit 125 using an erosion mask having the form of a 3*3 or 5*5 matrix. Accordingly, as shown in Fig. 5, spots included in the background region 20, that is, portions erroneously detected as the skin region 10, are eliminated. In contrast, when such an erosion operation is performed, a dark portion in the actual skin region 10 is further dilated. Therefore, in order to correct this phenomenon, the noise elimination unit 130 dilates a bright portion of the gray image, on which the erosion operation has been performed, using a dilation mask, and the resulting image thereof is shown in Fig. 6.
The binarization unit 135 receives the gray image, from which the spots are eliminated, from the noise elimination unit 130, and converts the gray image into a binary image having values of 0 (black) and 1 (white), as shown in Fig. 7. At this time, the binarization unit 135 performs an operation called a threshold calculation, which denotes, in detail, an operation of re-setting data, having values equal to or greater than a preset value among values of 0 to 255 of the gray image, to 255 and re-setting the remaining data to 0, thus obtaining a binary image exactly represented by black and white.
The gravity center location unit 140 measures distances between boundary locations between black and white regions and respective white locations in the binary image, and sets a location having the maximum distance as the center of gravity of the skin region 10. In other words, the gravity center location unit 140 performs a distance measurement operation on respective locations of the binary image using a distance measurement mask having the form of a 3*3 or 5*5 matrix, and sets a location, having the maximum value as a result of the operation, as the center of gravity of the skin region 10. In this case, the center of gravity obtained in this way may be a philtrum, the ridge of the nose or the middle of the forehead.
The image composition unit 145 searches the camera image 30 for a change region 31, which will be replaced with a character image 40, on the basis of the center of gravity located by the gravity center location unit 140, and replaces the found change region 31 with the character image 40, thus generating a primary composite image, as shown in Fig. 8. For example, when the camera image 30 has a size of 640*480, the coordinates of the center of gravity located by the gravity center location unit 140 are (320, 240), and the character image 40 has a size of 320*240, the coordinates of the corners of the change region 31 are (160, 120), (160, 360), (480, 120) and (480, 360), respectively, and thus the image composition unit 145 replaces this change region 31 with the character image 40.
The image composition unit 145 searches the change region 31 replaced with the character image 40 for a return region which will be returned to the original camera image 30. That is, as shown in Fig. 8, the character image 40 is composed of a character part, which is actually required, such as glasses or a moustache, and a black part definitely distinguished from the character part. Accordingly, the image composition unit 145 is intended to search for this black part, that is, the return region. In this case, the image composition unit 145 can search for the return region using, for example, the following conditional statement 1. Further, when the character part in the character image 40 is of a dark, shaded color, the unnecessary part may have a white color so as to be contrasted with the character part.
Conditional statement 1 if(image->imageData[(640 * y + x) *3] == 0)
Next, the image composition unit 145 may generate a secondary composite image, as shown in Fig. 9, by replacing the return region with the original camera image 30, and may output the secondary composite image to the control unit 110. In this case, the image composition unit 145 may replace the return region with the original camera image 30 using, for example, the following execute statement with respect to the return region which satisfies the above conditional statement 1.
Execute statement image->imageData[(640 * y + x)] = image_copy->iraageData[(640 * y + x)];
Meanwhile, when the secondary composite image generated in this way is considered, there may occur a problem in that, during a return procedure, the pixels of the camera image 30 close to the character image 40 do not have their original values, thus resulting in defects.
Therefore, the image composition unit 145 preferably generates a final composite image, as shown in Fig. 10, by searching for such a defective region 32 and correcting the defective region 32 to the camera image 30, and outputs the final composite image to the control unit 100. In this case, the image composition unit 145 can search for the defective region 32 using, for example, the following conditional statement 2.
Conditional statement 2 if((unsigned char)image->imageData[(640 * y + x) *3] < 10 && (unsigned char)image->imageData[(640 * y + x) * 3 + l] < 10 && (unsigned char)image->imageData[(640 * y + x) * 3 + 2] < 10)
Figs. 3 and 4 are flowcharts showing an image composition method for video communication terminals according to an embodiment of the present invent ion.
First, as shown in Fig. 3, at step SIl, an image is acquired by the camera 115. Next, at step S13, a skin region is detected in the camera image by the background processing unit 120. Thereafter, the process proceeds to step S15 at which a background region is processed with a black color.
Next, at step S17, the brightness of the skin region is adjusted by the gray image conversion unit 125, and thus the skin region is converted into a gray image. At step S21, erosion and dilation operations are performed on the gray image by the noise elimination unit 130, and thus spots are eliminated.
Next, at step S23, the gray image is converted into a binary image by the binarization unit 135. At step S25, the center of gravity of the skin region is located by the gravity center location unit 140. Thereafter, as shown in Fig. 4, at step S27, a change region is found in the camera image by the image composition unit 145 on the basis of the center of gravity. Then, the process proceeds to step S29, at which the found change region is replaced with a character image, and thus a primary composite image is generated. Thereafter, at step S31, a return region is found in the primary composite image, and the process proceeds to step S33 at which the return region is replaced with the camera image, and thus a secondary composite image is generated.
Next, at step S35, a defective region is found in the secondary composite image by the image composition unit 145. At step S37, the defective region is replaced with the camera image, and thus the secondary composite image is corrected. Next, the process proceeds to step S39, at which data of a final composite image obtained through correction is output to the control unit 110.
(Another embodiment)
Fig. 11 is a block diagram showing a video communication terminal equipped with an image composition apparatus according to another embodiment of the present invention, and Figs. 14 to 19 are views illustrating the screen of images processed by the image composition apparatus of Fig. 11.
As shown in Fig. 11, a video communication terminal 600 according to the present invention includes a camera 615, a gray image conversion unit 620 for converting a camera image acquired by the camera 615 (refer to Fig. 14) into a gray image, a background image storage unit 665 for storing background images processed as gray images (refer to Fig. 15), a subtraction unit 625 and an addition unit 630 for performing subtraction and addition operations on the camera image, received from the gray image conversion unit 620, and a background image, stored in the background image storage unit 665, thus obtaining a binary image between the two images, a skin region searching unit 635 for searching the camera image for a skin region by comparing the binary image (refer to Fig. 16) received from the addition unit 630 with the camera image received from the camera 615, an eye region template storage unit 660 for storing eye region templates processed as gray images, an eye region searching unit 640 for searching the camera image for an eye region by performing template matching between the skin region found by the skin region searching unit 635 and the eye region templates stored in the eye region template storage unit 660, a gravity center location unit 645 for locating the center of gravity of the eye region found by the eye region searching unit 640, an image composition unit 650 for combining the camera image with a character image on the basis of the center of gravity located by the gravity center location unit 645, and outputting a composite image, a character image storage unit 655 for storing various character images downloaded from a character provision server 500, an RF transmission/reception unit 675 for transmitting/receiving RF signals to/from a base station through an antenna, a key input unit 685, a display unit 680, and a control unit 670 for controlling the entire operation of the video communication terminal 600, such as by controlling the RF transmission/reception unit 675 so that the composite image generated by the image composition unit 650 is transmitted.
In the above construction, the gray image conversion unit 620 performs a preprocessing operation of converting the camera image, which is a chromatic image, into a gray image, the pixel values (gray scale values) of which exist within, for example, a range from 0 to 255 so that external factors, such as lighting, are excluded at the time of locating the center of gravity.
The subtraction unit 625 performs a subtraction operation of subtracting the background image from the camera image by using respective pixel values of the camera image as a minuend and using respective pixel values of the background image as a subtrahend, and subsequently performs, on the contrary, a subtraction operation of subtracting the camera image from the background image by using the pixel values of the background image as a minuend and using the pixel values of the camera image as a subtrahend. In this way, the background region having the same pixel values between the camera image and the background image obtains a value of '0' through the above subtractions and is then processed as a black color.
Further, when a resulting value obtained from the two subtractions is a negative value, the subtraction unit 625 substitutes a positive value for the negative value by adding the maximum value of a gray scale range, that is, 255, to the negative value. Therefore, the subtraction unit 625 obtains a first subtraction image from a first subtraction operation, and obtains a second subtraction image from a second subtraction operation.
The addition unit 630 performs an addition operation on the two subtraction images obtained in this way, thus obtaining a binary image in which a personal region 70, which is a target for image composition, is white and the remaining background region is black, as shown in Fig. 16.
The skin region searching unit 635 converts the RGB components of the camera image, received from the camera 615, into an YCbCr format. Then, the skin region searching unit 635 searches the camera image for the skin region using components Cb and Cr, but searches only a portion corresponding to the personal region 70, received from the addition unit 630, for the skin region. Then, a skin region 80, as shown in Fig. 17, may be detected by the skin region searching unit 635.
The eye region searching unit 640 searches the camera image for an eye region by performing template matching between the skin region found by the skin region searching unit 635 and the eye region templates stored in the eye region template storage unit 660.
The gravity center location unit 650 locates the center of gravity of the eye region found by the eye region searching unit 640. For example, the coordinates (X, Y) of an upper left corner and the coordinates (X, Y) of a lower right corner are found in the eye region having a rectangular box shape, as shown in Fig. 18, and thereafter the coordinates of a middle point 91 of a diagonal line which connects the two corners to each other are set as the coordinates of the center of gravity. Then, the center of gravity obtained in this way will be the middle of the forehead in general.
As shown in Fig. 19, the image composition unit 650 combines the camera image 50 with a character image 92 on the basis of the center of gravity located by the gravity center location unit 645 and to output a composite image. The image composition unit 650 is identical to the image composition unit 145 described in the above embodiment, and thus a detailed description thereof is omitted.
Meanwhile, the background image is acquired by the camera 610 and is processed as a gray image by the gray image conversion unit 620, and the gray image is stored in the background image storage unit 665. Further, in the present embodiment, the present invention has been described on the basis of the construction in which background images are stored in advance, but is not limited to this construction. That is, the camera 610 may individually acquire a background image 60 (refer to Fig. 15) and a personal image 50 (refer to Fig. 14), and the gray image conversion unit 620 may receive the two images from the camera 610, process the two images as gray images, and output the two gray images to the subtraction unit 625.
Figs. 12 and 13 are flowcharts showing an image composition method for video communication terminals according to another embodiment of the present invention.
First, as shown in Fig. 12, at step S51, an image is acquired by the camera 615. Next, at step S13, the camera image is converted into a gray image by the gray image conversion unit 620.
Next, at step S55, subtraction and addition operations are performed on the camera image received from the gray image conversion unit 620 and a background image stored in the background image storage unit 665 by the subtraction unit 625 and the addition unit 630, and thus a binary image between the two images is obtained.

Claims

[CLAIMS] [Claim 1]
An image composition apparatus for video communication terminals, comprising: a gravity center location unit for locating a center of gravity of a skin region in a camera image acquired by a camera; and an image composition unit for generating a primary composite image by searching the camera image for a change region on a basis of the center of gravity located by the gravity center location unit and by replacing the change region with a prepared character image, and for generating a secondary composite image by searching the primary composite image for a return region and by replacing the return region with the camera image.
[Claim 2]
The image composition apparatus according to claim 1, wherein the character image includes a character part and a black or white part distinguished from the character part, and the black or white part is the return region.
[Claim 3]
The image composition apparatus according to claim 2, wherein the image composition unit corrects a defective region in the secondary composite image.
[Claim 4]
The image composition apparatus according to claim 3, further comprising: a background processing unit for detecting a skin region from the camera image, and processing a remaining background region as a black color; a gray image conversion unit for converting the skin region in an image received from the background processing unit into a gray image; a noise elimination unit for performing erosion and dilation operations, among binary morphology operations, thus eliminating a noise component from the gray image received from the gray image conversion unit; and a binarization unit for binarizing the gray image received from the noise elimination unit into a binary image, and outputting the binary image to the gravity center location unit.
[Claim 5]
The image composition apparatus according to any one of claims 2 to 4, wherein the gravity center location unit performs a distance measurement operation on respective locations of the binary image, received from the binarization unit, using a distance measurement mask having a matrix form, and sets a location having a maximum value as a result of the distance measurement operation as a center of gravity of the skin region.
[Claim 6]
An image composition method for video communication terminals, the image composition method being performed by a video communication terminal equipped with an image composition apparatus for combining a camera image, acquired by a camera, with a prepared character image, comprising the steps of:
(a) locating a center of gravity of a skin region in the camera image;
(b) searching the camera image for a change region on a basis of the center of gravity located at step (a) and replacing the change region with the character image, thus generating a primary composite image; and
(c) searching the primary composite image generated at step (b) for a return region and replacing the return region with the camera image, thus generating a secondary composite image.
[Claim 7]
The image composition method according to claim 6, wherein: the character image includes a character part and a black or white part distinguished from the character part, and at step (c), the searching for the return region is a procedure for searching for the black or white part.
[Claim 8] The image composition method according to claim 7, further comprising the step of (d) correcting a defective region in the secondary composite image generated at step (c).
[Claim 9]
The image composition method according to claim 8, further comprising the steps of:
(bal) detecting the skin region in the camera image and processing a remaining background region as a black color;
(ba2) converting the skin region in the camera image processed at step (bal) into a region of a gray image;
(ba3) performing erosion and dilation operations among binary morphology operations, thus eliminating a noise component from the gray image obtained as a result of performance of step (ba3) ; and
(ba4) binarizing the gray image obtained as a result of performance of step (ba3) into a binary image, wherein step (a) is performed by locating the center of gravity in the binary image obtained as a result of performance of step (ba4).
[Claim 10]
The image composition method according to any one of claims 7 to 9, wherein step (a) is performed by performing a distance measurement operation on respective locations of the binary image using a distance measurement mask having a matrix form, and setting a location having a maximum value as a result of the distance measurement operation as the center of gravity.
[Claim 11]
An image composition apparatus for video communication terminals, comprising: a skin region searching unit for searching a camera image acquired by a camera for a skin region; an eye region searching unit for searching the camera image for an eye region by performing template matching between the skin region found by the skin region searching unit and pre-stored eye region templates; a gravity center location unit for locating a center of gravity of the eye region found by the eye region searching unit; and an image composition unit for generating a primary composite image by searching the camera image for a change region on a basis of the center of gravity located by the gravity center location unit and by replacing the change region with a prepared character image, and for generating a secondary composite image by searching the primary composite image for a return region and by replacing the return region with the camera image.
[Claim 12]
The image composition apparatus according to claim 11, wherein the character image includes a character part and a black or white part distinguished from the character part, and the black or white part is the return region.
[Claim 13]
The image composition apparatus according to claim 12, wherein the image composition unit corrects a defective region in the secondary composite image.
[Claim 14]
The image composition apparatus according to claim 13, wherein the image composition unit replaces the defective region with the camera image.
[Claim 15]
The image composition apparatus according to any one of claims 11 to 14, wherein the camera acquires a personal image which includes a target person to be combined with the character image, and a background image which does not include the person, the image composition apparatus further comprising: a gray image conversion unit for converting the camera images acquired by the camera into gray images! a subtraction unit for performing a first subtraction operation of receiving the personal image and the background image from the gray image conversion unit and subtracting gray scale values of the background image from gray scale values of the personal image, obtaining a first subtraction image by adding a maximum value of a gray scale range to a negative value generated from the first subtraction operation, performing a second subtraction operation of subtracting the gray scale values of the personal image from the gray scale values of the background image, and obtaining a second subtraction image by adding the maximum value of the gray scale range to a negative value generated from the second subtraction operation; and an addition unit for performing an addition operation on the first subtraction image and the second subtraction image received from the subtraction unit, thus obtaining a binary image, wherein the skin region searching unit searches for the skin region using the binary image.
[Claim 16]
An image composition method for video communication terminals, the image composition method being performed by a video communication terminal equipped with an image composition apparatus for combining a camera image, acquired by a camera, with a prepared character image, comprising the steps of:
(a) searching the camera image acquired by the camera for a skin region;
(b) searching the camera image for an eye region by performing template matching between the skin region found at step (a) and pre-stored eye region templates;
(c) locating a center of gravity of the eye region found at step (b) ;
(d) searching the camera image for a change region on a basis of the center of gravity located at step (c) and replacing the change region with the character image, thus generating a primary composite image; and
(e) searching the primary composite image generated at step (d) for a return region and replacing the return region with the camera image, thus generating a secondary composite image.
[Claim 17] The image composition method according to claim 16, wherein: the character image includes a character part and a black or white part distinguished from the character part, and at step (e), the searching for the return region is a procedure for searching for the black or white part. [Claim 18]
The image composition method according to claim 17, further comprising the step of (f) correcting a defective region in the secondary composite image generated at step (e). [Claim 19]
The image composition method according to claim 18, wherein step (f) is performed by replacing the defective region with the camera image. [Claim 20]
The image composition method according to any one of claims 16 to 19, wherein the camera acquires a personal image which includes a target person to be combined with the character image, and a background image which does not include the person, the image composition method further comprising the steps of:
(bal) converting the camera images acquired by the camera into gray images;
(ba2) performing a first subtraction operation of subtracting gray scale values of the background image from gray scale values of the personal image, and obtaining a first subtraction image by adding a maximum value of a gray scale range to a negative value generated from the first subtraction operation;
(ba3) performing a second subtraction operation of subtracting the gray scale values of the personal image from the gray scale values of the background image, and obtaining a second subtraction image by adding the maximum value of the gray scale range to a negative value generated from the second subtraction operation; and
(ba4) performing an addition on the first subtraction image and the second subtraction image, thus obtaining a binary image, wherein step (a) is performed by searching for the skin region using the binary image obtained at step (ba4).
PCT/KR2008/006754 2007-11-21 2008-11-17 Apparatus and method for complexing image in visual communication terminal WO2009066910A2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
KR10-2007-0119382 2007-11-21
KR1020070119382A KR100948369B1 (en) 2007-11-21 2007-11-21 Apparatus and method for complexing image in visual communication terminal
KR1020080015409A KR100948370B1 (en) 2008-02-20 2008-02-20 Apparatus and method for complexing image in visual communication terminal
KR10-2008-0015409 2008-02-20

Publications (2)

Publication Number Publication Date
WO2009066910A2 true WO2009066910A2 (en) 2009-05-28
WO2009066910A3 WO2009066910A3 (en) 2009-07-30

Family

ID=40667972

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2008/006754 WO2009066910A2 (en) 2007-11-21 2008-11-17 Apparatus and method for complexing image in visual communication terminal

Country Status (1)

Country Link
WO (1) WO2009066910A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9398251B2 (en) 2011-04-19 2016-07-19 Samsung Electronics Co., Ltd Apparatus and method for compositing image in a portable terminal

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060100969A (en) * 2005-03-16 2006-09-22 엘지전자 주식회사 Methode and apparatus for composing pictures by camera built-in terminal engaged
KR100672748B1 (en) * 2005-12-14 2007-01-22 (주)티아이스퀘어 Method and apparatus for transmitting multimedia contents items using a terminal during video communication
KR20070025449A (en) * 2005-09-02 2007-03-08 주식회사 케이티 Method and system for video communication service using real-time image effecting

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060100969A (en) * 2005-03-16 2006-09-22 엘지전자 주식회사 Methode and apparatus for composing pictures by camera built-in terminal engaged
KR20070025449A (en) * 2005-09-02 2007-03-08 주식회사 케이티 Method and system for video communication service using real-time image effecting
KR100672748B1 (en) * 2005-12-14 2007-01-22 (주)티아이스퀘어 Method and apparatus for transmitting multimedia contents items using a terminal during video communication

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9398251B2 (en) 2011-04-19 2016-07-19 Samsung Electronics Co., Ltd Apparatus and method for compositing image in a portable terminal
US9792883B2 (en) 2011-04-19 2017-10-17 Samsung Electronics Co., Ltd Apparatus and method for compositing image in a portable terminal
US10482849B2 (en) 2011-04-19 2019-11-19 Samsung Electronics Co., Ltd Apparatus and method for compositing image in a portable terminal

Also Published As

Publication number Publication date
WO2009066910A3 (en) 2009-07-30

Similar Documents

Publication Publication Date Title
CN102884536B (en) The colour of skin and feature detection for video conference compression
KR100997061B1 (en) Video frame motion-based automatic region-of-interest detection
KR101194133B1 (en) Image processing method, display image processing method, image processing device, image processing program, and integrated circuit containing the image processing device
US8606037B2 (en) Techniques to improve contrast enhancement
JP2008109669A (en) Display device and method of improving flicker of image
CN101620679B (en) Method for eliminating red eye in image
WO2007024407A1 (en) Techniques to improve contrast enhancement using a luminance histogram
CN107846554B (en) Image processing method, terminal and computer readable storage medium
US20200234044A1 (en) Image Signal Processing Method, Apparatus, And Device
CN107920213A (en) Image synthesizing method, terminal and computer-readable recording medium
CN108337465A (en) Method for processing video frequency and device
CN105979195B (en) A kind of video image processing device and method
CN101317185B (en) Video sensor-based automatic region-of-interest detection
JP5003488B2 (en) Video communication system, terminal, and image conversion apparatus
CN108200421B (en) White balance processing method, terminal and computer readable storage medium
WO2009066910A2 (en) Apparatus and method for complexing image in visual communication terminal
US9196029B2 (en) Threshold setting device for setting threshold used in binarization process, object detection device, threshold setting method, and computer readable storage medium
CN108280813A (en) A kind of image processing method, terminal and computer readable storage medium
KR100948369B1 (en) Apparatus and method for complexing image in visual communication terminal
KR100958595B1 (en) apparatus and method for complexing image in visual communication terminal
CN108205795A (en) Face image processing process and system during a kind of live streaming
CN111160267A (en) Image processing method, terminal and storage medium
WO2019052449A1 (en) Skin color recognition method and apparatus, and storage medium
KR100948370B1 (en) Apparatus and method for complexing image in visual communication terminal
CN109783178A (en) A kind of color adjustment method of interface assembly, device, equipment and medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08852950

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase in:

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 14/09/10)

122 Ep: pct application non-entry in european phase

Ref document number: 08852950

Country of ref document: EP

Kind code of ref document: A2