KR20100059493A - Image synthesis system and image synthesis method - Google Patents

Abstract

PURPOSE: An image synthesis system and an image synthesis method are provided to synthesize a specific body portion of a user with composition object in photographed background image in real time, thereby arousing interest of the user. CONSTITUTION: An image synthesis server(410) detects a specific body part for composition in a video of specific body part of a user. The image synthesis server synchronizes a synthesis target in a selected background image and the specific body part. The image synthesis server composes a synchronized image of the specific body part and the composition object of the background image in real time. A background image DB(420) stores background images.

Description

IMAGE SYNTHESIS SYSTEM AND IMAGE SYNTHESIS METHOD}

The present invention relates to an image synthesizing system and a method thereof, and more particularly, to a system and a method for synthesizing in real time by synchronizing an image of a specific body part of a user with a synthesis target in a pre-made background image.

Chroma key is a technique that can remove a certain series of color from the background and synthesize it with other images. It is a technique used when composing a background image prepared in advance for a video of a photographed subject when producing a TV broadcast image.

FIG. 1 is a diagram for describing an image synthesis method using a chroma key technique according to the prior art.

Referring to FIG. 1, in the image synthesis method using the chroma key technique, a background image is produced, and then an image of a subject to be synthesized with the background image is captured. At this time, when photographing a subject, a photograph is taken with a blue screen as a background.

Subsequently, the image of the subject whose blue screen is removed using the chroma key method is synthesized from the image of the subject taken with the blue screen as a background with the previously produced background image.

However, in the conventional image synthesis method using the chroma key described above, the whole body or the specific body part of the subject synthesized with the chroma key appears to overlap on the background image, so that the background image and the image of the subject are unnatural. There was a problem of synthesis.

On the other hand, as a prior art, Korean Patent No. 10-0743352 discloses a "movie synthesizing vending machine" for automatically synthesizing an image of a specific body part captured by a user with animation content.

However, the prior art is a technique of synthesizing a user's face image captured by a camera with an animation in which a face has been previously removed, so that the still face image of the user synthesized in the animation has a uniform expression regardless of the story development. Therefore, there is a problem that the completeness of the 3D animation is poor.

In addition, the prior art has a problem in that the use of the video vending machine is available only in the place where the video vending machine is installed in order to synthesize the face image of the user photographed with the camera, the use is limited.

An embodiment of the present invention provides an image synthesizing system and method for synthesizing a body part of a moving photographic subject and a body part of a model in a pre-produced background image in real time.

As a technical means for achieving the above technical problem, the first aspect of the present invention, in the image synthesis method, (a) providing a user terminal with a selection of a background image for synthesizing an image of a specific body part of the user; (b) receiving, from the user terminal, a user image capturing the user's appearance using a camera with a blue screen as a background; and (c) receiving the user image from the user terminal received from the user terminal. Detecting a specific body part, and (d) synthesizing an image of the specific body part of the user to the synthesis area in the background image in real time by synchronizing the detected body part of the user with the synthesis area in the background image; And (d) providing a synthesized image obtained by synthesizing an image of a specific body part of the user to the user terminal. Provides an image combining method comprising the steps:

As a technical means for achieving the above technical problem, the second aspect of the present invention, in the image synthesis system, from a background image DB and a user terminal for storing a plurality of background images for synthesizing the image of a specific body part of the user Receives an image of a specific body part of the user, detects only a specific body part of the user to be synthesized from the image of the specific body part of the user and synchronizes with the synthesis object in the background image selected by the user, the synchronization It provides an image synthesis system comprising a composite image server for synthesizing the image of the specific body part of the user and the synthesis target in the background image in real time to the user terminal.

As a technical means for achieving the above technical problem, the third aspect of the present invention is a song accompaniment device capable of synthesizing images,

A data storage for storing accompaniment sound for outputting the accompaniment of the song selected by the user and music video data displayed in correspondence with the song selected by the user, a key input unit for receiving a selection of a song from the user, and a microphone A voice input unit for receiving the user's song voice through the voice input unit, a voice output unit for mixing the accompaniment sound of the song selected by the user to the user's song voice input through the voice input unit, and outputting it through a speaker; An image input unit for receiving a user's image of the user singing a real song from a camera, and removing the blue screen from the user's image received from the camera by using a chroma key technique. Synthetic region detection unit for detecting, The image synthesizer for synthesizing the face image of the user in real time to the face of the singer in the music video by synchronizing the detected face area of the user with the face of the singer in the music video, and the face image of the user is the music video. The present invention provides a song accompaniment apparatus including an image output unit configured to display a synthesized image synthesized in real time on a face of a singer on a screen.

According to one of the aforementioned problem solving means of the present invention, the user's interest may be induced by synthesizing the specific body part of the user in real time with the synthesis target in the photographed background image.

According to another one of the problem solving means of the present invention described above, the image synthesizing method can be produced and utilized not only for the entertainment industry, but also for education that can experience image synthesizing technology.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . In addition, when a part is said to "include" a certain component, this means that it may further include other components, except to exclude other components unless otherwise stated.

Hereinafter, a system and a method of synthesizing an image of a specific body part of a user on a background image in real time using a chroma key according to an embodiment of the present invention will be described with reference to FIGS. 2 to 6.

2 is a diagram illustrating a connection relationship of a system for synthesizing an image of a specific body part of a user in real time into a background image using a chroma key according to an embodiment of the present invention.

Referring to FIG. 2, the connection relationship of the image synthesizing system includes a user terminal 100, a camera 200, a network 300, and an image synthesizing system 400.

The user terminal 100 transmits the user video transmitted from the camera 200 to the image synthesis system 400 connected through the network 300. Here, the user video is taken with a blue screen using the camera 200 as a background, and the camera 200 is connected to the user terminal 100 with a universal serial bus (USB) or an IEEE 1394 cable, or the user. It may be embedded in the terminal 100.

The user terminal 100 may be implemented as a computer 110 and a portable terminal 120. In this case, the computer 110 may include a laptop, a desktop, a laptop, and the like equipped with a web browser, and the portable terminal 120 is a wireless communication device that ensures portability and mobility. Personal Communication System (PCS), Global System for Mobile communications (GSM), Personal Digital Cellular (PDC), Personal Handyphone System (PHS), Personal Digital Assistant (PDA), International Mobile Telecommunication (IMT) -2000, Code Division (CDMA) Handheld based wireless communication devices of all kinds, such as Multiple Access (2000) -2000, W-Code Division Multiple Access (W-CDMA), and Wireless Broadband Internet (WBRO) terminals.

The camera 200 may be implemented as, for example, a digital camera, and photographs an image input from a lens through an imaging device such as a CCD or a CMOS and outputs the image to the user terminal 100.

The network 300 may be a wired or mobile radio network such as a local area network (LAN), a wide area network (WAN) or a value added network (VAN), a mobile radio network, or a satellite network. It can be implemented in all kinds of wireless networks such as Bluetooth, Bluetooth, Wireless Broadband Internet (Wibro), and High Speed Downlink Packet Access (HSDPA).

The image synthesizing system 400 includes an image synthesizing server 410 and a background image DB 420, and removes a blue screen from a user image received from the user terminal 100 using a chroma key technique. The user terminal 100 detects a specific body part of the user to be used for synthesis, synchronizes the detected user's specific body part with a synthesis target in the background image, and synthesizes an image of the user's specific body part in real time with the synthesis target in the background image. )

3 is a detailed block diagram of an image synthesizing system according to an embodiment of the present invention.

Referring to FIG. 3, the image synthesizing system 400 includes an image synthesizing server 410 and a background image DB 420. The image synthesizing server 410 includes a communication network interface 411 and a background image selecting unit 412. , An image input unit 413, a synthesis region detector 414, an image synthesizer 415, and an image output unit 416.

The image synthesizing server 410 detects only a specific body part of the user to be synthesized from the image of the specific body part of the user received from the user terminal 100 and synchronizes it with the synthesis target in the background image selected by the user. The synthesis Daesung in the image of the specific body part and the background image is synthesized in real time and provided to the user terminal 100.

The background image DB 420 stores various background images to which an image of a specific body part of the user is synthesized. The background image includes various video contents such as animation, music video, movie, CF, drama, news, and sports.

The communication network interface 411 provides communication with the user terminal 100 through the network 300. The communication network interface 411 supports both a wired and wireless communication function, or only a wired communication function, but may adopt a method of substantially supporting wireless communication in connection with a mobile operator system (not shown). .

The background image selecting unit 412 provides the user terminal 100 with a list of a plurality of background images for receiving the selection of the background image to be synthesized from the user, and retrieves the background image selected by the user from the background image DB 420. Provided to the user terminal 100.

The image input unit 413 receives a user image captured by the camera 200 from the user terminal 100.

The synthesis region detector 414 removes a blue screen from the user image received from the user terminal 100 using a chroma key technique to detect a specific body part of the user, for example, a face region, to be synthesized with the background image.

The image synthesizer 415 synchronizes the face region of the user detected by the synthesis region detector 414 and the synthesis target in the background image selected by the user, and displays the face image of the user received from the user terminal 100 as the background image. It synthesizes in real time by matching the synthesis target of the genus. In this case, the synthesis object in the background image is a specific body part of the person of the person in the background image to be synchronized with the specific body part of the user to be synthesized, for example, a singer in the music video or the main character of the animation. Can be your face.

The image output unit 416 provides the synthesized image obtained by synthesizing an image of a specific body part of the user to the user terminal 100.

4 is a flowchart illustrating a method of synthesizing an image of a specific body part of a user on a background image in real time using a chroma key according to an embodiment of the present invention.

In operation S11, the image synthesizing system 400 classifies various background images for synthesizing an image of a specific body part of the user by category and stores them in the background image DB 420. The background image includes various video contents such as animation, music video, movie, CF, drama, news, and sports.

In operation S12, the user terminal 100 requests a list of background images from the image synthesis system 400 in order to select a background image to be used for synthesis. In operation S13, the image synthesis system 400 receives the user terminal ( A list of the background images requested from 100 is provided to the user terminal 100.

In step S14, the user terminal 100 displays a list of background images received from the image synthesizing system 400 on the screen, and then proceeds to step S15, where one of the list of background images is displayed by the user. If selected, the image synthesis system 400 requests the selected background image.

In operation S16, the image synthesizing system 400 retrieves the background image requested from the user terminal 100 from the background image DB 420 and provides it to the user terminal 100.

In operation S17, the user terminal 100 receives an image of a specific body part of the user photographed using the camera 200 from the camera 200, and then proceeds to operation S18 and captures the specific body of the user. The image of the part is transmitted to the image synthesis system 400. Here, the image of the specific body part of the user is photographed against a blue screen.

In operation S19, the image synthesizing system 400 removes a blue screen from an image of a specific body part of the user received from the user terminal 100 using a chroma key technique, and thus, a specific body part of the user to be used for synthesis. For example, only the face area is detected.

In step S20, the image synthesizing system 400 synchronizes the specific body part of the user detected in step S19 with the synthesis target in the background image, and then proceeds to step S21, where the specific body part of the synchronized user is synchronized. Video is synthesized into the synthesis target in the background image.

In operation S22, the image synthesizing system 400 provides a synthesized image obtained by synthesizing a specific body part of the user to the user terminal 100, and in operation S21, the user terminal 100 generates an image of the specific body part of the user. The synthesized background image is received from the image synthesizing system 400 and displayed on the screen.

5 is a diagram for describing a song accompaniment apparatus to which an image synthesis method is applied, according to an exemplary embodiment.

Referring to FIG. 5, the song accompaniment apparatus 500 according to an exemplary embodiment of the present invention may include a camera interface 510, a data storage unit 520, a key input unit 530, a voice input unit 540, and a voice output unit ( 550, an image input unit 560, a synthesis region detector 570, an image synthesizer 580, an image output unit 590, and a communication network interface 595.

The camera interface 510 provides communication with an external device such as the camera 200.

The data storage unit 520 may include accompaniment sound data for outputting the accompaniment of the song selected by the user, image data such as a music video displayed by matching the song selected by the user, and lyrics data displayed together with the image data. Stores song information of various songs to include.

The key input unit 530 receives various key input signals related to song selection, setting, and searching, and various key input signals for synthesizing a user's face image with a singer's face in a music video.

The voice input unit 540 receives a user's song voice through a microphone.

The voice output unit 550 mixes the accompaniment sound of the corresponding song to the song voice of the user input through the voice input unit 540 and outputs it through the speaker.

The image input unit 560 receives a user image from the camera 200 in which a picture of a user who is actually singing is photographed. At this time, the user's video is taken with the blue screen installed at a predetermined position in the karaoke.

The synthesis region detector 570 removes the blue screen from the user image received from the camera 200 using a chroma key technique to detect the face region of the user to be synthesized with the face of the singer in the music video.

The image synthesizer 580 synchronizes the face region of the user detected by the synthesis region detector 570 with the face region of the singer in the music video, and converts the face image of the user received from the camera 200 into the singer in the music video. Match faces and synthesize in real time.

The image output unit 590 displays a synthesized image of the user's actual singing face image synthesized in real time on the face of the singer in the music video on the screen.

The communication network interface 595 provides communication with a portable terminal, a computer, an external storage medium, a content providing server, and the like, and may be implemented by USB, infrared communication, Bluetooth, or the like. The communication network interface 595 supports both a wired and wireless communication function or only a wired communication function, but may adopt a method of substantially supporting wireless communication in connection with a mobile operator system (not shown). . For example, by using the communication network interface 595, the synthesized image obtained by synthesizing the face image of the user may be transmitted to the portable terminal of the user or the portable terminal of another person through the mobile communication service provider system. In addition, by using the communication interface 595, a synthesized image obtained by synthesizing a user's face image may be stored in a storage medium such as a CD, a DVD, a USB memory, or a flash memory through USB, infrared communication, or Bluetooth. have.

As described above, the song accompaniment apparatus 500 according to an embodiment of the present invention outputs the accompaniment of the selected song through the speaker when the user selects a predetermined song by using the selection key of the key input unit 530. do. In addition, the song accompaniment apparatus 500 synthesizes and outputs in real time a face image of a user who sings a real song to a face of a singer in a music video through a camera 200 installed in a karaoke room.

6 is a flowchart illustrating a method of synthesizing an image using a song accompaniment device according to an embodiment of the present invention.

In operation S31, the song accompaniment apparatus 500 selects a song from the user, and then proceeds to operation S32, and the accompaniment sound data of the song selected by the user and a music video matched to the song are stored in the data storage unit. Search from 520.

In operation S33, the song accompaniment apparatus 500 mixes the searched song accompaniment sound with the user's song voice received through the microphone and outputs it through the speaker.

In operation S34, when the song accompaniment apparatus 500 receives a user image of an actual song from the camera 200, the song accompaniment apparatus 500 proceeds to operation S35, and the user image received from the camera 200 using a chroma key technique. Detect the user's face area to be synthesized by removing the blue screen.

In operation S36, the song accompaniment apparatus 500 synchronizes the detected face region of the user with the face region of the singer in the music video, and then proceeds to step S37 to convert the synchronized user's face image into the music artist in the music video. On your face.

In operation S38, the song molecular apparatus 500 displays a music video on which a user's actual singing face image is synthesized.

One embodiment of the present invention can also be implemented in the form of a recording medium containing instructions executable by a computer, such as a program module executed by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery media.

The foregoing description of the present invention is intended for illustration, and it will be understood by those skilled in the art that the present invention may be easily modified in other specific forms without changing the technical spirit or essential features of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is shown by the following claims rather than the above description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

1 is a view for explaining an image synthesis method using a chroma key technique according to the prior art.

2 is a diagram illustrating a connection relationship of a system for synthesizing an image of a specific body part of a user in real time into a background image using a chroma key according to an embodiment of the present invention.

3 is a detailed block diagram of an image synthesizing system according to an embodiment of the present invention.

4 is a flowchart illustrating a method of synthesizing an image of a specific body part of a user on a background image in real time using a chroma key according to an embodiment of the present invention.

5 is a diagram for describing a song accompaniment apparatus to which an image synthesis method is applied, according to an exemplary embodiment.

6 is a flowchart illustrating a method of synthesizing an image using a song accompaniment device according to an embodiment of the present invention.

Claims (7)

In the image synthesis method, (a) providing a user terminal with a selection of a background image for synthesizing an image of a specific body part of the user; (b) receiving a user image of the user from the user terminal by using a camera with a blue screen as a background; (c) detecting a specific body part of the user to be used for synthesis in the user image received from the user terminal; (d) synthesizing the image of the specific body part of the user to the synthesis area in the background image by synchronizing the detected body part of the user with the synthesis area in the background image; (d) providing a synthesized image obtained by synthesizing an image of a specific body part of the user to the user terminal; Image synthesis method comprising a. The method of claim 1, The detecting of the specific body part of the user may include detecting a specific body part of the user to be synthesized by removing the blue screen from the user image received from the user terminal using a chroma key technique. Image Synthesis Method. In an image synthesis system, Background image DB for storing a plurality of background images for synthesizing the image of a specific body part of the user and Receiving an image of a specific body part of the user from a user terminal, detecting only a specific body part of the user to be synthesized from the received image of the specific body part of the user, and synchronizing with a synthesis object in the background image selected by the user, Synthetic image server for synthesizing the image of the specific body part of the synchronized user and the synthesis target in the background image in real time to the user terminal Image synthesis system comprising a. The method of claim 3, wherein The video synthesis server, Background image selection unit for providing a selection of the background image to the user terminal, Image input unit for receiving a user image from the user terminal by taking a picture of the user using a camera on a blue screen background, A synthesis region detector for detecting a specific body part of the user to be synthesized from the received user image; An image synthesizing unit synchronizing the detected body part of the user with the synthesis object in the background image, and synthesizing the image of the specific body part of the user synchronized with the synthesis object in the background image in real time; An image output unit for providing a composite image synthesized with the image of the specific body part of the user in the background image to the user terminal Image synthesis system comprising a. The method of claim 4, wherein The synthesis region detector, And detecting only a specific body part of the user to be synthesized by removing the blue screen from the user image received from the user terminal using a chroma key technique. In the song accompaniment device that can synthesize video, A data storage unit for storing accompaniment sound data for outputting the accompaniment sound of the song selected by the user and music video data that is displayed in correspondence with the song selected by the user; A key input unit for receiving a selection from the user, Voice input unit for receiving the user's song voice through the microphone, A voice output unit for mixing the accompaniment sound of the song selected by the user with the voice of the user's song input through the voice input unit and outputting it through a speaker; Image input unit for receiving a user image from the camera taking a picture of the user singing the actual song on the blue screen background, A synthesis region detector for detecting a face region of the user by removing the blue screen from the user image received from the camera using a chroma key technique; An image synthesizer configured to synchronize the detected face area of the user with the face of the singer in the music video, and synthesize the face image of the user in real time with the face of the singer in the music video; An image output unit configured to display a synthesized image of the user's face image synthesized in real time on the face of the singer in the music video on a screen; Song accompaniment device comprising a. The method of claim 6, Communication network interface for transmitting the synthesized image synthesized with the user's face image to an external storage medium, portable terminal and content providing server Song accompaniment device that further comprises.

Images