KR20140041998A - Method and apparatus for chromakey compositing of subject image and background image - Google Patents

Abstract

The present invention relates to a method and an apparatus for synthesizing a chromakey processed subject image and background image in an image synthesizer. The purpose of the present invention is to automatically adjust perspective on a background by controlling the scale of a subject image to be suited for a background image when the image synthesizer synthesizes the chromakey processed subject image and background image. The apparatus of the present invention includes: an external input device consisting of a camera, a microphone, and an infrared sensor; an image processing device consisting of a chromakey processing unit for processing a subject image using a chromakey and an image synthesizing unit for enlarging or reducing the size of a subject image at a predetermined position of a background image at a predetermined ratio; a voice processing device consisting of a voice synthesizing unit for synthesizing a singing voice signal of a user and accompaniment sound and a voice output unit for outputting sing and accompaniment sound; a control unit for controlling the recognition of a subject according to a signal inputted from the infrared sensor, and image synthesis and voice synthesis according to the selection of the user through a key input unit; an external output device consisting of a speaker for outputting voice under the control of the control unit, an image output unit for outputting an image where a subject image, in which the subject is processed using the chromakey, and a background image and a foreground image are synthesized, a mixer unit for storing sing and accompaniment sound, outputted through the speaker, and a synthesis image, and an external output unit for transmitting voice and image data by wireless; and a background and foreground image storage unit for classifying and storing a background image and a foreground image according to categories. [Reference numerals] (AA) Start; (BB) Obtain a subject video; (CC) Processing an image using a chromakey; (DD) Background image selected?; (EE,KK,OO,UU) No; (FF,LL,PP,TT) Yes; (GG) Allow a user to select a background image; (HH) Access the background image, and the synthesis position and size information of a subject; (II) Synthesize the subject image and the background image; (JJ) Path of the subject image moved?; (MM) Synthesize the subject image at a predetermined position of the background image at a predetermined size; (NN) Size of the subject image changed?; (QQ) Synthesize the subject image and the background image while moving and changing the position and size of the subject image on the background image at predetermined time intervals; (RR) Synthesize the subject image and the background image while moving and changing the position of the subject image on the background image at predetermined time intervals; (SS) Foreground image selected?; (VV) Synthesize the foreground image, the subject image, and the background image; (WW) Output the synthesis image, the voice of the user, and accompaniment sound in synchronization with one another; (XX) End

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for synthesizing a chroma key image of a video synthesizer,

In the present invention, when a video synthesizer is used, a perspective image is automatically or automatically adjusted so as to match a background image by adjusting the scale of the subject image so that the subject image in the chroma key processing of the subject and the background image are combined with the background video / The present invention relates to a chroma key object image and a background image synthesizing apparatus and method of a video synthesizer capable of manually manipulating a size.

Generally, in the image synthesis method using the chroma key technique, a specific color is keyed in a movie in which a subject is photographed so as to be edited as a single image by synthesizing a pre-set background image (a moving image or a photo image) The pixel corresponding to the key in the main image is replaced with another image.

A conventional apparatus and method for producing a video material using a chroma key are disclosed in Korean Patent Registration No. 10-0934471.

FIG. 1 is a block diagram of an apparatus for producing a video material using a chroma key according to the related art. FIG. 1 is a block diagram of a video material production apparatus using chroma key according to the related art, in which content control information is included in content information and chroma key is used to produce video content System.

The video material production system using the chroma key includes a background screen 10, a camera 20, an infrared sensor 30, a microphone 40, a portable terminal 50, and an image production apparatus 60.

The background screen 10 is installed behind the subject model at the time of capturing a moving image and uses a monochromatic background so that the image can be edited by a chroma key technique afterwards. Generally, a blue screen is applied to a blue screen .

The camera 20 photographs a subject, converts it into a digital video signal, and records the converted video signal. In order to photograph a subject at various angles, a plurality of cameras can be installed.

The infrared sensor 30 is an apparatus for detecting a physical quantity or a stoichiometric quantity such as temperature, pressure, and intensity of radiation using infrared rays and converting it into an electric quantity capable of signal processing, The infrared ray is emitted to the corresponding part of the background screen 10 and the detection signal is transmitted to the image producing apparatus 60 when the user's motion is detected.

The infrared sensor 30 may be integrated with the camera 20 or may be an independent device.

The microphone 40 converts a voice signal of a user into an electric signal, and is a means provided for recording a voice of a user together with the image photographed through the camera 20.

The portable terminal 50 may be a terminal such as a personal computer (PC), and is connected to a web server on which content information including the content control information is stored through a communication network.

The portable terminal 50 may request and receive the corresponding content information upon request from the user and provide the content information to the image production apparatus 60. The content information produced by the video production apparatus 60 is simultaneously stored in three files, that is, an ASF (Active Streaming Format) file, an MP4 file, and an image file, so that the MP4 player (video player) CD albums, and portable storage devices.

The image production apparatus 60 performs an overall function for producing a video image using the chroma key. If the content is selected by the user while the content type information is displayed, the video production apparatus 60 analyzes the content information and outputs the content information.

Here, the content information includes content content information (e.g., background sound content information, effect sound source content information, effect video content information, background video content information and content control information, that is, a sound packet signal (SPS) (Motion Packet Signal), MPS (Movie Packet Signal), MCPS (Movie Control Packet Signal), and / or alpha channel information And image information.

A first sound output unit for outputting the sound information generated by the first sound information generating unit and the second sound information generated by the second sound information generating unit, Layer image by combining the alpha channel information and the background image (effect image) information, and when the third behavior control generation signal is sensed, the infrared sensor 30 is driven, Upon receiving the motion detection signal from the sensor 30, the sound source content information and / or the effect video content information matched in advance to the behavior control occurrence signal is extracted and output. It also includes a function of mixing together the sound signal input from the microphone 40 or the sound to be used as background music.

When the output content is finished, the produced content information is simultaneously stored in three files: an ASF file, an MP3 file, and an image file. Also, the content information can be stored in real time while being produced.

Thus, in order to adjust the perspective of the synthesized chroma key image and the background image, a device for producing a video material using a chroma key according to the related art requires a skilled expert capable of artificially manipulating the program using the program, There is a problem in that it is not possible to apply natural image synthesis with perspective.

Accordingly, the present invention has been made in view of the above problems, and it is an object of the present invention to provide a method and apparatus for adjusting the scale of a subject image in accordance with a background moving image / a photograph image in synthesizing a subject moving image and a background moving image / And an object of the present invention is to provide an apparatus and method for synthesizing a chroma key object image and a background image of an image synthesizer that automatically adjusts the perspective of an image.

It is another object of the present invention to provide a method of synthesizing a perspective image of a chroma key image and a background image, which moves the chroma key-processed subject image to a predetermined area while maintaining a background image and perspective.

In order to accomplish the object of the present invention, a chroma key object image and a background image synthesizer of a video synthesizer combine a chroma key processed subject, a background image, and a foreground image in a perspective manner, and synthesize a song and an accompaniment sound source A chroma key object image and background image synthesizing apparatus of an image synthesizer for outputting a chroma key image and a background image synthesizing apparatus, comprising: a camera for acquiring a moving image of a subject; a microphone for acquiring a voice signal of a song called by the subject; and an infrared sensor Device; A chroma key processing unit for performing chroma key processing on the subject image input through the camera; and a synthesizer for synthesizing the subject image processed by the chroma key processing unit with the stored background image, wherein the size of the subject image is enlarged The image processing apparatus comprising: A voice processing unit composed of a voice synthesizing unit for synthesizing a user's singing voice signal and an accompaniment sound of the microphone, and a voice output unit for outputting a song and an accompaniment sound of the voice synthesizing unit; A control unit for recognizing a subject according to a signal input from the infrared sensor, and controlling composition according to selection of a user's key input unit for image synthesis and speech synthesis; A speaker for outputting sound by controlling the control unit, an image output unit for outputting a composite image of a chroma key processed image of the subject and a background image, a foreground image, and a song, An external output device including a mixer unit and an external output unit for transmitting voice and image data stored in the mixer unit as wireless data; And a background image and a foreground image storage unit for classifying and storing the background image and the foreground image by category and providing the background image and the foreground image to the image synthesizing unit.

Here, the image synthesizer may move the composite position of the chroma key-processed object image and the background image, and enlarge or reduce the object image at a predetermined magnification according to the movement position.

In order to accomplish the object of the present invention, a chroma key object image and a background image composition method of a video synthesizer combine a subject obtained by chroma key processing of a moving image obtained by a camera, a background image and a foreground image stored in an image database in perspective, A method of synthesizing a chroma key image and a background image of a video synthesizer for storing a song and an accompaniment sound source that are played by a user together with a video, the method comprising: synchronizing a moving picture of a subject obtained through the camera and a voice signal input through a microphone, A first step of storing the data in the memory; A second process of performing a chroma key process and a size of a subject on a moving image of a subject obtained through the camera; A third process of calculating the size of the subject by subjecting the subject image subjected to the chroma key processing, the background image selected by the user, and the subject in the image processing unit; And a fourth step of outputting a composite image in the third step and outputting a user's voice and an accompaniment sound synchronized with the synthesized image through a speaker.

Here, the third step may include providing a visual sense of perspective by synthesizing the subject image with the size and position of the subject set in the background image selected by the user, wherein the subject image is positioned at a predetermined time interval along the movement path set in the background image To move.

And the size of the moving picture of the subject is also converted into a predetermined ratio as the position of the subject image moves in the background image.

The chroma key object image and the background image synthesizer and method of the image synthesizer according to the present invention are synthesized in accordance with the size of the object image set for each background image in synthesizing the object image with the chroma key in the image synthesizer, There is an effect of providing the naturalness that the person who is singing is actually in the background image and the size of the image of the subject is automatically changed according to the movement route set in the background image, It has the same effect as watching music video singing while moving within the screen.

Also, when synthesizing a foreground image such as an eye, a rain, a star, etc. on the front side of a composite image of a background image and a subject image, the size of the foreground image is automatically changed according to the position of the subject image. .

FIG. 1 is a block diagram of an apparatus for producing a video material using a chroma key according to the prior art,
FIG. 2 is a block diagram of a chroma key object image and a background image synthesizer of the image synthesizer according to the embodiment of the present invention,
3 is a flowchart of a chroma key object image and background image synthesis process of the image synthesizer according to the embodiment of the present invention,
FIG. 4 is a diagram illustrating an example of displaying a position and a size of a subject image and a background image synthesized according to the present invention,
5A and 5B are views showing an embodiment of an image produced according to an embodiment of the present invention.

A chroma key object image and a background image composing method and apparatus of an image synthesizer according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 2 is a block diagram of a chroma key image and a background image synthesizer of a video synthesizer according to an embodiment of the present invention. When a user calls a song using a video synthesizer, a subject video camera 111, an infrared sensor 112 for sensing the user, and a microphone 113 for acquiring a voice signal of a song called by the user, and an external input device for inputting a subject image input through the camera 111, A chroma key processing unit 114 for processing the chroma key processed by the chroma key processing unit 114, a foreground image storage unit 115 (116) for categorizing and storing the background image and the foreground image, An image synthesizing unit 117 synthesizing the background image with the stored background image and enlarging or reducing the size of the object image at a set position of the background image at a predetermined ratio, And a video output unit 118 for outputting a moving picture synthesized by the video synthesis unit 117. The audio processing unit includes an accompaniment tone storage unit 121 for storing an accompaniment tone, A voice synthesizer 122 for synthesizing the accompaniment notes of the accompaniment tone storing unit 121 and a voice output unit 123 for outputting a song and an accompaniment tone of the voice synthesizer 122; A control unit 132 for recognizing a subject according to a signal input from the infrared sensor 112, and controlling image synthesis and speech synthesis according to a selection of a user's key input unit 131; A video output unit 118 for outputting a composite image of a chroma key processed image of the subject and a background image and a foreground image, a mixer unit 131 for synchronously storing a song and an accompaniment sound output from the speaker, And an external output unit 132 for transmitting voice and images stored in the mixer unit 119 as wireless data.

The background image storage unit 115 stores a plurality of background images (moving images or photo images) for various categories and provides the background images to users. In each background image, the size of the subject is set to be natural so that the subject And includes data in which the position of the object image is set in the background image, and the size of the object image is set in each of the positions.

In addition, each background image of the background image storage unit 115 is set to have a size of the foreground image so that the foreground image, the background image, and the subject image are natural when the user selects the foreground image.

The operation of the present invention will now be described in detail with reference to FIGS. 2 to 5. FIG.

First, when a user synthesizes his or her singing image and a selected background image by using a video synthesizer, the user who is singing is allowed to synthesize the perspective in a natural manner according to the background image. As a result, Synthesized.

FIG. 3 is a flowchart of a chroma key object image and a background image synthesis process of the image synthesizer according to an embodiment of the present invention. The user manipulates the image synthesizer through the key input unit 131 to select a song, a background image, And starts the operation of the image synthesizer to produce a moving image.

The background image displays a plurality of images stored in the background image storage unit 115 on the screen, and selects the foreground image by displaying various foreground images stored in the foreground image storage unit 116.

As the image synthesizer starts its operation, it acquires a moving picture of a subject through the camera 111.

The moving image obtained through the camera 111 is stored, and after the chroma key image processing is performed using the filter, only the subject image is extracted and stored.

The process of extracting the subject image obtained through the chroma key image processing of the subject moving image obtained through the camera 111 may be performed in real time or may be performed by delaying a predetermined time.

The control unit 132 recognizes the subject input from the infrared sensor 112 and determines the size of the recognized subject based on the size of the recognized subject .

The calculation of the size of the subject calculates the height of the subject in the number occupied by the vertical pixels of the input screen.

For example, when the input screen is '640 * 480 (pixcell)', the pixel is converted into 'cm pixel' to be '96 pixcell = 25.4 mm', and the height of the subject can be calculated as shown in Equation 1 below.

That is, an example in which the size is calculated using Equation (1) is shown in Table 1 below.

division Example 1 Example 2 Example 3 Remarks fixing Focal length 160 160 160 Auto focus fixing Shooting distance 2500 2500 2500 variable Number of pixels in pixels 250 120 66 VGA grade, up to 480p in portrait Request value Subject height 3,906 1,875 1,034 Unit: mm

The control unit 132 controls the image synthesizer 117 to synthesize the subject image and the background image.

At this time, the synthesized position and size are preset so that the perspective image becomes natural when the subject image to be synthesized is synthesized with the background image.

The size of the subject image is determined by metadata set in a fixed or subject size ratio for each background image.

In this manner, the image synthesizer 117 analyzes the metadata included in the background image and synthesizes the metadata at the corresponding position of the background image according to the set position and size of the set subject image.

That is, as shown in FIG. 4, the position and size of the background image are displayed. In the background image 101, the subject composition area 102 data set for the position and size of the subject image 100 is displayed on the background image Is stored in the metadata of the storage unit 115 so that the image synthesis unit 117 synthesizes the subject image 100 in the subject synthesis region 102 to which the subject size and position data set in the background image 101 are applied ≪ / RTI >

After the background image 101 and the subject image 100 are synthesized, the subject image 100 can be moved to a predetermined path in the background image. That is, an image such as a user singing while moving from the actual position of the background image 101 can be produced through the image synthesis process.

5A shows an embodiment for explaining the positional shift of an object image. The image synthesizing unit 117 synthesizes a composite image 102 of a subject image from a left side to a right side of the background image 101 at a predetermined interval As shown in FIG. As if the user is actually moving the position while singing in the field of the background image.

The movement position data is provided as metadata in the background image.

FIG. 5B is a diagram illustrating an embodiment for explaining the process of automatically moving the position and size of the subject image. When the background image 101 is a road, the subject image is moved to a position close to the far side of the road The position and size of the subject composition area 102 are synthesized while being automatically converted, so that the subject image can be produced as if it is singing while moving at a set time interval instead of a fixed position of the background image, The size is also automatically converted according to the position, so that perspective can be maintained, allowing the composite image to feel natural.

As described above, in addition to the combination of the movement path of the subject image 101 moving backward and backward on the background image, the size of the subject image in the case of moving the composite position from the front to the back is also changed depending on the metadata of the background image And automatically synthesizes the moved and reduced image of the subject.

The positional change of the position and size of the composite region 102 of the subject synthesized in the background image 101, the size of the subject composite region at each position, and the movement speed data of the composite region are stored in the background image storage unit 115, And is stored as metadata in each background image.

If foreground image synthesis such as eye, ratio, etc. is selected in the image synthesizer 117, a foreground image (not shown in the drawings) selected on the foreground of the background image 101 and the subject image 100 is divided into three layers And outputs the synthesized image to the screen through the image output unit 118.

The synthesized image obtained by synthesizing the background image, the subject image, and the foreground image is outputted through the image output unit 118 and the synthesized image synthesized through the speech synthesis unit 122 under the control of the control unit 132. [ And the accompaniment sound are output through the sound output unit 123 in synchronization with the subject composite image.

The synthesized video and audio signals synchronously output from the video output unit 118 and the audio output unit 123 are stored in the mixer unit 131 and transmitted to the external device through the external output unit 132 as wired or wireless data.

In other words, it is stored in a CD album, a USB memory, or an SD memory card through the external output unit 132, transmitted online, uploaded to the homepage, transmitted via e-mail, or transmitted to a mobile phone for storage.

  Although specific embodiments of the chroma key object image and the background image synthesizing apparatus and method of the image synthesizer of the present invention have been described and shown in the above, a method of determining the position and size for synthesizing the object image in the background image of the present invention It is obvious that various modifications can be made by those skilled in the art.

It should be understood, however, that such modified embodiments are not to be understood individually from the spirit and scope of the invention, and such modified embodiments are intended to be included within the scope of the appended claims.

100: subject 101: background image
102: subject composing area 111: camera
112: Infrared sensor 113: Microphone
114: chroma key processing unit 115: background image storage unit
116: foreground image storage unit 117:
118: Video output unit 121: Accompaniment sound storage unit
122: voice synthesizer 123: voice output unit
131: mixer unit 132: external output unit

Claims (10)

A chroma key object image and a background image synthesizer of a video synthesizer for synthesizing a chroma key processed subject, a background image, and a foreground image in a perspective manner, and outputting a song and an accompaniment sound source,
An external input device comprising a camera for acquiring a moving picture of the subject, a microphone for acquiring a voice signal of a song called by the subject, and an infrared sensor for sensing the subject;
A chroma key processing unit for chroma key processing the subject image input through the camera, and synthesizes the subject image processed by the chroma key processing unit with the stored background image, but enlarges or enlarges the size of the subject image at the set position of the background image at a set ratio. An image processing apparatus configured to reduce the image synthesis unit;
A voice processing unit composed of a voice synthesizing unit for synthesizing a user's singing voice signal and an accompaniment sound of the microphone, and a voice output unit for outputting a song and an accompaniment sound of the voice synthesizing unit;
A control unit for recognizing a subject according to a signal input from the infrared sensor, and controlling composition according to selection of a user's key input unit for image synthesis and speech synthesis; And
An image output section for outputting a composite image of a background image and a foreground image of a chroma key processed image of the subject, a song and an accompaniment note outputted from the speaker, and a composite image An external output device including a mixer unit and an external output unit for transmitting voice and image data stored in the mixer unit as wireless data;
And a background image and a foreground image storage unit for categorizing and storing the background image and the foreground image by the category and providing the background image and the foreground image to the image synthesizing unit.
The method according to claim 1,
Wherein the image synthesizer moves the composite position of the subject image subjected to the chroma key processing and the background image and enlarges or reduces the subject image at a predetermined magnification according to the movement position to synthesize the chroma key subject image and the background image Synthesizer.
The method according to claim 1,
Each of the background image and the foreground image stored in the image database includes the combined position data of the subject image, and the enlarged or reduced ratio data of the subject image for each synthesized position as metadata. Background image synthesis device.
A video synthesizer synthesizes a subject obtained by chroma key processing of a video obtained through a camera, a background image and a foreground image stored in the background and foreground image storage unit in a video synthesis unit, and stores a song and an accompaniment sound source, In a chroma key object image and a background image synthesis method,
A first step of acquiring and storing a moving image of a subject obtained through the camera;
A second process of performing a chroma key process and a size of a subject on a moving image of a subject obtained through the camera;
A third process of calculating the size of the subject by the chroma key processing in the image processing unit, the background image selected by the user, and the image synthesizing unit in a perspective manner; And
And a fourth step of outputting a synthesized image in the third step and outputting a user's voice and an accompaniment sound synchronized with the synthesized image through a speaker. Background image synthesis method. 5. The method of claim 4,
Wherein the third step is to synthesize the subject image with the size and position of the subject set in the background image selected by the user to give a visual perspective sense. 6. The method of claim 5,
Wherein the object image moves in a predetermined time interval along a movement path set in the background image. The method of claim 7, wherein
And converting a size of a moving picture of a subject into a predetermined ratio as the position of the subject image moves in the background image. 8. The method according to any one of claims 5 to 7,
Wherein the size of the synthesized object image set in the background image is determined in a fixed or ratio manner. 5. The method of claim 4,
And synthesizing the foreground image selected by the user in three layers in addition to the third process, wherein the foreground image is synthesized to a size set in the background image. The method of claim 9,
Wherein the size of the foreground image is changed according to the size of the synthesized object image.

Images