KR20160139681A - Apparatus and method for making augmented reality video used chromakey - Google Patents

Apparatus and method for making augmented reality video used chromakey Download PDF

Info

Publication number
KR20160139681A
KR20160139681A KR1020150074994A KR20150074994A KR20160139681A KR 20160139681 A KR20160139681 A KR 20160139681A KR 1020150074994 A KR1020150074994 A KR 1020150074994A KR 20150074994 A KR20150074994 A KR 20150074994A KR 20160139681 A KR20160139681 A KR 20160139681A
Authority
KR
South Korea
Prior art keywords
image
augmented reality
content
buffer
chroma key
Prior art date
Application number
KR1020150074994A
Other languages
Korean (ko)
Other versions
KR101688926B1 (en
Inventor
박수왕
김동욱
송성환
이봉규
Original Assignee
(주)소셜네트워크
제주대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by (주)소셜네트워크, 제주대학교 산학협력단 filed Critical (주)소셜네트워크
Priority to KR1020150074994A priority Critical patent/KR101688926B1/en
Priority to PCT/KR2015/005443 priority patent/WO2016190472A1/en
Publication of KR20160139681A publication Critical patent/KR20160139681A/en
Application granted granted Critical
Publication of KR101688926B1 publication Critical patent/KR101688926B1/en

Links

Images

Classifications

    • H04N13/004—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • H04N13/0007—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring

Abstract

An apparatus and method for generating an augmented reality image using a chroma key are disclosed.
The apparatus for generating an augmented reality image using a chroma key generates an augmented reality content image including an alpha channel before synthesizing a content image including a virtual object in a live image, And outputs it. Thus, the problem of synchronization by alpha channel synthesis can be solved.

Description

[0001] APPARATUS AND METHOD FOR MAKING AUGMENTED REALITY VIDEO USED CHROMAKEY [0002]

The present invention relates to an apparatus and method for generating an augmented reality image using a chroma key, and more particularly, to an apparatus and method for generating an augmented reality image using a chroma key, .

In general, the chroma key technique refers to a technique of generating a single image by combining an image of a subject photographed in a virtual space with a predetermined background image or image. A chroma key is used for image synthesis. After a specific color (chroma) is designated as a key, a pixel corresponding to a key in the main image is replaced with another image.

When the chroma key is applied to the augmented reality, the content transformation such as the position shift of the contents synthesized in the augmented reality occurs by the chroma key technique in real time by the action of the user or the feature point of the input image. At this time, the alpha channel is synthesized so that the unnaturalness due to the content modification can be covered by the image.

However, in the conventional augmented reality chroma key system, a synchronization problem occurs because the color channel and the alpha channel, which are prepared in advance, are synthesized in real time as shown in FIGS. 1A and 1B. When such a synchronization problem occurs, there is a problem that the part to be shown as shown in FIG. 1C is not visible or the black shade is visible to the part to be invisible. In addition, there is a problem that storage space is wasted because twice the space (the space occupied by the color channel and the alpha channel) is required for one output image.

Therefore, an augmented reality chroma key method that can solve the problems of synchronization of alpha channel and storage space is needed.

According to an aspect of the present invention, a content image for an augmented reality including an alpha channel is synthesized before a content image including a virtual object is synthesized in a live image, a live image is synthesized with a previously generated augmented reality content image, The present invention provides an apparatus and method for generating an augmented reality image using a chroma key.

According to an aspect of the present invention, there is provided a method of generating an augmented reality image using a chroma key, the method comprising: removing a background region of a previously generated content image using a chroma key technique; And generating a content image for augmented reality by combining the color channel and the alpha channel, and when a live image is inputted through a camera provided in advance, the live image is synthesized with the augmented reality content image, Generate chromakey images.

The extraction of the color channel and the alpha channel for the content image from which the background region has been removed may be performed by separating the content image from which the background region is removed into a plurality of still images, Can be extracted.

Generating a content image for the augmented reality comprises generating a plurality of combined still images by synthesizing a color channel and an alpha channel for each separated still image of the content image from which the background area is removed, You can create an image sequence by aligning images.

And performing rendering on each combined still image included in the image sequence.

When the augmented reality image output signal is inputted by the user after generating the augmented reality content image, the rendered combined still image corresponding to the size of the buffer provided in the augmented reality image production apparatus in the image sequence is extracted, As shown in FIG.

The generating of the augmented reality chroma key image comprises: when the rendered combined still image corresponding to the size of the buffer is loaded into the buffer and the live image is input, when the live image is input, The augmented reality chroma key image can be generated by synthesizing the still image and the live image.

When the synthesized position or size information of the augmented reality content is received, the position or size of each augmented reality content included in the loaded combined rendered still image is changed according to the received synthesized position or size information, And generating the augmented reality chroma key image by synthesizing the live image with each of the loaded and rendered combined still images whose sizes have been changed.

The difference between the last combined frame number loaded in the buffer and the frame number currently being output is calculated so that the augmented reality image can be continuously output, and whether the image size corresponding to the calculated difference is less than the size of the buffer And if the image size corresponding to the calculated difference is less than the size of the buffer, extracting the rendered combined still image corresponding to the calculated difference in the image sequence and loading the extracted combined still image into the buffer have.

According to an aspect of the present invention, there is provided an apparatus for generating an augmented reality image using a chroma key, the apparatus comprising: an input unit for inputting a previously produced content image; a chroma key method applied to the content image to remove a background region of the content image; An augmented reality control unit for controlling the augmented reality control unit to extract a color channel and an alpha channel for a content image from which an area has been removed and to generate a content image for augmented reality by synthesizing the color channel and the alpha channel, And a chroma key controller for generating an augmented reality chroma key image by synthesizing the live image with the content image for the augmented reality.

The augmented reality controller extracts the color channel and the alpha channel for the content image from which the background area is removed by dividing the content image from which the background area is removed into a plurality of still images, You can extract color channels and alpha channels.

Wherein the augmented reality control unit generates a plurality of combined still images by synthesizing a color channel and an alpha channel for each separated still image of the content image from which the background area is removed, The plurality of combined still images may be arranged to generate an image sequence.

The augmented reality controller may further include performing rendering on each combined still image included in the image sequence.

When the augmented reality image output signal is input by the user, the augmented reality controller extracts the rendered combined still image corresponding to the size of the buffer provided in the augmented reality image production apparatus in the image sequence, and loads the rendered combined still image into the buffer Quot;

Wherein the chromatic key control unit loads the rendered combined still image corresponding to the size of the buffer into the buffer by the augmented reality control unit and, when the live video is input, The augmented reality chroma key image may be generated by combining the combined still image and the live image, and the augmented reality chroma key image may be output through an output unit provided in the augmented reality image producing apparatus.

Wherein the chroma key control unit controls the chroma key control unit such that when the synthesized position or size information of the augmented reality content is received from the augmented reality control unit, And generating the augmented reality chroma key image by synthesizing the live image with each of the loaded and rendered combined still images whose position or size has been changed, by changing the position or size of the augmented reality chroma key image.

Wherein the augmented reality controller calculates a difference between a last combined image number loaded in the buffer and a currently output frame number so that the augmented reality image can be continuously output, And if the image size corresponding to the calculated difference is less than the size of the buffer, extracts the rendered combined still image corresponding to the calculated difference in the image sequence and loads the combined still image corresponding to the calculated difference into the buffer And < / RTI >

According to an aspect of the present invention, an augmented reality content image including an alpha channel is produced in advance, and then an augmented reality chroma key image is generated by synthesizing a live image, thereby solving a synchronization problem by alpha channel synthesis , Additional storage space for alpha channel synthesis can be used to save storage space.

1A, 1B, and 1C are diagrams for explaining a conventional alpha channel synthesis method.
2 is a block diagram of an apparatus for producing an augmented reality image using a chroma key according to an embodiment of the present invention.
3 is a view showing an image in which a color channel and an alpha channel are synthesized.
4 is a diagram showing a sequence of images synthesized in FIG.
FIG. 5 is a flowchart illustrating a method of generating a content image for an augmented reality in a method for generating an augmented reality image according to an embodiment of the present invention.
FIG. 6 is a detailed flowchart of a method of loading a content image for augmented reality into a buffer in FIG.
FIG. 7 is a flowchart illustrating a method of generating and outputting an augmented reality chroma key image in an augmented reality image production method according to an embodiment of the present invention.
8 is a flowchart illustrating a method of loading a content image for augmented reality into a buffer during an augmented reality chroma key image playback.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

FIG. 2 is a block diagram of an apparatus for producing an augmented reality image using a chroma key according to an embodiment of the present invention. FIG. 3 is a diagram illustrating an image obtained by combining a color channel and an alpha channel, Lt; / RTI >

In order to solve the synchronization problem due to alpha channel synthesis, an apparatus for generating an augmented reality image using a chroma key according to an exemplary embodiment of the present invention , An augmented reality image can be produced and output by synthesizing a pre-produced content image synthesized with an alpha channel and a live image captured through a camera in real time.

2, an augmented reality image production apparatus 1 according to an exemplary embodiment of the present invention includes an input unit 100, an interface unit 200, a control unit 300, an output unit 400, and a memory unit 500. [ . ≪ / RTI >

The input unit 100 can receive information necessary for the augmented reality image production. To this end, the input unit 100 may include a camera unit 110 and a sensor unit 120.

The camera unit 110 can capture a real-environment image, i.e., a live image, to be combined with the augmented reality image. The camera unit 110 may transmit the photographed live image to the control unit 300.

The sensor unit 120 can sense information on the external environment. For example, the sensor unit 120 can sense a movement of a user and can detect a change in the position of the augmented reality image production apparatus 1 by a user. The sensor unit 120 may transmit information about the sensed environment to the controller 300. [

The interface unit 200 may sense an input signal input from a user. The interface unit 200 serves as an interface with all the external devices connected to the augmented reality image production apparatus 1. [ For example, it can be connected to a device with a wired / wireless headset port, an external rechargeable battery port, a wired / wireless data port, a memory card port, an identification module, an audio I / O port, (Input / Output) port, an earphone port, and the like. The interface unit 200 may receive an input signal input by a user or input information from a connected external device, and may transmit the input signal to the controller 300.

The control unit 300 can generate a content image for an augmented reality in which the alpha channel is synthesized integrally, generate an augmented reality chroma key image by synthesizing the generated augmented reality content image with a live image, It is possible to load the generated chroma key image into the buffer so that the Maki image can be outputted seamlessly through the output unit 400. [ For this, the controller 300 may include an augmented reality controller 310 and a chroma key controller 320.

The augmented reality controller 310 may generate a content image for augmented reality to be combined with the augmented reality image.

Specifically, the augmented reality control unit 310 may receive a content image including content to be synthesized from the input unit 100, the interface unit 200, or the memory unit 500. The augmented reality controller 310 may remove the background area from the content image using the chroma key technique. At this time, the live image may be synthesized by the chroma key controller 320 in the background region removed from the content image.

On the other hand, a content change for the augmented reality contents image may occur such as a change of the position of the content image, a change of the size of the content image, or the like by a user's action or a real-time position movement of the input image by the minutiae point. In order to output a natural image when such a content change occurs, an alpha channel corresponding to the content video can be synthesized. However, as described above in the background art, a conventional augmented reality image production apparatus may synthesize an alpha channel in real time, thereby causing a synchronization problem. Therefore, the augmented reality controller 310 according to an exemplary embodiment of the present invention synthesizes an alpha channel in advance to a content image so that a synchronization problem according to an alpha channel does not occur when an augmented reality chromakey image is output, A content video for the content can be generated.

Specifically, the augmented reality controller 310 may separate the content image from which the background region has been removed into a plurality of still images. The augmented reality control unit 310 may extract the color channel and the alpha channel for each still image of the content image. As shown in FIG. 3, the augmented reality controller 310 may combine the extracted color channel and alpha channel to generate a combined still image. At this time, the combined still image in which the color channel and the alpha channel are synthesized integrally can be formed in the PNG file format. As shown in FIG. 4, the augmented reality controller 310 may generate an image sequence by aligning a plurality of combined still images in which a color channel and an alpha channel are combined. At this time, since the image means a continuous still image, the image sequence itself may be a content image for an augmented reality.

The augmented reality control unit 310 may transmit the generated image sequence to a renderer that is provided in advance and control rendering to be performed on a plurality of combined still images included in the image sequence.

The augmented reality control unit 310 may receive the augmented reality video output signal through the input unit 100 or the interface unit 200. [ When the augmented reality image output signal is received, the augmented reality controller 310 may load the combined still image included in the image sequence into a buffer provided in the output unit 400 in advance. At this time, the augmented reality controller 310 can confirm the size of the buffer. The augmented reality controller 310 may calculate the number of combined still images corresponding to the size of the confirmed buffer. The augmented reality controller 310 may extract a combined still image corresponding to the number of combined still images corresponding to the size of the buffer in the image sequence. At this time, extracting the combined still images from the image sequence can be sequentially extracted in the order in which they are arranged in the image sequence. The augmented reality controller 310 may load the extracted combined still image into the buffer.

The augmented reality controller 310 combines the combined still image loaded into the buffer with the live image by the chroma key controller 320 so that the augmented reality chroma key image can be continuously output, You can load an image into a buffer.

Specifically, if it is confirmed that the augmented reality chroma key image is being output to the output unit 400, the augmented reality control unit 310 can receive the timing information from the renderer. At this time, the timing information may include output timing information of a frame to be reproduced. The augmented reality controller 310 may calculate a current frame number to be reproduced using the received timing information. On the other hand, if the content information for the augmented reality includes sound information, the current sound number can be calculated together with the current frame number. The augmented reality controller 310 can check whether the current frame number is less than the total frame number to check whether all the combined still images included in the current image sequence are output. At this time, the total frame number may mean the total number of frames. On the other hand, when the current frame number exceeds the total frame number, it does not occur and is excluded. When it is determined that the current frame number is less than the total frame number, the augmented reality controller 310 can check whether the combined still image corresponding to the current frame is loaded in the buffer. If the combined still image corresponding to the current frame is not loaded, the augmented reality controller 310 may extract the image corresponding to the current frame from the image sequence and load the same into the buffer. After the image corresponding to the current frame is loaded, the augmented reality controller 310 calculates a difference between the number of the last image loaded in the buffer and the current frame number

Figure pat00001
) Can be calculated. The augmented reality controller 310 calculates the difference between the number of the last image loaded in the buffer and the current frame number
Figure pat00002
) Can be calculated. The augmented reality controller 310 determines the difference between the number of the last image loaded in the buffer and the current frame number
Figure pat00003
) Is smaller than the size of the buffer. At this time, the difference between the number of the last image loaded in the buffer and the current frame number
Figure pat00004
Is smaller than the size of the buffer, it is determined that an image smaller than the number of images that can be loaded into the current buffer is loaded in the buffer, and the number of images that can be additionally loaded into the buffer, The difference between the number of the last image loaded in the current frame and the current frame number
Figure pat00005
) Can be extracted from the image sequence. The augmented reality controller 310 may load the extracted combined still image into the buffer. The augmented reality control unit 310 can confirm whether the current frame number and the total frame number match whether the combined still image is synthesized with the live image by the chroma key control unit 320 and output. The augmented reality controller 310 recognizes that all the combined still images included in the image sequence are output when the current frame number matches the total frame number and can initialize the control for the corresponding image sequence. At this time, initialization may mean setting the number of the current frame to 0 or not loading the image into the buffer.

The augmented reality control unit 310 may analyze an input image input by the input unit 100 and detect an area to be synthesized in the input image. For example, if a person exists in the input image, the portion indicated by the human hand may be an area in which the content is synthesized. The augmented reality control unit 310 may track the position and size of the area in which the content is to be synthesized in the input image. The augmented reality controller 310 may transmit the tracking information of the area where the tracked content is to be synthesized to the chroma key controller 320. [ At this time, the tracking information may include position information or size information of an area in which the contents according to the frame flow are to be synthesized.

The chroma key controller 320 can generate an augmented reality chroma key image by combining the live still image and the combined still image loaded into the buffer by the augmented reality control unit 310 when the live image is input by the camera unit 110 have. At this time, the chroma key controller 320 may synthesize a live image in the background region of each combined still image loaded in the buffer. At this time, the background region of the combined still image may be a portion removed by the augmented reality control unit 310 using the chroma key technique, and the corresponding region may be set as a key to synthesize a live image.

On the other hand, when the tracking information is received from the augmented reality controller 310, the chroma key controller 320 converts the augmented reality content image according to the tracking information, synthesizes a live image to the augmented reality content image, have. At this time, the tracking information may include position and size information of the content according to the frame flow. Specifically, when the tracking information is transmitted, the chroma key control unit 320 may change the content included in the combined still image according to the tracking information received for each combined still image loaded in the buffer. At this time, the chroma key control unit 320 may transform each combined still image to correspond to the content position or size information according to the time included in the tracking information. For example, since the tracking information includes content position or size information for each frame, the chroma key controller 320 detects a combined still image corresponding to each frame included in the tracking information, and stores the combined still image in the detected combined still image The contents can be controlled to change according to the corresponding tracking information.

The output unit 400 may output the augmented reality chroma key image generated by the chroma key control unit 320. [ The output unit 400 may output an image to a screen provided in the augmented reality image production apparatus 1 and may output an image to another display apparatus connected to the augmented reality image production apparatus 1. [ Also, the output unit 400 may further include a speaker. If the augmented reality chroma key image to be output is matched in advance with the acoustic information, the output unit 400 may output the corresponding image and output the matched sound.

The memory unit 500 may store information necessary for the overall control of the controller 300. [ The memory unit 500 may store a plurality of content images to be combined with an augmented reality image to produce an augmented reality image. The memory unit 500 may temporarily store the augmented reality content image generated by the augmented reality control unit 310. [

Hereinafter, a method of generating a content image for an augmented reality according to an embodiment of the present invention will be described with reference to FIG.

First, a content image is received from the input unit 100, the interface unit 200, or the memory unit 500 (610), and the input content image is divided into a plurality of still images (620).

In this case, separating the input content image into a plurality of still images may mean separating a plurality of still images constituting the content image according to the order of constructing the content image.

The input content image is divided into a plurality of still images (620), and a color channel and an alpha channel for the separated still images are extracted (630).

A plurality of combined still images are generated by combining the color channel and the alpha channel extracted from each still image of the content image (640).

At this time, generating a combined still image including an alpha channel means that a synchronization problem may occur if the alpha channel is synthesized in real time when an augmented reality image is produced. Therefore, in order to fundamentally block the synchronization problem, And generates a content image.

(650) arranging a plurality of combined still images to generate an image sequence, and transmitting the generated image sequence to a renderer, rendering each combined still image included in the generated image sequence to generate augmented reality content And generates an image (660).

After a content image for augmented reality is generated (660), it is confirmed whether a user inputs an augmented reality image output signal (670). If it is confirmed that an augmented reality image output signal is input, (680).

At this time, loading of the content video for augmented reality into the buffer will be described in detail with reference to FIG.

Hereinafter, a method of loading a content video for augmented reality into a buffer will be described with reference to FIG.

When a content image for augmented reality is generated by the augmented reality control unit 310, the augmented reality content control unit 310 synthesizes the content image for augmented reality with a live image, and outputs the combined image to the augmented reality control unit 310. The augmented reality control unit 310 may load a content image for augmented reality .

To do this, the number of combined still images corresponding to the size of the buffer is calculated (681) so that the number of images that can be loaded into the buffer can be confirmed.

After calculating 681 the number of combined still images corresponding to the size of the buffer, a combined still image corresponding to the number of combined still images calculated in the image sequence is extracted (682).

At this time, extracting the combined still images from the image sequence can extract the combined still images in order according to the order in which they are arranged in the image sequence.

The extracted combined still image is loaded into the buffer (683).

A method of loading the finally combined still image into the buffer while the combined still image loaded in the buffer is synthesized with the live image by the chroma key control unit 320 and outputted is described in detail with reference to FIG.

FIG. 7 is a flowchart illustrating a method of generating and outputting an augmented reality chroma key image in an augmented reality image production method according to an embodiment of the present invention.

When the combined still image is loaded into the buffer by the augmented reality controller 310, it is determined whether a live image is inputted through the input unit 100 (710).

When a live image is input (710), an augmented reality chroma key image is generated by synthesizing a combined still image and an input live image loaded in a buffer (720), and the generated augmented reality chroma key image is output to an output unit (730).

8 is a flowchart illustrating a method of loading a content image for augmented reality into a buffer during an augmented reality chroma key image playback.

As described above, the augmented reality image production apparatus 1 according to the embodiment of the present invention loads the combined still image corresponding to the size of the buffer into the buffer, Can be loaded in a chain.

To do this, it is checked 810 whether an augmented reality chromakey image is being reproduced through the output unit 400. If the augmented reality chromakey image is currently being reproduced, timing information is received from the renderer 815.

At this time, the renderer receives the augmented reality content image by the augmented reality control unit 310 when constructing the augmented reality content image, and performs rendering on the received augmented reality content image to generate timing information for the augmented reality content image Can be obtained.

A current frame number to be reproduced is calculated using the received timing information (820).

On the other hand, if the content information for the augmented reality includes sound information, the current sound number can be calculated together with the current frame number.

After the current frame number is calculated 820, it is checked 825 whether the current frame number is less than the total frame number to check whether all the combined still images included in the current image sequence are output.

At this time, the total frame number may mean the total number of frames. On the other hand, when the current frame number exceeds the total frame number, it does not occur and is excluded.

If it is determined 825 that the current frame number is less than the total frame number, it is checked 830 whether a combined still image corresponding to the current frame is loaded in the buffer.

If it is determined that the combined still image corresponding to the current frame is not loaded (830), the image corresponding to the current frame is extracted from the image sequence and loaded into the buffer (835).

The image corresponding to the current frame is loaded (835), and the difference between the number of the last image loaded in the buffer and the current frame number

Figure pat00006
(840).

The difference between the number of the last image loaded in the buffer and the current frame number (

Figure pat00007
(845) whether the size of the image corresponding to the current frame number is less than the size of the buffer, and determines whether the difference between the number of the last image loaded in the buffer and the current frame number
Figure pat00008
) Is less than the size of the buffer, it is recognized that an image smaller than the number of images that can be loaded into the current buffer is loaded in the buffer, so that the number of the last image loaded in the buffer and the current frame number Difference(
Figure pat00009
) Is extracted from the image sequence and loaded into the buffer (850).

The difference between the number of the last image loaded in the buffer and the current frame number (

Figure pat00010
) Is equal to the size of the buffer, the current buffer is recognized as being full, and the process returns to step 840. [ At this time, the combined still image loaded into the buffer is synthesized with the live image by the chroma key controller 320, and the output combined still image is deleted from the buffer, so that the current frame number is changed, Difference between image number and current frame number (
Figure pat00011
) Changes with time, so the difference between the number of the last image loaded in the buffer and the current frame number
Figure pat00012
) Can be loaded into the buffer by checking whether the image size is less than the size of the buffer.

After the combined still image is loaded into the buffer, it is checked whether the current frame number and the total frame number match (855).

If the current frame number matches the total frame number, all the combined still images included in the image sequence are recognized as being outputted, and control for the corresponding image sequence is initialized (860).

At this time, initialization may mean setting the number of the current frame to 0 or not loading the image into the buffer.

Such a technique for generating an augmented reality image using a chroma key can be implemented in an application or implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination.

The program instructions recorded on the computer-readable recording medium may be ones that are specially designed and configured for the present invention and are known and available to those skilled in the art of computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made therein without departing from the spirit and scope of the invention as defined in the appended claims. It will be possible.

1: Augmented reality image production device using chroma key
100: Input unit
200:
300:
310: Augmented reality controller
320: Chroma key control unit
400: Output section
500: memory unit

Claims (16)

The background region of the previously produced content image is removed using the chroma key technique,
Extracting a color channel and an alpha channel for the content image from which the background region is removed, generating a content image for augmented reality by synthesizing the color channel and the alpha channel,
And generating augmented reality chroma key image by synthesizing the live image with the content image for the augmented reality when a live image is input through a camera provided in advance. The method according to claim 1,
Extracting a color channel and an alpha channel for the content image from which the background area is removed,
And separating the content image from which the background area is removed into a plurality of still images and extracting a color channel and an alpha channel for each separated still image. 3. The method of claim 2,
Generating the content image for the augmented reality includes:
Generating a plurality of combined still images by synthesizing a color channel and an alpha channel for each separated still image of the content image from which the background area has been removed and generating an image sequence by arranging the plurality of combined still images, A method for constructing an augmented reality image using The method of claim 3,
Further comprising performing rendering on each of the combined still images included in the image sequence. 5. The method of claim 4,
When the augmented reality image output signal is inputted by the user after generating the augmented reality content image, the rendered combined still image corresponding to the size of the buffer provided in the augmented reality image production apparatus in the image sequence is extracted, The method comprising the steps of: 6. The method of claim 5,
The generation of the augmented reality chroma key image is performed,
A rendered combined still image corresponding to the size of the buffer is loaded into the buffer, and when the live video is input, the loaded combined video still image and the live video are synthesized when the live video is input, A method for generating augmented reality image using chroma key to generate chroma key image. The method according to claim 6,
When the synthesized position or size information of the augmented reality content is received, the position or size of each augmented reality content included in the loaded combined rendered still image is changed according to the received synthesized position or size information, And generating the augmented reality chroma key image by synthesizing the live image with each of the loaded and rendered combined still images whose size is changed, and generating the augmented reality chroma key image. 6. The method of claim 5,
The difference between the last combined frame number loaded in the buffer and the frame number currently being output is calculated so that the augmented reality image can be continuously output, and whether the image size corresponding to the calculated difference is less than the size of the buffer And if the image size corresponding to the calculated difference is less than the size of the buffer, extracting the rendered combined still image corresponding to the calculated difference in the image sequence and loading the extracted combined still image into the buffer A Method for Producing Augmented Reality Video Using Marky. An input unit for receiving a pre-produced content image;
Extracting a color channel and an alpha channel for the content image from which the background area is removed, applying the chroma key technique to the content image, and combining the color channel and the alpha channel to enhance An augmented reality controller for controlling the generation of a content image for a real time; And
And a chroma key controller for controlling the chroma key controller to generate an augmented reality chroma key image by synthesizing the live image with the content image for augmented reality when a live image is input through a camera provided in advance. 10. The method of claim 9,
The augmented reality controller extracts a color channel and an alpha channel for the content image from which the background area is removed,
And separating the content image from which the background area is removed into a plurality of still images and extracting a color channel and an alpha channel for each separated still image. 11. The method of claim 10,
The augmented reality control unit generates the augmented reality content image,
Generating a plurality of combined still images by synthesizing a color channel and an alpha channel for each separated still image of the content image from which the background area has been removed and generating an image sequence by arranging the plurality of combined still images, Augmented Reality Video Production System Using. 12. The method of claim 11,
Wherein the augmented reality controller comprises:
Further comprising performing rendering on each combined still image included in the image sequence. 13. The method of claim 12,
Wherein the augmented reality controller comprises:
Further comprising: extracting the rendered combined still image corresponding to a size of a buffer provided in advance in the augmented reality image producing apparatus in the image sequence when the augmented reality image output signal is input by the user and loading the rendered combined still image into the buffer, Augmented Reality Video Production System Using. 14. The method of claim 13,
Wherein the chroma key control unit comprises:
Wherein the rendered combined still image corresponding to the size of the buffer is loaded into the buffer by the augmented reality controller, and when the live image is input, when the live image is input, And outputting the augmented reality chroma key image through an output unit provided in the augmented reality image production apparatus. 2. The augmented reality image production apparatus of claim 1, wherein the augmented reality chroma key image is generated by synthesizing an augmented reality chroma key image. 15. The method of claim 14,
Wherein the chroma key control unit comprises:
When the synthesized position or size information of the augmented reality content is received from the augmented reality control unit, the position or size of each augmented reality content included in the loaded combined rendered still image according to the received synthesized position or size information, And generating the augmented reality chroma key image by synthesizing the live image with each of the loaded and rendered combined still images whose positions or sizes are changed, and generating the augmented reality chroma key image. 14. The method of claim 13,
Wherein the augmented reality controller comprises:
The difference between the last combined frame number loaded in the buffer and the frame number currently being output is calculated so that the augmented reality image can be continuously output, and whether the image size corresponding to the calculated difference is less than the size of the buffer And if the image size corresponding to the calculated difference is less than the size of the buffer, extracting the rendered combined still image corresponding to the calculated difference in the image sequence and loading the extracted combined still image into the buffer Augmented Reality Video Production System Using Marquee.
KR1020150074994A 2015-05-28 2015-05-28 Apparatus and method for making augmented reality video used chromakey KR101688926B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020150074994A KR101688926B1 (en) 2015-05-28 2015-05-28 Apparatus and method for making augmented reality video used chromakey
PCT/KR2015/005443 WO2016190472A1 (en) 2015-05-28 2015-05-29 Device and method for producing augmented reality image by using chroma key

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150074994A KR101688926B1 (en) 2015-05-28 2015-05-28 Apparatus and method for making augmented reality video used chromakey

Publications (2)

Publication Number Publication Date
KR20160139681A true KR20160139681A (en) 2016-12-07
KR101688926B1 KR101688926B1 (en) 2016-12-22

Family

ID=57393288

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150074994A KR101688926B1 (en) 2015-05-28 2015-05-28 Apparatus and method for making augmented reality video used chromakey

Country Status (2)

Country Link
KR (1) KR101688926B1 (en)
WO (1) WO2016190472A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101944358B1 (en) 2017-05-26 2019-02-01 강다겸 Web Based Real Time Diustributed Streaming And Chromakey System And Method
KR102133735B1 (en) 2018-07-23 2020-07-21 (주)지니트 Panorama chroma-key synthesis system and method
KR102101966B1 (en) * 2018-10-19 2020-04-23 주식회사 산타 Device and method for chroma key image synthesis
GB2582917B (en) * 2019-04-05 2021-05-12 Holome Tech Limited Method and apparatus for generating three dimensional images
CN115086704A (en) * 2022-08-19 2022-09-20 广州市千钧网络科技有限公司 Method, device and storage medium for video image synthesis in live broadcast

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20120137021A (en) * 2011-06-10 2012-12-20 한남대학교 산학협력단 Authoring method of augmented reality base chroma-key using overlay layer

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100719101B1 (en) * 2005-08-05 2007-05-17 주식회사 오아시스미디어 Real-time multilayer image synthesis recording system and method, and the store device which records a method
KR100934471B1 (en) * 2008-04-17 2009-12-30 (주)아이스타티브이 Image Production Apparatus and Method Using Chroma Key
KR20100026240A (en) * 2008-08-29 2010-03-10 김상국 3d hair style simulation system and method using augmented reality
KR101264838B1 (en) * 2011-09-03 2013-05-15 (주)네오텍소프트 Real-time image synthesizing system
KR101414217B1 (en) * 2012-05-04 2014-07-04 임윤호 Real time image synthesis apparatus and image synthesis method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20120137021A (en) * 2011-06-10 2012-12-20 한남대학교 산학협력단 Authoring method of augmented reality base chroma-key using overlay layer

Also Published As

Publication number Publication date
WO2016190472A1 (en) 2016-12-01
KR101688926B1 (en) 2016-12-22

Similar Documents

Publication Publication Date Title
KR101688926B1 (en) Apparatus and method for making augmented reality video used chromakey
US20150381925A1 (en) Smart pause for neutral facial expression
US20120105657A1 (en) Image processing apparatus, image pickup apparatus, image processing method, and program
KR102113812B1 (en) Apparatus and method for implementing immersive augmented reality with RGB-D data
US9148564B2 (en) Image pickup apparatus, information processing system and image data processing method
US20070222858A1 (en) Monitoring system, monitoring method and program therefor
JP6060965B2 (en) Image processing apparatus, image processing method, and program
KR100981002B1 (en) Composite image output apparatus, and recording medium storing program
US20070024710A1 (en) Monitoring system, monitoring apparatus, monitoring method and program therefor
JP2002232783A (en) Image processor, method therefor and record medium for program
KR102459850B1 (en) Method and apparatus for processing 3-dimension image, and graphic processing unit
US9600160B2 (en) Image processing device, image processing method, and program
WO2021096339A1 (en) Method for transforming image
JP6137910B2 (en) Information processing apparatus, information processing method, and program
JP2012151544A (en) Imaging apparatus and program
US8269858B2 (en) Image pickup device, image pickup method, and integrated circuit
US9727993B2 (en) Image processing apparatus, image processing method, and program
TW201320727A (en) Image processing device, image processing method, and recording medium
KR102457176B1 (en) Electronic apparatus and method for generating contents
JP5894505B2 (en) Image communication system, image generation apparatus, and program
KR102299565B1 (en) Method for real time person object cognition and real time image processing on real time broadcasting image and apparatus for performing the method
JP6396932B2 (en) Image composition apparatus, operation method of image composition apparatus, and computer program
CN113973175A (en) Rapid HDR video reconstruction method
KR101803532B1 (en) Method Controlling Quality of Augmented Reality
JP2016129281A (en) Image processor

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
N231 Notification of change of applicant
E701 Decision to grant or registration of patent right
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20191125

Year of fee payment: 4