KR20210052884A - Personalized Video Production System and Method Using Chroma Key - Google Patents

Abstract

A personalized video production system using a chroma key includes: a server connected to a network; and a system for a plurality of users connected to the server via the network, wherein the server includes: an image conversion unit that divides an uploaded image by the number of cores of a CPU, stores the sum of frames calculated per core in a memory to form an alpha layer, obtains the shortest distance pixel of a broken outline in the alpha layer to arbitrarily connect the outline, calculates the non-transparent area by filling the area inside the outline with a random color, and transparently processes the background area; an image storage unit; a distributed processing unit for distributing the image obtained from the image storage unit without delay; and a plurality of video transmitting units.

Description

Personalized Video Production System and Method Using Chroma Key}

The present invention relates to a system and method for producing personalized images using chroma key, and more particularly, a self-introduction letter or an image obtained by automatically synthesizing a chroma key subject image and a real-time image using a specific video using a chroma key technique. It relates to a system and method for producing personalized video using chroma key that can be used for UCC, etc.

Image processing technology for video production includes image synthesis technology widely used in fields such as computer art, advertisements, and entertainment by combining or transforming actual video images with synthetic images of artificial images, and image synthesis in video images. Image analysis technology, which refers to a series of tasks that identify appearing objects, and Image Pattern Recognition technology, which refers to the work of identifying and recognizing objects such as characters, human fingerprints, and faces given as video images. There is this.

Among the image processing technologies described above, the image synthesis technology that users mainly use when creating video-based content such as UCC or PCC is a method of combining different types of images into one using a chroma-key method. Is typically used.

Chroma key is a technology that combines two images. If there are two images such as A and B, the'narrow range color' from the B image can be removed or made transparent so that the A image behind it is reflected. This technology can also be referred to as color keys, color separation overlays, green screens, and blue screens. It is usually used in weather forecast broadcasts, where the forecaster appears to be standing in front of a large map, but is actually in a studio with a blue or green background.

For real-time video or broadcast transmission, a video mixer that can use chroma keying is used, and for non-linear video editing such as post production (post production) used for film production, use multiple layers with a video editing program like Premier Pro. You can use the chroma key.

Chroma keys aren't necessarily blue and green. You can choose a color directly using a video mixer device or video editing software, or you can smooth the edges of the overlapping parts by using the "throttle function". In Korea, blue is usually used, but at this time, you can perform actions while wearing clothes other than blue. Then the blue color will become transparent and will be composited with the background. However, in the West, green is often used. The reason is that the eyes of Western people are sometimes blue, so using a blue screen can result in blue eyes becoming transparent.

Usually, this chroma key function is often used when moving images or movies cannot be transferred directly to reality. For example, the flying Superman from the movie also uses this method.

In order to synthesize an image using such a chroma key, conventionally, if a blue or green background portion as described above is designated, the computer extracts only the blue or green background portion of the still image for each frame of the image before image synthesis. There was a problem that had to go through the process of having to show.

In addition, when using a server to synthesize images using chroma key, it is a phenomenon that occurs when the number of simultaneous users to the server increases unlimitedly. A response to a message sent to the server arrives late, or access to the server. There were problems such as the process taking a very long time, the connection to the server was suddenly disconnected, or the connection to the server itself failed due to the timeout of TCP retransmission timeout or user-defined keep alive messaging.

In order to solve this problem, after analyzing the traffic of the server, there was a problem in that several computers were used to divide and process the data by subdividing the processing stage of data that one machine had to process.

The present invention has been conceived to solve the above problems, and an object of the present invention is to determine whether or not it is a background region through a change in an image without going through the process of extracting only the background portion and showing the transparent region. It is a system and method for producing a personalized image using a chroma key that differentiates and processes the background area transparently.

Another object of the present invention is to provide a system and method for producing a personalized video using chroma key using a VM (Virtual Machine) technology to solve an overload of a server.

Meanwhile, other objects not specified in the present invention will be additionally considered within a range that can be easily deduced from the detailed description and effects thereof to be described later.

The system and method for producing a personalized video using a chroma key according to the present invention for achieving the above object include a server 10 connected to a network, and a system 20 of a plurality of users connected to the server 10 through the network. ), and the server 10 divides the image taken and uploaded by the external camera 11 by the number of cores of the CPU, and the frequency of pixel change for each frame before and after each divided image, and the number of pixels surrounding each frame. An alpha layer is formed by calculating the average value and storing the sum of the frames calculated per core in the memory to form an alpha layer, obtaining the shortest-distance pixels of the outline broken from the alpha layer, and connecting randomly to form an outline, and An image conversion unit 12 filling in an arbitrary color to calculate a non-transparent area, calculating a background area excluding the calculated non-transparent area, and processing the background area to be transparent; An image storage unit 13 for storing the processed image in the image conversion unit 12; When there is a different request from a plurality of users, each container to transmit an image to the system 20 of the plurality of users is created, the image obtained from the image storage unit 13 is distributed without delay, and the container is managed. Giving distributed processing unit 14; And an image to be distributed and processed by the distributed processing unit 14 to be sent to a designated user's system 20 among the plurality of user's systems 20 as a temporary image inside its container, and the plurality of user's systems 20 ) To generate and transmit a streaming protocol, and the plurality of user's systems 20 include the plurality of images through the network. An image processing unit 21 processing the transparent area of the image in real time so as to separate the channel of the transparent area of the streaming image received from the transmission units 15-1, 15-2, 15-3; An image input unit 23 for recognizing a video device and streaming an image to a web browser; An image synthesizing unit 24 for synthesizing the image of the image input unit 23 in the transparent area of the image processing unit 21 and performing chroma key processing; And an image output unit 25 for outputting the image synthesized by the image synthesis unit 24 to the web browser.

The system and method for producing a personalized image using chroma key according to the present invention makes the background area transparent by distinguishing whether the background area is or not through the change of the image without going through the process of extracting only the background part and showing the transparent area. It is effective to provide a web-based real-time distributed streaming and chroma key image processing system using VM (Virtual Machine) technology to process and solve server overload.

1 is a block diagram of a personalized video production system using chroma key according to an embodiment of the present invention.
FIG. 2 is a diagram for giving an image image divided by octa cores of a personalized image production system using a chroma key according to an embodiment of the present invention.
3 is a diagram illustrating an alpha channel, which is a transparent area visually implemented through a system and method for producing a personalized image using a chroma key according to an embodiment of the present invention.
4 is a diagram illustrating a clear and clear transparent area through a system and method for producing a personalized image using a chroma key according to an embodiment of the present invention.

Objects, specific advantages and novel features of the present invention will become more apparent from the following detailed description and preferred embodiments associated with the accompanying drawings. In addition, the terms used are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of the user operator. Therefore, definitions of these terms should be made based on the overall contents of the present specification.

In adding reference numerals to elements of each drawing, it should be noted that the same elements are assigned the same numerals as possible, even if they are indicated on different drawings. In addition, in describing the present invention, when it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

In addition, in describing the constituent elements of the present invention, terms such as first, second, A, B, (a), (b) may be used. These terms are for distinguishing the constituent element from other constituent elements, and the nature, order, or order of the constituent element is not limited by the term. When a component is described as being "connected", "coupled" or "connected" to another component, the component may be directly connected or connected to the other component, but the other component is "connected" between each component. It is to be understood that it may be connected", "coupled" or "connected".

Hereinafter, the configuration and operation of an embodiment of the present invention will be described in detail with reference to the drawings.

1 is a block diagram of a personalized video production system using chroma key according to a preferred embodiment of the present invention.

Referring to FIG. 1, a personalized video production system using a chroma key according to a preferred embodiment of the present invention includes a server 10 and a user's system 20, wherein the server 10 is connected to a network. In addition, the user's system 20 is connected to the server 10 through the network, and only three are shown in FIG. 1, but the present invention is not limited thereto, and may be composed of one or more.

The server 10 includes an image conversion unit 12, an image storage unit 13, a distributed processing unit 14, and a plurality of image transmission units 15-1, 15-2, and 15-3.

The image conversion unit 12 analyzes the images of the front and rear frames of the image taken and uploaded by the external camera 11 to distinguish the background area excluding the changed pixel area, calculates a background area to be processed as transparent for each frame, and calculates the background area. It adds a video channel that makes the area transparent.

2 is a diagram showing an image image divided into octa cores.

Referring to FIG. 2, in relation to the image conversion unit 12, the image of FIG. 2 (ex> octa core [6 Core]) is divided according to the number of cores of the CPU, and each region is divided into the preceding frame and the rear frame The CPU cores of the CPU calculate the area where the pixel changes.

3 is a diagram illustrating an alpha channel, which is a visually implemented transparent area.

Referring to FIG. 3, as shown in FIG. 3 (visually implemented transparent area = alpha channel), the calculated area per frame of an image is loaded into a memory and stored in a storage space, and an alpha layer is generated. The entire transparent area is processed and smoothed (noise removal).

4 is a diagram showing a clear and clear transparent area.

In order to extract a clear and clear transparent area as shown in FIG. 4, a noise removal process is performed before area operation. Extracting the average value from the previously developed still image by considering the area in front of and behind the transparent area extracted as shown in Fig. 3 as an operation according to area allocation for each previous core (blue or green background part of the still image for each frame of the image) It removes noise and improves clarity and clarity of the area by calculating the frequency of change of the corresponding pixel for each frame of the front and back image and the average value of the surrounding pixels for each frame, not the process of extracting the bay and showing the transparent area. After that, for the calculation of the inner area, the shortest-distance pixel of the broken outline is obtained and randomly connected to complete the outline. Then, when the inner area is filled, the non-transparent area is calculated. After that, it is converted into an image codec including the supplemented transparent area (alpha channel).

The image storage unit 13 stores the processed image in the image conversion unit 12.

When there is a different request from a plurality of users, the distributed processing unit 14 creates each container to transmit an image to the system 20 of the plurality of users, and stores the image obtained from the image storage unit 13 without delay. It distributes and manages the container.

The distributed processing unit 14 performs an imaging process of containers of the image transmission units 15-1, 15-2, and 15-3 according to each situation, so as to generate and reduce and expand a fluid like a virtual machine (VM). It manages the streams to be transmitted and each user group in the media and the protocol generated by the video transmission unit (15-1, 15-2, 15-3), and manages each stream with resource management of the server of each stream. It helps distributed processing of each network resource through the idle time of stream transmission.

The plurality of image transmission units 15-1, 15-2, 15-3 are distributedly processed by the distributed processing unit 14 to transmit images to be sent to the system 20 of a designated user among the systems 20 of the plurality of users. It stores a temporary video in its own container and generates and transmits a streaming protocol to the systems 20 of the plurality of users.

In addition, the plurality of image transmission units (15-1, 15-2, 15-3) is a plug-in between web browsers through WebRTC (Web Real-Time Communication) that can communicate with each other without a plug-in based on JavaScript. It was produced in accordance with the standard of W3C, which enables video transmission without the need, and it is driven through a separate process for the creation of a protocol, so it is possible to efficiently manage server resources through flexible system resource management according to users.

The system 20 of the plurality of users includes an image processing unit 21, an image input unit 23, an image synthesis unit 24, and an image output unit 25.

The image processing unit 21 processes the transparent region of the image in real time to separate the transparent region channel of the streaming image received from the plurality of image transmission units 15-1, 15-2, and 15-3 through a network.

The image input unit 23 recognizes a video device and streams an image to a web browser.

The image synthesizing unit 24 synthesizes the image of the image input unit 23 in the transparent area of the image processing unit 21 and processes the chroma key, and the image streamed in real time is processed by a client based on the web browser. By performing graphic operation processing through language, the load concentrated on the server 10 is reduced. That is, the server 10 is in charge of the stream and the client reduces the delay for augmented reality synthesis through graphic operation, thereby reducing the load concentrated on the server 10.

The image output unit 25 outputs the image synthesized by the image synthesis unit 24 to the web browser.

Hereinafter, a web-based real-time distributed streaming and chroma key image processing method according to a preferred embodiment of the present invention is a pixel area changed by analyzing images of the front and rear frames of an image taken by an external camera 11 from a server connected to a network and uploaded. An image conversion step of calculating a background region to be transparently processed for each frame by distinguishing a background region except for and adding an image channel for transparently processing the background region; An image storage step of storing the processed image in the image conversion unit 12 by the server; A plurality of image transmission units (15-1, 15-2, 15-3) to transmit an image to be distributed and processed by the server by the distributed processing unit 14 to be sent to the system 20 of a designated user among the systems 20 of a plurality of users. ) Storing a temporary image inside the container and generating and transmitting a streaming protocol to the system 20 of the plurality of users; The transparent region of the image is processed in real time to separate the transparent region channel of the streaming image received from the plurality of image transmission units 15-1, 15-2, 15-3 through the network by the user's system 20. Image processing step; An image input step of recognizing a video device by the user's system 20 and streaming an image to a web browser; An image synthesizing step of synthesizing the image of the image input step in the transparent area of the image processing step by the user's system 20 and performing chroma key processing; And an image output step of outputting the image synthesized in the image synthesis step by the user's system 20 to the web browser.

Terms such as "include", "consist of", or "have" described above mean that the corresponding component may be included, unless otherwise stated, excluding other components Rather, it should be interpreted as being able to further include other components. All terms, including technical or scientific terms, unless otherwise defined, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms generally used, such as terms defined in the dictionary, should be interpreted as being consistent with the meaning of the context of the related technology, and are not interpreted as ideal or excessively formal meanings unless explicitly defined in the present invention.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

10: server
11: External camera
12: image conversion unit
13: video storage unit
14: distributed processing unit
15-1, 15-2, 15-3: video transmission unit
20: user's system
21: image processing unit
22: camera
23: video input unit
24: image synthesis unit
25: video output unit

Claims (2)

A server 10 connected to a network, and a system 20 of a plurality of users connected to the server 10 through the network,
The server 10,
An image channel that analyzes the image of the front and rear frames of the image taken and uploaded by the external camera 11, and calculates the background area to be transparent for each frame by distinguishing the background area excluding the changed pixel area, and processes the background area transparently An image conversion unit 12 for adding
An image storage unit 13 for storing the processed image in the image conversion unit 12;
When there is a different request from a plurality of users, each container to transmit an image to the system 20 of the plurality of users is created, the image obtained from the image storage unit 13 is distributed without delay, and the container is managed. Giving distributed processing unit 14; And
The distributed processing unit 14 stores the image to be sent to the designated user's system 20 among the plurality of user's systems 20 as a temporary image inside its container, and the plurality of user's systems 20 It includes a plurality of video transmission units (15-1, 15-2, 15-3) for generating and transmitting a streaming protocol to the,
The system 20 of the plurality of users,
An image processing unit 21 that processes a transparent region of an image in real time so as to separate a transparent region channel of the streaming image received from the plurality of image transmission units 15-1, 15-2, 15-3 through the network;
An image input unit 23 for recognizing a video device and streaming an image to a web browser;
An image synthesizing unit 24 for synthesizing the image of the image input unit 23 in the transparent area of the image processing unit 21 and performing chroma key processing; And
And an image output unit (25) for outputting the image synthesized by the image synthesis unit (24) to the web browser.
A server connected to the network analyzes the image of the front and back frames of the image taken and uploaded with an external camera 11 to distinguish the background area excluding the changed pixel area, calculate the background area to be processed as transparent for each frame, and make the background area transparent An image conversion step of adding an image channel to be processed accordingly;
An image storage step of storing the processed image in the image conversion unit 12 by the server;
A plurality of image transmission units (15-1, 15-2, 15-3) that are distributed by the server by the distributed processing unit (14) to transmit an image to be sent to a designated user's system (20) among a plurality of user's systems (20). ) Storing a temporary video inside the container and generating and transmitting a streaming protocol to the system 20 of the plurality of users;
The transparent region of the image is processed in real time to separate the transparent region channel of the streaming image received from the plurality of image transmission units 15-1, 15-2, 15-3 through the network by the user's system 20. Image processing step;
An image input step of recognizing a video device by the user's system 20 and streaming an image to a web browser;
An image synthesizing step of synthesizing the image of the image input step in the transparent area of the image processing step by the user's system 20 and performing chroma key processing; And
And an image output step of outputting the image synthesized in the image synthesis step by the user's system 20 to the web browser.

Images