KR20230148285A - System for Ultra High-Definition image using chroma-key - Google Patents

Abstract

The present invention relates to a chroma key system for ultra-high definition images, and more specifically, to ultra-high definition images of UHD or higher.
In applying the chroma key technique to the blue screen, the deviation that occurs between each HD video that makes up the ultra-high definition video is used.
This is about a chroma key system for ultra-high definition images that can easily extract composite objects excluding lean. This foot
The chroma key system for ultra-high-definition video according to the standard provides color information for one HD video before the encoder/decoder unit.
As a standard, the deviation from the color information for the remaining HD video is provided to the encoder/decoder unit, or the encoder/decoder unit
Separately, the color of the HD video is equalized using the above deviation, which not only provides a natural ultra-high definition video, but also provides a
Through a simple algorithm, the chroma key processing time can be shortened and the system load can be greatly reduced.

Description

Chroma key system for ultra high-definition images {System for Ultra High-Definition image using chroma-key}

The present invention relates to a chroma key system for ultra-high definition images. More specifically, in applying the chroma key technique to ultra-high definition images of UHD or higher, the blue screen is created by using the deviation that occurs between each HD image that makes up the ultra-high definition image. This is about a chroma key system for ultra-high definition images that can easily extract excluded composite objects.

Currently, the analog video technology is moving from the analog SD (Standard-Definition) to the digital HD (Hi-Definition) technology, and this digital video method makes it easy to adjust sharpness, saturation, and brightness, providing images with image quality that is closer to reality. is providing.

Moreover, the development of hardware that supports various compression technologies and video transmission technologies for processing the digital video is promoting the popularization of HD video and at the same time accelerating the development of video at a level higher than HD. Full HD, which is a higher level than the currently available HD, supports a resolution of 1920×1080 and consists of 2 million pixels, providing significantly higher quality images than HD.

Meanwhile, recent video technology is growing beyond Full HD to UHD (Ultra High-Definition), and UHD, which supports ultra-high definition and ultra-high resolution, is attracting attention as a next-generation media environment. The UHD supports 4K (3840×2160) and 8K (7680×4320) resolutions and up to 22.2 channels of surround audio. Compared to the above HD, UHD provides 4 times clearer picture quality even based on 4K UHD, and in the case of 8K UHD, it provides 16 times clearer picture quality than the above HD.

Meanwhile, along with the development of resolution as mentioned above, video technology is also advancing greatly, and representative examples include applying a 3D virtual studio to the background of an announcer or compositing a background created virtually by graphic designers to the background of an actor acting. The chroma key technique, which enables video synthesis, is being actively applied.

The chromakey technique consists of extracting the blue screen excluding the announcer from the video of the actor or announcer acting against the background of the blue screen, and synthesizing the graphic video through a mixer.

Therefore, this chroma key technique basically extracts the object by removing pixel values corresponding to a color range similar to blue in order to naturally extract the image of the subject against the background of a blue screen. In addition, various improved chromakey techniques have recently been provided, for example, by extracting a closed curve on the subject, extracting the part within the closed curve, removing the background other than the closed curve, or removing a continuously changing color similar to the removed color to relax background restrictions. Various additional technologies such as shikigaki are being applied.

The above chroma key technique is mainly used to process images below HD level, and when applying it to the above-mentioned UHD level, one screen is obtained through multiple cameras, so the color difference and color temperature difference between each screen , deviations in sharpness and color space changes occur and the image must be equalized before background removal, which delays processing time and increases the load on the hardware that encodes the image due to the large volume of UHD images.

Therefore, there is a need to develop a system that enables high-speed processing while reducing the complexity of the chroma key system for these UHD-level ultra-high definition images.

The present invention was proposed to solve the above-mentioned problems, and when multiple HD images are received and an ultra-high definition image is composed, deviations such as color, color temperature, sharpness, color space change, etc. that occur between each HD image are easily corrected. The purpose is to provide clear ultra-high definition images so that there are no boundaries between HD images.

In addition, the present invention can use the result derived from the deviation correction process to delete the blue screen included in each HD image through the chroma key technique and simultaneously correct the object included in each HD image. The purpose is to greatly reduce the processing time for encoding.

In order to achieve the above object, the chroma key system for ultra-high definition video according to the present invention includes an interface unit for dividing UHD video into raw data (law data) corresponding to a plurality of HD videos and receiving the data, and receiving the data through the interface unit. The raw data for the HD images are aligned to correspond to the UHD images, and the reference color information for a single color area of one of the plurality of HD images is extracted and compared with the solid color areas of the remaining HD images, and the reference color information is used as the standard. a pre-processing unit that generates deviation information for each HD video, and codes the raw data for the UHD video provided from the pre-processing unit into a preset format, and when the deviation information is received from the pre-processing unit, before the coding. It includes an encoder/decoder unit that corrects the deviation of each HD video based on the deviation information and deletes the monochromatic area of each HD video corresponding to the reference color information.

At this time, the chroma key system for ultra-high definition video further includes a high-speed storage medium for the UHD video and an output unit for outputting the UHD video to an external output device, and the encoder/decoder unit stores the coded UHD video at high speed. It can be stored in a medium, and the coded video from the high-speed storage medium can be decoded and output through the output unit.

Additionally, the high-speed storage medium may include an E-IDE hard disk drive (HDD) or solid state disk (SSD).

In addition, the pre-processing unit includes a data sorting unit that sorts the raw data for the plurality of HD images received through the interface unit into raw data of the UHD image, and extracts reference color information for the monochromatic area of the first received HD image. It includes a deviation measuring unit that compares the color information of the monochromatic area of other HD images provided by the data alignment unit, and a deviation information generating unit that receives the difference in color information between the HD images compared by the deviation measuring unit and generates deviation information. can do.

At this time, the deviation information may include at least one of color difference, color temperature difference, and color space change.

In addition, it may further include a deviation correction unit that corrects the HD video constituting the raw data of the UHD video sorted by the data sorting unit based on the deviation information.

Meanwhile, the deviation correction unit synchronizes based on time information included in the raw data of each HD video and can correct each HD video frame on a UHD frame basis.

In addition, it may further include a specific color area deletion unit that deletes or blanks pixels corresponding to the color information among the UHD images corrected through the deviation correction unit.

Additionally, the specific color area deletion unit may delete pixels with color information falling within a preset range based on the reference color information.

The chroma key system for ultra-high definition images according to the present invention to achieve the above object includes a plurality of modules that are connected through a LAN to form a network, and each module is connected to a network with other modules through the LAN. A NIC (Network Interface Card) that supports, an interface unit that divides and receives UHD video into raw data (law data) corresponding to multiple HD video frames, a synchronization unit that generates a synchronization clock signal, and the interface unit. The raw data for the HD video received through the HD video is sorted to form a UHD video, and the reference color information for a single color area of one of the HD images is extracted and compared with the monochromatic area of the remaining HD images, and the reference color information is used as the basis. A pre-processing unit that generates deviation information for each HD video and corrects raw data for each HD video based on the deviation information, and a high-speed storage medium that stores raw data for the UHD video corrected by the pre-processing unit. Contains,

It may further include an integrated processing unit that arranges raw data for UHD images stored in the high-speed storage media of each module into raw data for 8K UHD or higher images and processes them in a preset format.

At this time, the line processing unit may blank or delete the monochromatic area of the raw data for each corrected HD image based on the reference color information.

In addition, each module is set as at least one of a master or a slave, and the preprocessor of the master module transmits the reference color information and synchronization information to the LAN through the NIC, and the preprocessor of the slave module transmits the preprocessor of the master module. HD video can be corrected by receiving the reference color information and synchronization information through the NIC.

The chroma key system for ultra-high definition video according to the present invention provides the encoder/decoder with a deviation from the color information for the remaining HD images based on the color information for one HD video before the encoder/decoder, or Separately from the encoder/decoder unit, it not only provides a natural ultra-high definition video by using the above-mentioned deviation to equalize the color of the HD video, but also shortens the chroma key processing time through a simple algorithm, which has the effect of greatly reducing the system load. there is.

In addition, due to the simple system configuration as described above, the present invention can obtain images to which more precise chroma key techniques are applied by adding a separate chroma key algorithm to the encoder/decoder unit, and adjusts the load of the system to each component. By dispersing, system efficiency can be greatly increased.

In addition, the present invention arranges a plurality of modules that apply the algorithm according to the present invention for 4K UHD or higher video, and connects each module through LAN, so that each module displays the image based on the color information generated by one module. By processing, consistent and natural images can be obtained when applying the chroma key technique.

Figure 1 is a configuration diagram of a chroma key system for UHD video.
Figure 2 is a diagram showing the process of generating deviation information according to an embodiment of the present invention.
Figure 3 is a configuration diagram of a preprocessing unit according to an embodiment of the present invention.
Figure 4 is a configuration diagram of a deviation information generator according to an embodiment of the present invention.
Figure 5 is a diagram illustrating the correction process of the line processing unit and the chroma key function according to an embodiment of the present invention.
Figure 6 is a configuration diagram of a chroma key system for UHD video using NIC and LAN according to the present invention.

The present invention is proposed to solve the processing time delay, hardware complexity, and increased hardware load that occur when applying the pre-processing function for applying the chroma key technique to existing HD images to UHD images.

The chromakey technique electronically reads the signal from the camera and systematically removes specific components of the image to create a blank space. In other words, it is a technique of photographing a subject with a background and deleting the background excluding the subject to create a blank space. . By constructing a single screen through video compositing, which combines different images to fill this void with video signals from other cameras, a virtual video screen that does not actually exist can be created.

In other words, in the existing hardware configuration that performs image synthesis using images of resolution below HD, the encoder/decoder part is responsible for all processing functions related to the chroma key technique, but for large-capacity images of UHD or higher that are composed of multiple HD images. It is difficult to apply due to processing time delay due to image uniformity, hardware complexity, and increased load.

Therefore, the present invention can reduce the load caused by applying the chroma key to each HD video when applying the chroma key technique to the encoder/decoder unit, and also simplifies the configuration to reduce the load of the encoder/decoder unit, thereby reducing the load on the encoder/decoder unit. Since various algorithms applied to chroma key can be additionally added to the unit, clearer and more precise UHD images can be provided.

[0032] Based on the above, embodiments of the present invention will be examined in detail with reference to the drawings.

1 is a diagram illustrating a chroma key system for ultra-high definition images according to the present invention, which includes an interface unit 110 that acquires raw data for HD images from a plurality of cameras 10, and the interface unit 110. A pre-processing unit 120 that sorts the raw data for the HD image obtained from the HD image to correspond to the UHD image to extract or chroma key process deviation information for performing chroma key processing, and the pre-processed data through the pre-processing unit 120. It may include an encoder/decoder unit 130 that encodes or decodes UHD video in a preset format, and a high-speed storage medium 140 that stores the video encoded through the encoder/decoder unit 130.

In the present invention, through the above-mentioned configuration, the line processing unit 120 determines the color difference, color temperature difference, and color between each HD image so that the encoder/decoder unit 130 can easily apply the chroma key technique to the UHD image. Deviation information on the degree of space change, etc. is provided, and the encoder/decoder unit 130 compensates for the difference in picture quality between each HD video based on the deviation information and at the same time applies a chroma key technique using the deviation information, The processing time of the encoder/decoder unit 130 according to the synthesis process can be greatly reduced and the load can be reduced.

In addition, when generating the deviation information, the line processing unit 120 compares the portion corresponding to the monochromatic area between each HD video frame, that is, the blue screen area between each HD video frame, and determines the overall degree of deviation between each HD video frame. Therefore, the processing time and load of the pre-processing unit 120 can also be greatly reduced to the point where real-time provision of UHD video is satisfactory.

Meanwhile, in order to reduce the load caused by the chroma key processing of the encoder/decoder unit 130, the pre-processing unit 120 directly corrects the deviation due to the picture quality or color difference between each HD video frame using the deviation information. Chroma key processing can be performed to provide a corrected UHD video frame to the encoder/decoder unit 130.

In addition, the pre-processing unit 120 can synchronize the HD video frames of each camera 10 by comparing the time information of the raw data for the HD video provided in real time through the interface unit 110, thereby synchronizing the multiple synchronized HD video frames. After arranging the raw data for HD video frames, deviation information for each HD video can be extracted or chroma keyed for each UHD video frame.

Figure 2 is a diagram illustrating a process in which the line processing unit 120 generates deviation information according to an embodiment of the present invention. 1 to 4, which are a plurality of HD video frames constituting one UHD video frame, are a plurality of cameras. It is obtained through

Therefore, assuming that the first received HD video frame is HD video frame 1, the preprocessor 120 extracts a portion consisting of a monochromatic area from HD video frame 1 to obtain reference color information, and uses the reference color information as As a basis, the deviation information can be generated by comparing the monochromatic areas of each remaining HD video frame 2 to 4 received after HD video frame 1.

Thereafter, the encoder/decoder unit 130 performs correction on the entire HD video frame based on the deviation information, and through this, a plurality of uniform HD images without color differences can be obtained. Additionally, the encoder/decoder unit 130 can align the raw data of each corrected HD video frame into UHD video frames and then apply a chroma key technique to delete or blank the blue screen area.

At this time, the line processing unit 120 may include reference color information, which is a reference value of the deviation information, in the deviation information and provide it to the encoder/decoder unit 130. By applying the chroma key technique to each HD video frame based on color information to blank the blue screen, separate settings and processing steps for detecting the blue screen area can be omitted, significantly reducing processing time and load. .

In addition, the pre-processing unit 120 provides different values of reference color information for uniforming the image quality of each HD video for each UHD video frame constituting the video, so unlike hardware optimized for existing HD video, the present invention provides UHD video. It is optimized for the frame and easily extracts reference color information from the monochromatic area of one HD video frame without the need to analyze the entire screen and performs a uniformization process based on this, so it can provide UHD video with optimized picture quality. It can satisfy the real-time processing level of HD video even for large-capacity, high-definition video, such as UHD-level video, which had limitations in real-time video processing with existing hardware that supports HD video.

3 to 5 are diagrams showing various embodiments of the configuration of the preprocessing unit as described above. Looking first at FIG. 3, the preprocessing unit 120 includes a data alignment unit 121, a deviation measurement unit 122, It may be comprised of a deviation information generating unit 123.

The data alignment unit 121 receives raw data for a plurality of HD video frames from the interface unit 110 and sorts them to form a UHD video frame or specifies a position to be configured in the UHD video frame, and the deviation measurement unit (122) can analyze the monochromatic area based on one of the HD video frames aligned in the data sorting unit (121).

Thereafter, the deviation measuring unit 122 extracts the reference color information of the monochromatic region and calculates the deviation by comparing the reference color information with the color information of the monochromatic region included in the remaining HD video frames.

At this time, elements included in the reference color information may include color information, color temperature information, brightness information, etc., and the reference color information may be an average value of color information included in pixels included in a single color area.

The calculated deviation is transmitted to the deviation information generating unit 123, and based on this, the deviation information generating unit 123 generates deviation information and provides it to the encoder/decoder unit 130. At this time, the deviation information generator 123 may provide the reference color information together with the deviation information.

Therefore, as shown in FIG. 4, the deviation information generator 123 uses the reference color information, which is a reference value generated by the deviation measurement unit 122, based on a single color area of one of a plurality of HD video frames to measure the deviation. After receiving from the unit 122, deviation information including color difference information, color temperature difference, color space change, etc. is generated according to the calculation of the deviation measurement unit 122.

Meanwhile, the data alignment unit 121 transmits raw data for a plurality of HD video frames aligned to correspond to the UHD video frames to the encoder/decoder unit 130, and the encoder/decoder unit 130 Based on the reference color information, the image quality or color of the raw data of each HD video frame is corrected according to the above deviation information, and through this, the image quality or color of each HD video frame is mutually uniform. Raw data can be configured.

Thereafter, the encoder/decoder unit 130 converts the raw data for the corrected UHD video frame into a preset format, and converts pixels corresponding to the monochromatic area to blank or deletes them based on the reference color information. Apply chromakey technique. Thereafter, image synthesis can be performed by filling the blank portion through mixing with various images received through the interface unit 110.

Meanwhile, the correction process and chroma key function of the encoder/decoder unit 130 may be processed in the preprocessor 120. Looking at this through FIG. 5, in the configuration of FIG. 3, the deviation correction unit 124 and the specific A color area deletion unit 125 may be further configured.

The deviation correction unit 124 obtains the deviation information from the deviation information generating unit 123 and stores the raw data of each HD video frame constituting the UHD video frame sorted by the data sorting unit 121. Color correction is performed based on reference color information and deviation information, and the specific color area deletion unit 125 blanks or deletes pixels included in the raw data of each HD video frame based on the reference color information. Maki can be performed.

In this way, the pre-processing unit 120 provides raw data for UHD video frames on which both correction and chroma key have been processed to the encoder/decoder unit 130, and the encoder/decoder unit 130 provides the raw data. Carry out the formatting process according to the video format.

Therefore, processing performance can be further improved by greatly reducing the processing process of the encoder/decoder unit 130 and reducing data traffic exchanged between the pre-processing unit 120 and the encoder/decoder unit 130.

Thereafter, the pre-processing unit 120 may perform correction and chroma key processing for each continuously received UHD video frame through the process described above.

Meanwhile, the above-mentioned UHD video is explained based on 4K UHD, and in the case of 8K UHD, it can be processed with the following system configuration.

That is, as shown in FIG. 6, the interface unit 110, the pre-processing unit 120, and the high-speed storage medium 140 are configured as one module 200, and the module 200 is used to process 8K UHD images. ) and may include a separate integrated processing unit 400 that is connected to the plurality of modules through the LAN and performs the function of the encoder/decoder unit.

In addition, each of the modules 200 includes a network interface card (NIC) 300 and can exchange control signals through a LAN connected to the NIC 300.

At this time, each module 200 may further include a synchronization unit, and the pre-processing unit 120 is configured to perform synchronization for each HD video frame processed by each module 200 through the NIC 300. It can be connected to the synchronization unit to transmit a synchronization signal, and can provide a control signal for the reference color information.

Through this, the pre-processing unit 120 of each module 200 that processes 4K UHD images extracts deviation information for each HD image included in the 4K UHD image based on the reference color information included in the control signal. After performing the correction and chroma key processing, the UHD video frame can be stored in the form of raw data on the high-speed storage medium 140.

In addition, one of the plurality of modules 200 may be set as a master and the remaining modules 200 may be set as slaves, and the master module 200 may transmit the synchronization signal and the control signal to set the remaining slave modules ( 200) performs synchronization of HD video frames according to the synchronization signal, and can correct image quality or color according to the control signal.

Through this, the integrated processing unit 400 receives raw data of 4K UHD video frames for which correction and chroma key processing has been completed in order to generate 8K UHD images from the high-speed storage medium 140 of each module 200 through LAN. After that, it can be arranged to form an 8K UHD video frame for video synthesis or conversion to a preset format. Through this, it is possible to equalize the image quality or color of each HD video frame included in the 4K UHD video that makes up the 8K UHD video, and provide 8K UHD video with clear picture quality.

In this way, chroma key processing can be easily performed not only on 4K UHD video but also on 8K UHD video or higher video by configuring and connecting a plurality of modules 200.

10: Camera 110: Interface unit
120: Pre-processing unit 121: Data sorting unit
122: Deviation measurement unit 123: Deviation information generation unit
124: Deviation correction unit 125: Specific color area deletion unit
130: Encoder/decoder unit 140: High-speed storage medium
200: module 300: NIC
400: Integrated processing department

Claims (12)

An interface unit that receives UHD video by dividing it into raw data (law data) corresponding to a plurality of HD videos;
Raw data for HD images received through the interface unit is aligned to correspond to UHD images, and reference color information for a single color area of one of the plurality of HD images is extracted and compared with the monochromatic areas of the remaining HD images. A pre-processing unit that generates deviation information for each HD image based on reference color information; and
The raw data for the UHD video provided from the pre-processing unit is coded in a preset format, and when deviation information is received from the pre-processing unit, the deviation of each HD video is corrected based on the deviation information before coding and the Chromakey system for ultra-high definition video that includes an encoder/decoder unit that deletes the monochromatic area of each HD video corresponding to color information In claim 1,
A high-speed storage medium for storing the UHD video; and
Output unit that outputs UHD video to an external output device
It further includes,
The encoder/decoder unit stores the coded UHD video in the high-speed storage medium, decodes the coded video from the high-speed storage medium, and outputs the coded image through the output unit. In claim 1,
A chroma key system for ultra-high definition images, wherein the high-speed storage medium includes an E-IDE hard disk drive (HDD) or solid state disk (SSD). In claim 1,
The preprocessing unit
a data sorting unit that sorts raw data for the plurality of HD images received through the interface unit into raw data of UHD images;
a deviation measuring unit that extracts reference color information for a monochromatic area of an initially received HD image and compares it with color information for a monochromatic area of another HD image provided by the data sorting unit; and
A chroma key system for ultra-high definition images, including a deviation information generator that generates deviation information by receiving differences in color information between HD images compared by the deviation measurement unit. In claim 4,
The chroma key system for ultra-high definition images, wherein the deviation information includes at least one of color difference, color temperature difference, and color space change. In claim 4,
A chroma key system for ultra-high definition video, further comprising a deviation correction unit that corrects the HD video constituting the raw data of the UHD video sorted by the data alignment unit based on the deviation information. In claim 6,
The deviation correction unit synchronizes based on time information included in the raw data of each HD video and corrects each HD video frame on a UHD frame basis. A chroma key system for ultra-high definition video. In claim 4,
A chroma key system for ultra-high definition images, further comprising a specific color area deletion unit that deletes or blanks out pixels corresponding to the color information among the UHD images corrected through the deviation correction unit. In claim 8,
The chroma key system for ultra-high definition images, wherein the specific color area deletion unit deletes pixels with color information falling within a preset range based on the reference color information. It includes a plurality of modules that are connected via LAN to form a network,
Each module above is
A NIC (Network Interface Card) that supports network connection with other modules through the LAN;
An interface unit that receives UHD video by dividing it into raw data (law data) corresponding to a plurality of HD video frames;
A synchronization unit that generates a synchronization clock signal;
The raw data for the HD images received through the interface unit is sorted to form a UHD image, and the reference color information for a single color area of one of the HD images is extracted and compared with the monochromatic areas of the remaining HD images to determine the reference color. a pre-processing unit that generates deviation information for each HD image based on information and corrects raw data for each HD image based on the deviation information; and
It includes a high-speed storage medium that stores raw data for the UHD image corrected by the pre-processing unit,
A chroma key system for ultra-high definition video, further comprising an integrated processing unit that arranges the raw data for the UHD video stored in the high-speed storage medium of each module into raw data for the 8K UHD video or higher and processes it in a preset format. In claim 10,
The chroma key system for ultra-high definition images, wherein the pre-processing unit blanks or deletes monochromatic areas of raw data for each corrected HD image based on the reference color information. In claim 10,
Each of the modules is set to at least one of master or slave,
The pre-processing unit of the master module transmits the reference color information and synchronization information to the LAN through the NIC, and the pre-processing unit of the slave module receives the reference color information and synchronization information transmitted by the pre-processing unit of the master module through the NIC to produce HD video. A chroma key system for ultra-high definition images that is characterized by correction.