WO2016006746A1

WO2016006746A1 - Device for super-resolution image processing

Info

Publication number: WO2016006746A1
Application number: PCT/KR2014/006285
Authority: WO
Inventors: 이상설; 장성준; 현상진; 김제우
Original assignee: 전자부품연구원
Priority date: 2014-07-11
Filing date: 2014-07-11
Publication date: 2016-01-14
Also published as: KR20160008011A

Abstract

The present invention relates to a device for super-resolution image processing, and the present invention provides a device for image processing, comprising: a stream generation unit for dividing a source image into at least two cores and generating and outputting a stream for each of the divided core images; an encoder provided with at least two multi-core processing units compressively processing the at least two divided image cores, and receiving, from each of the multi-core processing units, a core image stream outputted through the stream generation unit so as to output the core image stream after compressing the same; and a first-in-first-out (FIFO) unit for receiving the compressed core images outputted through the encoder and FIFO-processing the compressed core images.

Description

Device for Ultra High Resolution Image Processing

The present invention relates to an apparatus for processing an ultra high resolution image, and more particularly, an ultra high resolution image processing capable of processing, compressing, and restoring multi-core based data for encoding and decoding an input ultra high resolution image in real time. It relates to a device for.

Recently, image media having various functions and high quality images have been rapidly developed, and as the demands of consumers watching the image media have also increased, the necessity for ultra-high definition images is emerging.

In this situation, it goes beyond the 2K (1,920 × 1,080) HD (High Definition) broadcasting that is currently being broadcast, and it has been applied to digital cinema and ultra high definition television (UHDTV). Technology development is active.

The digital cinema has a 4K (4,096 × 2,096) resolution, and the ultra-high definition TV has an 8K (7,680 × 4,320) resolution. Viewers who watch these 8K-class ultra-high resolution videos can view the images more realistically and realistically, so that both video producers and consumers can have a positive effect.

Currently, most of the codecs generally used for image processing are single-core and use real-time compression technology of HD-quality video.

However, for 8K-class processing, the frame buffer needed to process the image input from the frame buffer must be continuously buffered until one image is processed. That is, the structure of the frame buffer currently used mainly requires that the image is stored in the buffer for a time until the completion of one frame compression and reconstruction to process the image of the input frame.

In addition, since the image is compressed and reconstructed sequentially from the first pixel of the first line, the frame buffer size increases and the image processing time is delayed, thereby causing a high occupancy time of the image stored in the frame buffer.

That is, when the image is processed in the above-described manner in the 8K image, there is a problem that requires 16 times as much buffer size and 16 times longer image processing time than the above-described processing of the HD class image.

Accordingly, there is an urgent need to develop a technology for processing in real time by reducing processing time for ultra-high resolution images.

Accordingly, an object of the present invention in view of the above-described point is to provide an apparatus for processing ultra-high resolution images for compressing and reconstructing an ultra-high resolution image in real time.

In addition, another object of the present invention is to provide an apparatus for processing ultra-high resolution image to be compressed and reconstructed in real time through a multi-core based structure when processing the ultra-high resolution image.

An apparatus for image processing according to a preferred embodiment of the present invention for achieving the above object is a stream for dividing the source image into at least two cores, and generates and outputs a stream for each divided core image Generation unit; An encoder having at least two multi-core processing units for compressing at least two divided image cores, and receiving and compressing a core image stream output through the stream generating unit from each of the multi-core processing units; And a first-in first-out (FIFO) unit for receiving a compressed core image output through the encoder and performing first-in first-out on the compressed core image.

In the apparatus for image processing according to the present invention, the encoder comprises: at least one buffer for temporarily storing a core stream for the divided core image; At least one predictor for motion compensation in a corresponding partition; And at least one image compressor which performs compression on each core image whose motion is compensated through the predictor.

In the apparatus for image processing according to the present invention, the stream generation unit is provided in front of an input / output terminal of an image and generates a stream according to the ITU-R standard according to an interface representing a cable and a terminal for transmitting an image. Characterized in that provided with.

In the apparatus for image processing according to the present invention, when the source image is an 8K image, the stream generation unit divides the HD image into 16 core images, and generates and outputs 16 divided core image streams. It is done.

In accordance with an aspect of the present invention, there is provided an apparatus for image processing, comprising: a buffer configured to store at least two inputted and compressed core images; A distributor for distributing and outputting the core image compressed in two or more parts stored in the buffer; A decoder for decoding each compressed core image output through the distributor and outputting the compressed core image in the form of an output stream; When the output stream output through the decoder is input, it characterized in that it comprises a stream generating unit for sequentially outputting the output stream for each core image to be a video image.

In the apparatus for image processing according to the present invention, the decoder comprises: at least one reconstruction unit for decoding the compressed core image input through the distributor; And at least one output stream generator for outputting the decoded core image output through the at least one reconstruction unit as an output stream.

As described above, according to an exemplary embodiment of the present invention, for real-time compression and reconstruction of an image having an ultra high resolution of 8K or more, compression and reconstruction may be performed on a multi-core basis instead of a single core. As such, as the image is processed based on the multi-core, the usage of the frame buffer can be reduced.

In addition, the raw image to be compressed may be buffered in a small frame buffer size in comparison with a single core to perform real-time processing.

In addition, it is possible to provide an input / output stream technology capable of replacing various video codecs.

1 is a block diagram illustrating an apparatus for ultra-high resolution image processing according to an embodiment of the present invention;

2 is a block diagram illustrating a detailed internal configuration of an encoder in an apparatus for processing ultra high resolution images according to an embodiment of the present invention;

3 is a block diagram for explaining a detailed internal configuration of a decoder in an apparatus for ultra-high resolution image processing according to an embodiment of the present invention; And

4 is an exemplary diagram illustrating a source image divided into a multi-core form in order to explain an embodiment of the present invention.

Prior to the description of the present invention, the terms or words used in the specification and claims described below should not be construed as being limited to the ordinary or dictionary meanings, and the inventors should consider their own invention in the best way. For the purpose of explanation, it should be interpreted as meaning and concept corresponding to the technical idea of the present invention on the basis of the principle that it can be appropriately defined as the concept of term. Therefore, the embodiments described in the present specification and the configuration shown in the drawings are only the most preferred embodiments of the present invention, and do not represent all of the technical idea of the present invention, and various equivalents may be substituted for them at the time of the present application. It should be understood that there may be water and variations.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this case, it should be noted that like elements are denoted by like reference numerals as much as possible. In addition, detailed descriptions of well-known functions and configurations that may blur the gist of the present invention will be omitted. For the same reason, some components in the accompanying drawings are exaggerated, omitted, or schematically illustrated, and the size of each component does not entirely reflect the actual size.

Hereinafter, in the present invention, in order to edit and post-process an ultra high resolution image of 8K class in a broadcasting and entertainment system, multi-core format real-time compression and reconstruction for compressing an image of 60 frames or more in real time during compression of input image data is performed. We would like to propose a device that can do this. It also defines the input and output streams of each of the multicores.

1 is a block diagram illustrating an apparatus for processing ultra high resolution images according to an exemplary embodiment of the present invention.

Referring to FIG. 1, an apparatus 100 for image processing according to an exemplary embodiment of the present invention includes an encoder 200 and a decoder 300.

The encoder 200 performs compression on the input source image and outputs the compressed image. The encoding unit 200 according to the present invention divides the input source image into at least two cores, compresses each of the divided cores through an SDI interface having a parallel processing structure, and outputs the same. Here, the SDI interface serves as a multi-core processor for processing a multi-core image.

The encoder 200 is configured to include a stream generator 210, an encoder 220, and a FIFO unit 230.

The stream generator 210 splits the source image into at least two cores. According to the present invention, in order to process an image based on a multi-core, an original image having an ultra high resolution is divided into at least two core images. This image segmentation will be described with reference to FIG. 4.

Referring to FIG. 4, assuming that an original image has a size of 7680x4320, it may be divided into four UHD (3840x2160), which may be divided into 16 HD-class images (1920x1080). In addition, if the size of the 8K image or more size may be applied to the structure that is divided into more cores. That is, the core image divided into four is Qa, Qb, Qc, and Qd, and the core image divided into sixteen is divided into core 0 (core0), core1 (core1), which are divided from each of Qa, Qb, Qc, and Qd. It will be core2 and core3.

The stream generator 210 generates and outputs a stream for each core image divided with respect to the original image as described above.

The stream generator 210 is provided in front of an input / output terminal of an image, and may be provided according to the ITU-R standard according to a type of an interface representing a cable and a terminal for transmitting an image. In general, most SDI interface types are used in broadcasting equipment. Therefore, BT.1120, a standard for SDI input, is used to handle this. If the interface is changed due to HDMI input, the BT is placed in front of the input / output terminal of the video. .1120 Apply stream generator (ITU-R standard) to input or output.

The encoder 220 includes at least two or more multicore processors for compressing at least two image cores divided into at least two cores, and receives a core image stream output through the stream generator 210 into each of the multicore processors. Output after compressing. Such a detailed internal configuration of the encoder 220 according to the present invention will be described in detail with reference to FIG. 2.

FIG. 2 is a block diagram illustrating a detailed internal configuration of an encoder in an apparatus for processing ultra high resolution images according to an embodiment of the present invention.

Referring to FIG. 2, the encoder 220 includes a

buffer memory

20, 21, 22,..., 23, a

prediction unit

30, 31, 32,..., 33, and an image compressor.

Processors

40, 41, 42, ..., 43 are included. Here, the buffer 20, the predictor 30, and the image compressor 40 have one single core structure, and the single core structure is formed in parallel with the number of cores obtained by dividing the original image, as shown in FIG. Will be

The

buffers

20, 21, 22, ..., 23 temporarily store core streams for the divided core images.

Prediction units

30, 31, 32,..., 33 perform motion compensation on the split core image.

An

image encoding unit

40, 41, 42, ..., 43 performs compression on each core image that is motion-compensated through the

prediction units

30, 31, 32, ..., 33. .

Referring back to FIG. 1, the first-in first-out (FIFO) unit 230 receives the compressed core image output through the encoder 220 and processes the first-in first-out on the compressed core image.

The decoding unit 300 decodes the input compressed image. The decoding unit 300 according to the present invention includes a buffer BUF 310, a memory controller 320, a decoder 330, and a stream generator 340.

The buffer memory 310 stores a core image which is divided and compressed into at least two input signals.

The memory controller 320 distributes and outputs the core image compressed into two or more parts stored in the buffer 310.

The decoder 330 decodes each compressed core image output through the distributor 320 and outputs the compressed core image in the form of an output stream. A detailed internal configuration of the decoder 330 according to the present invention will be described in detail with reference to FIG. 3.

3 is a block diagram illustrating a detailed internal configuration of a decoder in an apparatus for processing ultra high resolution images according to an embodiment of the present invention.

Referring to FIG. 3, the decoder 330 may decode the

decoding processors

50, 51, 52,..., 53 and the

output stream generators

60, 61, 62,. It is configured to include.

The

decoding processors

50, 51, 52,..., 53 decode the compressed core image input through the distributor 320.

The

output stream generator

60, 61, 62,..., 63 outputs the decoded core image output through the at least one

restoration unit

50, 51, 52,..., 53 as an output stream. Output

Referring back to FIG. 1, when the output stream output through the decoder 330 is input, the stream generator 340 sequentially outputs an output stream for each core image to be one video image.

In other words, the structure for processing a single core includes an encoder 220 and a FIFO unit 230 including a stream generator 210, a buffer 20, a predictor 30, and an image compressor 40. Decoder including an encoder 200, a buffer 310, a divider 320, a restorer 50, a decoder 330 composed of an output stream generator 60 and a stream generator 340 including a; 330 can be viewed and processed as one set, and a structure capable of storing or acquiring the processed data to the FIFO unit 230 when each core is required by providing the FIFO unit 230. to be.

The single core structure configured as described above processes data based on HD-class video in the SDI interface, which is a current input / output type, and when processing 8K video with a single core in an apparatus for processing ultra-high resolution images according to the present invention. A separate process is required to combine the images from each of the 16 SDI interfaces. Accordingly, the multi-core processing unit is configured in parallel in each encoder 220, and the reconstruction unit and the output stream generator are configured in parallel in order to process each divided compressed core image in the decoder 330.

As described above, when the image is processed based on the multi core, the compression rate may be improved.

In addition, as the input / output interfaces are unified, it is unnecessary to develop an interface for the processing and display of a codec to be developed in the future, so that only the encoder 220 block and the decoder 330 block are replaced and used to increase the reusability. Can be.

In addition, the encoding unit 220 according to the present invention has a configuration of a compression codec that supports a variable parallel processing structure. In the case of using four, the UHD (4K) system can be applied to a movie theater that outputs images horizontally in the case of placing eight images horizontally.

While the invention has been described using some preferred embodiments, these embodiments are illustrative and not restrictive. As such, those of ordinary skill in the art will appreciate that various changes and modifications may be made according to equivalents without departing from the spirit of the present invention and the scope of rights set forth in the appended claims.

Claims

A stream generator for dividing the source image into at least two cores and generating and outputting a stream for each of the divided core images;

An encoder having at least two multi-core processing units for compressing at least two divided image cores, and receiving and compressing a core image stream output through the stream generating unit from each of the multi-core processing units;

A first-in first-out (FIFO) unit for receiving a compressed core image output through the encoder and first-in first-out processing on the compressed core image;

Apparatus for image processing comprising a.
The method of claim 1, wherein the encoder,

At least one buffer to temporarily store a core stream for the split core image;

At least one predictor for motion compensation in a corresponding partition; And

At least one image compressor for compressing each core image motion-compensated through the predictor;

Apparatus for image processing comprising a.
The method of claim 1, wherein the stream generating unit,

An apparatus for image processing, comprising: a stream generating unit according to the ITU-R standard according to a type of an interface representing a cable and a terminal for transmitting an image.
The method of claim 1, wherein the stream generating unit,

When the source image is an 8K-class video image, the HD image is divided into 16 core images, and the image processing apparatus for generating a core image stream divided into 16 outputs.
A buffer for storing at least two divided compressed core images input;

A distributor for distributing and outputting the core image compressed in two or more parts stored in the buffer;

A decoder for decoding each compressed core image output through the distributor and outputting the compressed core image in the form of an output stream;

A stream generator for sequentially outputting an output stream for each core image to be a video image when an output stream output through the decoder is input;

Apparatus for image processing comprising a.
The method of claim 5, wherein the decoder,

At least one reconstruction unit configured to decode a compressed core image input through the distributor;

At least one output stream generator configured to output a decoded core image output through the at least one reconstruction unit as an output stream;

Apparatus for image processing comprising a.