KR20090089960A

KR20090089960A - A method for encoding and decoding of ultra-high definition image and apparatus thereof

Info

Publication number: KR20090089960A
Application number: KR1020080015144A
Authority: KR
Inventors: 오승균
Original assignee: 엘지전자 주식회사
Priority date: 2008-02-20
Filing date: 2008-02-20
Publication date: 2009-08-25

Abstract

The present invention relates to a method for efficiently compressing and reconstructing an ultra high resolution image and an apparatus using the same.

More specifically, in order to compress and reconstruct an ultra-high resolution image, one picture is divided into a plurality of subpictures, and compression and reconstruction are independently performed on each subpicture. In this case, each of the divided subpictures is macrosliced. Super-resolution video encoding and decoding method that enables information sharing between adjacent macroslices within a single picture to enable deblocking filtering of macroslice boundaries and to perform motion compensation more efficiently. To the device.

Description

A method for encoding and decoding of ultra-high definition image and apparatus

Recently, image media having various functions and high quality images have been rapidly developed, and as the demands of consumers watching the image media have also increased, the necessity for ultra-high definition images is emerging.

In this situation, it goes beyond the 2K (1,920 × 1,080) HD (High Definition) broadcasting that is currently being broadcast, and it has been applied to digital cinema and ultra high definition television (UHDTV). Technology development is active.

The digital cinema has a 4K (4,096 × 2,096) resolution, and the ultra-high definition TV has an 8K (7,680 × 4,320) resolution. Viewers who watch these ultra-high resolution videos can view the images more realistically and realistically, so that both video producers and consumers can have a positive effect.

However, some of the following problems occur when dealing with such ultra-high resolution images.

Since the current technology cannot encode or decode an ultra high resolution image into a single unit using a single system, each picture included in the sequence must be divided into a plurality of subpictures, and each encoding and decoding process must be performed independently. .

That is, after splitting a super resolution video input from an ultra high resolution camera into a plurality of subpictures, encoding each subpicture independently using a compression system, multiplexing each image bitstream, and then performing a transmission path. Send it through.

In this case, each image compression unit compresses only a sub picture that is in charge of itself independently of other image compression units, outputs a corresponding sub picture bitstream, and manages each sub picture completely independently.

In addition, each subpicture constituting one picture is regarded as a separate independent sequence by mapping each divided subpicture into a sequence.

Therefore, since each subpicture is regarded as a separate sequence, a single picture is divided into a plurality of sequences by the number of divided subpictures, and an encoding and decoding process is performed.

This conventional configuration uses a compression system that is independent of each other between subpictures, which wastes a bit amount. For example, to construct an 8K video system of UHD using a 2K compression system of HD, which is mainly used, the image processing speed is significantly reduced since 16 2K compression systems are required. In addition, since 16 picture headers are required for each picture, an additional bit amount is increased.

In addition, in the case of using the H / 264 / AVC or VC-1 and the compression standard, since the ultra-high resolution image is encoded and decoded independently for each subpicture, it is necessary to decode near the boundary of the divided subpicture. There is a problem that blocking filtering cannot be performed.

Therefore, in the case of realizing 4K or 8K images using a 2K compression system, the deblocking filtering may not be performed on the image boundary, so that the vicinity of the boundary may be illuminated with a grid, which is a major cause of deterioration of image quality. It becomes the cause.

Although the post-processing operation can reduce the distortion between the image boundaries to some extent, the post-processing operation is not performed in the image reconstruction loop, so the deblocking filtering efficiency is low.

In addition, if a motion vector pointing to the outside of the image is set in the outer portion of each divided subpicture, the information using neighboring subpictures cannot be used efficiently in a configuration using a conventional independent encoding and decoding method. There is a problem that motion compensation cannot be performed.

1 to 3 illustrate a problem when processing an ultra high resolution image in the prior art.

1 is a schematic diagram showing a state of a divided picture during encoding and decoding of a high resolution image according to the related art.

As described above, one picture 10 constituting the ultra-high resolution image includes a plurality of subpictures 11 to 16, and each subpicture is independently encoded and decoded and treated as a separate sequence. Therefore, the macroblock existing at the boundary of each subpicture does not perform deblocking filtering to cause deterioration of image quality.

FIG. 2 is a diagram for explaining a case in which a motion vector faces outside of a subpicture in the prior art.

Each picture 10 constituting the ultra-high resolution image includes a plurality of subpictures 11 to 14, each of which is independently encoded and decoded. Therefore, information in other subpicture regions cannot be used.

For example, as shown in FIG. 2, the object 15 that existed in subpicture 0 (11) in the N-1 picture is a subpicture different from the subpicture 0 (11) that existed in the N-1 picture in N picture. An object 15 'exists in subpicture 3 (14).

Then, as shown in the N picture, the motion vector points outside the sub picture 3 (14), and the sub picture 3 (14) is performed independently to be considered as a separate sequence. Since the area of) cannot be used, the estimation error inevitably becomes large.

To solve this problem, a method of pixel padding is used as will be described later with reference to FIG. 3.

FIG. 3 is a diagram for describing pixel padding to compensate for a motion vector when the motion vector faces the outside of the subpicture in the prior art.

In H.264 / AVC, if the motion vector points outside of the picture, or points to another subpicture, the value of that region is not available, so the pixel value with the outermost pixel of the picture or subpicture is used as the estimated value. use. In FIG. 3, the outermost pixel values are padded in the same rows and columns as the outermost layers.

However, the use of such a pixel padding method does not use the value of the region actually indicated by the motion vector, but uses an estimate using the outermost pixel value, resulting in a large error and thus a large bit rate.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to efficiently encode and decode an ultra-high resolution image to effectively compensate for distortion between images and to minimize an error in motion compensation to provide an image with high quality. The present invention proposes a method and apparatus for encoding and decoding for ultra high resolution.

In addition, another object of the present invention is to provide a syntax required for a video compression standard to effectively compress and reconstruct a segmented image.

In order to solve the above problems, the following is proposed.

According to an embodiment of the present invention, there is provided a method of encoding an ultra high resolution video, comprising: dividing a picture into a plurality of subpictures; Mapping the divided subpictures into macro slices, respectively; Encoding each macroslice; And multiplexing the bitstream generated by the encoding.

According to another exemplary embodiment of the present invention, there is provided a method of decoding a high resolution video, the method comprising: parsing an input bitstream in units of macro slices; Decoding each parsed macroslice; And synthesizing the decoded image.

According to another embodiment of the present invention, an apparatus for encoding an ultra high resolution image includes: an image divider configured to divide a picture of an ultra high definition image into a plurality of subpictures; A mapping unit for mapping the plurality of subpictures divided by the image splitter into macro slices, respectively; An image compressor for encoding respective macro slices mapped by the mapping unit to generate respective bitstreams; A bitstream multiplexer configured to receive and multiplex the bitstream generated by the image compressor; And a memory unit for storing information of the macroblock existing at the boundary between the bitstream generated by the image compression unit and each macroslice.

According to the present invention, when a super high resolution image is divided into multiple divided images and compressed using a multiprocessor in an ultra high resolution imaging system, efficient compression can be performed using the concept of macroslice.

In addition, according to the present invention, a single image bitstream can be generated by not using a plurality of conventional picture lasers using a macro slice.

In addition, according to the present invention, it is possible to obtain high quality image quality by enabling deblocking filtering at each subpicture boundary.

In addition, according to the present invention, even when the motion vector is out of each subpicture region, the compression efficiency can be increased by performing effective motion compensation.

In addition, according to the present invention, by using a multiprocessor to implement an ultra-high resolution video system, the burden of processing speed and throughput of a single processor can be reduced.

Advantageously, said divided subpictures have the same size and said macroslice includes one or more slices.

Preferably, the rightmost macroblock and the bottommost macroblock of each macroslice are encoded in intra mode, and macroblocks present at the boundaries of each macroslice are encoded before each macroslice.

Preferably, the method further comprises performing deblocking filtering on each macroslice, wherein the macroblocks existing at the boundaries of each macroslice are decoded before each macroslice.

Preferably, the decoding of each macro slice further includes performing motion compensation of each macro slice, and if the motion vector for the motion compensation points outside of the macro slice, the value of the macro slice is determined. To perform the motion compensation.

Preferably, when the motion vector for the motion compensation points to the outside of the picture, the motion compensation is performed using the padding value of the outermost pixel.

According to still another embodiment of the present invention, there is provided an apparatus for encoding an ultra high definition image, the apparatus comprising: an image divider for dividing a picture constituting an ultra high definition image into a plurality of subpictures; A mapping unit for mapping the plurality of subpictures divided by the image divider into macro slices, respectively; An image compressor for encoding respective macro slices mapped by the mapping unit to generate respective bitstreams; A bitstream multiplexer configured to receive and multiplex the bitstream generated by the image compressor; And a memory unit for storing information of the macroblock existing at the boundary between the bitstream generated by the image compression unit and each macroslice.

Preferably, the memory unit may include: a temporary memory unit for storing macroblock information, pixel information, and pointer information existing at the boundary of each macroslice; A sub memory unit for referring to the information stored in the temporary memory and storing an image of each macro slice; And a main memory unit configured to store all picture information including each macro slice and syntax information above the macro slice layer.

Preferably, the plurality of subpictures divided by the image splitter have the same size, and the rightmost macroblock and the bottommost macroblock of each macro slice are encoded in an intra mode.

Preferably, the apparatus further includes a main compression unit that encodes the upper layer syntax of the macroslice layer, and the image compression unit shares the upper syntax.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily understand and implement the present invention.

4 is a block diagram showing the configuration of an ultra-high resolution video encoding apparatus according to an embodiment of the present invention.

An ultra-high resolution video encoding apparatus according to an embodiment of the present invention includes an image splitter 100 for dividing each picture constituting an ultra-high resolution video into a plurality of subpictures; A mapping unit 200 for mapping each of the plurality of subpictures divided by the image division unit 100 into macro slices; An image compressor (300) for encoding respective macro slices mapped by the mapping unit (200) to generate respective bit streams; A bitstream multiplexer 400 for receiving and multiplexing the bitstream generated by the image compressor 300; And a memory unit 500 in which information of the macroblock existing at the boundary between the bitstream generated by the image compressor 300 and each macroslice is stored.

The memory unit 500 refers to a temporary memory unit 530 in which information of macroblocks, pixel information, pointer information, and the like, which exist at the boundary of each macroslice, is stored in the temporary memory 530. A sub memory unit 520 for storing images of each macro slice, a main memory unit 510 for storing entire picture information including each macro slice and syntax information above the macro slice layer.

When an ultra high resolution image, for example, a 4K or 8K image, is received, a picture is divided into a plurality of subpictures by the image divider 100. In this case, the image divider 100 may divide the received image picture into the same size. Alternatively, the image may be divided into subpictures having different sizes or different resolutions.

In the case of dividing a video picture received with the same size, the video picture can be divided into various sizes, but the resolution is standardized so as to make the best use of the conventional video compression system, for example, the resolution size of 2K video used in HD video. The picture can be divided.

Therefore, when using a 2K image system, the 4K image is divided into four processes, and the 8K image is divided into sixteen to perform the process. FIG. 4B exemplarily shows an image of 16K divided into 8K class pictures using a 2K class video system.

The drawings are illustrated by way of example in order to facilitate describing the present invention, and the present invention is not limited thereto, and it will be apparent to those skilled in the art that the size and number of divisions of the image may be changed according to embodiments.

Each subpicture divided by the image splitter 100 is mapped to a macroslice by the mapping unit 200. The macro slice includes one or more slices, and the layer of the macro slice is on the middle layer of the picture layer and the slice layer.

The ultra high resolution image according to the embodiment of the present invention may be implemented by combining the plurality of macro slices.

Accordingly, the present invention does not generate a plurality of picture layers for a single picture by using the concept of macroslice, but deblocking a subpicture boundary by sharing information between each divided subpicture by generating a single image bitstream. Deblocking filtering may be performed, and an effective motion compensation (MC) may be performed when the motion vector is out of the region of each subpicture.

Each of the sub pictures mated into macro slices in the mating unit 200 may be independently compressed by the plurality of image compression units 300. Each macroslice is compressed by a single multi-processor.

Each image compressor 300 compresses syntax and data of a macroslice layer or less. In this case, one of the image compressors 300 may be designated as a master compressor to encode a syntax above the macroslice layer.

As described above, the master compressor may designate any one of the plurality of image compressors 300 or further include a separate main compressor (not shown) that performs encoding for the macroslice layer upper syntax. can do.

The content of the upper syntax is stored in the memory unit 500 so that each image compression unit 300 can share and use the same syntax. The bitstreams passed through the image compressor 300 are aligned and multiplexed according to the macroslice numbering.

As described above, a compression process is performed by a single multiprocessor and each subpicture is mapped to a macro slice so that information between the subpictures can be shared, and the shared information is stored in the memory unit 500.

The various pieces of information stored in the memory unit are configured to be accessed by each image compression unit 300, and when information of the other subpicture is to be used in the process of compressing and restoring the current macro slice, information of the corresponding area is arbitrarily selected. Can be accessed.

Therefore, if there is a motion vector pointing to the outside of the macroslice and the motion vector is inside the picture, motion compensation may be performed using the corresponding area of the macroslice including the area indicated by the motion vector.

In addition, when a motion vector pointing to the outside of the macro slice exists and the motion vector points to the outside of the picture, the padding value is used by padding the outermost pixel value of the picture.

In FIG. 4B, the rightmost macroblock and the lowest macroblock of each macroslice are preferably compressed and reconstructed in an intra mode that does not use adjacent information.

In addition, the region of the macroblock that is the rightmost macroblock of the picture and does not belong to the bottommost macroblock of the macro slice and the bottommost macroblock of the picture and does not belong to the rightmost macroblock of the macro slice is an intra mode or an inter mode. It is preferred to be compressed and restored.

The macroblocks existing at the boundary of the macroslice may determine whether to perform deblocking filtering according to an embodiment (as described later, whether it is standalone or composite).

For example, in an embodiment in which deblocking filtering of macroblocks existing at the boundaries of macroslices is required, it is preferable that estimation, compression, and reconstruction are performed in advance since they can be used for deblocking filtering in adjacent macroslices.

The bit stream compressed by each image compressor 300 multiplexes the images of each macroslice through the bitstream multiplexer 400. The image of each macro slice is reconstructed using the decoding apparatus shown in FIG. 7 and reconstructed to an image having an original resolution through the image synthesizing unit 800.

In the ultra-high resolution image processing method according to an embodiment of the present invention, a plurality of divided subpictures are mapped to macroslices to have a hierarchical structure different from that of the prior art.

That is, a video bitstream, a sequence layer, a picture layer, a macroslice layer, a slice layer, a macroblock layer, a block layer, By forming a hierarchical structure of a block layer, it is possible to apply information of a macroslice to an upper layer of a slice and use information of another subpicture region even when encoding / decoding each subpicture independently.

Macro slices according to an embodiment of the present invention can be divided into macro slices of the independent type and macro slices of the composite type according to the type thereof.

5 is an exemplary diagram illustrating a case in which the macro slice is configured in an independent form according to an embodiment of the present invention, and FIG. 6 is a diagram illustrating a case in which the macro slice is configured in a composite form according to an embodiment of the present invention. It is an exemplary diagram shown by.

5 and 6 exemplarily show that a single picture 50, 60 is composed of four macro slices 51-54, 61-64.

The independent format refers to a format in which images of each macroslice are composed of independent separate images, and each macroslice contains different contents, so it is inappropriate to use an image of another macroslice region as a reference image. It is desirable to compress and restore independently.

According to an embodiment, the deblocking filtering process of the macroslice boundary may not be performed on the picture including the macroslice of the independent type.

In this case, since it is difficult to handle all the information of the subcomponents in the picture layer header, the macroslice may form one picture layer by including the corresponding information in the macroslice header. A sequence parameter, a picture parameter, reference picture information, etc. may be recorded in the macroslice header.

In the composite format, an image containing single information is divided into a plurality of macro slices, and thus, the information does not need to be included in the macro slice header, respectively.

According to an embodiment, in the picture layer, macroslices having a composite or independent format may be grouped together to share information included in the picture header.

As in FIG. 6, when the macro slices 61-64 have a composite format, the shaded area 65 represents an area to be compressed in the intra mode. Macroblock processing is required to perform encoding and decoding at high speed, and to reduce memory space and delay time.

For this purpose, the deblocking filtering should be completed. For example, the macroblock on the leftmost side of the macroslice 1 62 of FIG. 6 may be deblocked only after the macroblock on the rightmost side of the macroslice 061 is restored. Can be. This causes a delay, which increases as the number of macro slices increases.

Thus, in the embodiment according to the present invention, the shaded area 65 is previously determined to be in the intra mode so as to have the reconstruction pixel in advance. The reconstructed pixel is stored in the memory unit shown in FIG. 4 and used for encoding and decoding of adjacent macro slices.

As described above, when the motion vector points to another macroslice region in the macro slice, motion compensation may be performed using information of the corresponding region. Therefore, since peripheral pixel information can be used, the bit amount of an error due to motion estimation can be minimized.

In the case of an independent macroslice, the padding value using the pixel information of the corresponding region or the outermost pixel of the macroslice may be used for motion compensation similarly to the composite format.

7 is a block diagram illustrating a configuration of a high resolution image decoding apparatus according to an embodiment of the present invention.

The high resolution image decoding apparatus according to an embodiment of the present invention includes a macroslice parser 600 for parsing a received bitstream in units of macroslices, and each macroslice parsed by the macroslice parser 600. An image restoring unit 700 for restoring, an image synthesizing unit 800 for generating an ultra-high resolution image by synthesizing the macro slices restored in the image restoring unit 700, and image information restored in the image restoring unit 700 And a memory unit 900 for storing macroblock information and pixel information.

The memory unit 900 may include a main memory unit 910 for storing image information reconstructed by the image reconstructor 700, and a main memory controller 920 for storing deblocking information of macroblocks existing at a macroslice boundary. ).

The macroslice parser 600 parses the macroslice layer and separates the macroslice information. The bitstreams divided in units of macro slices are respectively input to the image reconstructor 700 to perform decoding.

Unlike encoding, decoding does not require reconstruction of macroblocks of intra mode that may exist at the boundary of an image, and deblocking information of the macroblocks is stored in the main memory controller 920 and then deblocked. Filtering is performed when filtering is possible.

So far, the present invention has been described with reference to the preferred embodiments, and those skilled in the art to which the present invention pertains to the detailed description of the present invention and other forms of embodiments within the essential technical scope of the present invention. Could be implemented.

Here, the essential technical scope of the present invention is shown in the claims, and all differences within the equivalent range will be construed as being included in the present invention.

FIG. 2 is a diagram for explaining a case where a motion vector faces outside of a subpicture in the prior art. FIG.

FIG. 3 is a diagram for explaining pixel padding for compensating for a motion vector facing outward of a subpicture according to the prior art; FIG.

4 is a block diagram showing a configuration of a high resolution video encoding apparatus according to an embodiment of the present invention.

5 is an exemplary view showing a case in which the macro slice is configured in an independent form according to an embodiment of the present invention.

6 is a diagram illustrating a case in which a macro slice is configured in a composite form according to an embodiment of the present invention.

7 is a block diagram showing the configuration of a high resolution image decoding apparatus according to an embodiment of the present invention.

※ Description of the main parts of the drawings ※

100: image segmentation unit 200: mapping unit

300: image compression unit 400: bitstream multiplexer

500: memory unit 600: macro slice parser

700: image restoration unit 800: image synthesis unit

900 memory section

Claims

Dividing one picture into a plurality of subpictures;

Mapping the divided subpictures into macro slices, respectively;

Encoding each macroslice; And

And multiplexing the bitstream generated by the encoding.

The method of claim 1,

And the divided subpictures have the same size.

The method of claim 1,

And the macroslice comprises one or more slices.

The method of claim 1,

And the rightmost macroblock and the bottommost macroblock of each macro slice are encoded in an intra mode.

The method of claim 1,

And a macroblock existing at the boundary of each macroslice is encoded before each macroslice.

Parsing the input bitstream in units of macro slices;

Decoding each parsed macroslice; And

And synthesizing the decoded image.

The method of claim 6,

And performing deblocking filtering on each of the macro slices.

The method of claim 6,

And a macroblock existing at the boundary of each macroslice is decoded before each macroslice.

The method of claim 6,

Decoding the respective macroslice further comprises performing motion compensation of each macroslice,

And if the motion vector for the motion compensation points to the outside of the macro slice, performing the motion compensation using the value of the corresponding macro slice.

The method of claim 9,

And if the motion vector for the motion compensation points to the outside of the picture, performing the motion compensation using the padding value of the outermost pixel.

An image dividing unit dividing each picture constituting an ultra high resolution image into a plurality of sub-pictures;

A mapping unit for mapping the plurality of subpictures divided by the image divider into macro slices, respectively;

An image compressor for encoding respective macro slices mapped by the mapping unit to generate respective bitstreams;

A bitstream multiplexer configured to receive and multiplex the bitstream generated by the image compressor; And

And a memory unit for storing information of a macroblock existing at a boundary of each macro slice and a bitstream generated by the image compression unit.

The method of claim 11,

The memory unit,

A temporary memory unit for storing macroblock information, pixel information, and pointer information existing at the boundary of each macroslice;

A sub memory unit for referring to the information stored in the temporary memory and storing an image of each macro slice; And

And a main memory unit which stores all picture information including each macro slice and syntax information above the macro slice layer.

The method of claim 11,

And the plurality of subpictures divided by the image splitter have the same size.

The method of claim 11,

And the uppermost macroblock and the lowest macroblock of each macro slice are encoded in an intra mode.

The method of claim 11,

And a main compression unit for encoding the macroslice layer upper syntax, wherein the image compression unit shares the upper syntax.

A macroslice parsing unit for parsing the received bitstream in units of macroslices;

An image restoring unit configured to restore each macro slice parsed by the macro slice parsing unit;

An image synthesizer configured to synthesize the macro slices reconstructed by the image reconstructor to generate an ultra high resolution image; And

And a memory unit (900) for storing the image information, macroblock information, and pixel information reconstructed by the image reconstruction unit (700).

The method of claim 16,

The memory unit,

A main memory unit which stores the image information restored by the image restoration unit; And

And a main memory controller for storing deblocking information of macroblocks existing at a macroslice boundary.