KR20190088635A - Apparatus and Method for Extracting Thumbnail Video for High Efficiency Video Coding - Google Patents

Abstract

An apparatus and a method for extracting thumbnail images for high efficiency video coding (HEVC) improve the quality of the thumbnail images by using weights based on results of partial reduction and intra prediction modes to reduce computational complexity in HEVC, and effectively eliminate aliasing side effects while not significantly increasing the computational complexity of the generation of the entire thumbnail images.

Description

[0001] The present invention relates to an apparatus and a method for extracting a thumbnail image for an HEVC,

The present invention relates to a thumbnail image extracting apparatus, and more particularly, to a method and apparatus for extracting a thumbnail image for HEVC, which improves the image quality of a thumbnail image by using a weight based on a result of a partial reduction and an intra- And an extraction device and method.

Thumbnail image extraction from an encoded video bitstream requires complex computation processes including decoding by downsampling.

In order to solve such a problem of thumbnail image extraction, in the MPEG-2, a thumbnail image can directly generate a DC value of each macroblock in the DCT domain.

This is because the DC value of the DCT domain is the average value of the pixel domain.

High compression rate can be obtained by adding intra prediction in H.264 / AVC. The MPEG-2 thumbnail extraction method can not be used in H.264 / AVC because DC value of DCT domain is residual DC value due to intraprediction method.

As a result, H.264 / AVC thumbnail extraction algorithm has been developed. The H.264 / AVC thumbnail extraction algorithm generates a thumbnail image using a lookup table in the DCT domain. However, the proposed thumbnail extraction algorithm in H.264 / AVC requires 264 two-dimensional look-up tables and these are generated by combining block size and mode number.

The DC value, which is the average value of the block, is extracted directly from the DCT domain, and only one DC value is extracted regardless of the block size.

When an 8 × 8 DCT is performed, one DC value is extracted and shared at adjacent 4 × 4 locations (typically, the thumbnail image is 1/16 of the original image size, requiring four averages in the 8 × 8 block).

Recently, the High Efficiency Video Coding (HEVC) standard provides a higher compression ratio than H.264 / AVC.

HEVC requires a low-complexity thumbnail extraction algorithm. However, the H.264 / AVC thumbnail extraction method can not be applied to HEVC because a large block size must be used to achieve a new intra prediction mode, various intra mode directions, and high compression.

Thumbnail extraction algorithm proposed in H.264 / AVC can cause severe image quality degradation in HEVC with large block size.

A planar mode, which is a new intraprediction mode, must be added to the HEVC, and a conventional thumbnail extraction method should be able to define a lookup table in planar mode.

To define a lookup table requires very large memory, that is, 264 two-dimensional lookup tables. To apply this to the HEVC, additional intra prediction modes and various block sizes should be considered. Therefore, a total of 1680 two-dimensional look-up tables must be stored.

For this reason, the thumbnail extraction algorithm proposed in H.264 / AVC can not be applied to HEVC.

Korean Patent No. 10-0907707 (entitled " Method for High-speed Thumbnail Image Extraction in H.264 / AVC Compression Area, Apparatus and Recording Medium Recorded Thereof ")

In order to reduce computational complexity in HEVC (High Efficiency Video Coding), the present invention proposes a method for extracting a thumbnail image for HEVC, which enhances the quality of a thumbnail image by using a weight based on a result of a partial reduction and an intra- And an object of the present invention is to provide an apparatus and method.

According to an aspect of the present invention, there is provided an apparatus for extracting a thumbnail image for an HEVC,

The N × N inverse discrete cosine transform is performed on the bit stream of the compressed moving image according to the HEVC (High Efficiency Video Coding) standard, and the residual value of the pixel of the 4 × 4 specific boundary in the bit stream subjected to the inverse discrete cosine transform is A partial decoding unit for performing partial conversion by reconstruction;

A block generator for generating a prediction block using the intra mode in a 4x4 pixel and generating a residual block by subtracting the prediction block from the current block; And

Calculating a weighted average predicted value for the prediction block using a predetermined weight table, calculating a weighted average residual value of the residual block using the weight table, and adding the weighted average residual value and the weighted average predicted value And a downsampling unit for generating a thumbnail pixel.

A thumbnail image extracting method for an HEVC according to an aspect of the present invention includes:

Performing a NxN inverse discrete cosine transform on a bit stream of a compressed moving image according to a HEVC (High Efficiency Video Coding) standard;

Performing a partial conversion by reconstructing a residual value of a pixel of a 4x4 specific boundary in the bit stream in which the inverse discrete cosine transform is performed;

Generating a prediction block using a 4x4 pixel using an intra mode, and generating a residual block by subtracting the prediction block from a current block;

Calculating a weighted average predicted value for the prediction block using a predetermined weighting table and calculating a weighted average residual value of the residual block using the weighting table; And

And generating a thumbnail pixel by adding the weighted average residual value and the weighted average predicted value.

With the above-described configuration, the present invention effectively eliminates the side effect of the aliasing without significantly increasing the computational complexity of generating the entire thumbnail image.

The present invention has the effect of improving visual quality of a thumbnail image while maintaining low complexity in extracting an HEVC thumbnail image.

FIG. 1 is a diagram illustrating a configuration of a thumbnail image extracting apparatus in high-efficiency video coding according to an embodiment of the present invention.
FIG. 2 is a diagram showing cumulative distribution of residual values of intra prediction mode 10 in the embodiment of the present invention.
FIG. 3 is a diagram showing a state in which prediction blocks are generated in the intra mode 10 of the embodiment of the present invention.
4 is a diagram showing an example of a thumbnail result of various sequences.

Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise.

The thumbnail image extracting method in the high efficiency video coding of the present invention provides an algorithm for improving the quality of a thumbnail image using a predetermined weight table.

Conventional thumbnail extraction methods in High Efficiency Video Coding (HEVC) cause thumbnail quality degradation by generating a thumbnail image with low complexity using a simple downsampling method.

The method of extracting a thumbnail image in high-efficiency video coding of the present invention estimates a source average value of each thumbnail pixel using a weighted average value of certain pixels based on an intra mode direction.

The thumbnail image extraction method of the present invention improves the visual quality of a thumbnail image while maintaining low complexity.

In order to generate a thumbnail image from the encoded HEVC bitstream, the HEVC decoder sequentially performs entropy decoding, dequantization, intraprediction, inverse transform, in-loop filtering, and downsampling. However, the thumbnail extraction method of the present invention decodes a 4x4 boundary pixel using only a partial decoding process, thereby reducing calculation complexity and omitting an in-loop.

FIG. 1 is a diagram illustrating a configuration of a thumbnail image extracting apparatus in high-efficiency video coding according to an embodiment of the present invention.

The thumbnail marquee extraction apparatus 100 in the high efficiency video coding according to the embodiment of the present invention includes a partial decoding unit 110, a block generation unit 120, a weight storage unit 130, and a downsampling unit 140 .

(1) Partial decoding for transform and prediction unit

The decoding process of the present invention is performed on a conversion unit and a prediction unit.

The partial decoding unit 110 performs N × N inverse discrete cosine transform on the bit stream of the moving image compressed according to the HEVC (High Efficiency Video Coding) standard by the following Equation (1) The residual value of the pixel of the 4x4 specific boundary in the bitstream is reconstructed by the following equation (2) to perform partial conversion.

The N × N inverse discrete cosine transform of High Efficiency Video Coding (HEVC) is expressed by the following Equation (1).

은 픽셀 영역과 변환 영역의 잔여 블록을 나타낸다.Where N and A represent the block size and the transformation matrix,

Represents a pixel block and a residual block of the transform block.

In order to generate a thumbnail image for the HEVC video, an inverse discrete cosine transform can be performed using Equation (1).

Inverse discrete cosine transform refers to a process of returning a value of a frequency domain to a luminance value when a compressed image is expanded (decoded) according to the MPEG standard.

The entire residual value of the residual block must be reconstructed to produce an intraframe for the thumbnail image.

However, the entire residual value of the residual block need not be extracted for the thumbnail image. For this reason, partial transformations are proposed, and thumbnail images with low computational complexity can be generated.

The partial conversion reconstructs the residual value at the 4x4 boundary as shown in the following equation (2).

는 각각 정수, 요소별 곱셈 및 XOR 연산을 나타내고, k는 벡터의 위치이고, 4×4 내지 32×32 크기의 경우, k는 {3}, {3, 7}, {3, 7, 11, 15}, {3, 7, 11,15, 19, 23, 27, 31}로 각각 표시된다.here,

3, 7, 3, 7, 11, 2, 3, 4, 5, 6, 15}, {3, 7, 11, 15, 19, 23, 27, 31}.

For example, if the block size is 4 × 4, n and k are integers, so n can be 0 and k can be 3. The present invention does not require the entire prediction value in the prediction block generated by the intra prediction.

As such, the partial intra prediction is used to reconstruct the prediction values on the 4x4 boundary of the prediction block, and these results are used to generate the values needed for the thumbnail extraction.

(2) Downsampling method for thumbnail generation

In order to generate a thumbnail image, downsampling is required because the thumbnail image size is required.

Thumbnails are generated using the subsampling method, and one pixel is extracted per 4 × 4 area. However, this method can cause excessive aliasing.

The thumbnail image can be generated using the average value of all the pixels of the 4x4 block. This method can reduce the aliasing but has a high computational complexity.

The conventional thumbnail extraction method generates thumbnails using the average value of 7 pixels at the 4x4 boundary, so there is a problem that the aliasing can not be reduced much.

The thumbnail pixel is generated by adding the weighted average value to the residual block and the prediction block.

The block generator 120 generates a prediction block using the intra mode in a 4x4 pixel, and subtracts the prediction block from the current block to generate a residual block.

2 (a) shows the cumulative distribution of the residual values of the intra prediction mode 10.

Intra prediction uses a intra mode to generate a prediction block and subtracts the prediction block from the current block to obtain a residual block

In the intra mode 10, the residual block is generated through the left reference pixels (Reference Pixels).

The distribution of the remaining blocks indicates a constant growth based on the direction of the intra mode 10. That is, there is a correlation between the distribution of the residual values and the direction of the intra prediction mode 10.

The average value of the 4x4 pixels can be calculated using a weight based on the intra mode direction (Table 1). The difference in cumulative distribution of the 4x4 size becomes too small as shown in Fig. 2 (b).

The average residual value of a 4 × 4 pixel can be estimated by combining the intra mode direction at the 4 × 4 boundary with the weight obtained from the residual values.

The weight storage unit 130 stores a weight table made up of predetermined weighted values with respect to the average residual value, as shown in Table 1 below.

Mode 0 and Mode 1 use the average value of 7 pixels on a 4 × 4 boundary. The weight of mode 6 is defined as the average value of mode 2 and mode 10.

The weight of mode 4 can be calculated as the average value of mode 2 and mode 6.

The weights of the remaining modes may be defined similar to the above-described method.

The downsampling unit 140 calculates the weighted average predicted value for the prediction block using the predetermined weight table, and calculates the weighted average residual value of the residual block using the weight table.

The weighted average residual value is expressed by the following Equation (3) using the predetermined weight table (Table 1).

은 [표 1]의 평균 잔여값에 대한 가중치이다.Here, M is an intra mode,

Is the weight of the average residual value in [Table 1].

In a similar manner, the weighted average predictive value is expressed as Equation (4) using the predefined weighting table (Table 1).

은 예측 블록이고,

은 평균 예측값에 대한 기정의된 가중치이다.here,

Is a prediction block,

Is a predefined weight for the average predicted value.

For example, as shown in FIG. 3, assuming that a 4x4 prediction block is generated using reference pixels having an intra prediction mode 10, in order to calculate an average value of a prediction block, a block is generated using a reference pixel do. The average value is obtained by adding all prediction values of the size of 4 × 4 and dividing by 16.

Hence, the average value of the prediction block can be calculated by multiplying the reference pixels by the predetermined weights WPs.

The four reference pixels on the left line are used, and the average value of the prediction blocks is expressed by the following equation (5).

Here, P _{4 × 4} and index -1 indicate the positions of the prediction block and the reference pixel.

As shown in Equation (5), four weights for the average value of the prediction block can be obtained. The weights for the other modes may be defined in a similar manner.

The present invention does not refer to the block size but defines only a 4 × 4 weight value.

The downsampling unit 140 generates a thumbnail pixel by adding the weighted average residual value and the weighted average predicted value.

The thumbnail can be generated by summing WARV and WARV as shown in the following Equation (6).

The present invention was tested using High Efficiency Video Coding (HEVC) reference software. All HEVC test sequences except Class C and Class D were used in the experiment.

Generally, a thumbnail is generated at 1/16 the size of the original frame.

For Class C and Class D, the resolution of the sequence is too small to be recognized as 832 × 480 and 416 × 240, respectively.

All frames are I-frames and Table 2 compares the thumbnail extraction times of various methods for various sequences as a measure of computation cost.

In Table 2, FD is a method of extracting thumbnails having an average value of 16 pixel values in a 4 × 4 size after pull-coating using a full decoding method.

PDSS is a thumbnail extraction method using subsampling, and subsampling extracts one pixel per 4 × 4 area.

PDAVG7 is a thumbnail generation method that uses an average value of 7 pixels on a 4x4 boundary. The results shown in [Table 2] are the results obtained by repeating an average of 50 times.

As shown in Table 2, it can be seen that the thumbnail extraction method of the present invention requires an average time of 29.34% less than FD.

Table 3 compares the peak signal-to-noise ratios (PSNRs) of the luminance components of the generated thumbnails.

The thumbnail extraction method of the present invention has an average PSNR of 8.2 dB lower than that of the FD. The performance of the thumbnail extraction method of the present invention shows much better performance than the conventional method and proves the use of the predefined weight table with the intra prediction direction.

4 is a diagram showing an example of a thumbnail result of various sequences.

4A is a cut-out thumbnail image result of the FD, FIG. 4B is a cut-out thumbnail image result of the PDSS, FIG. 4C is a cut- FIG. 4D is a thumbnail image result of the present invention. FIG.

Figure 4 shows that the thumbnail extraction method of the present invention produces fewer side effects.

PDSS and PDAVG7, while the thumbnail extraction method of the present invention has an advantage of reducing the side effect of aliasing as shown in FIG. 4 (d).

The present invention proposes a method of improving the visual quality of a thumbnail image by using a partially reduced and predefined weight table to reduce the computational complexity of thumbnail generation.

The present invention improves the image quality of the thumbnail image by using the weight based on the result of the intra-prediction mode.

The thumbnail extraction method of the present invention effectively eliminates the side effects of the aliasing without significantly increasing the computational complexity of the overall thumbnail image generation.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, And such an embodiment can be easily implemented by those skilled in the art from the description of the embodiments described above.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

100: Thumbnail image extracting device
110: partial decoding unit
120:
130: weight storage unit
140: Downsampling unit

Claims (9)

The N × N inverse discrete cosine transform is performed on the bit stream of the compressed moving image according to the HEVC (High Efficiency Video Coding) standard, and the residual value of the pixel of the 4 × 4 specific boundary in the bit stream subjected to the inverse discrete cosine transform is A partial decoding unit for performing partial conversion by reconstruction;
A block generator for generating a prediction block using the intra mode in a 4x4 pixel and generating a residual block by subtracting the prediction block from the current block; And
Calculating a weighted average predicted value for the prediction block using a predetermined weight table, calculating a weighted average residual value of the residual block using the weight table, and adding the weighted average residual value and the weighted average predicted value And a downsampling unit for generating a thumbnail pixel. The method according to claim 1,
Wherein the partial decoding unit performs the NxN inverse discrete cosine transform on a bit stream of a moving image compressed according to a HEVC (High Efficiency Video Coding) standard by the following equation (1).
[Equation 1]

Where N and A represent the block size and the transformation matrix,

Is the remaining block of the pixel region and transform region. The method according to claim 1,
Wherein the partial decoding unit reconstructs the residual value of the pixels of the 4x4 specific boundary in the bit stream subjected to the inverse discrete cosine transform by the following Equation 2 to perform the partial conversion: .
&Quot; (2) "

here,

3, 7, 3, 7, 11, 2, 3, 4, 5, 6, 15}, {3, 7, 11, 15, 19, 23, 27, 31}. The method according to claim 1,
Wherein the weighted mean predicted value for the prediction block is expressed as Equation (3) below using the weight table.
&Quot; (3) "

Here, M is an intra mode,

Is the weight of the average residual value in [Table 1]. The method according to claim 1,
Wherein the weighted average residual value of the residual block is expressed as Equation (4) using the weight table.
&Quot; (4) "

here,

Is a prediction block,

Is a predefined weight for the average predicted value. The method according to claim 1,
Wherein weight values corresponding to respective average residual values of the weight table are set differently according to an intra prediction mode. Performing a N × N inverse discrete cosine transform on a bit stream of a moving image compressed according to a HEVC (High Efficiency Video Coding) standard by Equation (1);
Performing a partial conversion by reconstructing a residual value of a pixel of a 4x4 specific boundary in a bitstream obtained by performing the inverse discrete cosine transform according to Equation (2);
Generating a prediction block using a 4x4 pixel using an intra mode, and generating a residual block by subtracting the prediction block from a current block;
Calculating a weighted average predicted value for the prediction block using a predetermined weighting table and calculating a weighted average residual value of the residual block using the weighting table; And
And generating a thumbnail pixel by adding the weighted average residual value and the weighted average predicted value to generate a thumbnail pixel.
[Equation 1]

Where N and A represent the block size and the transformation matrix,

Is the remaining block of the pixel region and transform region.
&Quot; (2) "

here,

3, 7, 3, 7, 11, 2, 3, 4, 5, 6, 15}, {3, 7, 11, 15, 19, 23, 27, 31}. 8. The method of claim 7,
Wherein the weighted average predicted value for the prediction block is expressed as Equation (3) using the weight table.
&Quot; (3) "

Here, M is an intra mode,

Is the weight of the average residual value in [Table 1]. 8. The method of claim 7,
Wherein the weighted average residual value of the residual block is expressed as Equation (4) using the weight table.
&Quot; (4) "

here,

Is a prediction block,

Is a predefined weight for the average predicted value.

Images