WO2017010664A1

WO2017010664A1 - Method and apparatus for encoding/decoding scalable image at high speed

Info

Publication number: WO2017010664A1
Application number: PCT/KR2016/004645
Authority: WO
Inventors: 김용환; 권기원; 우용제
Original assignee: 전자부품연구원
Priority date: 2015-07-10
Filing date: 2016-05-03
Publication date: 2017-01-19
Also published as: KR20170006926A

Abstract

Provided is a method and apparatus for encoding/decoding a scalable image at high speed. A scalable image processing method according to embodiments of the present invention comprises: calculating a position of an RL block in an RL picture; up-sampling the calculated RL block; and encoding or decoding a current PU by referring to the up-sampled RL block. Accordingly, the present invention can enhance an encoding/decoding speed through reducing an amount of consumption of a memory by an encoder/decoder and decreasing an amount of calculation, by performing an up-sampling process on only a required block in the PU encoding/decoding process.

Description

Method and apparatus for fast encoding / decoding of scalable video

The present invention relates to fast encoding and decoding of an image, and more particularly, to a method for speeding up pixel block upsampling, which is an essential module for supporting spatial scalability in a layer-based scalable image encoder and a decoder.

1.Scalable High Efficiency Video Coding (SHVC)

SHVC is video coding with scalable coding based on HEVC codec. Scalable coding is based on a multi-layer structure, and the biggest difference from the existing single layer coding standard is that an inter-layer prediction process is added as shown in FIG. Therefore, the SHVC encoder and the decoder add memory and arithmetic operations required for inter-layer prediction as compared to the conventional single layer HEVC coding. In particular, pixel interpolation calculation through an up-sampling filter is essential to support spatial scalability.

2. Picture-Based Upsampling Process

In the SHVC standardization process, the SHM reference software, which is announced for algorithm verification, implements the 'picture-based upsampling process'.

First, the picture-based upsampling process can be divided into two modules. The first module is a module that precomputes the upsampling pixel coordinates of FIG. 2 (that is, the RL pixel coordinates corresponding to the EL pixel coordinates), and the second module 3 is a picture based upsampling operation module. In FIG. 2, EL stands for Enhancement Layer and RL stands for Reference Layer. In addition, el_width and el_height mean width and length of an EL picture, respectively. CroppingChangeFlag is a flag indicating a case where cropping offsets of EL and RL pictures vary for each picture in one sequence. The x and y pixel coordinates of the luma (Y) plane computed by the formulas defined in the SHVC standard are stored in the m_xPosY [] and m_yPosY [] arrays, and the coordinates of the chroma (Cb / Cr) planes are m_xPosC [] and m_yPosC []. Stored in an array. Through the plane common preliminary preparation module, up-sampling pixel coordinate calculation may be performed in advance on only the first picture for one sequence, or only when the cropping offset information is changed, thereby reducing the amount of computation. The size of the luma pixel coordinate m_xPosY [] is el_width and the size of m_yPosY [] is el_height. The size of m_xPosC [], which is the chroma pixel coordinates, is the width of the EL chroma picture, and the size of m_yPosC [] is the height of the EL chroma picture.

The picture-based upsampling method of FIG. 3 is a method of upsampling the entire RL picture and storing it in a separate picture buffer before all EL picture decoding or encoding. The upsampling filtering operations of (1) and (3) are performed using the values of m_xPosY [], m_yPosY [], m_xPosC [], and m_yPosC [] calculated in the common preparation process of FIG.

This picture-based upsampling method has two disadvantages as a result.

1) In addition, two picture size buffer memories are required. That is, a temporary buffer having a picture size to store the horizontal up-sampling result is required, and a picture buffer to store the vertical up-sampling result is required.

2) There is a computational load that upsamples all pixel block regions that are not actually used for inter-layer prediction.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to perform an upsampling process on only necessary blocks in a PU (Prediction Unit) encoding / decoding process, thereby using the memory usage of the encoder / decoder and The present invention provides a fast encoding / decoding method that reduces computation amount.

According to one or more exemplary embodiments, a scalable image processing method includes: calculating a position of an RL block in a reference layer (RL) picture; Up-sampling the calculated RL block; And encoding or decoding the current PU (Prediction Unit) with reference to the up-sampled RL block.

The RL block may include RL pixels to be referred to for encoding or decoding of a current PU of an EL (Enhancement Layer) picture.

In addition, the up-sampling may include: up-sampling the RL block horizontally; And vertically up-sampling the RL block.

The up-sampling may further include padding at least one of a top and a bottom of the horizontally up-sampled RL block, wherein the vertical up-sampling step vertically ups the padded RL block. -sampling

In addition, the size of the RL block may be determined by the structure of the up-sampling filter.

On the other hand, according to another embodiment of the present invention, a computer-readable recording medium comprising: calculating the position of the RL block in the reference layer (RL) picture; Up-sampling the calculated RL block; And encoding or decoding the current PU (Prediction Unit) with reference to the up-sampled RL block. The program capable of performing the scalable image processing method according to claim 1 is included.

As described above, according to embodiments of the present invention, in the PU encoding / decoding process, the upsampling process is performed on only necessary blocks, thereby reducing the memory usage of the encoder / decoder and reducing the amount of computation, thereby increasing the encoding / decoding rate. It can be improved.

1 is a SHVC encoder block diagram,

2 is a common preliminary preparation process for pixel upsampling,

3 is a picture-based upsampling method,

4 is a flowchart of a PU based upsampling method;

5 is block coordinates (for Y PB case) of an RL picture corresponding to a current PU of an EL picture,

6 is a block diagram conceptual diagram of PU-based upsampling filtering;

7 is C pseudo code of PU based luma upsampling filtering,

8 is C pseudo code of PU based chroma upsampling filtering, and

9 is a block diagram of an image encoding / decoding apparatus according to another embodiment of the present invention.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

The most significant difference from the conventional HEVC is that SHVC performs inter-layer prediction. However, for spatial scalability, the picture upsampling process is essential for inter-layer prediction. The upsampling process is computationally expensive because it uses an 8-tap polyphase FIR filter for luma and a 4-tap polyphase FIR filter for chroma.

In the embodiment of the present invention, in the PU (Prediction Unit) encoding / decoding process, the fast sampling that reduces the memory usage and the amount of computation of the encoder / decoder by performing an upsampling process only when necessary, i.e., only when inter-layer prediction is required. Provide a decryption method.

In addition, in the exemplary embodiment of the present invention, unlike the conventional 'picture-based upsampling method', upsampling is performed only on blocks necessary in a PU (Prediction Unit) decoding or encoding process without performing upsampling on the entire picture. do.

The conventional 'picture-based upsampling method' is inefficient in terms of calculation amount and memory consumption because upsampling is performed on reference layer (RL) pixel blocks that are not referred to by EL (Enhancement Layer). .

In the embodiment of the present invention, the upsampling is performed only on the RL pixel blocks referred to in the EL to avoid unnecessary computation, and the memory usage is greatly reduced as compared with the upsampling in the picture unit by the upsampling in the PU unit.

4 is a flowchart provided to explain a PU-based upsampling method according to an embodiment of the present invention.

As shown in FIG. 4, in the PU-based upsampling method, the entire process is divided into picture unit processing and PU unit processing. The relatively low complexity preprocessing portion is performed in advance in picture units. This process is the same as the common preliminary preparation process for upsampling in FIG.

On the other hand, upsampling filtering requiring high complexity and a lot of memory is performed in units of PU only when necessary according to the inter prediction type of the EL PU.

In the PU-based upsampling method according to an embodiment of the present invention, it is assumed that left / right border padding of an RL picture has already been performed during RL picture encoding or decoding. PU-based upsampling filtering is performed only when a reference picture is an RL picture through a flag in a reference picture stored in a coded picture buffer (CPB) or a decoded picture buffer (DPB) at the time of Inter PU encoding or decoding of an EL picture. For reference, Intra PU does not perform inter-layer prediction in the SHVC standard.

The texture up-sampling procedure is as follows.

(1) calculate the RL block position corresponding to the current PU

-Coordinates of the corresponding block of the RL picture to be referenced in upsampling by the current PU currently being encoded or decoded in the EL picture, and coordinates necessary for filtering. 5 shows the block coordinates of the RL picture corresponding to the current PU of the EL picture.

In Fig. 5, y coordinates yRef0 and yRef1 of the RL luma (Y) plane corresponding to the y coordinates cur_y0 and cur_y0 + H of the current PU of the EL are calculated based on the SHVC standard. Since the calculation has already been performed through the common preliminary preparation process of FIG. 2, the value is simply obtained by using a table look-up. Since the luma upsampling uses an 8-tap polyphase filter, the y coordinate of the RL luma block actually used for PU-based upsampling becomes ref_y0 and ref_y1 of FIG. 5, which is obtained using Equation 1 below.

ref_y0 = yRef 0-3

ref_y1 = yRef1 + 4 (Equation 1)

In the case of chroma (Cb / Cr), a 4-tap polyphase filter is used, and the y coordinate of the RL chroma block is obtained using the following equation (2).

ref_cy0 = cyRef0-1

ref_cy1 = cyRef1 + 2 (Equation 2)

In FIG. 5, W and H respectively representing the width and height of the EL PU may be the same value or different values depending on the PU type. The maximum value for each of W and H is 64, the maximum width or height of the luma coding tree block (CTB). In addition, the width and height of the RL block required for upsampling filtering can be expressed by W_RL and H_RL, and can be obtained from Equation 3.

W_RL = ref_x1-ref_x0

H_RL = ref_y1-ref_y0 (expression 3)

Next, (2) Horizontal upsampling, (3) Padding the top or bottom of the horizontally upsampled RL block, and (4) Upsampling vertically (Vertical up-sampling) In FIG. 6, the concept of the procedure '(2)', '(3)' and '(4)' is shown from a block buffer perspective.

C pseudo code of the PU-based luma upsampling method by the algorithm shown in FIG. 4 is shown in FIG.

In FIG. 7, the size of m_pTempBuf [] is 64 * (64 + 4 * 2) regardless of the size of the picture and is a 16-bit data type. m_lumaFilter [16] [8] is an 8-tap FIR filter coefficient array with 16 phases. Filter coefficient values are specified in the SHVC standard.

The sumLumaHor8 (src, pFC) function of FIG. 7 is a filtering function that multiplies and adds 8 pixels and 8 filter coefficients in the horizontal direction, respectively. C pseudo code is as follows.

result = (src [0] * pFC [0] + src [1] * pFC [1] + src [2] * pFC [2] + src [3] * pFC [3] + src [4] * pFC [ 4] + src [5] * pFC [5] + src [6] * pFC [6] + src [7] * pFC [7]);

The sumLumaVer8 (src, pFC, stride) function of FIG. 7 is a filtering function of multiplying 8 pixels and 8 filter coefficients in a vertical direction, respectively, and C pseudo code is as follows.

result = (src [0] * pFC [0] + src [1 * stride] * pFC [1] + src [2 * stride] * pFC [2] + src [3 * stride] * pFC [3] + src [4 * stride] * pFC [4] + src [5 * stride] * pFC [5] + src [6 * stride] * pFC [6] + src [7 * stride] * pFC [7]);

In FIG. 7, shift1Y is a value obtained by subtracting 8 from luma bit depth of RL as specified in the SHVC standard.

Meanwhile, the C pseudo code of the PU-based chroma upsampling method by the algorithm shown in FIG. 4 is shown in FIG. 8.

M_TempBuf [] of FIG. 8 is the same buffer as FIG. 7, and m_chromaFilter [16] [4] is a 4-tap FIR filter coefficient array having 16 phases. Filter coefficient values are specified in the SHVC standard.

The sumChromaHor4 (src, pFC) function of FIG. 8 is a filtering function that multiplies and adds four pixels and four filter coefficients in a horizontal direction, respectively. C pseudo code is as follows.

result = (src [0] * pFC [0] + src [1] * pFC [1] + src [2] * pFC [2] + src [3] * pFC [3]);

The sumChromaVer4 (src, pFC, stride) function of FIG. 8 is a filtering function of multiplying four pixels and four filter coefficients in a vertical direction, respectively, and the C pseudo code is as follows.

result = (src [0] * pFC [0] + src [1 * stride] * pFC [1] + src [2 * stride] * pFC [2] + src [3 * stride] * pFC [3]);

Shift1C of FIG. 8 is obtained by subtracting 8 from the chroma bit depth of the RL as specified in the SHVC standard.

The algorithm of FIG. 8 should be applied to Cb and Cr PB respectively. In other words, the same routine should be executed twice after changing only the RL picture pixel.

As shown in FIGS. 7 and 8, the memory size of the temporary buffer of the PU-based upsampling method is the size of the maximum PB size plus 4 for top / down padding. That is, 64 * (64 + 4 * 2) * sizeof (int16_t) = 9216 bytes. Also, for example, the memory size of a block in which pixels are stored as a result of final upsampling in a YCbCr420 format image is 64 * 64 * 1.5 = 6144 bytes. On the other hand, the memory usage of the temporary buffer of the conventional technology 'picture-based upsampling method' is el_width * el_height * sizeof (int16_t), and the luma picture memory size in which pixels are stored as the result of upsampling is el_width * el_height * 1.5.

In the UHD image encoder and the decoder, the conventional technology and the embodiment of the present invention show a big difference in memory usage. That is, the picture-based upsampling method requires 3840 * 2160 * 2 + 3840 * 2160 * 1.5 = 28304640 bytes of memory, but the PU-based upsampling method according to the embodiment of the present invention only uses 9216 + 6144 = 15360 bytes of memory. need.

In addition, since the upsampling filtering operation is performed only when inter-layer prediction is required, it is possible to design a fast SHVC encoder and decoder by reducing the amount of computation.

9 is a block diagram of an image encoding / decoding apparatus according to another embodiment of the present invention. An image encoding / decoding apparatus according to an embodiment of the present invention, as illustrated in FIG. 9, includes an image input unit 110, an SHVC codec 120, an image output unit 130, and a memory 140.

The SHVC codec 120 performs encoding or decoding on the image input through the image input unit 110 and outputs the image through the image output unit 130. In this process, the SHVC codec 120 selectively up-samples a specific block in the RL according to the PU-based upsampling method shown in FIG. 4, and encodes or decodes the current PU with reference to the up-sampled RL block. Thus, the minimum memory 140 is used.

At this time, since the size of the RL block is determined by the structure of the up-sampling filter, the size of the RL block at the time of luma upsampling and the size of the RL block at the time of chroma upsampling are different from each other.

Up to now, preferred embodiments of the fast encoding / decoding method of the scalable image have been described in detail. In addition to the proposed method and apparatus, it should be noted that the implementation of the fast coding / decoding method of the scalable image in software as well as the implementation of only the upsampling method in software may be included in the scope of the present invention.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

Claims

Calculating a position of the RL block in the reference layer (RL) picture;

Up-sampling the calculated RL block; And

And encoding or decoding the current PU (Prediction Unit) with reference to the up-sampled RL block.
The method according to claim 1,

The RL block is,

A scalable image processing method comprising: RL pixels to be referred to for encoding or decoding of a current PU of an enhancement layer (EL) picture.
The method according to claim 1,

The up-sampling step,

Horizontally up-sampling the RL block; And

And vertically up-sampling the RL block.
The method according to claim 3,

The up-sampling step,

Padding at least one of the top and bottom of the horizontally up-sampled RL block;

The vertical up-sampling step,

And up-sampling the padded RL block vertically.
The method according to claim 1,

The size of the RL block is,

A scalable image processing method, characterized in that determined by the structure of an up-sampling filter.
Calculating a position of the RL block in the reference layer (RL) picture;

Up-sampling the calculated RL block; And

Encoding or decoding the current PU (Prediction Unit) with reference to the up-sampled RL block; and includes a program capable of performing a scalable image processing method. Record carrier.