KR20200090716A - Method and Apparatus for image encoding - Google Patents

Abstract

The present invention relates to a method for encoding an image and, more specifically, to a method capable of encoding an image at high-speeds by reducing complexity in a coding process. The method comprises the steps of: comparing depth information of at least one surrounding coding unit of a current coding unit with a threshold value; and performing a division operation of the coding unit according to a result of the comparison step.

Description

Image encoding method and apparatus{Method and Apparatus for image encoding}

The present invention relates to an image encoding method and apparatus.

H.264/AVC is a video compression standard technology with high performance compression efficiency. In H.264/AVC, the original signal can be predictively coded through intra-prediction technology for removing correlation between images and inter-prediction technology for removing correlation between images. The H.264/AVC encoder performs discrete cosine transform encoding and quantization on a difference value that is a difference value between an original signal and a prediction signal. Then, the quantized signal is aligned by a zigzag scanning method and then entropy-encoded.

Recently, ITU-T Video Coding Experts Group (VCEG) and Moving Picture Experts Group (ISO/IEC MPEG) have formed Joint Collaborative Team on Video Coding (JCT-VC), a new video compression standard, High Efficiency Video Coding (HEVC) The standardization of is in progress, which is known to improve the compression efficiency of about 40% or more compared to the established H.264/AVC. H.264/AVC and HEVC are basically the same as block-based image encoders, but unlike H.264/AVC, which performs encoding in units of MB (Macro Block), which is a fixed size of 16x16, HEVC is within the common test condition. The encoding is performed according to the basis of a CU (Coding Unit) having various sizes ranging from a maximum size of 64x64 to 8x8.

However, HEVC, which has been standardized to date, has excellent compression efficiency, but uses a method of selecting a CU having the highest compression efficiency by encoding for CUs of all sizes, so it requires a long encoding time with very high computational complexity. There is a problem. This can cause problems in viewing real-time video or providing high-definition content, so we need a way to solve it.

The present invention has been devised to solve the above problems, and has an object, in particular, to provide a video encoding method and apparatus that simplifies operations during encoding and shortens encoding time.

In addition, an object of the present invention is to provide an image encoding method and apparatus for providing high-quality image information in real time through high-speed encoding by improving the performance of an encoder.

An embodiment of the present invention devised to achieve the above object is a step of comparing the depth information and a threshold value of at least one neighboring coding unit of the current coding unit, and performing a division of the coding unit according to the result of the comparison step It provides a video encoding method comprising the step of.

Here, the peripheral coding unit depth information is preferably depth information of the left coding unit adjacent to the left side of the current coding unit and depth information of the upper coding unit adjacent to the upper side.

In addition, it is preferable that the depth information of the peripheral coding unit is depth information of the current coding unit in the previous frame.

Further, the depth information of the peripheral coding unit is preferably the depth information of the left coding unit adjacent to the left side of the current coding unit, the depth information of the upper coding unit adjacent to the upper side, and the depth information of the current coding unit in the previous frame. Do.

Meanwhile, when the size of the coding unit is 64x64, the comparison step may not be performed.

In addition, when the size of the coding unit is 32x32, it is preferable to set the threshold to 1, and in the case of 16x16, to set the threshold to 3.

The method, as a result of the comparison step, if the depth information of the surrounding coding unit is smaller than the threshold, performs a step of encoding a next largest coding unit, and as a result of the comparison step, the depth information of the surrounding coding unit If is greater than the threshold value, the step of encoding the coding unit to the next depth may be further included.

According to the present invention, it is possible to reduce the complexity of the encoder by omitting the coding process of the smaller coding unit unit using the coding unit information around the current coding unit.

In addition, according to the present invention, it is possible to provide high-speed image coding, thereby providing real-time images and high-definition images more efficiently.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.
2 is a block diagram showing the configuration of an image decoding apparatus according to another embodiment of the present invention.
3 shows an LCU splitting structure used in an image encoding method according to an embodiment of the present invention.
4 illustrates CU division of a video encoding method according to an embodiment of the present invention.
5 illustrates CU division of a video encoding method according to another embodiment of the present invention.
6 is a flowchart illustrating an image encoding method according to an embodiment of the present invention.

The present invention can be applied to various changes and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals are used for similar components.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from other components. For example, the first component may be referred to as a second component without departing from the scope of the present invention, and similarly, the second component may be referred to as a first component. The term and/or includes a combination of a plurality of related described items or any one of a plurality of related described items.

When an element is said to be "connected" to or "connected" to another component, it is understood that other components may be directly connected to or connected to the other component, but there may be other components in between. It should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that no other component exists in the middle.

The terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “include” or “have” are intended to indicate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, and that one or more other features are present. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the image encoding apparatus 100 includes a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, and a conversion unit 130 , A quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference image buffer 190.

The image encoding apparatus 100 encodes an input image in an intra mode or an inter mode and outputs a bitstream. Hereinafter, in the embodiment of the present invention, intra prediction may be used in the same sense as intra-prediction and inter-prediction. In order to determine an optimal prediction method for a prediction unit, an intra-screen prediction method and an inter-screen prediction method may be selectively used for the prediction unit. The image encoding apparatus 100 generates a prediction block for the original block of the input image, and then encodes the difference between the original block and the prediction block.

In the intra prediction mode, the intra prediction unit 120 (or the intra prediction unit may also be used as a term having the same meaning.) by performing spatial prediction using pixel values of an already coded block around the current block. Create predictive blocks.

In the inter-prediction mode, the motion predicting unit 111 finds a region that best matches the input block in the reference image stored in the reference image buffer 190 in the motion prediction process to obtain a motion vector. The motion compensation unit 112 generates a prediction block by performing motion compensation using a motion vector.

The subtractor 125 generates a residual block by the difference between the input block and the generated prediction block. The transform unit 130 performs transform on the residual block and outputs a transform coefficient. Then, the quantization unit 140 quantizes the input transform coefficient according to a quantization parameter and outputs a quantized coefficient. The entropy encoding unit 150 entropy-encodes the input quantized coefficient according to a probability distribution, and outputs a bit stream.

Since HEVC performs inter prediction encoding, that is, inter-picture prediction encoding, the currently encoded image needs to be decoded and stored in order to be used as a reference image. Therefore, the quantized coefficients are inversely quantized by the inverse quantization unit 160 and inversely transformed by the inverse transformation unit 170. The inverse quantized and inverse transformed coefficients are added to the prediction block through the adder 175 and a reconstructed block is generated.

The restoration block passes through the filter unit 180, and the filter unit 180 applies at least one of a deblocking filter, SAO (Sample Adaptive Offset), and ALF (Adaptive Loop Filter) to the restoration block or restoration picture. can do. The filter unit 180 may also be referred to as an adaptive in-loop filter. The deblocking filter can remove block distortion caused by the boundary between blocks. The SAO can add an appropriate offset value to the pixel value to compensate for the coding error. ALF may perform filtering based on a value obtained by comparing the reconstructed image with the original image, or may be performed only when high efficiency is applied. The reconstructed block that has passed through the filter unit 180 is stored in the reference image buffer 190.

2 is a block diagram showing the configuration of an image decoding apparatus according to another embodiment of the present invention.

Referring to FIG. 2, the image decoding apparatus 200 includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, and a filter unit. 260 and a reference image buffer 270.

The image decoding apparatus 200 receives the bitstream output from the encoder and performs decoding in intra mode or inter mode, and outputs a reconstructed image, that is, a reconstructed image. In the intra mode, the prediction block is generated using the intra prediction mode, and in the inter mode, the prediction block is generated using the inter prediction method. The video decoding apparatus 200 obtains a residual block from the received bitstream, generates a prediction block, and then adds the residual block and the prediction block to generate a reconstructed block, that is, a reconstructed block.

The entropy decoding unit 210 entropy-decodes the input bitstream according to a probability distribution, and outputs a quantized coefficient. The quantized coefficients are inversely quantized by the inverse quantization unit 220 and inversely transformed by the inverse transformation unit 230. As a result of the inverse quantization/inverse transformation of the quantized coefficients, a residual block is generated.

In the intra prediction mode, the intra prediction unit 240 (or the inter prediction unit) generates a prediction block by performing spatial prediction using pixel values of an already coded block around the current block.

In the inter prediction mode, the motion compensation unit 250 generates a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference image buffer 270.

The residual block and the prediction block are added through the adder 255, and the added block is passed through the filter unit 260. The filter unit 260 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or reconstructed picture. The filter unit 260 outputs a reconstructed image, that is, a reconstructed image. The reconstructed image may be stored in the reference image buffer 270 and used for inter-screen prediction.

Methods for improving the prediction performance of an encoding/decoding device include a method of increasing the accuracy of an interpolation image and a method of predicting a difference signal. Here, the difference signal is a signal indicating a difference between the original image and the predicted image.

In the present invention, "differential signal" can be used as a "differential signal", "remaining block" or "differential block" depending on the context, and a person having ordinary knowledge in the art may influence the idea and nature of the invention. You will be able to distinguish it within the scope not given.

In an embodiment of the present invention, for convenience of explanation, a coding unit (hereinafter referred to as “CU”) is used as a coding unit, but may be a unit for performing decoding as well as coding. Hereinafter, the image encoding method described in FIGS. 3 to 6 according to an embodiment of the present invention may be implemented in accordance with the functions of each module described in FIGS. 1 and 2, and these encoders and decoders are within the scope of the present invention. Is included. That is, the image encoding/decoding method to be described later in the embodiment of the present invention may be performed in each component included in the image encoder and the image decoder described above in FIGS. 1 and 2. The meaning of the configuration unit may include not only a hardware meaning but also a software processing unit that can be performed through an algorithm.

In the image encoding/decoding according to the above-described embodiments of FIGS. 1 and 2, a CU structure in which a macroblock of a single size is extended to various sizes may be defined for efficient encoding of an image. The CU is a unit in which encoding is performed in a video encoder, and may be hierarchically divided with depth information based on a quad tree structure. CU can have various sizes such as 8×8, 16×16, 32×32, and 64×64. In addition, the largest sized CU is called a Large Coding Unit (LCU), and the smallest sized CU is called a Smallest Coding Unit (SCU). All CUs except SCU allocate split_flag information to indicate whether the CU is a divided area or not according to the value. The encoder may adjust the size of the LCU in the encoding process according to various video signal characteristics.

3 illustrates an LCU splitting structure used in a video encoding method according to an embodiment of the present invention.

According to FIG. 3, the size of a CU can be divided into 8×8, 16×16, 32×32, and 64×64, the size of the LCU is 64×64, and the size of the SCU is 8×8.

The division of the CU is determined by the characteristics of the image corresponding to the region of the corresponding LCU, and the conventional HEVC encoder performs CU unit-based encoding of all possible sizes from LCU to SCU and selects the CU structure having the best compression efficiency. Consists of. The selection of the CU structure is made using rate-distortion optimization, which finds the most efficient structure by comparing the relationship between data volume and image quality. The size of the CU is expressed through depth information, and can be expressed from an LCU at depth 0 to an SCU at depth 3. In addition, it can be informed whether or not it is split through 1 bit split_flag for each CU.

4 is an example of CU division of an image encoding method according to an embodiment of the present invention. As a result of encoding using the HM-5.0 version, which is a reference software of HEVC,'basketballDrill', which is one of the test sequences of HEVC 'It shows an example of how the image is divided into CU units when the image is actually encoded.

In FIG. 4, each square indicates each CU, the largest rectangle indicates that the LCU itself is not divided, and the smallest rectangle indicates an SCU in which the LCU is subdivided to the maximum depth. That is, as a result of encoding for all CU sizes, it means that the split structure having the most compression efficiency is shown in FIG. 4. In general, if the activity of a region is high or the movement is dynamic, the CU is subdivided. In the case of a region with little movement or a flat region, encoding is performed with a large CU.

Looking at the relationship between neighboring CUs in the image encoding process through FIG. 4, the CU shows a similar trend to the size of the neighboring CUs. That is, it is difficult to find a sudden size change in which the difference between depth information of a corresponding CU and a neighboring CU differs by 2 or more, and it is observed that adjacent CUs have similar depth information. Since the coding order for the CU in the HEVC encoding process is performed in the shape of an alphabet Z in the frame, left CU and upper CU can be used for depth information about the neighboring CU.

Table 1 shows the surrounding CU (left or upper position) for the final CU partition structure obtained by simulating four HEVC Test Sequences of WVGA (832x480 size) size in accordance with QP=22, 27, 32, and 37 for low_delay_p conditions. The difference from CU) is the result of obtaining a probability value of 0 or 1. When looking at the probability values for Table 1, it can be confirmed that the difference between the depth information of the neighboring CU and the current CU is less than 1 for the remaining CU sizes excluding 64x64, and has a high probability of 0.9 or more. It can be seen that the size between CUs does not change rapidly. Table 1 below shows the probability that the depth information difference between neighboring CUs by CU size is 2 or less.

Sequence CU size 64x64 32x32 16x16 8x8 Basketball Drill 0.8706 0.9656 0.9994 0.9970 BQMall 0.8630 0.9380 0.9992 0.9965 PartyScene 0.6900 0.9136 0.9998 0.9994 RaceHorse 0.8008 0.9247 0.9995 0.9984 average 0.8061 0.9354 0.9994 0.9978

5 is an example of CU division of a video encoding method according to another embodiment of the present invention. Referring to FIG. 5, FIG. 5 is a segmentation of a CU when actually encoding a'BasketballDrill' sequence, one of HEVC's test sequences. This is an excerpt of a part of the image for, and it can be confirmed that the CU size between the current frame and the reference frame is similar. That is, it can be also confirmed that the LCU at the same position between the current frame (b) and the reference frame (a) has a similar CU splitting structure with a depth information difference of 1 or less. 4 and 5, the current CU is closely related to the neighboring CU.

Here, the peripheral CU may be expressed as Equation 1, including the left CU and the upper CU, which are the spatial peripheral CUs, and the temporal peripheral CUs, as shown in the embodiment of FIG.

[Equation 1]

: 주변 CU,

: 좌측 CUhere,

: CU around,

: Left CU

: 상측 CU,

: 시간적 주변 CU

: Upper CU,

: Temporary peripheral CU

The depth information between the current CU and the neighboring CUs does not show a large difference, and when the size of the neighboring CU is large, the current CU size is very likely to be a CU size having the same or small difference (around 0 or 1) depth information. That is, when the current CU is called and the sum of the depth information with the neighboring CU is D, it can be defined as Equation (2).

[Equation 2]

The smaller the D value defined in Equation 2, the larger the size of the neighboring CU where encoding is completed. Since the difference between the size of the neighboring CU and the current CU is not large, when the size of the CU currently being coded has a D value below a certain threshold, it can be said that the size of the neighboring CU is sufficiently large. Since the probability of occurrence is very low, even if the encoding process is omitted, there will be no significant difference in the encoding result.

As described above, CU depth information is closely related to neighboring CUs spatially and temporally. Therefore, if information about neighboring CUs having similar CU depth information is used well in encoding, the depth of the CU can be predicted in advance. As a result, it can be concluded that the computational complexity can be reduced. In addition, if the size of the CU to be encoded has a D value below a certain threshold, the encoding process can be omitted, and high-speed image encoding is possible.

6 is a flowchart illustrating an image encoding method according to an embodiment of the present invention.

Referring to FIG. 6, after determining the optimal mode through rate-distortion optimization (S110 ), comparing the threshold and D values (S120 ), and comparing and encoding after CU division according to the result of the comparison (S120) It includes a step S130 and a step S140 of encoding the next LCU.

Determining the optimal mode through rate-distortion optimization (S110) is a process of encoding each CU and calculating a rate-distortion cost for the encoded image. The rate-distortion cost is a value calculated by considering both the distortion and the data rate of the coded image. The video encoding apparatus selects a prediction mode having a low rate-distortion cost as an optimal prediction mode.

After step S130 of encoding by dividing into CUs of different sizes, step S110 of determining an optimal mode through rate-distortion optimization is a recursive process that is repeatedly performed.

In the video encoding method according to an embodiment of the present invention, since the compression efficiency is determined through the step of comparing the threshold value and the D value (S120), it is very important to determine the threshold value. When the threshold is increased, a reduction in compression efficiency occurs, but high-speed encoding is performed. When the threshold is decreased, the compression efficiency is preserved, but the effect of improving the speed of encoding is reduced.

Table 2 shows the probability density function (PDF) value of the D value for 64x64, 32x32, 16x16, and 8x8 sizes that can occur in the encoding process. The coding conditions are the same as those used in Table 1.

To preserve the high compression efficiency of HEVC, the allowable threshold should be set small. According to Table 2, since the probability of 0, which is the minimum value that D can have, is also very high for a 64x64 sized CU, it is desirable to exclude the 64x64 CU size from the fast CU depth determination method to preserve compression efficiency. On the other hand, since the PDF value of D=0 is very low in the size of 32x32, it is preferable because the compression efficiency will be high even if a fast CU depth determination method is performed by setting the 1 value to the threshold. In the case of the size of 16x16, when the value of 3 is used as a threshold for the same reason, compression efficiency can be preserved and the encoding speed can be improved. Table 2 below shows the PDF of the D value according to the CU size.

D(CU _curr ) CU size 64x64 32x32 16x16 8x8 0 0.178733 0.000077 0 0 One 0.281027 0.009407 0.000108 0 2 0.235615 0.088288 0.000344 0 3 0.162091 0.267736 0.004863 0.000012 4 0.080105 0.262355 0.032249 0.000208 5 0.039865 0.210848 0.142129 0.001796 6 0.022377 0.104354 0.306077 0.013485 7 0.000094 0.056857 0.346464 0.066986 8 0.000094 0.000077 0.167765 0.257488 9 0 0 0 0.660025

On the other hand, in order to verify the effect of the image encoding method according to an embodiment of the present invention, the proposed method is implemented in the HM-5.0 reference software (which is the HEVC test model software), and the aspect of the original HM-5.0 encoding, compression efficiency and encoding time According to the comparison. The low_delay_p condition was used, and the evaluation sequence was simulated for 4 situations, QP={ 22, 27, 32, 37} for 4 HEVC Test Sequences. The relative results were measured based on the original HM-5.0, and changes in compression efficiency due to BD-Rate were observed. While it can be reduced by about 7.27%, the reduction in compression efficiency is 0.31%, which confirms that the encoding speed can be effectively reduced with little loss in compression efficiency. Table 3 below shows the performance results of the proposed method compared to the HM-5.0 method.

Sequence QP HM-5.0 Proposed Method kbps Y psnr(dB) encoding time(sec) kbps Y psnr(dB) encoding time(sec) BD-Rate(%) Encoding Time Saving ratio(%) Basketball
Drill 22 3954.127 40.2313 10544.294 3962.923 40.222 9858.329 0.55
10.3278 27 1814.71 37.0832 9564.932 1820.552 37.0747 8674.401 32 859.42 34.2795 8786.686 861.8896 34.2697 7727.878 37 438.4184 31.8825 7888.079 439.5768 31.8686 6724.411 BQmall 22 4587.61 40.0975 12127.375 4592.274 40.0961 11391.121 0.52
10.1526 27 1975.417 37.2956 10914.187 1978.004 37.2894 9924.755 32 934.9152 34.4257 10042.36 939.1368 34.4144 8861.654 37 470.64 31.6129 9201.962 471.8352 31.5854 7815.234 PartyScene


22 9346.673 38.2242 11624.99 9345.91 38.2237 11408.216 0.07
4.4443 27 3716.226 34.3846 10129.876 3719.254 34.3877 9867.292 32 1563.51 31.0648 9048.42 1564.386 31.0616 8644.687 37 661.3352 27.9502 8026.486 660.5616 27.9387 7183.871 RaceHorses


22 6127.539 39.9923 9104.682 6132.694 39.9971 8944.073 0.12 3.2775 27 2374.563 36.2113 8122.996 2377.785 36.2129 7914.524 32 1020.782 33.0676 7358.795 1021.681 33.0597 7059.057 37 462.768 30.2441 6476.457 462.1144 30.2417 6127.18 AVERAGE 2519.291 34.8779 9310.1610 2521.911 34.87146 8632.9176 0.31 7.2742

100: video encoding device 111: motion prediction unit
112: motion compensation unit 120: intra prediction unit
125: subtractor 130: conversion unit
140: quantization unit 150: entropy encoding unit

Claims (4)

Decoding a split flag (split_flag) for the first coding block from a bitstream based on depth information of a neighboring block adjacent to the first coding block;
Dividing the first coding block into a plurality of second coding blocks based on the decoded split flag;
Decoding the second coding block in an intra mode to generate a prediction block;
Entropy decoding the bitstream to obtain a quantized coefficient;
Generating a residual block by performing inverse quantization and inverse transformation on the obtained quantized coefficients; And
Generating a reconstruction block of the second coding block based on the prediction block and the residual block,
The segmentation flag is decoded considering whether the size of the first coding block is smaller than a predetermined threshold,
The size of the first coding block is expressed through depth information,
The threshold is a fixed value pre-set in the decoding apparatus, and the fixed value is two. According to claim 1,
The peripheral block includes at least one of a left peripheral block or an upper peripheral block of the first coding block. Encoding a split flag (split_flag) for the first coding block based on depth information of a neighboring block adjacent to the first coding block;
Dividing the first coding block into a plurality of second coding blocks based on the coded split flag;
Decoding the second coding block in an intra mode to generate a prediction block;
Generating a residual block based on a difference between the original block of the second coding block and the prediction block;
Transforming and quantizing the generated residual block to obtain a quantized coefficient; And
Generating a bitstream by encoding the obtained quantized coefficients,
The segmentation flag is encoded considering whether the size of the first coding block is smaller than a predetermined threshold,
The size of the first coding block is expressed through depth information,
The threshold is a fixed value pre-set in the encoding device, and the fixed value is two. According to claim 3,
The neighboring block includes at least one of a left peripheral block or an upper peripheral block of the first coding block.

Images