JP6459761B2 - Moving picture coding apparatus, moving picture coding method, and moving picture coding computer program - Google Patents

Description

  The present invention relates to, for example, a moving image encoding apparatus, a moving image encoding method, and a moving image encoding computer program.

  The moving image data generally has a very large amount of data. Therefore, a device that handles moving image data compresses the moving image data by encoding it when transmitting the moving image data to another device or when storing the moving image data in the storage device. . Typical moving picture coding methods include the Moving Picture Experts Group phase 2 (MPEG- 2 / H.262), MPEG-4, or H.264 MPEG-4 Advanced Video Coding (H.264 MPEG-4 AVC) is used. MPEG-4 AVC / H.264 and later are being standardized by the jointly established (Joint Collaborative Team on Video Coding (JCT-VC). H / H.265, the official name is ISO / IEC 23008, or ITU-T H.265).

  In HEVC, the degree of freedom of the size of a block that divides each picture included in the moving image data is improved as compared with the conventional moving image encoding method. FIG. 1 is a diagram illustrating an example of picture division by HEVC.

  As shown in FIG. 1, a picture 100 is divided in coding block coding tree unit (CTU) units, and each CTU 101 is encoded in raster scan order. The size of the CTU 101 can be selected from 64 × 64 to 16 × 16 pixels. However, the size of the CTU 101 is constant for each sequence.

  The CTU 101 is further divided into a plurality of Coding Units (CU) 102 in a quadtree structure. Each CU 102 in one CTU 101 is encoded in the Z scan order. The size of the CU 102 is variable, and the size is selected from 8 × 8 to 64 × 64 pixels in the CU division mode. The CU 102 is a unit for selecting an intra prediction encoding mode and an inter prediction encoding mode, which are examples of the encoding mode. The CU 102 is individually processed in units of Prediction Unit (PU) 103 or Transform Unit (TU) 104. The PU 103 is a unit for performing prediction according to the encoding mode. For example, the PU 103 is a unit to which the prediction mode is applied in the intra prediction encoding mode, and is a unit for performing motion compensation in the inter prediction encoding mode. For example, when the intra prediction encoding mode is applied, the size of the PU 103 can be selected from 2Nx2N and NxN (N is CU size / 2).

On the other hand, the TU 104 is a unit of orthogonal transformation. In the intra prediction encoding mode, the TU 104 is also a prediction block generation unit. The size of the TU 104 is selected from 4 × 4 pixels to 32 × 32 pixels. The TU 104 is divided by a quadtree structure and processed in the Z scan order.
Note that the intra prediction encoding mode is an encoding mode that uses the fact that moving image data has a high correlation in the spatial direction, and the encoding target block of the encoding target picture has already been encoded. This is an encoding mode in which encoding is performed using the information of the area. On the other hand, the inter prediction encoding mode is an encoding mode that uses the fact that moving image data has a high correlation in the time direction, and the encoding target block of the encoding target picture is changed to that of another already encoded picture. This is an encoding mode for encoding using information.

  In HEVC, when determining the size of the CU, PU, and TU for the CTU of interest, for example, the encoding cost is calculated for each possible combination of the size of CU, PU, and TU and the encoding mode. . Then, the combination of the size and the encoding mode of each CU, PU, and TU that minimizes the encoding cost is applied when encoding the CTU of interest.

  In addition, a technique has been proposed that uses a spatial activity value representing the complexity of a target block when determining the size of a block to be encoded (see, for example, Patent Document 1). In the moving picture coding method disclosed in Patent Document 1, when determining the coding condition of the target block, the first spatial activity value representing the complexity of at least a part of the target block is more than the first threshold value. If it is smaller, the first coding condition for subdivision is set as the coding condition for the target block. On the other hand, when the first spatial activity value is greater than or equal to the first threshold, the moving image encoding method uses the second encoding condition for large partitioning as the encoding condition for the target block.

International Publication No. 2010/150486

  In the technique disclosed in Patent Document 1, the larger the complexity of the encoding target block, the larger the size of the encoding target block.

  On the other hand, rate distortion optimization (Rate distortion) is used as a method for determining the encoding mode to be applied so as to optimize the balance between the image quality and compression efficiency of the moving image obtained by decoding the encoded moving image data. optimization, RDO) method has been proposed. In the RDO scheme, when determining an encoding mode, a distortion amount, which is an error statistic before and after encoding, and a rate, which is a code amount of a block to be encoded, are considered. Then, the encoding mode in which the RD characteristic representing the relationship between the distortion amount and the rate is the best is selected.

  Here, as described in Patent Document 1, when the size of the encoding target block is determined based on the complexity of the encoding target block, the RD characteristic for the determined size is not necessarily the best, In some cases, the RD characteristics with respect to the size of the film became better.

  Therefore, an object of the present specification is to provide a moving image encoding apparatus capable of appropriately determining the size of a block as a unit for performing encoding.

  According to one embodiment, a video encoding device is provided. This moving image encoding apparatus calculates an offset value according to the complexity of each of a plurality of sub-blocks and the degree of similarity between the sub-blocks obtained by dividing a block on a picture included in the moving image data. The first encoding cost when the block is encoded as the encoding unit is the sum of the second encoding costs of the plurality of subblocks when the block is encoded using the subblock as the encoding unit. A coding mode determining unit that selects a block as a coding unit, and selects a sub-block as a coding unit when the first coding cost is greater than the sum, And an encoding unit that encodes a block for each selected encoding unit.

The objects and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.
It should be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention as claimed.

  The moving picture encoding apparatus disclosed in this specification can appropriately determine the size of a block serving as a unit for performing encoding.

It is a figure which shows an example of the division | segmentation of the picture by HEVC. (A) And (b) shows an example of block division, respectively. (A) And (b) is a figure which shows the RD characteristic when making TU size into the value set beforehand about the picture shown by Fig.2 (a) and FIG.2 (b), respectively. It is a schematic block diagram of the moving image encoder by one Embodiment. It is an operation | movement flowchart of an encoding mode determination process. It is an operation | movement flowchart of a moving image encoding process. It is a block diagram of the computer which operate | moves as a moving image encoding apparatus by the computer program which implement | achieves the function of each part of the moving image encoding apparatus by said embodiment or its modification.

  Hereinafter, the moving picture coding apparatus will be described with reference to the drawings. First, an example in which a picture is divided into blocks according to HEVC will be described.

FIG. 2A and FIG. 2B each show an example of block division. A row of trees is shown in the picture 200 shown in FIG. Also, the picture 210 shown in FIG. 2B represents the wide noise. Each block 201 shown on the picture 200 and each block 211 shown on the picture 210 are obtained when the encoding cost for each combination of the sizes of CU, PU, and TU is calculated by the following equation: Represents the TU that minimizes the coding cost. In this example, the coding cost is calculated on the assumption that the picture is coded in the intra prediction coding mode in which information of other pictures is not referred to.

Here, the prediction error is calculated, for example, as the sum of absolute values of errors between the corresponding pixels of the block that is the encoding cost calculation target and the prediction block. The mode information amount represents the information amount of information used in the encoding mode that is the encoding cost calculation target. Λ is Lagrange's undetermined multiplier.
As shown in FIGS. 2A and 2B, the smaller the TU is selected, the more complicated the TU is. In particular, for the picture 210, the smallest TU is selected for the entire picture. However, in a picture in which such a complicated scene is shown, the RD characteristic of the picture is not necessarily improved as the picture division size is smaller.

  3 (a) and 3 (b) are diagrams showing RD characteristics when the TU size is set to a preset value for the pictures shown in FIGS. 2 (a) and 2 (b), respectively. It is. 3 (a) and 3 (b), the horizontal axis represents the amount of generated information, that is, the rate (unit: Mbps), and the vertical axis represents the peak signal-to-noise ratio (PSNR). That is, it represents the amount of distortion (unit: dB). Therefore, the RD characteristics closer to the upper left are better. 3A, graphs 301 to 304 represent RD characteristics for the picture 200 when the TU sizes are 32, 16, 8, and 4, respectively. Similarly, in FIG. 3B, graphs 311 to 314 represent the RD characteristics for the picture 210 when the TU sizes are 32, 16, 8, and 4, respectively.

  As shown in FIG. 3A and FIG. 3B, the RD characteristic when the TU size is 32 is the best for both the picture 200 and the picture 210, and the smaller the TU size, the RD The characteristics are degraded.

  As described above, when the size of a block into which a picture is divided is determined based on the encoding cost calculated according to the equation (1), the RD characteristic is not necessarily optimal. On the other hand, in the picture 200 and the picture 210, the entire picture is relatively complicated. Therefore, when the block size is determined based on the complexity as described in Patent Document 1, depending on the local position on the picture, even if the smaller block size has better RD characteristics, the picture There is a high possibility that a large block size is selected as a whole.

Here, as a result of diligent research, the present inventor has the possibility that the RD characteristic is better when the block that can be divided into a plurality of sub-blocks is not divided into sub-blocks when the following two conditions are satisfied. I found that the nature is high.
(1) The subject in each sub-block is complex (2) The subject in each sub-block is similar to each other

  Therefore, when determining the coding mode and the size of the block that is the coding unit, the moving image coding apparatus according to the present embodiment is configured to determine the complexity of each of a plurality of sub-blocks into which the block can be divided. Degree and similarity between sub-blocks are calculated. The video encoding apparatus calculates an offset having a positive value when the complexity for each sub-block and the similarity between sub-blocks satisfy the above conditions (1) and (2). Furthermore, the moving image encoding apparatus encodes the block of interest when encoding the block of interest using the subblock as an encoding unit and the encoding cost for each subblock and the block itself as the encoding unit. Calculation cost. Then, the moving image coding apparatus adds an offset to the sum of the coding costs of each sub-block when comparing the sum of the coding costs for each sub-block with the coding cost of the block of interest. As a result, the moving picture encoding apparatus makes it difficult for a block that satisfies the above conditions (1) and (2) to be divided into sub-blocks.

  In the present embodiment, the video encoding device is assumed to be compliant with HEVC. The moving picture coding apparatus divides a picture into CTU units, and selects a combination of a CU size, a PU size, a TU size, and a coding mode for each CTU. Then, the moving image encoding apparatus encodes the CTU for each CTU according to the selected combination.

  FIG. 4 is a schematic configuration diagram of a video encoding apparatus according to one embodiment. The moving image encoding apparatus 1 includes a motion vector calculation unit 11, an encoding mode determination unit 12, an encoding unit 13, and a storage unit 14.

  Each of these units included in the moving image encoding apparatus 1 is formed as a separate circuit. Alternatively, these units included in the video encoding device 1 may be mounted on the video encoding device 1 as one integrated circuit in which circuits corresponding to the respective units are integrated. Furthermore, each of these units included in the moving image encoding device 1 may be a functional module realized by a computer program executed on a processor included in the moving image encoding device 1.

  A picture to be encoded is divided into a plurality of CTUs, for example, by a control unit (not shown) that controls the entire moving image encoding apparatus 1. Each CTU is input to the moving image encoding apparatus 1 in the raster scan order, for example. Then, the moving image encoding apparatus 1 performs encoding for each CTU. Hereinafter, each part which the moving image encoder 1 has is demonstrated.

When the encoding target picture is a picture to which the inter-prediction encoding mode is applicable, such as a P picture or a B picture, the motion vector calculation unit 11 performs a motion vector for each PU applicable to the encoding target CTU. Is calculated. At that time, the motion vector calculation unit 11 performs block matching on a reference picture that has already been encoded and can be referred to by the encoding target picture for each PU, and the reference picture that most closely matches the PU, and its reference picture Determine the location of the region on the reference picture. Then, the motion vector calculation unit 11 calculates a vector representing a spatial movement amount between the PU and the area as a motion vector.
For each PU, the motion vector calculation unit 11 determines the PU and the corresponding region when the sum of the absolute value of the difference between the corresponding pixels of the PU and the corresponding region on the reference picture and the sum of the motion vector code amounts are minimum. The amount of movement between them may be a motion vector.
The motion vector calculation unit 11 outputs information indicating a motion vector and a reference picture referred to by the motion vector to the encoding mode determination unit 12 for each PU.

  The encoding mode determination unit 12 obtains a combination of applicable CU size, PU size, TU size, and encoding mode in the CTU to be encoded. Then, the encoding mode determination unit 12 calculates an encoding cost that is an estimated value of the code amount for each combination. Then, the encoding mode determination unit 12 determines a combination of the CU size, the PU size, the TU size, and the encoding mode to be applied to the encoding target CTU based on the encoding cost of each combination.

The encoding mode determination unit 12 generates a prediction block for each combination of applicable CU size, PU size, TU size, and encoding mode. The prediction block is generated from the encoded reference picture or another encoded block according to the encoding mode included in the combination of interest. The encoding mode includes, for example, an intra prediction encoding mode and an inter prediction encoding mode. Further, when a prediction block is created based on the intra prediction coding mode, the coding mode determination unit 12 predicts a prediction block for each of a plurality of modes that stipulate, for example, a method for creating a prediction block defined in HEVC. Create In addition, when a prediction block is generated based on the inter prediction encoding mode, the encoding mode determination unit 12 performs each applicable motion vector generation method or prediction method (for example, direct mode or merge mode). Create a prediction block. In addition, when the encoding target picture is a B picture capable of bidirectional prediction, the encoding mode determination unit 12 not only uses a prediction block based on a motion vector based on unidirectional prediction but also two motions based on prediction in each direction. A prediction block based on a vector is also created.
In the following, for the sake of convenience, not only a mode that defines a method for creating a prediction block in intra prediction coding mode, but also a combination of a motion vector generation method or prediction method and unidirectional prediction or bidirectional prediction in inter prediction coding mode. Is called a prediction mode.

ここで、OrgPixelは着目する組み合わせに含まれるTU内の画素の値であり、PredPixelは、予測ブロックの対応画素の値である。 For example, in order to calculate the encoding cost of the combination of interest, the encoding mode determination unit 12 calculates a prediction error for the TUs included in the combination. In the present embodiment, the encoding mode determination unit 12 calculates the pixel difference absolute value sum SAD as a prediction error according to the following equation.

Here, OrgPixel is the value of the pixel in the TU included in the combination of interest, and PredPixel is the value of the corresponding pixel of the prediction block.

ここで、AveBは、TU全体の画素値の平均値である。 Note that the encoding mode determination unit 12 calculates, as a prediction error, an absolute value sum SATD of the values of each pixel after Hadamard transform of the difference image between the focused TU and the prediction block, instead of calculating SAD. May be. In addition, for a TU for which a prediction block is not created, as in the case where the TU is located at the upper left corner of the picture and the encoding mode of interest is the intra prediction encoding mode, the encoding mode determination unit 12 performs prediction The activity may be calculated as an error. The activity ACT is calculated according to the following formula, for example.

Here, AveB is an average value of the pixel values of the entire TU.

ここで、SADは、着目する組み合わせに含まれるTUについて算出された予測誤差である。またRは、モード情報量であり、動きベクトル、予測モードを表すフラグなど、直交変換係数以外の項目についての符号量の推定値である。そしてλはラグランジュの未定乗数である。 The encoding mode determination unit 12 calculates the encoding cost Cost according to the following expression for the combination of interest.

Here, SAD is a prediction error calculated for TUs included in the combination of interest. R is a mode information amount, which is an estimated value of a code amount for items other than orthogonal transform coefficients, such as a motion vector and a flag indicating a prediction mode. Λ is Lagrange's undetermined multiplier.

  The coding mode determination unit 12 obtains a combination of a CU size, a PU size, a TU size, and a prediction mode to be applied separately for the inter prediction coding mode and the intra prediction coding mode. At that time, for example, the encoding mode determination unit 12 determines a PU size and a prediction mode that minimize the encoding cost for each TU of the same size. Further, the encoding mode determination unit 12 determines the encoding cost when using the PU size and prediction mode determined for the TU and each TU obtained by dividing the TU into four (hereinafter referred to as sub-TUs for convenience). Compare the sum of coding costs when using PU size and prediction mode. Then, the encoding mode determination unit 12 selects the TU size according to the comparison result of the encoding costs. The encoding mode determination unit 12 recursively repeats the above processing in order from the largest TU size, so that the CU size, the PU size, and the TU size for each of the inter prediction encoding mode and the intra prediction encoding mode. And a combination of prediction modes. Then, for each CU, the encoding mode determination unit 12 applies, to the CTU to be encoded, the one with the lower encoding cost among the combinations obtained for the inter prediction coding mode and the combinations obtained for the intra prediction coding mode. It is set as a combination.

  As described with reference to FIG. 1, the size of the TU is selected from 4 × 4 pixels to 32 × 32 pixels. Therefore, for example, the encoding mode determination unit 12 determines the encoding cost for a TU having a size of 32 × 32 pixels and the encoding cost of each of four sub TUs having a size of 16 × 16 pixels obtained by dividing the TU into four. Compare the sum. Similarly, the encoding mode determination unit 12 determines the encoding cost for a TU having a size of 16 × 16 pixels, and the encoding cost for each of four sub TUs having a size of 8 × 8 pixels obtained by dividing the TU into four. Compare the sum. Further, the encoding mode determination unit 12 sums the encoding cost for the TU having the size of 8x8 pixels and the encoding cost of each of the four sub TUs having the size of 4x4 pixels obtained by dividing the TU into four. And compare.

  In this embodiment, when comparing the encoding costs for different TU sizes, the encoding mode determination unit 12 divides the TU of interest when the above conditions (1) and (2) are satisfied. The target TU is more easily selected than the obtained four sub-TUs.

ここでoffsetは、着目するTUを４分割した４個のTUのそれぞれの複雑度及び４個のTU間の類似度に基づいて決定されるオフセット値である。 For example, the encoding cost of the target TU is C0, and the encoding costs of four sub-TUs obtained by dividing the target TU into four are C1 [i] (i = 1, 2, 3, 4). In this case, the encoding mode determination unit 12 selects the size of the focused TU, that is, the larger TU size as the TU size when the following equation is satisfied. On the other hand, when the following equation is not satisfied, the encoding mode determination unit 12 selects the sub TU size, that is, the smaller TU size as the TU size.

Here, offset is an offset value determined based on the complexity of each of the four TUs obtained by dividing the TU of interest into four and the similarity between the four TUs.

  The larger the coding cost C1 [i] is, the larger the prediction error between the corresponding sub-TU and its prediction block is, so the subject in the sub-TU is assumed to be more complicated. Further, it is assumed that the subject shown in each sub-TU is more similar as the variance of the encoding cost C1 [i] of each sub-TU is smaller. Therefore, in the present embodiment, the encoding mode determination unit 12 sets the encoding cost C1 [i] calculated for each sub TU as the complexity for the sub TU, and the encoding cost C1 [i of each sub TU. ] Is the similarity between sub-TUs.

ここでμは定数であり、例えば、0よりも大きく、かつ、1以下の値に設定される。またVar(C1[i])は、各サブTUの符号化コストC1[i]の分散であり、次式で算出される。閾値Th1は、例えば、サブTUの画素数に定数α（例えば、1〜5）を乗じた値に設定される。また、閾値Th2は、着目するTUに含まれるサブTUの数（本実施形態では、4）に画素の取り得る値の最大値（例えば、255）と定数β（例えば、0.05〜0.1）を乗じた値に設定される。

すなわち、各サブTUが一定以上複雑である場合、かつ、サブTU間のシーンが類似しているほどoffsetは大きな値となり、その結果として大きい方のTUのサイズが選択され易くなる。 In this case, offset is calculated according to the following equation, for example.

Here, μ is a constant, and is set to a value greater than 0 and 1 or less, for example. Var (C1 [i]) is a variance of the coding cost C1 [i] of each sub-TU, and is calculated by the following equation. The threshold value Th1 is set to, for example, a value obtained by multiplying the number of sub-TU pixels by a constant α (for example, 1 to 5). The threshold Th2 is obtained by multiplying the number of sub-TUs included in the TU of interest (4 in this embodiment) by the maximum value (for example, 255) that a pixel can take and a constant β (for example, 0.05 to 0.1). Value is set.

That is, when each sub TU is more than a certain level and the scenes between sub TUs are more similar, offset becomes a larger value, and as a result, the larger TU size is easily selected.

ここでt1[i](i=1,2,3,4)は、各サブTUについて、そのサブTUと対応する予測ブロック間の差分画像をアダマール変換した後の各画素の値の絶対値和SATDである。そしてVar(t1[i])は、各サブTUのSATDの分散である。閾値Th3は、例えば、サブTUの画素数に定数α（例えば、1〜5）を乗じた値に設定される。また、閾値Th4は、着目するTUに含まれるサブTUの数（本実施形態では、4）に画素の取り得る値の最大値（例えば、255）と定数β（例えば、0.05〜0.1）を乗じた値に設定される。SATDは、周波数領域でのサブTUの成分の大きさを表している。そのため、Var(t1[i])が小さいほど、各サブTUに写っている被写体はより類似していると想定される。この変形例における定数μの値は、上記の（６）式におけるμの値よりも大きい方が好ましい。 Note that, according to the modification, the encoding mode determination unit 12 sets the prediction error of each sub TU as the complexity for the sub TU, and sets the variance of the prediction error of each sub TU as the similarity between the sub TUs. It is good. In this case, the encoding mode determination unit 12 calculates offset according to the following equation, for example.

Here, t1 [i] (i = 1, 2, 3, 4) is the sum of the absolute values of the values of each pixel after Hadamard transform of the difference image between the sub TU and the prediction block corresponding to the sub TU. SATD. Var (t1 [i]) is the variance of the SATD of each sub-TU. For example, the threshold value Th3 is set to a value obtained by multiplying the number of pixels of the sub TU by a constant α (for example, 1 to 5). The threshold Th4 is obtained by multiplying the number of sub-TUs included in the TU of interest (4 in this embodiment) by the maximum value (for example, 255) that a pixel can take and a constant β (for example, 0.05 to 0.1). Value is set. SATD represents the size of the sub-TU component in the frequency domain. For this reason, it is assumed that the subject shown in each sub-TU is more similar as Var (t1 [i]) is smaller. The value of the constant μ in this modification is preferably larger than the value of μ in the above equation (6).

ここでs1[i](i=1,2,3,4)は、各サブTUについて、そのサブTUと対応する予測ブロック間のSADである。また、関数max(s1[i])は、s1[i](i=1,2,3,4)のうちの最大値を出力する関数であり、関数min(s1[i])は、s1[i](i=1,2,3,4)のうちの最小値を出力する関数である。閾値Th5は、例えば、サブTUの画素数に定数α（例えば、1〜5）を乗じた値に設定される。また、閾値Th6は、着目するTUに含まれるサブTUの数（本実施形態では、4）に画素の取り得る値の最大値（例えば、255）と定数β（例えば、0.05〜0.1）を乗じた値に設定される。 According to another modification, the similarity between sub-TUs may be evaluated by the absolute value of the difference in SAD between sub-TUs. In this case, the encoding mode determination unit 12 calculates offset according to the following equation, for example.

Here, s1 [i] (i = 1, 2, 3, 4) is the SAD between the prediction blocks corresponding to the sub TU for each sub TU. The function max (s1 [i]) is a function that outputs the maximum value of s1 [i] (i = 1, 2, 3, 4), and the function min (s1 [i]) is s1 [i] A function that outputs the minimum value of (i = 1, 2, 3, 4). The threshold value Th5 is set to, for example, a value obtained by multiplying the number of pixels of the sub TU by a constant α (for example, 1 to 5). The threshold Th6 is obtained by multiplying the number of sub-TUs included in the target TU (in this embodiment, 4) by the maximum value (for example, 255) that a pixel can take and a constant β (for example, 0.05 to 0.1). Value is set.

If (max (s1 [i])-min (s1 [i])) is 0, the scenes shown in each sub-TU may be the same. Therefore, in this case, it is preferable that the encoding mode determination unit 12 selects the larger TU size regardless of the equation (9).
According to still another modification, the similarity between the sub TUs may be calculated as a difference between the maximum value and the minimum value of the encoding costs C1 [i] of the sub TUs. Further, the complexity of each sub-TU may be calculated as an activity calculated by equation (3).

  FIG. 5 is an operation flowchart of the encoding mode determination process. For each CTU, the encoding mode determination unit 12 determines a combination of a CU size, a PU size, a TU size, and an encoding mode to be applied to the encoding target CTU according to the following operation flowchart.

  The encoding mode determination unit 12 sets the intra prediction encoding mode to the encoding mode in which attention is paid (step S101). Then, the encoding mode determination unit 12 sets the maximum TU size as a focused TU size (step S102).

  The encoding mode determination unit 12 calculates the minimum values C0 and C1 [i] of the encoding cost for each TU having the TU size of interest and each of the sub TUs obtained by dividing the TU into four (step S103). .

  The encoding mode determination unit 12 calculates an offset for each TU having a target TU size based on the complexity of each sub-TU obtained by dividing the TU into four and the similarity between sub-TUs (step S104).

  The encoding mode determination unit 12 determines, for each TU having a target TU size, whether the encoding cost C0 of the TU is less than or equal to the sum of the encoding costs ΣC1 [i] of each sub TU and offset (Step S105). Then, when the encoding cost C0 is equal to or less than the sum of the encoding costs ΣC1 [i] and offset of each sub TU (step S105—Yes), the encoding mode determination unit 12 selects the size of the TU of interest. (Step S106). On the other hand, when the encoding cost C0 is larger than the sum of the encoding costs ΣC1 [i] of each sub TU and offset (No in step S105), the encoding mode determination unit 12 selects the size of the sub TU ( Step S107).

  The encoding mode determination unit 12 determines whether there is a TU for which the size of the sub TU is selected (step S108). If there is one or more TUs for which the sub TU size is selected (step S108—Yes), the encoding mode determination unit 12 determines whether the size of the sub TU is the minimum size that the TU can take (step S109). Note that the minimum size that can be taken by the TU may be 4 × 4 pixels defined by HEVC, or may be a size set in advance by 4 × 4 pixels or more. If the size of the sub TU is not the minimum size that can be taken by the TU (No in step S109), the encoding mode determination unit 12 sets the size of the sub TU to the size of the TU of interest (step S110). Then, the encoding mode determination unit 12 repeats the processing from step S103 onward as the sub TU and TU for each TU for which the size of the sub TU is selected.

  On the other hand, in step S108, when there is no TU for which the size of the sub TU is selected (step S108-No), the encoding mode determination unit 12 ends the division of the TU. Also, when the size of the sub TU is the minimum size that can be taken by the TU in step S109 (step S109—Yes), the encoding mode determination unit 12 ends the division of the TU. Thereafter, the coding mode determination unit 12 determines whether or not the coding mode of interest is the inter prediction coding mode (step S111). If the encoding mode of interest is the intra prediction encoding mode (step S111-No), the encoding mode determination unit 12 sets the inter prediction encoding mode to the encoding mode of interest (step S112). Then, the encoding mode determination unit 12 repeats the processing after step S102.

  On the other hand, if the encoding mode of interest is the inter prediction encoding mode (step S111-Yes), the encoding mode determination unit 12 is based on the TU division result and the corresponding PU and prediction mode for each CU. Thus, the encoding cost of the inter prediction encoding mode and the encoding cost of the intra prediction encoding mode are calculated. Then, the coding mode determination unit 12 sets the coding mode to be applied to the CU of the inter prediction coding mode and the intra prediction coding mode with the smaller coding cost (step S113). Note that the coding mode determination unit 12 also has the CU coding cost and the sum of the coding costs of the four sub CUs obtained by dividing the CU into four CUs in order from the CU having the largest possible size. And the one with the lower coding cost may be selected. However, since the TU and PU are not set across multiple CUs, if the CU of interest cannot be divided into four sub CUs as a result of the above TU partitioning, the CU of interest is no more than that. Not divided. Then, the encoding mode determination unit 12 ends the encoding mode determination process.

  In addition, when the encoding target picture is an I picture to which the inter prediction encoding mode is not applied, the encoding mode determination unit 12 ends the encoding mode determination process without performing the processes after step S110. To do.

  When the encoding mode determination unit 12 determines a combination of the CU size, the PU size, the TU size, and the encoding mode to be applied to the encoding target CTU, the encoding mode determination unit 12 outputs the combination to the encoding unit 13. Note that, for a CU to be inter-predictively encoded, the encoding mode determination unit 12 also outputs a motion vector calculated for a PU included in the CU to the encoding unit 13.

The encoding unit 13 divides the CTU to be encoded according to the CU size, PU size, and TU size determined by the encoding mode determination unit 12. Then, the encoding unit 13 creates a prediction block for each TU included in the CU according to the encoding mode for each CU determined by the encoding mode determination unit 12. Then, the encoding unit 13 quantizes and variable-length encodes the orthogonal transform coefficient obtained by orthogonally transforming the prediction error signal between the prediction blocks corresponding to each TU included in the CTU to be encoded for each TU. Thus, the CTU to be encoded is encoded.
Referring to FIG. 4 again, the encoding unit 13 includes a prediction block generation unit 21, a prediction error signal calculation unit 22, an orthogonal transformation unit 23, a quantization unit 24, a decoding unit 25, and a variable length encoding unit. 26.

  The prediction block generation unit 21 generates a prediction block. When the target CU included in the CTU to be encoded is inter-predictively encoded, the prediction block generation unit 21 uses the motion vector set in the PU for each PU included in the CU to generate a reference picture. A prediction block is generated by performing motion compensation. In addition, when the target CU is subjected to intra-prediction encoding, the prediction block generation unit 21 encodes blocks around the PU according to the prediction mode applied to the PU for each PU included in the CU. A prediction block is generated from the pixels included in.

  The prediction block generation unit 21 outputs the prediction block to the prediction error signal calculation unit 22.

  The prediction error signal calculation unit 22 performs a difference calculation for each pixel in the CTU to be encoded with the corresponding pixel of the prediction block. And the prediction error signal calculation part 22 makes the difference value corresponding to each pixel obtained by the difference calculation a prediction error signal. The prediction error signal calculation unit 22 outputs the prediction error signal to the orthogonal transformation unit 23.

The orthogonal transform unit 23 obtains a set of orthogonal transform coefficients by performing orthogonal transform on the prediction error signal for each TU in the CTU to be encoded. For example, the orthogonal transform unit 23 obtains a set of DCT coefficients for each TU as orthogonal transform coefficients by using discrete cosine transform (DCT) as orthogonal transform processing. Alternatively, the orthogonal transform unit 23 may use Hadamard transform as the orthogonal transform process.
The orthogonal transform unit 23 outputs a set of orthogonal transform coefficients for each TU to the quantization unit 24.

  For each TU, the quantization unit 24 quantizes the orthogonal transform coefficient to calculate a quantization coefficient of the orthogonal transform coefficient. This quantization process is a process that represents a signal value included in a certain section as one signal value. The fixed interval is called a quantization width. For example, the quantization unit 24 quantizes the orthogonal transform coefficient by truncating a predetermined number of lower bits corresponding to the quantization width from the orthogonal transform coefficient. The quantization width is determined by the quantization parameter. For example, the quantization unit 24 determines a quantization width to be used according to a function representing a quantization width value with respect to a quantization parameter value. The function can be a monotonically increasing function with respect to the value of the quantization parameter, and is set in advance.

Further, the quantization unit 24 may determine the quantization parameter according to any of various quantization parameter determination methods corresponding to a moving image coding standard such as HEVC. The quantization unit 24 can use, for example, a quantization parameter calculation method related to the MPEG-2 standard test model 5. For the quantization parameter calculation method for MPEG-2 standard test model 5, refer to the URL specified at http://www.mpeg.org/MPEG/MSSG/tm5/Ch10/Ch10.html, for example. I want to be.
Since the quantization unit 24 can reduce the number of bits used to represent the orthogonal transform coefficient by executing the quantization process, the code amount can be reduced. The quantization unit 24 outputs the quantized coefficient to the decoding unit 25 and the variable length coding unit 26.

  The decoding unit 25 is referred from the quantization coefficient of the CTU to be encoded in order to encode a picture subsequent to the CTU after the CTU and a picture including the CTU to be encoded in the encoding order. Generate a reference picture. For this purpose, the decoding unit 25 dequantizes the quantization coefficient by multiplying the quantization coefficient by a predetermined number corresponding to the quantization width determined by the quantization parameter. By this inverse quantization, a set of orthogonal transform coefficients of each TU, for example, a set of DCT coefficients is restored. Thereafter, the decoding unit 25 performs inverse orthogonal transform processing on the set of orthogonal transform coefficients for each TU. For example, when the orthogonal transform unit 23 uses DCT, the decoding unit 25 performs inverse DCT processing on each TU. By performing the inverse quantization process and the inverse orthogonal transform process on the quantized signal, a prediction error signal having the same level of information as the prediction error signal before encoding is reproduced.

The decoding unit 25 adds the reproduced prediction error signal corresponding to the pixel to each pixel value of the prediction block. By executing these processes for each prediction block, the decoding unit 25 restores a block that is referred to in order to generate a prediction block for a PU to be encoded thereafter. Further, the decoding unit 25 may perform a deblocking filter process on the restored block.
Each time the decoding unit 25 restores a block, the decoding unit 25 stores the restored block in the storage unit 14.

  The storage unit 14 temporarily stores the restored block. By combining the restored blocks for one picture according to the coding order of each block, a picture to be referred to when coding the subsequent picture is obtained. The storage unit 14 stores a predetermined number of pictures that may be referred to by the encoding target picture. When the number of stored pictures exceeds the predetermined number, the encoding order is old. Discard in order from the picture.

  Furthermore, the storage unit 14 stores a motion vector for each of the PUs subjected to inter prediction encoding.

  The variable length coding unit 26 performs variable length coding on the quantization coefficient included in the CTU to be coded. Furthermore, the variable length coding unit 26 also performs variable length coding on a motion vector or the like used to create a prediction block. Then, the variable length coding unit 26 outputs a bit stream in which coded bits obtained by the variable length coding are arranged in a predetermined order according to HEVC or the like. The variable length coding unit 26 can use arithmetic coding processing such as Context-based Adaptive Binary Arithmetic Coding (CABAC) as the variable length coding method. Alternatively, the variable length coding unit 26 may use Huffman coding processing such as Context-based Adaptive Variable Length Coding (CAVLC) as the variable length coding method.

  A control unit (not shown) combines the output bit streams in a predetermined order and adds header information according to HEVC or the like to obtain a bit stream including encoded moving image data.

  FIG. 6 is an operation flowchart of the moving image encoding process performed by the moving image encoding device 1. The moving image encoding apparatus 1 performs encoding for each CTU according to the following operation flowchart.

  When the encoding target picture is a picture to which the inter-prediction encoding mode is applicable, such as a P picture or a B picture, the motion vector calculation unit 11 performs a motion vector for each PU applicable to the encoding target CTU. Is calculated (step S201).

  The encoding mode determination unit 12 determines a combination of a CU size, a PU size, a TU size, and an encoding mode to be applied to the encoding target CTU (step S202).

  The prediction block generation unit 21 of the encoding unit 13 calculates a prediction block for each PU according to the encoding mode included in the determined combination (step S203). And the prediction error signal calculation part 22 of the encoding part 13 calculates a prediction error signal for every TU by calculating the difference for every pixel between TU and the prediction block corresponding to the TU (step S204).

  Thereafter, the orthogonal transform unit 23 of the encoding unit 13 performs orthogonal transform on the prediction error signal for each TU to calculate a set of orthogonal transform coefficients (step S205). Then, the quantization unit 24 of the encoding unit 13 quantizes each orthogonal transform coefficient (step S206).

  The decoding unit 25 of the encoding unit 13 adds the prediction error signal reproduced by inverse quantization and inverse orthogonal transform of the quantized coefficient and the prediction block to generate a reference block (step S207). Then, the decoding unit 25 stores the reference block in the storage unit 14. On the other hand, the variable length coding unit 26 of the coding unit 13 performs variable length coding on each quantization coefficient (step S208). Then, the moving image encoding apparatus 1 ends the moving image encoding process.

  As described above, in this moving image encoding apparatus, for a block to be encoded, the complexity of each of a plurality of sub-blocks obtained by dividing the block is greater than or equal to a predetermined level, and the similarity between each sub-block is high. The higher the value, the harder the block is divided. Thereby, this moving image encoding apparatus can appropriately determine the size of the encoding unit, for example, the size of the unit of orthogonal transform.

  For pictures to which the inter prediction coding mode is applied, even if the complexity of each sub-TU obtained by dividing the TU of interest is high and the similarity between the sub-TUs is high, each sub-TU is displayed on the reference picture. There may be regions that match well with the TU. In such a case, even if the target TU is divided into sub-TUs, the prediction error for each sub-TU is very small, and the RD characteristics may be relatively good. Therefore, the encoding mode determination unit 12 may determine the TU size with offset always set to 0 when the inter prediction encoding mode is applied as the encoding mode.

  According to another modification, the moving image encoding apparatus may apply the above process not only when determining the unit for selecting the encoding mode but also for the unit of orthogonal transform. For example, when comparing the coding cost between a focused CU and a sub-CU obtained by dividing the CU into four, the moving image coding apparatus determines according to the complexity of each sub-CU and the similarity between the sub-CUs. The offset to be added may be added to the sum of the coding costs of each sub CU.

  FIG. 7 is a configuration diagram of a computer that operates as a moving image encoding apparatus when a computer program that realizes the functions of the respective units of the moving image encoding apparatus according to the above-described embodiment or its modification is operated.

  The computer 100 includes a user interface unit 101, a communication interface unit 102, a storage unit 103, a storage medium access device 104, and a processor 105. The processor 105 is connected to the user interface unit 101, the communication interface unit 102, the storage unit 103, and the storage medium access device 104 via, for example, a bus.

  The user interface unit 101 includes, for example, an input device such as a keyboard and a mouse, and a display device such as a liquid crystal display. Alternatively, the user interface unit 101 may include a device such as a touch panel display in which an input device and a display device are integrated. Then, the user interface unit 101 outputs, for example, an operation signal for selecting moving image data to be encoded to the processor 105 in accordance with a user operation.

  The communication interface unit 102 may include a communication interface for connecting the computer 100 to a device that generates moving image data, for example, a video camera, and a control circuit thereof. Such a communication interface may be, for example, High-Definition Multimedia Interface (HDMI) (registered trademark) or Universal Serial Bus (Universal Serial Bus, USB).

  Furthermore, the communication interface unit 102 may include a communication interface for connecting to a communication network according to a communication standard such as Ethernet (registered trademark) and a control circuit thereof.

  In this case, the communication interface unit 102 acquires moving image data to be encoded from another device connected to the communication network, and passes the data to the processor 105. Further, the communication interface unit 102 may output the encoded moving image data received from the processor 105 to another device via the communication network.

  The storage unit 103 includes, for example, a readable / writable semiconductor memory and a read-only semiconductor memory. The storage unit 103 stores a computer program for executing a moving image encoding process executed on the processor 105, and data generated during or as a result of these processes.

  The storage medium access device 104 is a device that accesses a storage medium 106 such as a magnetic disk, a semiconductor memory card, and an optical storage medium. For example, the storage medium access device 104 reads a computer program for moving image encoding processing executed on the processor 105 stored in the storage medium 106 and passes the computer program to the processor 105.

  The processor 105 generates encoded moving image data by executing the computer program for moving image encoding processing according to the above-described embodiment or modification. The processor 105 stores the generated encoded moving image data in the storage unit 103 or outputs it to another device via the communication interface unit 102.

  Note that the computer program capable of executing the functions of the respective units of the moving image encoding device 1 on the processor may be provided in a form recorded on a computer-readable medium. However, such a recording medium does not include a carrier wave.

  All examples and specific terms listed herein are intended for instructional purposes to help the reader understand the concepts contributed by the inventor to the present invention and the promotion of the technology. It should be construed that it is not limited to the construction of any example herein, such specific examples and conditions, with respect to showing the superiority and inferiority of the present invention. Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention.

The following supplementary notes are further disclosed regarding the embodiment described above and its modifications.
(Appendix 1)
The block on the picture included in the moving image data is divided, and an offset value is calculated according to the complexity of each of the plurality of sub-blocks and the similarity between the sub-blocks. The first encoding cost when encoding is the sum of the second encoding cost of each of the plurality of sub-blocks and the offset value when the block is encoded using the sub-block as the encoding unit. Encoding mode selection for selecting the block as the encoding unit, and for selecting the sub-block as the encoding unit when the first encoding cost is greater than the total. And
An encoding unit that encodes the block for each of the selected encoding units;
A moving picture encoding apparatus having:
(Appendix 2)
The moving picture encoding apparatus according to appendix 1, wherein the encoding mode determination unit sets, for each of the plurality of sub-blocks, the second encoding cost of the sub-block as the complexity of the sub-block.
(Appendix 3)
The encoding mode determination unit, for each of the plurality of sub-blocks, the difference absolute value sum between corresponding pixels of the sub-block and the prediction block for the sub-block as the complexity of the sub-block The moving image encoding apparatus according to 1.
(Appendix 4)
The moving picture coding apparatus according to Supplementary Note 2 or 3, wherein the coding mode determination unit calculates a variance of the complexity of each of the plurality of sub-blocks as the similarity.
(Appendix 5)
The moving picture coding apparatus according to appendix 2 or 3, wherein the coding mode determination unit calculates a difference between a maximum value and a minimum value of the complexity of each of the plurality of sub-blocks as the similarity.
(Appendix 6)
The encoding mode determination unit sets the offset to 0 when encoding the block based on an inter prediction encoding mode in which the block is encoded with reference to another encoded picture. When encoding based on an intra prediction encoding mode in which encoding is performed with reference to an encoded region of a picture, according to the complexity of each of the plurality of sub-blocks and the similarity between the sub-blocks The moving image encoding apparatus according to any one of appendices 1 to 5, which calculates the offset value.
(Appendix 7)
The encoding unit orthogonally transforms a prediction error between the block and a prediction block for the block for each of the selected coding units to calculate an orthogonal transform coefficient for each of the coding units, and the coding unit The moving image encoding device according to any one of appendices 1 to 6, wherein the orthogonal transform coefficient is encoded for each.
(Appendix 8)
The block on the picture included in the moving image data is divided, and an offset value is calculated according to the complexity of each of the plurality of sub-blocks and the similarity between the sub-blocks,
The first encoding cost when the block is encoded using the block as an encoding unit is the second encoding cost of the plurality of sub-blocks when the block is encoded using the sub-block as the encoding unit. If the block is equal to or less than the sum of the encoding costs and the offset value, the block is selected as the encoding unit. On the other hand, if the first encoding cost is greater than the total, the encoding unit is Select the sub-block,
Encoding the block for each of the selected encoding units;
A moving picture encoding method including the above.
(Appendix 9)
The block on the picture included in the moving image data is divided, and an offset value is calculated according to the complexity of each of the plurality of sub-blocks and the similarity between the sub-blocks,
The first encoding cost when the block is encoded using the block as an encoding unit is the second encoding cost of the plurality of sub-blocks when the block is encoded using the sub-block as the encoding unit. If the block is equal to or less than the sum of the encoding costs and the offset value, the block is selected as the encoding unit. On the other hand, if the first encoding cost is greater than the total, the encoding unit is Select the sub-block,
Encoding the block for each of the selected encoding units;
A computer program for encoding a moving image for causing a computer to execute the above.

DESCRIPTION OF SYMBOLS 1 Moving image encoder 11 Motion vector calculation part 12 Encoding mode determination part 13 Encoding part 14 Storage part 21 Prediction block generation part 22 Prediction error signal calculation part 23 Orthogonal transformation part 24 Quantization part 25 Decoding part 26 Variable length code | symbol Conversion unit 100 computer 101 user interface unit 102 communication interface unit 103 storage unit 104 storage medium access device 105 processor

Claims (7)

The block on the picture included in the moving image data is divided, and an offset value is calculated according to the complexity of each of the plurality of sub-blocks and the similarity between the sub-blocks. The first encoding cost when encoding is the sum of the second encoding cost of each of the plurality of sub-blocks and the offset value when the block is encoded using the sub-block as the encoding unit. Encoding mode selection for selecting the block as the encoding unit, and for selecting the sub-block as the encoding unit when the first encoding cost is greater than the total. And
An encoding unit that encodes the block for each of the selected encoding units;
A moving picture encoding apparatus having:   The moving picture encoding apparatus according to claim 1, wherein the encoding mode determination unit sets the second encoding cost of the subblock as the complexity of the subblock for each of the plurality of subblocks. .   The encoding mode determination unit, for each of the plurality of sub-blocks, the difference absolute value sum between corresponding pixels of the sub-block and the prediction block for the sub-block is set as the complexity of the sub-block. Item 4. The moving image encoding device according to Item 1.   The moving image encoding apparatus according to claim 2, wherein the encoding mode determination unit calculates a variance of the complexity of each of the plurality of sub-blocks as the similarity.   The moving picture coding apparatus according to claim 2 or 3, wherein the coding mode determination unit calculates a difference between a maximum value and a minimum value of the complexity of each of the plurality of sub-blocks as the similarity. . The block on the picture included in the moving image data is divided, and an offset value is calculated according to the complexity of each of the plurality of sub-blocks and the similarity between the sub-blocks,
The first encoding cost when the block is encoded using the block as an encoding unit is the second encoding cost of the plurality of sub-blocks when the block is encoded using the sub-block as the encoding unit. If the block is equal to or less than the sum of the encoding costs and the offset value, the block is selected as the encoding unit. On the other hand, if the first encoding cost is greater than the total, the encoding unit is Select the sub-block,
Encoding the block for each of the selected encoding units;
A moving picture encoding method including the above. The block on the picture included in the moving image data is divided, and an offset value is calculated according to the complexity of each of the plurality of sub-blocks and the similarity between the sub-blocks,
The first encoding cost when the block is encoded using the block as an encoding unit is the second encoding cost of the plurality of sub-blocks when the block is encoded using the sub-block as the encoding unit. If the block is equal to or less than the sum of the encoding costs and the offset value, the block is selected as the encoding unit. On the other hand, if the first encoding cost is greater than the total, the encoding unit is Select the sub-block,
Encoding the block for each of the selected encoding units;
A computer program for encoding a moving image for causing a computer to execute the above.

Images