JP2014045434A - Image encoder, image decoder and programs thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance coding efficiency in inter-encoding.SOLUTION: An image encoder for encoding an image by using inter-screen prediction for an encoding block, where an image is divided into a plurality of blocks, by a prediction block of a size equal to or smaller than that of the encoding block, includes a selection unit for selecting one orthogonal transformation among a plurality of orthogonal transformations, respectively in the horizontal direction and vertical direction, on the basis of the positional relation between the prediction block and a conversion block where the encoding block is divided into one or more by hierarchy division, and a conversion unit performing orthogonal transformation in the horizontal direction and orthogonal transformation in the vertical direction selected by the selection unit. The selection unit selects DST-II when both sides of the conversion block facing each other do not match the sides of the prediction block, in both horizontal direction and vertical direction.

Description

  The present invention relates to an image encoding device, an image decoding device, and a program thereof that perform inter-screen prediction.

  In recent years, with the advance of multimedia, there are many opportunities to handle moving images. Image data has a large amount of information, and a moving image, which is a set of images, has an information amount exceeding 1 Gbit / sec in high-definition (1920 × 1080) broadcasting, which is the mainstream of television broadcasting.

  Therefore, the moving image data is H.264. H.264 / AVC (Advanced Video Coding) (see, for example, Non-Patent Document 1) and HEVC (High Efficiency Video Coding) being standardized, etc. Has been done.

  HEVC does not simply divide the screen into blocks that are encoding units from the upper left as in the conventional encoding technique, but newly adopts hierarchical division to enable block division of a plurality of hierarchies. A coding unit divided into a plurality of blocks is also referred to as a CU (Coding Unit) or a coding block.

  FIG. 1 is a diagram illustrating an example of block division by HEVC. As shown in FIG. 1, in HEVC, an image can be divided into 4 × 4, 8 × 8, 16 × 16, 32 × 32, etc. and encoded.

  In HEVC, in order to efficiently express the prediction error signal, the CU can be divided into layers and divided into conversion units. The transform unit is also referred to as a TU (Transform Unit) or a transform block.

  FIG. 2 is a diagram illustrating a relationship between a CU and a TU. As shown in FIG. 2, for example, for a 32 × 32 CU, a TU with 0 layer division is the same block as a 32 × 32 CU, and a TU with 1 layer division is a block divided into 16 × 16 It is. In addition, a TU with two hierarchical divisions is a block divided into 8 × 8.

  Since TUs are hierarchically divided according to the size of CUs, the TU sizes in each layer are different depending on the size of CUs.

  In HEVC, a CU is divided into prediction units that are a plurality of types of rectangular areas, and prediction processing is performed in each prediction unit. This prediction unit is also referred to as a PU (Prediction Unit) or a prediction block.

  Here, the following encoding is performed on the divided blocks. In-screen coding (intra coding) encodes an image by performing spatial signal prediction within the screen. This technique is also referred to as intra prediction (intra prediction). Also, inter-screen prediction (inter-coding) searches for similar regions between different frames and encodes an image for prediction. This technique is also referred to as motion compensation prediction (inter prediction).

  FIG. 3 is a diagram illustrating an example of a motion compensation prediction error signal in a block. As shown in FIG. 3, the prediction residual signal based on motion compensation prediction tends to have a slightly larger error at the boundary of the region used for prediction than at the center of the region (see Non-Patent Document 2). ).

  In HEVC currently under consideration for standardization, a block size can be selected flexibly for each of the motion compensated prediction block (prediction block) and the orthogonal transform block (transform block). Therefore, an appropriate block size can be determined for each of the motion compensation prediction and the orthogonal transform, and efficient compression can be realized.

  In consideration of the signal characteristics, an encoding method using DCT (Discrete Cosine Transform) -IV or DST (Discrete Sine Transform) -VII has been proposed (see, for example, Non-Patent Document 3). Hereinafter, this technique is also referred to as technique 1. This method 1 utilizes the fact that the error is large at the boundary portion of the prediction block and the error is small inside.

  That is, Method 1 uses an orthogonal transform in which a low-frequency component orthogonal transform base is an open end and a reverse end is a short-circuited end in a transform block having a boundary that matches the prediction block. FIG. 4 is a diagram illustrating an example of a relationship between a transform block and a prediction block. As shown in FIG. 4, the error is small at the center of the prediction block, and the error is large at the periphery. Therefore, in the conversion block shown in FIG. 4, DCT-IV / Flipped DST-VII is performed in the vertical direction, and Flipped DCT-IV / DST-VII is performed in the horizontal direction. The waveform shown in FIG. 4 is a DCT-IV waveform. For example, in the case of DCT-IV, “Flipped” means that DCT-IV is performed by switching the signal line of the input signal between upper and lower.

  Note that “Flipped DCT-IV / DST-VII” means a function obtained by inverting the direction of DCT-IV / DST-VII. FIG. 5 is a diagram showing the relationship between DCT-IV and Flipped DCT-IV. As shown in FIG. 5, the flipped DCT-IV is a function in which the direction of the DCT-IV is inverted.

  DCT-IV and DST-VII satisfy the condition and are used for coding the transform block of the motion compensated prediction residual block. When the boundary is not shared between the transform block and the prediction block, DCT is selected.

  In Method 1, DST-VII may be used instead of DCT-IV when the block size is 4 × 4, for example. Therefore, in Method 1, the orthogonal transform can be switched based on the block size or the boundary condition between the transform block and the prediction block.

  FIG. 6 is a diagram illustrating an example of orthogonal transform in each transform block in the case of Method 1. The transform blocks b101 to b103 shown in FIG. 6 use DCT-IV or flipped DCT-IV orthogonal transform for the boundary direction that coincides with the prediction block boundary. For the transform block b104 that does not coincide with the boundary of the prediction block, DCT-II orthogonal transform is used in the horizontal direction and the vertical direction.

  Further, the advantage in implementation of Method 1 can be realized by using DCT-II butterfly computation when DCT-IV is used as asymmetric orthogonal transformation. Thereby, an increase in hardware cost can be suppressed.

  FIG. 7 is a diagram illustrating a method for realizing DCT-IV. As shown in FIG. 7, DCT-IV can be realized by diverting part of DCT-II.

  In addition, paying attention to ease of mounting, a method of encoding using DST-II having a complementary relationship with DCT-II has been proposed (for example, see Non-Patent Document 4). Hereinafter, this technique is also referred to as technique 2.

  Method 2 is a method of encoding the residual signal of motion compensation prediction by switching between DCT-II and DST-II by an RD (Rate-Distortion) optimization method. Thereby, an optimal orthogonal transform can be selected for each transform block, and a highly efficient compression effect can be realized.

  FIG. 8 illustrates an example of transform block division. For each transform block shown in FIG. 8, in Method 2, RD optimization is performed to switch between DCT and DST. FIG. 9 is a diagram illustrating an implementation example of DST-II using DCT-II. As shown in FIG. 9, DST-II can be mounted relatively easily.

  On the other hand, in Method 2, it is necessary to transmit to the decoding side which orthogonal transformation is applied, and identification information indicating which orthogonal transformation is a factor that hinders the effect of reducing the code amount (non-conversion). (See Patent Document 5 and Patent Documents 1 and 2). For this reason, in the method 2, the optimal orthogonal transform is not switched in the horizontal and vertical directions, but the two-dimensional orthogonal transform is switched in units of blocks.

Supervised by Satoshi Okubo, “Revised Third Edition H.264 / AVC Textbook”, Impress R & D, January 1, 2009 J. Schmidt, B. Edler and J. Ostermann, "Prediction od DCT Coefficients Considering Motion Compensation Error Distributions", VCIP 2011 J. An, X. Zhao X. Guo, and S. Lei "CE7: Boundary-dependent Transform for Inter-Predicted Residue," JCTVC-H0309 (Jan.2012) A. Ichigaya and S. Sakaida, "Performance report of adaptive DCT / DST selection" JCTVC-D182 (Jan.2011) Goro Ichigaya, Yasuko Sugito, Shinichi Sakaida, "Inter-coding using DCT and DST adaptively", Eiji-gakugakuho, vol.36, no.9 ME2012-59, PP.213-218,2012

JP 2011-205602 A JP 2011-2223068 A

  However, in Method 1, the transform block that does not share the boundary with the prediction block uses only the conventional DCT as orthogonal transform, and is not improved. In Method 2, identification information is assigned to each block, and the increase in the identification information is a factor that hinders the effect of reducing the code amount.

  Therefore, an object of the present invention is to provide an image encoding device, an image decoding device, and a program capable of improving encoding efficiency in inter encoding.

  An image encoding apparatus according to an aspect of the present invention encodes an image using inter-frame prediction using a prediction block having a size equal to or smaller than the encoded block for an encoded block obtained by dividing the image into a plurality of blocks. An apparatus comprising: a plurality of orthogonal transforms in each of a horizontal direction and a vertical direction based on a positional relationship between a transform block obtained by dividing the coding block into one or a plurality of layers by hierarchical partitioning and the prediction block. A selection unit that selects one orthogonal transformation; and a conversion unit that performs the orthogonal transformation in the horizontal direction and the orthogonal transformation in the vertical direction selected by the selection unit in each direction. If the opposite sides of the transform block do not coincide with the sides of the prediction block in each of the vertical directions, DST-II is selected.

  In addition, the selection unit selects DCT-II when both opposite sides of the transform block coincide with the sides of the prediction block in each of the horizontal direction and the vertical direction, and one side of the transform block is the prediction block. DCT-IV, Flipped DCT-IV, DST-VII, or Flipped DST-VII may be selected.

  In addition, a setting unit that sets either the first mode that prioritizes implementation or the second mode that prioritizes performance is further provided, and the selection unit is configured to set one side of the conversion block when the first mode is set. When DCT-IV or Flipped DCT-IV is selected and the second mode is set, when one side of the transform block matches the side of the prediction block DST-VII or Flipped DST-VII may be selected.

  In addition, the image encoding program according to another aspect encodes an image using inter-screen prediction using a prediction block having a size equal to or smaller than the encoded block for an encoded block obtained by dividing the image into a plurality of blocks. Based on the positional relationship between the prediction block and the transform block obtained by dividing the coding block into one or more by hierarchical partitioning, one of a plurality of orthogonal transforms in the horizontal direction and the vertical direction is stored in the computer. An image encoding program that executes a selection step of selecting an orthogonal transformation, and a transformation step of performing an orthogonal transformation in a horizontal direction and an orthogonal transformation in a vertical direction selected by the selection unit in each direction, In the selection step, both opposing sides of the transform block do not coincide with sides of the prediction block in each of the horizontal direction and the vertical direction. If you select the DST-II.

  An image decoding apparatus according to another aspect is an image decoding apparatus that decodes a stream encoded by the image encoding apparatus described above, wherein the stream is entropy-decoded and decoded by the decoding section. And a selection unit that selects one inverse orthogonal transform from among a plurality of inverse orthogonal transforms in each of the horizontal direction and the vertical direction based on the positional relationship between the transform block and the prediction block used at the time of intra prediction. An inverse transform unit that performs an inverse orthogonal transform in the horizontal direction and an inverse orthogonal transform in the vertical direction selected by the selection unit in each direction, the selection unit in each of the horizontal direction and the vertical direction, IDST-II may be selected when both opposing sides of the transform block do not coincide with the sides of the prediction block.

  Further, an image decoding program according to another aspect includes a decoding step of decoding the stream, the decoded conversion block, and intra-screen prediction to a computer that decodes the stream encoded by the image encoding device described above. A selection step of selecting one inverse orthogonal transform from a plurality of inverse orthogonal transforms in each of the horizontal direction and the vertical direction based on the positional relationship with the prediction block used at the time, and the selected horizontal orthogonal inverse An image decoding program that performs an inverse transform step for performing transform and inverse orthogonal transform in the vertical direction in each direction, wherein the selection step includes both sides of the transform block facing in the horizontal direction and the vertical direction, respectively. If both do not coincide with the side of the prediction block, IDST-II is selected.

  According to the present invention, coding efficiency can be improved in inter coding.

The figure which shows an example of the block division | segmentation by HEVC. The figure which shows the relationship between CU and TU. The figure which shows an example of the prediction error signal of the motion compensation in a block. The figure which shows an example of the relationship between a conversion block and a prediction block. The figure which shows the relationship between DCT-IV and Flipped DCT-IV. The figure which shows the example of the orthogonal transformation in each transformation block in the case of the method 1. The figure which shows the implementation | achievement method of DCT-IV. The figure which shows the example of a division | segmentation of a conversion block. The figure which shows the example of mounting of DST-II using DCT-II. 1 is a block diagram illustrating an example of a schematic configuration of an image encoding device according to Embodiment 1. FIG. FIG. 3 is a block diagram illustrating an example of a configuration of an orthogonal transform unit according to the first embodiment. The figure which shows the example of application distribution of the method by the position of a conversion block. FIG. 6 is a diagram illustrating an application example of horizontal orthogonal transformation in the first embodiment. 5 is a flowchart illustrating an example of inter prediction orthogonal transform processing according to the first embodiment. FIG. 6 is a block diagram illustrating an example of a configuration of an orthogonal transform unit according to the second embodiment. FIG. 10 is a diagram illustrating an application example of horizontal orthogonal transformation in the second embodiment. 9 is a flowchart illustrating an example of inter prediction orthogonal transform processing according to the second embodiment. FIG. 10 is a block diagram illustrating an example of a schematic configuration of an image decoding device according to a third embodiment. FIG. 10 is a block diagram illustrating an example of a configuration of an inverse orthogonal transform unit according to the third embodiment. FIG. 10 is a block diagram illustrating an example of a configuration of an image processing apparatus according to a fourth embodiment.

  In the following, each embodiment that improves the coding efficiency over Technique 1 and Technique 2 will be described in detail with reference to the accompanying drawings.

[Example 1]
In the first embodiment, method 1 is used for a transform block whose boundary matches the prediction block, and method 2 is used for a transform block whose boundary does not match the prediction block. Thereby, since orthogonal transform suitable for all the transform blocks can be performed, encoding efficiency can be improved.

<Configuration>
FIG. 10 is a block diagram illustrating an example of a schematic configuration of the image encoding device 10 according to the first embodiment. In the example illustrated in FIG. 10, the image encoding device 10 includes a preprocessing unit 100, a prediction error signal generation unit 101, an orthogonal transformation unit 102, a quantization unit 103, an entropy encoding unit 104, an inverse quantization unit 105, and an inverse orthogonal. A conversion unit 106, a decoded image generation unit 107, a loop filter unit 109, a decoded image storage unit 110, an intra prediction unit 111, an inter prediction unit 112, a motion vector calculation unit 113, and a predicted image selection unit 115 are included. An outline of each part will be described below.

  The preprocessing unit 100 rearranges the pictures in accordance with the picture type, and sequentially outputs the picture type and the frame image for each frame. The preprocessing unit 100 also performs block division and the like.

  The prediction error signal generation unit 101 acquires block data obtained by dividing an encoding target image of input moving image data into blocks such as 32 × 32, 16 × 16, and 8 × 8 pixels.

  The prediction error signal generation unit 101 generates a prediction error signal based on the block data and the block data of the prediction image output from the prediction image selection unit 115. The prediction error signal generation unit 101 outputs the generated prediction error signal to the orthogonal transformation unit 102.

  The orthogonal transform unit 102 performs orthogonal transform processing on the input prediction error signal. The orthogonal transform unit 102 switches the orthogonal transform on the prediction error signal in the case of inter prediction based on the positional relationship between the prediction block and the transform block. Details of the orthogonal transform unit 102 will be described later with reference to FIG. The orthogonal transform unit 102 outputs a signal separated into horizontal and vertical frequency components by the orthogonal transform process to the quantization unit 103.

  The quantization unit 103 quantizes the output signal from the orthogonal transform unit 102. The quantization unit 103 reduces the code amount of the output signal by quantization, and outputs this output signal to the entropy encoding unit 104 and the inverse quantization unit 105.

  The entropy coding unit 104 entropy codes and outputs the output signal from the quantization unit 103, the motion vector information output from the motion vector calculation unit 113, the filter coefficient from the loop filter unit 109, and the like. Entropy coding is a method of assigning variable-length codes according to the appearance frequency of symbols.

  The inverse quantization unit 105 dequantizes the output signal from the quantization unit 103 and then outputs the output signal to the inverse orthogonal transform unit 106. The inverse orthogonal transform unit 106 performs an inverse orthogonal transform process on the output signal from the inverse quantization unit 105 and then outputs the output signal to the decoded image generation unit 107. By performing decoding processing by the inverse quantization unit 105 and the inverse orthogonal transform unit 106, a signal having the same level as the prediction error signal before encoding is obtained.

  The decoded image generation unit 107 adds the block data of the image subjected to motion compensation by the inter prediction unit 112 and the prediction error signal decoded by the inverse quantization unit 105 and the inverse orthogonal transform unit 106. The decoded image generation unit 107 outputs the decoded image block data generated by addition to the loop filter unit 109.

  The loop filter unit 109 is, for example, an ALF (Adaptive Loop Filter) or a deblocking filter, and may include either or both.

  For example, the loop filter unit 109 divides the input image into groups for each predetermined size, and generates an appropriate filter coefficient for each group. The loop filter unit 109 divides the filtered decoded image into groups for each predetermined size, and performs filter processing for each group using the generated filter coefficients. The loop filter unit 109 outputs the filter processing result to the decoded image storage unit 110 and accumulates it as a reference image. The predetermined size is, for example, an orthogonal transformation size.

  The decoded image storage unit 110 stores the input block data of the decoded image as new reference image data, and outputs the data to the intra prediction unit 111, the inter prediction unit 112, and the motion vector calculation unit 113.

  The intra prediction unit 111 generates block data of the predicted image from the already-encoded reference pixels for the processing target block of the encoding target image. The intra prediction unit 111 performs prediction using a plurality of prediction directions and determines an optimal prediction direction.

  The inter prediction unit 112 performs motion compensation on the reference image data acquired from the decoded image storage unit 110 with the motion vector provided from the motion vector calculation unit 113. Thereby, block data as a motion-compensated reference image is generated. The inter prediction unit 112 outputs information on the prediction block used for motion compensation to the orthogonal transform unit 102.

  The motion vector calculation unit 113 obtains a motion vector using the block data in the encoding target image and the reference image acquired from the decoded image storage unit 110. The motion vector is a value indicating a spatial deviation in units of blocks obtained using a block matching technique or the like that searches for a position most similar to the processing target block from the reference image in units of blocks.

  The motion vector calculation unit 113 outputs the obtained motion vector to the inter prediction unit 112, and outputs motion vector information including information indicating the motion vector and the reference image to the entropy coding unit 104.

  The block data output from the intra prediction unit 111 and the inter prediction unit 112 are input to the predicted image selection unit 115.

  The predicted image selection unit 115 selects one of the block data acquired from the intra prediction unit 111 and the inter prediction unit 112 as a predicted image. The selected prediction image is output to the prediction error signal generation unit 101.

  Note that the configuration of the image encoding device 10 illustrated in FIG. 10 is an example, and the configurations may be combined or the configurations may be appropriately changed as necessary.

<Configuration of orthogonal transform unit>
Next, the configuration of the orthogonal transform unit 102 will be described. FIG. 11 is a block diagram illustrating an example of the configuration of the orthogonal transform unit 102 according to the first embodiment. An orthogonal transform unit 102 illustrated in FIG. 11 illustrates a configuration for orthogonal transform for a prediction error signal of inter prediction according to the present invention.

  11 includes a selection unit 201, a cost calculation unit 202, a conversion unit 203, and a setting unit 204. The selection unit 201 acquires prediction block information from the inter prediction unit 112. In addition, the selection unit 201 acquires a prediction error signal from the prediction error signal generation unit 101.

  Based on the relationship between the position of the transform block that is the target of orthogonal transform and the position of the predicted block based on the obtained prediction block information, the selection unit 201 performs the horizontal direction and the vertical direction respectively. One orthogonal transformation is selected from among a plurality of orthogonal transformations. The plurality of orthogonal transforms are, for example, DCT-II, DCT-IV, Flipped DCT-IV, DST-II, DST-VII, and Flipped DST-VII.

  If at least one boundary (side) between the transform block and the prediction block matches in each of the horizontal direction and the vertical direction, the selection unit 201 selects orthogonal transform by the same method as Method 1. The selection unit 201 notifies the transformation unit 203 of the selected orthogonal transformation.

  In addition, if the sides of the transform block and the prediction block do not match, the selection unit 201 causes the cost calculation unit 202 to select the optimal one of DCT-II and DST-II in the same manner as the method 2. . For example, the selection unit 201 instructs the conversion unit 203 to perform DCT-II and DST-II, and instructs the cost calculation unit 202 to calculate an encoding cost (also simply referred to as a cost).

  The cost calculation unit 202 uses a rate distortion optimization method (see, for example, http://www.tom.comm.waseda.ac.jp/~tsukuba/source/avm20050609.pdf), and performs orthogonal processing. The cost (for example, code amount) of the transform (for example, DCT-II and DST-II) is calculated, and the orthogonal transform with the smallest cost is selected. For convenience of explanation, it is assumed that the cost calculation unit 202 includes a quantization unit and an entropy encoding unit and calculates the code amount in each orthogonal transform. However, the code amount is fed back from the entropy encoding unit 104 in implementation. May be obtained. Further, the cost calculation unit 202 may be included in the selection unit 201.

  The transform unit 203 includes a DCT unit 205 and a DST unit 206, and has a plurality of orthogonal transforms. The DCT unit 205 performs DCT-based orthogonal transformation. The DST unit 206 performs orthogonal transformation of the DST system. The conversion unit 203 performs one orthogonal transformation selected by the selection unit 201 on the prediction error signal to be converted acquired from the prediction error signal generation unit 101, and outputs the converted coefficient to the quantization unit 103.

  Note that the conversion unit 203 has been described with a configuration in which the DCT unit 205 and the DST unit 206 are separated for easy understanding. However, in terms of mounting, as shown in FIGS. 7 and 9, the conversion unit 203 can mount DCT-IV and DST-II using DCT-II as a core. Therefore, the conversion unit 203 performs part of the DCT-II processing when performing DCT-IV (see FIG. 7), and performs DCT-II with slight modifications to the input and output when performing DST-II. Each process can be realized by performing the process II (see FIG. 9). That is, the conversion unit 203 may be divided into a processing unit that performs DCT-II, DCT-IV, and DST-II, and a processing unit that performs DST-VII. For example, the implementation of the conversion unit 203 can be freely changed as appropriate.

  In addition, when the selection unit 201 instructs the DCT-II and DST-II processes, the conversion unit 203 executes the DCT-II and DST-II processes. The transform unit 203 outputs the respective orthogonal transform coefficients to the cost calculation unit 202 when the DCT-II and DST-II processes are performed.

  The conversion unit 203 outputs the transform coefficient of the orthogonal transform selected by the cost calculation unit 202 and identification information for identifying the selected orthogonal transform to the quantization unit 103. Note that the transform unit 203 may output the selected entropy-encoded data of the orthogonal transform to the entropy encoder 104 when the cost calculation unit 202 performs processing. Further, when the cost calculation unit 202 is fed back from the entropy encoding unit 104, the orthogonal transform unit 102 may instruct the entropy encoding unit 104 of information indicating which encoded data is used.

  Note that the selection unit 201 may select whether to use DCT-IV or DST-VII based on the mode set by the setting unit 204. For example, the selection unit 201 selects DCT-IV when the first mode with mounting priority is set by the setting unit 204, and DST-VII when the second mode with performance priority is set by the setting unit 204. Is used. DST-VII is said to have better performance than DCT-IV. Therefore, DCT-IV may be used when priority is given to mounting, and DST-VII may be used when priority is given to performance.

  The setting unit 204 sets either the first mode or the second mode. The mode may be set in advance by the user. When the conversion unit 203 has only one of DCT-IV and DST-VII, the setting unit 204 is not necessarily a necessary configuration.

  The setting unit 204 may set a predetermined block size and process the DST-VII, for example, for the set block size. The predetermined block size is 4 × 4, for example.

<Application example of method>
Next, an application example of the method for the transform block will be described. FIG. 12 is a diagram illustrating an application distribution example of the technique based on the position of the transform block. In the example shown in FIG. 12, the prediction block is divided into vertically long rectangles. Since the transform block b201 illustrated in FIG. 12 does not match the sides of the prediction block on all sides, the method 2 is applied. Since transform blocks other than the transform block b201 illustrated in FIG. 9 have at least one side that matches the side of the prediction block, the method 1 is applied.

<Application example of orthogonal transformation>
Next, an application example of orthogonal transform to the transform block will be described. FIG. 13 is a diagram illustrating an application example of the orthogonal transformation in the horizontal direction according to the first embodiment. In the example illustrated in FIG. 13, the selection unit 201 selects orthogonal transform as follows. The transform block shown in FIG. 13 has a square shape, but is not limited to a square and may be a rectangle. In the example illustrated in FIG. 13, it is assumed that the conversion unit 203 does not have DST-VII or the first mode is set by the setting unit 204. The thick line side shown in FIG. 13 indicates a side that matches the side of the prediction block.

(1) Both sides coincidence selection unit 201 selects DCT-II as orthogonal transformation when both the left and right sides of the transform block coincide with the sides of the prediction block.

(2) The left side match selection unit 201 selects DCT-IV as orthogonal transform when the left side of the transform block matches the side of the prediction block.

(3) When the right side of the transform block matches the side of the prediction block, the right side match selection unit 201 selects the flipped DCT-IV as orthogonal transform.

(4) When both sides of the transform block do not match the sides of the prediction block, the mismatch selection unit 201 selects DCT-II and DST-II as orthogonal transform. The cost calculation unit 202 selects an orthogonal transform with a low cost.

  Note that the selection unit 201 may perform processing in the vertical direction in the same manner as in the horizontal direction. In the vertical direction, the selection unit 201 may perform processing by replacing the left side with the upper side and the right side with the lower side.

<Operation>
Next, the operation of the image encoding device 10 in the first embodiment will be described. FIG. 14 is a flowchart illustrating an example of inter prediction orthogonal transform processing according to the first embodiment. The process shown in FIG. 14 is executed in each of the horizontal direction and the vertical direction.

  In step S101, the selection unit 201 determines whether any of the boundaries between the transform block and the prediction block matches. That is, the selection unit 201 determines whether at least one side of the transform block matches the side of the prediction block. If at least one side matches (step S101—YES), the process proceeds to step S102. If one side does not match (step S101—NO), the process proceeds to step S107.

  In step S102, the selection unit 201 determines whether only the left side (or upper side) of the transform block matches the side of the prediction block. If only the left side (or top side) of the transform block matches the side of the prediction block (step S102-YES), the process proceeds to step S103, and if only the left side (or top side) of the transform block does not match the side of the prediction block (step S102). (S102-NO) It progresses to step S104.

  In step S103, the conversion unit 203 performs DCT-IV processing on the conversion block.

  In step S104, the selection unit 201 determines whether only the right side (or the lower side) of the transform block matches the side of the prediction block. If only the right side (or bottom side) of the transform block matches the side of the prediction block (step S104-YES), the process proceeds to step S105, and if only the right side (or bottom side) of the transform block does not match the side of the prediction block (step S104-NO) Proceed to step S106.

  In step S105, the conversion unit 203 performs a flipped DCT-IV process on the conversion block.

  In step S106, the conversion unit 203 performs DCT-II processing on the conversion block.

  In step S107, the conversion unit 203 performs DCT-II processing on the conversion block.

  In step S108, the conversion unit 203 performs DST-II processing on the converted block.

  In step S109, the cost calculation unit 202 calculates the costs of DCT-II and DST-II, performs RD determination, and selects the orthogonal transform with the lower cost.

  Therefore, the orthogonal transform unit 102 can perform appropriate orthogonal transform using the method 1 in steps S101 to S106, and can perform appropriate orthogonal transform using the method 2 in steps S107 to S109.

  As described above, according to the first embodiment, encoding efficiency can be improved in inter-coding. In addition, according to the first embodiment, Method 1 is used for a transform block whose boundary matches the prediction block, and Method 2 is used for a transform block whose boundary does not match the prediction block. An orthogonal transform suitable for the block can be performed.

[Example 2]
Next, an image coding apparatus according to the second embodiment will be described. In the second embodiment, orthogonal transforms of all transform blocks are selected based on the positional relationship between transform blocks and prediction blocks without using RD determination. The RD determination has a larger processing amount than other processes. Therefore, in the second embodiment, the coding efficiency can be made substantially equal to that of the first embodiment while reducing the processing amount by not performing the RD determination.

<Configuration>
Since the configuration of the image coding apparatus according to the second embodiment is the same as that illustrated in FIG. 10 according to the first embodiment, the description will be made using the same reference numerals. In the second embodiment, since the processing of the orthogonal transform unit 102 is different from that of the first embodiment, the orthogonal transform unit 102 of the second embodiment will be mainly described below.

<Orthogonal transformation unit>
FIG. 15 is a block diagram illustrating an example of the configuration of the orthogonal transform unit 102 according to the second embodiment. Among the configurations of the orthogonal transform unit 102 illustrated in FIG. 15, the same components as those illustrated in FIG. 11 are denoted by the same reference numerals. Hereinafter, the selection unit 250 will be mainly described.

  The selection unit 250 has a plurality of horizontal and vertical directions based on only the positional relationship between the transform block acquired from the prediction error signal 101 and the prediction block based on the prediction block information acquired from the inter prediction unit 112. One orthogonal transform is selected from the orthogonal transforms. For example, the selection unit 250 is smaller than DCT-II because the DC component of the error signal in the prediction block is small when both sides in the horizontal or vertical direction of the transform block do not match the sides of the prediction block. Select DST-II suitable for error expression.

  In addition, the selection unit 250 selects DCT-II when both opposing sides of the transform block match the prediction block side, and DCT-IV, Flipped when one side of the transform block matches the prediction block side. Select either DCT-IV, DST-VII, or Flipped DST-VII. Further, the selection unit 250 selects one of the orthogonal transforms according to the position of the side where the transform block and the prediction block match. The selection unit 250 notifies the conversion unit 203 of the selected orthogonal transformation.

  Similarly to the first embodiment, the selection unit 250 selects the DCT-IV if the setting mode 204 sets the first mode in which the mounting priority is set, and selects the DST if the second mode in which the performance priority is set. -Select VII.

<Application example of orthogonal transformation>
Next, an application example of orthogonal transform to the transform block will be described. FIG. 16 is a diagram illustrating an application example of horizontal orthogonal transform in the second embodiment. In the example illustrated in FIG. 16, the selection unit 250 selects orthogonal transform as follows. The transform block shown in FIG. 16 has a square shape, but is not limited to a square and may be a rectangle. In the example illustrated in FIG. 16, it is assumed that the conversion unit 203 does not have DST-VII or the first mode is set by the setting unit 204. The thick line side shown in FIG. 16 indicates a side that matches the side of the prediction block.

(1) Both sides coincidence selection unit 250 selects DCT-II as orthogonal transformation when both the left and right sides of the transformation block coincide with the sides of the prediction block.

(2) The left side match selection unit 250 selects DCT-IV as orthogonal transform when the left side of the transform block matches the side of the prediction block.

(3) When the right side of the transform block matches the side of the prediction block, the right side coincidence selection unit 250 selects the flipped DCT-IV as orthogonal transform.

(4) When both sides of the transform block do not match the sides of the prediction block, the mismatch selection unit 250 selects DST-II as orthogonal transform.

  This is because DST-II, which has a short-circuited base at both ends, is suitable when the boundary between both sides in the horizontal direction or the vertical direction does not match between the prediction block and the transform block.

  Note that the selection unit 250 may perform processing in the vertical direction in the same manner as in the horizontal direction. In the vertical direction, the selection unit 250 may perform processing by replacing the left side with the upper side and the right side with the lower side.

  Thereby, since the selection part 250 can select orthogonal transformation only based on the positional relationship between a transformation | blocking block and a prediction block, it is not necessary to use RD determination, Furthermore, the identification information which shows an orthogonal transformation classification is added. There is no need.

<Operation>
Next, the operation of the image encoding device 10 according to the second embodiment will be described. FIG. 17 is a flowchart illustrating an example of inter prediction orthogonal transform processing according to the second embodiment. The process shown in FIG. 17 is executed in each of the horizontal direction and the vertical direction.

  In step S201, the selection unit 250 determines whether both ends of the boundary between the transform block and the prediction block match. If both ends match (step S201—YES), the process proceeds to step S202. If one side matches or both ends do not match (step S201—NO), the process proceeds to step S203.

  In step S202, the selection unit 250 selects DCT-II, for example, according to FIG. The selection unit 250 notifies the conversion unit 203 of the selected orthogonal transformation. The conversion unit 203 performs DCT-II processing on the conversion block.

  In step S203, the selection unit 250 determines whether both ends of the boundary between the transform block and the prediction block coincide with each other. If both ends do not match (step S203—YES), the process proceeds to step S204, and if at least one side matches (step S203—NO), the process proceeds to step S205.

  In step S204, the selection unit 250 selects DST-II, for example, according to FIG. The selection unit 250 notifies the conversion unit 203 of the selected orthogonal transformation. The conversion unit 203 performs DST-II processing on the conversion block.

  In step S205, the selection unit 250 determines whether the left side (or upper side) of the transform block matches the side of the prediction block. If the left side (or top side) of the transform block matches the side of the prediction block (step S205—YES), the process proceeds to step S206, and if the left side (or top side) of the transform block does not match the side of the prediction block (step S205— NO) Proceed to step S207.

  In step S206, the conversion unit 203 performs DCT-IV processing on the conversion block.

  In step S207, the conversion unit 203 performs a flipped DCT-IV process on the conversion block. Note that DCT-IV in FIGS. 16 and 17 may use DST-VII.

  As described above, according to the second embodiment, encoding efficiency can be improved in inter-coding. Also, according to the second embodiment, since orthogonal transform can be selected based only on the positional relationship between the transform block and the prediction block, it is not necessary to use RD determination, and identification information indicating the orthogonal transform type is added. There is no need to do. Note that the encoding efficiency in the second embodiment can be substantially equal to that in the first embodiment.

[Example 3]
Next, the image decoding device 30 according to the third embodiment will be described. The image decoding device 30 according to the third embodiment is a device that decodes the bitstream encoded by the image encoding device 10 according to the first or second embodiment.

<Configuration>
FIG. 18 is a block diagram illustrating an example of a schematic configuration of the image decoding device 30 according to the third embodiment. As illustrated in FIG. 18, the image decoding device 30 includes an entropy decoding unit 301, an inverse quantization unit 302, an inverse orthogonal transform unit 303, an intra prediction unit 304, a decoded information storage unit 305, an inter prediction unit 306, and a predicted image selection unit. 307, a decoded image generation unit 308, a loop filter unit 310, and a frame memory 311. An outline of each part will be described below.

  When a bit stream is input, the entropy decoding unit 301 performs entropy decoding corresponding to the entropy encoding of the image encoding device 10. The prediction error signal decoded by the entropy decoding unit 301 is output to the inverse quantization unit 302. Also, the decoded filter coefficient, the decoded motion vector in the case of inter prediction, and the like are output to the decoded information storage unit 305.

  In the case of intra prediction, the entropy decoding unit 301 notifies the intra prediction unit 304 to that effect. In addition, the entropy decoding unit 301 notifies the prediction image selection unit 307 whether the decoding target image is inter predicted or intra predicted.

  The inverse quantization unit 302 performs an inverse quantization process on the output signal from the entropy decoding unit 301. The inversely quantized output signal is output to the inverse orthogonal transform unit 303.

  The inverse orthogonal transform unit 303 performs an inverse orthogonal transform process on the decoded block of the output signal from the inverse quantization unit 302 to generate a residual signal. In the case of inter prediction, the inverse orthogonal transform unit 303 controls whether to perform inverse DCT (IDCT) processing or inverse DST (IDST) processing. Details of the inverse orthogonal transform unit 303 will be described later with reference to FIG. The residual signal is output to the decoded image generation unit 308.

  The intra prediction unit 304 generates a predicted image using a plurality of prediction directions from the peripheral pixels already decoded of the decoding target image acquired from the frame memory 311.

  The decoding information storage unit 305 stores decoding information such as filter coefficients, motion vectors, and division modes of the decoded loop filter.

  The inter prediction unit 306 performs motion compensation on the reference image data acquired from the frame memory 311 using the motion vector and the division mode acquired from the decoded information storage unit 305. Thereby, block data as a motion-compensated reference image is generated. The inter prediction unit 306 notifies the inverse orthogonal transform unit 303 of the prediction block information.

  The predicted image selection unit 307 selects either the intra predicted image or the inter predicted image. The selected block data is output to the decoded image generation unit 308.

  The decoded image generation unit 308 adds the predicted image output from the predicted image selection unit 307 and the residual signal output from the inverse orthogonal transform unit 303 to generate a decoded image. The generated decoded image is output to the loop filter unit 310.

  The loop filter unit 310 applies a filter for reducing block distortion to the decoded image output from the decoded image generation unit 308, and outputs the decoded image after the loop filter processing to the frame memory 311. Note that the decoded image after the loop filter may be output to a display device or the like.

  The frame memory 311 stores a decoded image that serves as a reference image. The decoded information storage unit 305 and the frame memory 311 are configured separately, but they may be the same storage unit.

<Configuration of inverse orthogonal transform unit>
Next, the configuration of the inverse orthogonal transform unit 303 will be described. FIG. 19 is a block diagram illustrating an example of the configuration of the inverse orthogonal transform unit 303 according to the third embodiment. The inverse orthogonal transform unit 303 illustrated in FIG. 19 includes a selection unit 401 and an inverse transform unit 402.

  The selection unit 401 acquires information on the prediction block from the inter prediction unit 306 and acquires information on the transform block from the inverse quantization unit 302. The selection unit 401 has the same functions as the selection unit shown in FIGS. 11 and 15 in other processes. Note that, when the identification information indicating orthogonal transformation is acquired from the entropy decoding unit 301, the selection unit 401 selects an inverse orthogonal transformation corresponding to the identification information. The selection unit 401 notifies the inverse transform unit 402 of the selected inverse orthogonal transform.

  The inverse transform unit 402 performs inverse orthogonal transform on the signal obtained by inverse quantization of the transform block acquired from the inverse quantization unit 302. The inverse conversion unit 402 includes an IDCT unit 403 and an IDST unit 404.

  The IDCT unit 403 performs horizontal or vertical IDCT-II, IDCT-IV, or Flipped IDCT-IV processing selected by the selection unit 401 on the inversely quantized signal, and sends it to the decoded image generation unit 308. Outputs an inverse orthogonal transformed signal.

  The IDST unit 404 performs the process of IDST-II, IDST-VII, or Flipped IDST-VII selected by the selection unit 401 with respect to the horizontal direction or vertical direction of the inversely quantized signal, and the signal subjected to inverse orthogonal transform Is output to the decoded image generation unit 308.

  Thereby, the selection unit 401 and the inverse orthogonal transform unit 402 can perform the same processing as the processing of the image encoding device 10.

<Operation>
The inverse orthogonal transform processing in the image decoding device 30 is basically the same as the processing shown in FIGS. The difference is that the orthogonal transform is an inverse orthogonal transform and that IDCT-II or IDST-II is selected based on the identification information indicating the orthogonal transform after step S101-NO in FIG.

  As described above, according to the third embodiment, it is possible to appropriately decode the bitstream encoded by the image encoding device 10 according to the first and second embodiments.

[Example 4]
FIG. 20 is a block diagram illustrating an example of the configuration of the image processing apparatus 50 according to the fourth embodiment. An image processing apparatus 50 illustrated in FIG. 20 is an example of an apparatus in which the image encoding process described in the first or second embodiment or the image decoding process described in the third embodiment is implemented by software.

  As illustrated in FIG. 20, the image processing apparatus 50 includes a control unit 501, a main storage unit 502, an auxiliary storage unit 503, a drive device 504, a network I / F unit 506, an input unit 507, and a display unit 508. These components are connected to each other via a bus so as to be able to transmit and receive data.

  The control unit 501 is a CPU (Central Processing Unit) that controls each device, calculates data, and processes in a computer. In addition, the control unit 501 performs an operation for executing an image encoding process program including an orthogonal transform control process stored in the main storage unit 502 or the auxiliary storage unit 503 or an image decoding process program including an inverse orthogonal transform control process. Device. The control unit 501 receives data from the input unit 507 and the storage device, calculates and processes the data, and then outputs the data to the display unit 508 and the storage device.

  In addition, the control unit 501 can realize the processing described in each embodiment by executing a program for image encoding processing including orthogonal transformation control processing or image decoding processing including inverse orthogonal transformation control processing. .

  The main storage unit 502 is a ROM (Read Only Memory), a RAM (Random Access Memory), or the like. The main storage unit 502 is a storage device that stores or temporarily stores programs and data such as OS (Operating System) and application software which are basic software executed by the control unit 501.

  The auxiliary storage unit 503 is an HDD (Hard Disk Drive) or the like, and is a storage device that stores data related to application software or the like.

  The drive device 504 reads the program from the recording medium 505, for example, a flexible disk, and installs it in the storage unit.

  A predetermined program is stored in the recording medium 505, and the program stored in the recording medium 505 is installed in the image processing apparatus 50 via the drive device 504. The installed predetermined program can be executed by the image processing apparatus 50.

  The network I / F unit 506 has a communication function connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network) constructed by a data transmission path such as a wired and / or wireless line. This is an interface between the device and the image processing apparatus 50.

  The input unit 507 includes a keyboard having cursor keys, numeric input, various function keys, and the like, a mouse and a slide pad for selecting keys on the display screen of the display unit 508, and the like. The display unit 508 is configured by an LCD (Liquid Crystal Display) or the like, and performs display according to display data input from the control unit 501.

  Note that the decoded image storage unit 111 illustrated in FIG. 10 is realized by, for example, the main storage unit 502 or the auxiliary storage unit 503, and configurations other than the decoded image storage unit 111 illustrated in FIG. 10 include, for example, a control unit 501 and a work memory. This can be realized by the main storage unit 502.

  Further, the decoding information storage unit 305 and the frame memory 311 illustrated in FIG. 18 are realized by, for example, the main storage unit 502 or the auxiliary storage unit 503. The configurations other than the decoding information storage unit 305 and the frame memory 311 illustrated in FIG. It can be realized by the control unit 501 and the main storage unit 502 as a work memory.

  The program executed by the image processing apparatus 50 has a module configuration including each unit described in the first to third embodiments. As actual hardware, when the control unit 501 reads out and executes a program from the auxiliary storage unit 503, one or more of the above-described units are loaded onto the main storage unit 502, and one or more of the units are main. It is generated on the storage unit 502.

  As described above, the image encoding process described in the first embodiment and the second embodiment and the image decoding process described in the third embodiment may be realized as a program for causing a computer to execute. The processing described in each embodiment can be realized by installing this program from a server or the like and causing the computer to execute the program.

  It is also possible to record the program on the recording medium 505 and cause the computer or portable terminal to read the recording medium 505 on which the program is recorded, thereby realizing the above-described processing. The recording medium 505 is a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk, and information is electrically stored such as a ROM or flash memory. Various types of recording media such as a semiconductor memory for recording can be used. Further, the processes described in the above embodiments may be implemented in one or a plurality of integrated circuits.

  Note that the image processing device 50 according to the fourth embodiment has functions as the image encoding device 10 and the image decoding device 30 as described above.

  Each embodiment has been described in detail above. However, the present invention is not limited to the specific embodiment, and various modifications and changes other than the above-described modification are possible within the scope described in the claims. .

DESCRIPTION OF SYMBOLS 10 Image coding apparatus 30 Image decoding apparatus 50 Image processing apparatus 101 Predictive image generation part 102 Orthogonal transformation part 103 Quantization part 104 Entropy encoding part 105 Inverse quantization part 106 Inverse orthogonal transformation part 107 Decoded image generation part 109 Loop filter part 110 Decoded Image Storage Unit 111 Intra Prediction Unit 112 Inter Prediction Unit 113 Motion Vector Calculation Unit 115 Predicted Image Selection Unit 201, 250 Selection Unit 202 Cost Calculation Unit 203 Conversion Unit 204 Setting Unit 303 Inverse Orthogonal Transformation Unit 401 Selection Unit 402 Inverse Conversion Unit 501 Control unit 502 Main storage unit 503 Auxiliary storage unit 504 Drive device

Claims (6)

An image encoding device that encodes an image using inter-frame prediction with a prediction block having a size equal to or smaller than the encoded block for an encoded block obtained by dividing an image into a plurality of blocks,
Based on the positional relationship between the prediction block and the transform block obtained by dividing the encoded block into one or more by hierarchical division, one orthogonal transform is performed from among a plurality of orthogonal transforms in each of the horizontal direction and the vertical direction. A selection section to select;
A conversion unit that performs the orthogonal transformation in the horizontal direction and the orthogonal transformation in the vertical direction selected by the selection unit in each direction, and
The selection unit includes:
An image coding apparatus that selects DST-II when both sides of the transform block facing each other do not coincide with the sides of the prediction block in each of a horizontal direction and a vertical direction. The selection unit is in each of a horizontal direction and a vertical direction,
If both sides of the transform block coincide with the sides of the prediction block, select DCT-II;
The image encoding apparatus according to claim 1, wherein when one side of the transform block matches the side of the prediction block, one of DCT-IV, Flipped DCT-IV, DST-VII, and Flipped DST-VII is selected. . A setting unit that sets either the first mode that prioritizes implementation or the second mode that prioritizes performance;
The selection unit includes:
When the first mode is set, when one side of the transform block matches the side of the prediction block, select DCT-IV or Flipped DCT-IV,
The image coding apparatus according to claim 2, wherein when the second mode is set, DST-VII or Flipped DST-VII is selected when one side of the transform block coincides with a side of the prediction block. In order to encode an image using inter-frame prediction with a prediction block having a size equal to or smaller than the encoded block for an encoded block obtained by dividing the image into a plurality of blocks,
Based on the positional relationship between the prediction block and the transform block obtained by dividing the encoded block into one or more by hierarchical division, one orthogonal transform is performed from among a plurality of orthogonal transforms in each of the horizontal direction and the vertical direction. A selection step to select;
A conversion step of performing horizontal orthogonal transformation and vertical orthogonal transformation selected by the selection unit in each direction,
The selection step includes
An image encoding program for selecting DST-II when both sides of the transform block facing each other in the horizontal direction and the vertical direction do not coincide with the sides of the prediction block. An image decoding apparatus for decoding a stream encoded by the image encoding apparatus according to any one of claims 1 to 3,
A decoding unit for entropy decoding the stream;
One inverse orthogonal transform from among a plurality of inverse orthogonal transforms in each of the horizontal direction and the vertical direction based on the positional relationship between the transform block decoded by the decoding unit and the prediction block used at the time of intra prediction. A selection section for selecting
An inverse transform unit that performs an inverse orthogonal transform in the horizontal direction and an inverse orthogonal transform in the vertical direction selected by the selection unit in each direction, and
The selection unit includes:
An image decoding apparatus that selects IDST-II when both opposing sides of the transform block do not coincide with the sides of the prediction block in each of a horizontal direction and a vertical direction. In order to decode the stream encoded by the image encoding device according to any one of claims 1 to 3,
A decoding step of decoding the stream;
Selection for selecting one inverse orthogonal transform from a plurality of inverse orthogonal transforms in each of the horizontal direction and the vertical direction based on the positional relationship between the decoded transform block and the prediction block used for intra prediction. Steps,
An image decoding program that executes an inverse transform step of performing a selected horizontal inverse orthogonal transform and vertical inverse orthogonal transform in each direction,
The selection step includes
An image decoding program that selects IDST-II when both opposing sides of the transform block do not coincide with the sides of the prediction block in each of a horizontal direction and a vertical direction.