JP2014135742A - Image processing device and method, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable encoding efficiency in intra prediction to be improved.SOLUTION: In the event that the optimal intra prediction mode is mode 0, adjacent pixels to be used for prediction of the target block are pixels A0, A1, A2, and A3. According to these pixels and a 6-tap FIR filter, pixels a-0.5, a+0.5, and so on with 1/2 pixel precision are generated, and further, pixels a-0.75, a-0.25, a+0.25, and a+0.75 with 1/4 pixel precision are generated by linear interpolation. Subsequently, the optimal shift amount is determined with a value of -0.75 through +0.75 that is phase difference between an integer pixel and generated fractional pixel precision serving as a candidate of the shift amount in the horizontal direction. The present invention may be applied to an image encoding device which performs encoding using the H.264/AVC system, for example.

Description

  The present invention relates to an image processing apparatus and method, and a recording medium, and more particularly, to an image processing apparatus and method, and a recording medium that improve encoding efficiency in intra prediction.

  In recent years, image information has been handled as digital data, and at that time, for the purpose of efficient transmission and storage of information, encoding is performed by orthogonal transform such as discrete cosine transform and motion compensation using redundancy unique to image information. An apparatus that employs a method to compress and code an image is becoming widespread. This encoding method includes, for example, MPEG (Moving Picture Experts Group).

  In particular, MPEG2 (ISO / IEC 13818-2) is defined as a general-purpose image coding system, and is a standard that covers both interlaced scanning images and progressive scanning images, as well as standard resolution images and high-definition images. For example, MPEG2 is currently widely used in a wide range of applications for professional and consumer applications. By using the MPEG2 compression method, for example, a code amount (bit rate) of 4 to 8 Mbps is assigned to an interlaced scanned image having a standard resolution of 720 × 480 pixels. Further, by using the MPEG2 compression method, for example, in the case of a high-resolution interlaced scanned image having 1920 × 1088 pixels, a code amount (bit rate) of 18 to 22 Mbps is allocated. As a result, a high compression rate and good image quality can be realized.

  MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, but did not support encoding methods with a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. With the widespread use of mobile terminals, the need for such an encoding system is expected to increase in the future, and the MPEG4 encoding system has been standardized accordingly. Regarding the image coding system, the standard was approved as an international standard as ISO / IEC 14496-2 in December 1998.

  Furthermore, in recent years, for the purpose of image coding for the initial video conference, The standardization of 26L (ITU-T Q6 / 16 VCEG) is in progress. H. 26L is known to achieve higher encoding efficiency than the conventional encoding schemes such as MPEG2 and MPEG4, although a large amount of calculation is required for encoding and decoding. In addition, as part of MPEG4 activities, this H. Based on 26L, H. Standardization to achieve higher coding efficiency by incorporating functions not supported by 26L is being carried out as Joint Model of Enhanced-Compression Video Coding. As for the standardization schedule, H. H.264 and MPEG-4 Part10 (Advanced Video Coding, hereinafter referred to as H.264 / AVC).

  As an extension, FRExt (including RGB, 4: 2: 2, 4: 4: 4 coding tools necessary for business use, 8x8DCT and quantization matrix specified by MPEG-2) Fidelity Range Extension) standardization was completed in February 2005. As a result, H.C. Using 264 / AVC, it became an encoding method that can express film noise contained in movies well, and it has been used in a wide range of applications such as Blu-Ray Disc (trademark).

  However, nowadays, there is an increasing need for further high-compression encoding such as wanting to compress an image of about 4000 × 2000 pixels, which is four times the high-definition image. Alternatively, there is a growing need for further high compression rate encoding such as the desire to deliver high-definition images in an environment with a limited transmission capacity such as the Internet. For this reason, in the above-mentioned VCEG (= Video Coding Expert Group) under the ITU-T, studies on improving the coding efficiency are being continued.

  For example, in the MPEG2 system, motion prediction / compensation processing with 1/2 pixel accuracy is performed by linear interpolation processing. On the other hand, H. In the H.264 / AVC format, prediction / compensation processing with 1/4 pixel accuracy using a 6-tap FIR (Finite Impulse Response Filter) filter is performed.

  In recent years, the H.264 pixel accuracy prediction / compensation process has been developed. Studies are underway to further improve the efficiency of the H.264 / AVC format. As one encoding method for this purpose, Non-Patent Document 1 proposes motion prediction with 1/8 pixel accuracy.

  That is, in Non-Patent Document 1, the interpolation process with 1/2 pixel accuracy is performed by the filter [−3,12, −39,158,158, −39,12, −3] / 256. The interpolation processing with 1/4 pixel accuracy is performed by the filter [-3,12, -37,229,71, -21,6, -1] / 256, and the interpolation processing with 1/8 pixel accuracy is performed by linear interpolation. Is called.

  In this way, by performing motion prediction using interpolation processing with higher pixel accuracy, it is possible to improve prediction accuracy and improve coding efficiency, particularly in a relatively slow motion sequence having a texture with high resolution. Improvements can be realized.

  H. One of the factors that realize the high encoding efficiency of the H.264 / AVC format compared to the conventional MPEG2 format is the adoption of the intra prediction method described below.

  H. In the H.264 / AVC format, nine types of 4 × 4 pixel and 8 × 8 pixel block units and four types of 16 × 16 pixel macroblock unit intra prediction modes are defined for luminance signals. For color difference signals, four types of 8 × 8 pixel block-unit intra prediction modes are defined. The color difference signal intra prediction mode can be set independently of the luminance signal intra prediction mode. Note that the types of prediction modes correspond to the directions indicated by the numbers 0, 1, 3 to 8 in FIG. Prediction mode 2 is average value prediction.

  By adopting such an intra prediction method, improvement in prediction accuracy is realized. However, H.C. In the H.264 / AVC format, as shown in the direction of FIG. 1, only intra prediction in units of 22.5 ° is performed. Therefore, when the inclination of the edge is other than that, the improvement of the encoding efficiency is limited.

  Therefore, in order to further improve the coding efficiency, Non-Patent Document 2 proposes that prediction is performed at an angle smaller than the unit of 22.5 °.

“Motion compensated prediction with 1 / 8-pel displacement vector resolution”, VCEG-AD09, ITU-Telecommunications Standardization Sector STUDY GROUP Question 6 Video coding Experts Group (VCEG), 23-27 Oct 2006 Virginie Drugeon, Thomas Wedi, and Torsten Palfner, “High Precision Edge Prediction for Intra Coding”, 2008

  However, H.C. In the intra prediction of the H.264 / AVC method, a predetermined adjacent pixel of the block to be encoded is used for the prediction, whereas in the proposal described in Non-Patent Document 2, the adjacent pixel of the block to be encoded is used. Other pixels must also be used for prediction.

  Therefore, in the proposal described in Non-Patent Document 2, even if the prediction is performed at an angle smaller than the unit of 22.5 °, the number of memory accesses and the processing increase.

  The present invention has been made in view of such a situation, and further improves the encoding efficiency in intra prediction without increasing the number of memory accesses and processing.

  An image processing apparatus according to an aspect of the present invention is directed to an image to be encoded, a memory that accumulates adjacent pixels that are referred to when performing intra prediction on pixels of a target block that is an object of encoding processing, The phase of adjacent pixels read from the memory is shifted or the phase of adjacent pixels read from the memory is not shifted according to the prediction direction and block size when performing intra prediction on the pixels of the target block. A selection unit that selects the image, an intra prediction unit that performs intra prediction on the pixel of the target block using the adjacent pixel, and generates a prediction image, and a prediction image generated by the intra prediction unit And an encoding unit for encoding the image.

  When the selection unit selects that the phase of the adjacent pixel read from the memory is shifted, the intra prediction unit uses the adjacent pixel whose phase is shifted to perform intra prediction on the pixel of the target block. It can be performed.

  When the selection unit selects that the phase of the adjacent pixel read from the memory is not shifted, the intra prediction unit uses the adjacent pixel whose phase has not been shifted to the intra of the pixel of the target block. Predictions can be made.

  An image processing method according to an aspect of the present invention is directed to a prediction direction and a block size when an image processing apparatus performs intra prediction on a pixel of a target block that is a target of encoding processing for an image to be encoded. In response to this, the phase of the adjacent pixel read from the memory that stores the adjacent pixel to be referred to when performing intra prediction on the pixel of the target block is shifted, or the phase of the adjacent pixel read from the memory is shifted. Whether to perform the intra prediction on the pixels of the target block using the adjacent pixels, generates a prediction image, and encodes the image using the generated prediction image.

  When it is selected that the phase of the adjacent pixel read from the memory is shifted, intra prediction can be performed on the pixel of the target block using the adjacent pixel whose phase is shifted.

  When it is selected that the phase of the adjacent pixel read from the memory is not shifted, intra prediction can be performed on the pixel of the target block using the adjacent pixel whose phase is not shifted.

  A recording medium according to one aspect of the present invention is a recording medium that records an encoded stream generated by the image processing apparatus according to one aspect of the present invention.

  In one aspect of the present invention, for an image to be encoded, the target block is selected according to a prediction direction and a block size when performing intra prediction on pixels of the target block to be encoded. It is selected whether to shift the phase of the adjacent pixel read from the memory that stores the adjacent pixel to be referred to when intra prediction is performed on this pixel, or not to shift the phase of the adjacent pixel read from the memory. Then, intra prediction is performed on the pixels of the target block using the adjacent pixels, a predicted image is generated, and the image is encoded using the generated predicted image.

  Note that each of the above-described image processing apparatuses may be an independent apparatus, or may be an internal block constituting one image encoding apparatus or image decoding apparatus.

  According to the present invention, a prediction image can be generated by intra prediction. Further, according to the present invention, it is possible to improve the encoding efficiency without increasing the number of memory accesses or processing.

It is a figure explaining the direction of 4 * 4 pixel intra prediction. It is a block diagram which shows the structure of one Embodiment of the image coding apparatus to which this invention is applied. It is a figure explaining the motion prediction / compensation process of 1/4 pixel precision. It is a figure explaining the motion prediction and compensation system of a multi reference frame. It is a figure explaining the example of the production | generation method of motion vector information. It is a block diagram which shows the structural example of an intra estimation part and an adjacent pixel interpolation part. 3 is a flowchart for describing an encoding process of the image encoding device in FIG. 2. It is a flowchart explaining the prediction process of step S21 of FIG. It is a figure explaining the processing order in the case of 16 * 16 pixel intra prediction mode. It is a figure which shows the kind of 4 * 4 pixel intra prediction mode of a luminance signal. It is a figure which shows the kind of 4 * 4 pixel intra prediction mode of a luminance signal. It is a figure explaining the direction of 4 * 4 pixel intra prediction. It is a figure explaining intra prediction of 4x4 pixels. It is a figure explaining encoding of the 4 * 4 pixel intra prediction mode of a luminance signal. It is a figure which shows the kind of 16 * 16 pixel intra prediction mode of a luminance signal. It is a figure which shows the kind of 16 * 16 pixel intra prediction mode of a luminance signal. It is a figure explaining the 16 * 16 pixel intra prediction. It is a figure explaining the operation | movement for implement | achieving the intra prediction of decimal pixel precision. It is a figure explaining the example of an effect of intra prediction of decimal pixel accuracy. It is a flowchart explaining the intra prediction process of step S31 of FIG. It is a flowchart explaining the adjacent pixel interpolation process of step S45 of FIG. It is a flowchart explaining the inter motion prediction process of step S32 of FIG. It is a block diagram which shows the other structural example of an intra estimation part and an adjacent pixel interpolation part. It is a flowchart explaining the other example of the intra prediction process of FIG.8 S31. It is a flowchart explaining the adjacent pixel interpolation process of step S101 of FIG. It is a block diagram which shows the structure of one Embodiment of the image decoding apparatus to which this invention is applied. It is a block diagram which shows the other structural example of an intra estimation part and an adjacent pixel interpolation part. It is a flowchart explaining the decoding process of the image decoding apparatus of FIG. It is a flowchart explaining the prediction process of step S138 of FIG. It is a block diagram which shows the structural example of the hardware of a computer.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Configuration Example of Image Encoding Device]
FIG. 2 shows a configuration of an embodiment of an image encoding apparatus as an image processing apparatus to which the present invention is applied.

  This image encoding device 51 is, for example, H.264. 264 and MPEG-4 Part 10 (Advanced Video Coding) (hereinafter referred to as H.264 / AVC) format is used for compression coding.

  In the example of FIG. 2, the image encoding device 51 includes an A / D conversion unit 61, a screen rearrangement buffer 62, a calculation unit 63, an orthogonal transformation unit 64, a quantization unit 65, a lossless encoding unit 66, a storage buffer 67, Inverse quantization unit 68, inverse orthogonal transform unit 69, operation unit 70, deblock filter 71, frame memory 72, switch 73, intra prediction unit 74, adjacent pixel interpolation unit 75, motion prediction / compensation unit 76, prediction image selection The unit 77 and the rate control unit 78 are configured.

  The A / D conversion unit 61 performs A / D conversion on the input image, outputs it to the screen rearrangement buffer 62, and stores it. The screen rearrangement buffer 62 rearranges the stored frame images in the display order in the order of frames for encoding in accordance with GOP (Group of Picture).

  The calculation unit 63 subtracts the prediction image from the intra prediction unit 74 or the prediction image from the motion prediction / compensation unit 76 selected by the prediction image selection unit 77 from the image read from the screen rearrangement buffer 62, The difference information is output to the orthogonal transform unit 64. The orthogonal transform unit 64 subjects the difference information from the calculation unit 63 to orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform, and outputs the transform coefficient. The quantization unit 65 quantizes the transform coefficient output from the orthogonal transform unit 64.

  The quantized transform coefficient that is the output of the quantization unit 65 is input to the lossless encoding unit 66, where lossless encoding such as variable length encoding and arithmetic encoding is performed and compressed.

  The lossless encoding unit 66 acquires information indicating intra prediction from the intra prediction unit 74 and acquires information indicating inter prediction mode from the motion prediction / compensation unit 76. Note that information indicating intra prediction is hereinafter also referred to as intra prediction mode information. In addition, information indicating an information mode indicating inter prediction is hereinafter also referred to as inter prediction mode information.

  The lossless encoding unit 66 encodes the quantized transform coefficient and also encodes information indicating intra prediction, information indicating inter prediction mode, and the like, and uses it as a part of header information in the compressed image. The lossless encoding unit 66 supplies the encoded data to the accumulation buffer 67 for accumulation.

  For example, the lossless encoding unit 66 performs lossless encoding processing such as variable length encoding or arithmetic encoding. Examples of variable length coding include H.264. CAVLC (Context-Adaptive Variable Length Coding) defined in H.264 / AVC format. Examples of arithmetic coding include CABAC (Context-Adaptive Binary Arithmetic Coding).

  The accumulation buffer 67 converts the data supplied from the lossless encoding unit 66 to H.264. As a compressed image encoded by the H.264 / AVC format, for example, it is output to a recording device or a transmission path (not shown) in the subsequent stage.

  Further, the quantized transform coefficient output from the quantization unit 65 is also input to the inverse quantization unit 68, and after inverse quantization, the inverse orthogonal transform unit 69 further performs inverse orthogonal transform. The output subjected to the inverse orthogonal transform is added to the predicted image supplied from the predicted image selection unit 77 by the calculation unit 70, and becomes a locally decoded image. The deblocking filter 71 removes block distortion from the decoded image, and then supplies the deblocking filter 71 to the frame memory 72 for accumulation. The image before the deblocking filter processing by the deblocking filter 71 is also supplied to the frame memory 72 and accumulated.

  The switch 73 outputs the reference image stored in the frame memory 72 to the motion prediction / compensation unit 76 or the intra prediction unit 74.

  In the image encoding device 51, for example, the I picture, the B picture, and the P picture from the screen rearrangement buffer 62 are supplied to the intra prediction unit 74 as images for intra prediction (also referred to as intra processing). Further, the B picture and the P picture read from the screen rearrangement buffer 62 are supplied to the motion prediction / compensation unit 76 as an image to be inter predicted (also referred to as inter processing).

  The intra prediction unit 74 performs intra prediction processing of all candidate intra prediction modes based on the image to be intra predicted read from the screen rearrangement buffer 62 and the reference image supplied from the frame memory 72, and performs prediction. Generate an image.

  The intra prediction unit 74 calculates a cost function value for the intra prediction mode in which the predicted image is generated, and selects an intra prediction mode in which the calculated cost function value gives the minimum value as the optimal intra prediction mode. The intra prediction unit 74 supplies the adjacent pixel of the target block to be intra-predicted and the information of the optimal intra prediction mode to the adjacent pixel interpolation unit 75.

  The adjacent pixel interpolation unit 75 shifts the phase of the adjacent pixel in the shift direction according to the optimal intra prediction mode from the intra prediction unit 74 by a candidate shift amount. Actually, the adjacent pixel interpolation unit 75 applies a 6-tap FIR filter to the adjacent pixels in the shift direction corresponding to the optimal intra prediction mode, and linearly interpolates the adjacent pixels to obtain the phase of the adjacent pixels. Shift to decimal pixel accuracy. Therefore, for convenience of explanation, adjacent pixels whose phases are shifted by a 6-tap FIR filter and linear interpolation will be appropriately described as adjacent pixels that have been interpolated or whose phases have been shifted, but they agree. It is.

  The adjacent pixel interpolation unit 75 supplies the adjacent pixel whose phase has been shifted to the intra prediction unit 74.

  The intra prediction unit 74 uses the pixel value of the adjacent pixel from the adjacent image buffer 81 and the pixel value of the adjacent pixel whose phase is shifted by the adjacent pixel interpolation unit 75 to calculate the optimum phase shift amount for the adjacent pixel. decide. Further, the intra prediction unit 74 generates a predicted image of the target block using the pixel value of the adjacent pixel whose phase is shifted by the determined optimal shift amount, and the generated predicted image and the corresponding optimal intra prediction mode. The cost function value calculated for is supplied to the predicted image selection unit 77.

  When the predicted image generated in the optimal intra prediction mode is selected by the predicted image selection unit 77, the intra prediction unit 74 supplies information indicating the optimal intra prediction mode and information on the optimal shift amount to the lossless encoding unit 66. To do. When information is sent from the intra prediction unit 74, the lossless encoding unit 66 encodes this information and uses it as a part of header information in the compressed image.

  The motion prediction / compensation unit 76 performs motion prediction / compensation processing for all candidate inter prediction modes. That is, the inter prediction image read from the screen rearrangement buffer 62 and the reference image from the frame memory 72 are supplied to the motion prediction / compensation unit 76 via the switch 73. The motion prediction / compensation unit 76 detects motion vectors of all candidate inter prediction modes based on the inter-processed image and the reference image, performs compensation processing on the reference image based on the motion vector, and obtains the predicted image. Generate.

  Also, the motion prediction / compensation unit 76 calculates cost function values for all candidate inter prediction modes. The motion prediction / compensation unit 76 determines the prediction mode that gives the minimum value among the calculated cost function values as the optimal inter prediction mode.

  The motion prediction / compensation unit 76 supplies the predicted image generated in the optimal inter prediction mode and its cost function value to the predicted image selection unit 77. When the predicted image generated in the optimal inter prediction mode is selected by the predicted image selection unit 77, the motion prediction / compensation unit 76 transmits information indicating the optimal inter prediction mode (inter prediction mode information) to the lossless encoding unit 66. Output.

  If necessary, motion vector information, flag information, reference frame information, and the like are also output to the lossless encoding unit 66. The lossless encoding unit 66 performs lossless encoding processing such as variable length encoding and arithmetic encoding on the information from the motion prediction / compensation unit 76 and inserts the information into the header portion of the compressed image.

  The predicted image selection unit 77 determines an optimal prediction mode from the optimal intra prediction mode and the optimal inter prediction mode based on each cost function value output from the intra prediction unit 74 or the motion prediction / compensation unit 76. Then, the predicted image selection unit 77 selects a predicted image in the determined optimal prediction mode and supplies the selected predicted image to the calculation units 63 and 70. At this time, the predicted image selection unit 77 supplies the prediction image selection information to the intra prediction unit 74 or the motion prediction / compensation unit 76.

  The rate control unit 78 controls the rate of the quantization operation of the quantization unit 65 based on the compressed image stored in the storage buffer 67 so that overflow or underflow does not occur.

[H. Explanation of H.264 / AVC format]
FIG. 3 is a diagram illustrating an example of a block size for motion prediction / compensation in the H.264 / AVC format. FIG. H. In the H.264 / AVC format, motion prediction / compensation is performed with a variable block size.

  In the upper part of FIG. 3, macroblocks composed of 16 × 16 pixels divided into 16 × 16 pixels, 16 × 8 pixels, 8 × 16 pixels, and 8 × 8 pixel partitions are sequentially shown from the left. ing. Further, in the lower part of FIG. 3, from the left, 8 × 8 pixel partitions divided into 8 × 8 pixel, 8 × 4 pixel, 4 × 8 pixel, and 4 × 4 pixel subpartitions are sequentially shown. Yes.

  That is, H. In the H.264 / AVC format, one macroblock is divided into any partition of 16 × 16 pixels, 16 × 8 pixels, 8 × 16 pixels, or 8 × 8 pixels, and independent motion vector information is obtained. It is possible to have. In addition, an 8 × 8 pixel partition is divided into 8 × 8 pixel, 8 × 4 pixel, 4 × 8 pixel, or 4 × 4 pixel subpartitions and has independent motion vector information. Is possible.

  FIG. It is a figure explaining the prediction and compensation process of the 1/4 pixel precision in a H.264 / AVC system. H. In the H.264 / AVC format, prediction / compensation processing with 1/4 pixel accuracy using a 6-tap FIR (Finite Impulse Response Filter) filter is performed.

  In the example of FIG. 4, the position A is the position of the integer precision pixel, the positions b, c, and d are the positions of the 1/2 pixel precision, and the positions e1, e2, and e3 are the positions of the 1/4 pixel precision. Yes. First, in the following, Clip () is defined as the following equation (1).

なお、入力画像が８ビット精度である場合、max_pixの値は255となる。

When the input image has 8-bit precision, the value of max_pix is 255.

The pixel values at the positions b and d are generated by the following equation (2) using a 6-tap FIR filter.

なお、Clip処理は、水平方向および垂直方向の積和処理の両方を行った後、最後に１度のみ実行される。 The pixel value at the position c is generated as in the following Expression (3) by applying a 6-tap FIR filter in the horizontal direction and the vertical direction.

The clip process is executed only once at the end after performing both the horizontal and vertical product-sum processes.

The positions e1 to e3 are generated by linear interpolation as in the following equation (4).

  H. In the H.264 / AVC format, a large amount of motion vector information is generated by performing the motion prediction / compensation process described above with reference to FIG. 3 and FIG. Will be reduced. In contrast, H. In the H.264 / AVC format, motion vector encoding information is reduced by the method shown in FIG.

  FIG. It is a figure explaining the production | generation method of the motion vector information by a H.264 / AVC system.

  In the example of FIG. 5, a target block E to be encoded (for example, 16 × 16 pixels) and blocks A to D that have already been encoded and are adjacent to the target block E are shown.

  That is, the block D is adjacent to the upper left of the target block E, the block B is adjacent to the upper side of the target block E, the block C is adjacent to the upper right of the target block E, and the block A is , Adjacent to the left of the target block E. It should be noted that the blocks A to D are not divided represent blocks having any one of the 16 × 16 pixels to 4 × 4 pixels described above with reference to FIG.

  For example, motion vector information for X (= A, B, C, D, E) is represented by mvX. First, the predicted motion vector information pmvE for the target block E is generated by the median prediction using the motion vector information regarding the blocks A, B, and C as shown in the following equation (5).

pmvE = med (mvA, mvB, mvC) (5)

The motion vector information related to the block C may be unavailable (unavailable) because it is at the edge of the image frame or is not yet encoded. In this case, the motion vector information regarding the block C is substituted with the motion vector information regarding the block D.

Data mvdE added to the header portion of the compressed image as motion vector information for the target block E is generated as shown in the following equation (6) using pmvE.

mvdE = mvE-pmvE (6)

  Actually, processing is performed independently for each of the horizontal and vertical components of the motion vector information.

  As described above, the motion vector information is generated, and the data mvdE that is the difference between the motion vector information and the motion vector information predicted by the correlation with the adjacent block is used as motion vector information in the header portion of the compressed image. By adding, motion vector information can be reduced.

  Here, as described above with reference to FIG. The prediction / compensation processing with 1/4 pixel accuracy in the H.264 / AVC format is executed by the motion prediction / compensation unit. In the image encoding device 51 in FIG. 2, prediction with 1/4 pixel accuracy is performed by intra prediction. Also performed in This intra prediction with decimal pixel accuracy is executed by an intra prediction unit 74 and an adjacent pixel interpolation unit 75 described below.

[Configuration Example of Intra Prediction Unit and Adjacent Pixel Interpolation Unit]
FIG. 6 is a block diagram illustrating a detailed configuration example of the intra prediction unit and the adjacent pixel interpolation unit.

  In the case of the example of FIG. 6, the intra prediction unit 74 includes an adjacent image buffer 81, an optimal mode determination unit 82, an optimal shift amount determination unit 83, and a predicted image generation unit 84.

  The adjacent pixel interpolation unit 75 includes a mode determination unit 91, a horizontal direction interpolation unit 92, and a vertical direction interpolation unit 93.

  The adjacent image buffer 81 accumulates adjacent pixels of the target block for intra prediction from the frame memory 72. In the case of FIG. 6, the illustration of the switch 73 is omitted, but the adjacent pixels are supplied from the frame memory 72 to the adjacent image buffer 81 via the switch 73.

  The image for intra prediction read from the screen rearrangement buffer 62 is input to the optimum mode determination unit 82. The optimum mode determination unit 82 reads out adjacent pixels corresponding to the target block to be intra predicted from the adjacent image buffer 81.

  The optimal mode determination unit 82 performs intra prediction processing of all candidate intra prediction modes using adjacent pixels corresponding to the image of the target block to be intra predicted, and generates a prediction image. The optimal mode determination unit 82 calculates a cost function value for the intra prediction mode in which the predicted image is generated, and determines an intra prediction mode in which the calculated cost function value gives the minimum value as the optimal intra prediction mode. Information on the determined prediction mode is supplied to the mode determination unit 91, the optimum shift amount determination unit 83, and the predicted image generation unit 84. The predicted image generation unit 84 is also supplied with a cost function value corresponding to the supplied prediction mode.

  The optimal shift amount determination unit 83 receives the image for intra prediction read from the screen rearrangement buffer 62 and information on the prediction mode determined to be optimal by the optimal mode determination unit 82. In addition, the optimum shift amount determination unit 83 receives adjacent pixels whose phases are linearly interpolated by the horizontal direction interpolation unit 92 and the vertical direction interpolation unit 93 in accordance with the optimum intra prediction mode. The optimum shift amount determination unit 83 reads adjacent pixels corresponding to the target block to be intra-predicted from the adjacent image buffer 81.

  The optimum shift amount determination unit 83 uses the image of the target block to be intra-predicted, the corresponding adjacent pixel, and the pixel value of the corresponding interpolated adjacent pixel for the prediction mode determined by the optimal mode determination unit 82, Determine the optimal shift amount. The optimal shift amount determination unit 83 calculates, for example, a prediction error (residual) and determines the calculated small shift as the optimal shift amount. Information on the optimal shift amount determined by the optimal shift amount determination unit 83 is supplied to the predicted image generation unit 84.

  The predicted image generation unit 84 receives the cost function value corresponding to the prediction mode information determined by the optimal mode determination unit 82 and the information of the optimal shift amount determined by the optimal shift amount determination unit 83. The predicted image generation unit 84 reads the adjacent pixel corresponding to the target block to be intra-predicted from the adjacent image buffer 81, and sets the read adjacent pixel in the phase direction according to the prediction mode with the optimum shift amount and the phase of the adjacent pixel. shift.

  The predicted image generation unit 84 performs intra prediction in the optimal intra prediction mode determined by the optimal mode determination unit 82 using adjacent pixels whose phases are shifted, and generates a predicted image of the target block. The predicted image generation unit 84 outputs the generated predicted image and the corresponding cost function value to the predicted image selection unit 77.

  Further, when the predicted image generated in the optimal intra prediction mode is selected by the predicted image selection unit 77, the predicted image generation unit 84 converts the information indicating the optimal intra prediction mode and the shift amount information into the lossless encoding unit 66. To supply.

  The mode determination unit 91 outputs a control signal corresponding to the prediction mode determined by the optimal mode determination unit 82 to the horizontal direction interpolation unit 92 and the vertical direction interpolation unit 93. For example, a control signal indicating ON of interpolation processing is output according to the prediction mode.

  The horizontal direction interpolation unit 92 and the vertical direction interpolation unit 93 read adjacent pixels from the adjacent image buffer 81 in response to a control signal from the mode determination unit 91. The horizontal direction interpolation unit 92 and the vertical direction interpolation unit 93 shift the phase in the horizontal direction and the vertical direction, respectively, by a 6-tap FIR filter and linear interpolation with respect to the read adjacent pixels. Information on adjacent pixels interpolated by the horizontal direction interpolation unit 92 and the vertical direction interpolation unit 93 is supplied to the optimum shift amount determination unit 83.

[Description of Encoding Process of Image Encoding Device]
Next, the encoding process of the image encoding device 51 of FIG. 2 will be described with reference to the flowchart of FIG.

  In step S11, the A / D converter 61 performs A / D conversion on the input image. In step S12, the screen rearrangement buffer 62 stores the image supplied from the A / D conversion unit 61, and rearranges the picture from the display order to the encoding order.

  In step S13, the calculation unit 63 calculates the difference between the image rearranged in step S12 and the predicted image. The prediction image is supplied from the motion prediction / compensation unit 76 in the case of inter prediction, and from the intra prediction unit 74 in the case of intra prediction, to the calculation unit 63 via the prediction image selection unit 77, respectively.

  The difference data has a smaller data amount than the original image data. Therefore, the data amount can be compressed as compared with the case where the image is encoded as it is.

  In step S <b> 14, the orthogonal transform unit 64 performs orthogonal transform on the difference information supplied from the calculation unit 63. Specifically, orthogonal transformation such as discrete cosine transformation and Karhunen-Loeve transformation is performed, and transformation coefficients are output. In step S15, the quantization unit 65 quantizes the transform coefficient. At the time of this quantization, the rate is controlled as described in the process of step S25 described later.

  The difference information quantized as described above is locally decoded as follows. That is, in step S <b> 16, the inverse quantization unit 68 inversely quantizes the transform coefficient quantized by the quantization unit 65 with characteristics corresponding to the characteristics of the quantization unit 65. In step S <b> 17, the inverse orthogonal transform unit 69 performs inverse orthogonal transform on the transform coefficient inversely quantized by the inverse quantization unit 68 with characteristics corresponding to the characteristics of the orthogonal transform unit 64.

  In step S18, the calculation unit 70 adds the predicted image input via the predicted image selection unit 77 to the locally decoded difference information, and outputs the locally decoded image (for input to the calculation unit 63). Corresponding image). In step S <b> 19, the deblock filter 71 filters the image output from the calculation unit 70. Thereby, block distortion is removed. In step S20, the frame memory 72 stores the filtered image. Note that an image that has not been filtered by the deblocking filter 71 is also supplied to the frame memory 72 from the computing unit 70 and stored therein.

  In step S21, the intra prediction unit 74 and the motion prediction / compensation unit 76 each perform image prediction processing. That is, in step S21, the intra prediction unit 74 performs an intra prediction process in the intra prediction mode. The motion prediction / compensation unit 76 performs motion prediction / compensation processing in the inter prediction mode.

  The details of the prediction process in step S21 will be described later with reference to FIG. 8. With this process, prediction processes in all candidate prediction modes are performed, and cost functions in all candidate prediction modes are obtained. Each value is calculated. Then, based on the calculated cost function value, the optimal intra prediction mode is selected, and the predicted image generated by the intra prediction in the optimal intra prediction mode and its cost function value are supplied to the predicted image selection unit 77.

  Specifically, at this time, the intra prediction unit 74 uses adjacent pixels whose phase is shifted by the optimal shift amount in the shift direction according to the optimal intra prediction mode by the 6-tap FIR filter and linear interpolation. Then, the predicted image generated by the intra prediction is supplied to the predicted image selection unit 77. A cost function value for the optimal intra prediction mode is also supplied to the predicted image selection unit 77 together with the predicted image.

  On the other hand, the optimal inter prediction mode is determined from the inter prediction modes based on the calculated cost function value, and the predicted image generated in the optimal inter prediction mode and its cost function value are sent to the predicted image selection unit 77. Supplied.

  In step S <b> 22, the predicted image selection unit 77 optimizes one of the optimal intra prediction mode and the optimal inter prediction mode based on the cost function values output from the intra prediction unit 74 and the motion prediction / compensation unit 76. Determine the prediction mode. Then, the predicted image selection unit 77 selects the predicted image of the determined optimal prediction mode and supplies it to the calculation units 63 and 70. As described above, this predicted image is used for the calculations in steps S13 and S18.

  The prediction image selection information is supplied to the intra prediction unit 74 or the motion prediction / compensation unit 76. When a prediction image in the optimal intra prediction mode is selected, the intra prediction unit 74 converts information indicating the optimal intra prediction mode (that is, intra prediction mode information) and information on the shift amount determined to be optimal into a lossless encoding unit. 66.

  When the prediction image of the optimal inter prediction mode is selected, the motion prediction / compensation unit 76 sends information indicating the optimal inter prediction mode and, if necessary, information corresponding to the optimal inter prediction mode to the lossless encoding unit 66. Output. Information according to the optimal inter prediction mode includes motion vector information, flag information, reference frame information, and the like. That is, when a prediction image in the inter prediction mode is selected as the optimal inter prediction mode, the motion prediction / compensation unit 76 outputs the inter prediction mode information, motion vector information, and reference frame information to the lossless encoding unit 66. .

  In step S23, the lossless encoding unit 66 encodes the quantized transform coefficient output from the quantization unit 65. That is, the difference image is subjected to lossless encoding such as variable length encoding and arithmetic encoding, and is compressed. At this time, the intra prediction mode information from the intra prediction unit 74 or the information corresponding to the optimal inter prediction mode from the motion prediction / compensation unit 76, which is input to the lossless encoding unit 66 in step S22 described above, is also encoded. And added to the header information.

  In step S24, the accumulation buffer 67 accumulates the difference image as a compressed image. The compressed image stored in the storage buffer 67 is appropriately read and transmitted to the decoding side via the transmission path.

  In step S <b> 25, the rate control unit 78 controls the quantization operation rate of the quantization unit 65 based on the compressed image stored in the storage buffer 67 so that overflow or underflow does not occur.

[Explanation of prediction processing]
Next, the prediction process in step S21 in FIG. 7 will be described with reference to the flowchart in FIG.

  When the processing target image supplied from the screen rearrangement buffer 62 is an image of a block to be intra-processed, the decoded image to be referred to is read from the frame memory 72, and the intra prediction unit 74 via the switch 73. To be supplied.

  In step S31, the intra prediction unit 74 performs intra prediction on the pixels of the block to be processed in all candidate intra prediction modes, using the supplied image. Note that pixels that have not been deblocked filtered by the deblocking filter 71 are used as decoded pixels that are referred to.

  The details of the intra prediction process in step S31 will be described later with reference to FIG. 20. With this process, intra prediction is performed in all candidate intra prediction modes. Then, cost function values are calculated for all candidate intra prediction modes, and the optimal intra prediction mode is determined based on the calculated cost function values.

  Then, by the 6-tap FIR filter and linear interpolation, the phase of the adjacent pixel is shifted by the optimal shift amount in the shift direction corresponding to the determined optimal intra prediction mode. Adjacent pixels whose phases are shifted are used, and a prediction image is generated by intra prediction in the optimal intra prediction mode. The generated predicted image and the cost function value of the optimal intra prediction mode are supplied to the predicted image selection unit 77.

  When the processing target image supplied from the screen rearrangement buffer 62 is an image to be inter-processed, the referenced image is read from the frame memory 72 and supplied to the motion prediction / compensation unit 76 via the switch 73. The Based on these images, in step S32, the motion prediction / compensation unit 76 performs an inter motion prediction process. That is, the motion prediction / compensation unit 76 refers to the image supplied from the frame memory 72 and performs motion prediction processing for all candidate inter prediction modes.

  Details of the inter motion prediction process in step S32 will be described later with reference to FIG. 22. With this process, the motion prediction process is performed in all candidate inter prediction modes, and all candidate inter prediction modes are set. On the other hand, a cost function value is calculated.

  In step S33, the motion prediction / compensation unit 76 compares the cost function value for the inter prediction mode calculated in step S32, and determines the prediction mode that gives the minimum value as the optimal inter prediction mode. Then, the motion prediction / compensation unit 76 supplies the predicted image generated in the optimal inter prediction mode and its cost function value to the predicted image selection unit 77.

[H. Explanation of Intra Prediction Processing in H.264 / AVC System]
Next, H.I. Each mode of intra prediction defined in the H.264 / AVC format will be described.

  First, the intra prediction mode for the luminance signal will be described. In the luminance signal intra prediction mode, three methods are defined: an intra 4 × 4 prediction mode, an intra 8 × 8 prediction mode, and an intra 16 × 16 prediction mode. This is a mode for determining a block unit, and is set for each macroblock. For color difference signals, an intra prediction mode independent of the luminance signal can be set for each macroblock.

  Further, in the case of the intra 4 × 4 prediction mode, one prediction mode can be set from nine types of prediction modes for each target block of 4 × 4 pixels. In the case of the intra 8 × 8 prediction mode, one prediction mode can be set from nine types of prediction modes for each target block of 8 × 8 pixels. In the case of the intra 16 × 16 prediction mode, one prediction mode can be set from four types of prediction modes for a target macroblock of 16 × 16 pixels.

  Note that, hereinafter, the intra 4 × 4 prediction mode, the intra 8 × 8 prediction mode, and the intra 16 × 16 prediction mode will be referred to as 4 × 4 pixel intra prediction mode, 8 × 8 pixel intra prediction mode, and 16 ×, respectively. This is also referred to as a 16-pixel intra prediction mode as appropriate.

  In the example of FIG. 9, the numbers −1 to 25 attached to each block indicate the bit stream order (processing order on the decoding side) of each block. For the luminance signal, the macroblock is divided into 4 × 4 pixels, and DCT of 4 × 4 pixels is performed. Only in the case of the intra 16 × 16 prediction mode, as shown in the block of −1, the DC components of each block are collected to generate a 4 × 4 matrix, and further, orthogonal transformation is performed on this. Is done.

  On the other hand, for the color difference signal, after the macroblock is divided into 4 × 4 pixels and the DCT of 4 × 4 pixels is performed, the DC components of each block are collected as shown in the blocks 16 and 17. A 2 × 2 matrix is generated, and is further subjected to orthogonal transformation.

  Note that this can be applied to the intra 8 × 8 prediction mode only when the target macroblock is subjected to 8 × 8 orthogonal transformation with a high profile or higher profile.

  FIG. 10 and FIG. 11 are diagrams illustrating nine types of luminance signal 4 × 4 pixel intra prediction modes (Intra — 4 × 4_pred_mode). Each of the eight modes other than mode 2 indicating average value (DC) prediction corresponds to the direction indicated by the numbers 0, 1, 3 to 8 in FIG.

  Nine types of Intra_4x4_pred_mode will be described with reference to FIG. In the example of FIG. 12, pixels a to p represent pixels of a target block to be intra-processed, and pixel values A to M represent pixel values of pixels belonging to adjacent blocks. That is, the pixels a to p are images to be processed that are read from the screen rearrangement buffer 62, and the pixel values A to M are pixel values of a decoded image that is read from the frame memory 72 and referred to. It is.

  In the case of each intra prediction mode shown in FIGS. 10 and 11, the predicted pixel values of the pixels a to p are generated as follows using the pixel values A to M of the pixels belonging to the adjacent blocks. Note that the pixel value “available” indicates that the pixel value can be used without any reason such as being at the end of the image frame or not yet encoded. On the other hand, the pixel value “unavailable” indicates that the pixel value is not usable because it is at the end of the image frame or has not been encoded yet.

Mode 0 is the Vertical Prediction mode, and is applied only when the pixel values A to D are “available”. In this case, the predicted pixel values of the pixels a to p are generated as in the following Expression (7).

Predicted pixel value of pixels a, e, i, m = A
Predicted pixel value of pixels b, f, j, n = B
Predicted pixel value of pixels c, g, k, o = C
Predicted pixel value of pixels d, h, l, and p = D (7)

Mode 1 is a horizontal prediction mode and is applied only when the pixel values I to L are “available”. In this case, the predicted pixel values of the pixels a to p are generated as in the following Expression (8).

Predicted pixel value of pixels a, b, c, d = I
Predicted pixel value of pixels e, f, g, h = J
Predicted pixel value of pixels i, j, k, l = K
Predicted pixel value of pixels m, n, o, p = L (8)

Mode 2 is a DC Prediction mode. When the pixel values A, B, C, D, I, J, K, and L are all “available”, the predicted pixel value is generated as shown in Expression (9).

(A + B + C + D + I + J + K + L + 4) >> 3 (9)

Also, when the pixel values A, B, C, and D are all “unavailable”, the predicted pixel value is generated as in Expression (10).

(I + J + K + L + 2) >> 2 (10)

Further, when the pixel values I, J, K, and L are all “unavailable”, the predicted pixel value is generated as in Expression (11).

(A + B + C + D + 2) >> 2 (11)

  When the pixel values A, B, C, D, I, J, K, and L are all “unavailable”, 128 is used as the predicted pixel value.

Mode 3 is a Diagonal_Down_Left Prediction mode, and is applied only when the pixel values A, B, C, D, I, J, K, L, and M are “available”. In this case, the predicted pixel values of the pixels a to p are generated as in the following Expression (12).

Predicted pixel value of pixel a = (A + 2B + C + 2) >> 2
Predicted pixel value of pixels b and e = (B + 2C + D + 2) >> 2
Predicted pixel value of pixels c, f, i = (C + 2D + E + 2) >> 2
Predicted pixel value of pixels d, g, j, m = (D + 2E + F + 2) >> 2
Predicted pixel value of pixels h, k, n = (E + 2F + G + 2) >> 2
Predicted pixel value of pixels l and o = (F + 2G + H + 2) >> 2
Predicted pixel value of pixel p = (G + 3H + 2) >> 2
(12)

Mode 4 is a Diagonal_Down_Right Prediction mode, and is applied only when the pixel values A, B, C, D, I, J, K, L, and M are “available”. In this case, the predicted pixel values of the pixels a to p are generated as in the following Expression (13).

Predicted pixel value of pixel m = (J + 2K + L + 2) >> 2
Predicted pixel value of pixels i and n = (I + 2J + K + 2) >> 2
Predicted pixel value of pixels e, j, o = (M + 2I + J + 2) >> 2
Predicted pixel value of pixels a, f, k, p = (A + 2M + I + 2) >> 2
Predicted pixel value of pixels b, g, l = (M + 2A + B + 2) >> 2
Predicted pixel value of pixels c and h = (A + 2B + C + 2) >> 2
Predicted pixel value of pixel d = (B + 2C + D + 2) >> 2
... (13)

Mode 5 is a Diagonal_Vertical_Right Prediction mode, and is applied only when the pixel values A, B, C, D, I, J, K, L, and M are “available”. In this case, the predicted pixel values of the pixels a to p are generated as in the following Expression (14).

Predicted pixel value of pixels a and j = (M + A + 1) >> 1
Predicted pixel value of pixels b and k = (A + B + 1) >> 1
Predicted pixel value of pixels c and l = (B + C + 1) >> 1
Predicted pixel value of pixel d = (C + D + 1) >> 1
Predicted pixel value of pixels e and n = (I + 2M + A + 2) >> 2
Predicted pixel value of pixels f and o = (M + 2A + B + 2) >> 2
Predicted pixel value of pixels g and p = (A + 2B + C + 2) >> 2
Predicted pixel value of pixel h = (B + 2C + D + 2) >> 2
Predicted pixel value of pixel i = (M + 2I + J + 2) >> 2
Predicted pixel value of pixel m = (I + 2J + K + 2) >> 2
(14)

Mode 6 is a Horizontal_Down Prediction mode, and is applied only when the pixel values A, B, C, D, I, J, K, L, and M are “available”. In this case, the predicted pixel values of the pixels a to p are generated as in the following Expression (15).

Predicted pixel value of pixels a and g = (M + I + 1) >> 1
Predicted pixel value of pixels b and h = (I + 2M + A + 2) >> 2
Predicted pixel value of pixel c = (M + 2A + B + 2) >> 2
Predicted pixel value of pixel d = (A + 2B + C + 2) >> 2
Predicted pixel value of pixels e and k = (I + J + 1) >> 1
Predicted pixel value of pixels f and l = (M + 2I + J + 2) >> 2
Predicted pixel value of pixels i and o = (J + K + 1) >> 1
Predicted pixel value of pixels j and p = (I + 2J + K + 2) >> 2
Predicted pixel value of pixel m = (K + L + 1) >> 1
Predicted pixel value of pixel n = (J + 2K + L + 2) >> 2
... (15)

Mode 7 is a Vertical_Left Prediction mode, and is applied only when the pixel values A, B, C, D, I, J, K, L, and M are “available”. In this case, the predicted pixel values of the pixels a to p are generated as in the following Expression (16).

Predicted pixel value of pixel a = (A + B + 1) >> 1
Predicted pixel value of pixels b and i = (B + C + 1) >> 1
Predicted pixel value of pixels c and j = (C + D + 1) >> 1
Predicted pixel value of pixels d and k = (D + E + 1) >> 1
Predicted pixel value of pixel l = (E + F + 1) >> 1
Predicted pixel value of pixel e = (A + 2B + C + 2) >> 2
Predicted pixel value of pixels f and m = (B + 2C + D + 2) >> 2
Predicted pixel value of pixels g and n = (C + 2D + E + 2) >> 2
Predicted pixel value of pixels h and o = (D + 2E + F + 2) >> 2
Predicted pixel value of pixel p = (E + 2F + G + 2) >> 2
... (16)

Mode 8 is a Horizontal_Up Prediction mode, and is applied only when the pixel values A, B, C, D, I, J, K, L, and M are “available”. In this case, the predicted pixel values of the pixels a to p are generated as in the following Expression (17).

Predicted pixel value of pixel a = (I + J + 1) >> 1
Predicted pixel value of pixel b = (I + 2J + K + 2) >> 2
Predicted pixel value of pixels c and e = (J + K + 1) >> 1
Predicted pixel value of pixels d and f = (J + 2K + L + 2) >> 2
Predicted pixel value of pixels g and i = (K + L + 1) >> 1
Predicted pixel value of pixels h and j = (K + 3L + 2) >> 2
Predicted pixel value of pixels k, l, m, n, o, p = L
... (17)

  Next, a 4 × 4 pixel intra prediction mode (Intra — 4 × 4_pred_mode) encoding method for luminance signals will be described with reference to FIG. In the example of FIG. 13, a target block C that is 4 × 4 pixels and is an encoding target is illustrated, and a block A and a block B that are 4 × 4 pixels adjacent to the target block C are illustrated.

  In this case, it is considered that Intra_4x4_pred_mode in the target block C and Intra_4x4_pred_mode in the block A and the block B have a high correlation. By using this correlation and performing encoding processing as follows, higher encoding efficiency can be realized.

That is, in the example of FIG. 13, Intra_4x4_pred_mode in the block A and the block B is set as Intra_4x4_pred_modeA and Intra_4x4_pred_modeB, respectively, and MostProbableMode is defined as the following equation (18).

MostProbableMode = Min (Intra_4x4_pred_modeA, Intra_4x4_pred_modeB)
... (18)

  That is, among blocks A and B, the one to which a smaller mode_number is assigned is referred to as MostProbableMode.

  In the bitstream, two values, prev_intra4x4_pred_mode_flag [luma4x4BlkIdx] and rem_intra4x4_pred_mode [luma4x4BlkIdx], are defined as parameters for the target block C. And the values of Intra_4x4_pred_mode and Intra4x4PredMode [luma4x4BlkIdx] for the target block C can be obtained.

if (prev_intra4x4_pred_mode_flag [luma4x4BlkIdx])
Intra4x4PredMode [luma4x4BlkIdx] = MostProbableMode
else
if (rem_intra4x4_pred_mode [luma4x4BlkIdx] <MostProbableMode)
Intra4x4PredMode [luma4x4BlkIdx] = rem_intra4x4_pred_mode [luma4x4BlkIdx]
else
Intra4x4PredMode [luma4x4BlkIdx] = rem_intra4x4_pred_mode [luma4x4BlkIdx] + 1
... (19)

  Next, the 16 × 16 pixel intra prediction mode will be described. FIG. 14 and FIG. 15 are diagrams showing 16 × 16 pixel intra prediction modes (Intra_16 × 16_pred_mode) of four types of luminance signals.

  The four types of intra prediction modes will be described with reference to FIG. In the example of FIG. 16, the target macroblock A to be intra-processed is shown, and P (x, y); x, y = −1,0,..., 15 are pixels adjacent to the target macroblock A. It represents a pixel value.

Mode 0 is a Vertical Prediction mode, and is applied only when P (x, -1); x, y = -1,0,..., 15 is “available”. In this case, the predicted pixel value Pred (x, y) of each pixel of the target macroblock A is generated as in the following equation (20).

Pred (x, y) = P (x, -1); x, y = 0, ..., 15
... (20)

Mode 1 is a horizontal prediction mode and is applied only when P (-1, y); x, y = -1,0,..., 15 is “available”. In this case, the predicted pixel value Pred (x, y) of each pixel of the target macroblock A is generated as in the following equation (21).

Pred (x, y) = P (-1, y); x, y = 0, ..., 15
... (21)

Mode 2 is a DC Prediction mode, and when P (x, -1) and P (-1, y); x, y = -1,0, ..., 15 are all "available", the target macroblock A The predicted pixel value Pred (x, y) of each pixel is generated as in the following Expression (22).

When P (x, -1); x, y = -1,0, ..., 15 is "unavailable", the predicted pixel value Pred (x, y) of each pixel of the target macroblock A is Is generated as shown in Equation (23).

When P (-1, y); x, y = −1,0,..., 15 is “unavailable”, the predicted pixel value Pred (x, y) of each pixel of the target macroblock A is expressed by the following equation: It is generated as in (24).

  When P (x, −1) and P (−1, y); x, y = −1,0,..., 15 are all “unavailable”, 128 is used as the predicted pixel value.

Mode 3 is a plane prediction mode, and is applied only when P (x, -1) and P (-1, y); x, y = -1,0, ..., 15 are all "available". In this case, the predicted pixel value Pred (x, y) of each pixel of the target macroblock A is generated as in the following Expression (25).

  Next, the intra prediction mode for color difference signals will be described. FIG. 17 is a diagram illustrating four types of color difference signal intra prediction modes (Intra_chroma_pred_mode). The color difference signal intra prediction mode can be set independently of the luminance signal intra prediction mode. The intra prediction mode for the color difference signal is in accordance with the 16 × 16 pixel intra prediction mode of the luminance signal described above.

  However, the 16 × 16 pixel intra prediction mode for the luminance signal is intended for a block of 16 × 16 pixels, whereas the intra prediction mode for a color difference signal is intended for a block of 8 × 8 pixels. Furthermore, as shown in FIGS. 14 and 17 described above, the mode numbers do not correspond to each other.

  Here, the definition of the pixel value of the target macroblock A and the adjacent pixel value in the 16 × 16 pixel intra prediction mode of the luminance signal described above with reference to FIG. 16 is applied. For example, pixel values of pixels adjacent to the target macroblock A to be intra-processed (8 × 8 pixels in the case of a color difference signal) are P (x, y); x, y = −1,0,. To do.

Mode 0 is DC Prediction mode, and when P (x, -1) and P (-1, y); x, y = -1,0, ..., 7 are all "available", the target macroblock A The predicted pixel value Pred (x, y) of each pixel is generated as in the following Expression (26).

Further, when P (−1, y); x, y = −1,0,..., 7 is “unavailable”, the predicted pixel value Pred (x, y) of each pixel of the target macroblock A is (27).

When P (x, -1); x, y = -1,0,..., 7 is “unavailable”, the predicted pixel value Pred (x, y) of each pixel of the target macroblock A is Is generated as shown in Equation (28).

Mode 1 is a Horizontal Prediction mode, and is applied only when P (-1, y); x, y = -1,0,..., 7 is “available”. In this case, the predicted pixel value Pred (x, y) of each pixel of the target macroblock A is generated as in the following Expression (29).

Pred (x, y) = P (-1, y); x, y = 0, ..., 7
... (29)

Mode 2 is the Vertical Prediction mode, and is applied only when P (x, -1); x, y = -1,0, ..., 7 is "available". In this case, the predicted pixel value Pred (x, y) of each pixel of the target macroblock A is generated as in the following equation (30).

Pred (x, y) = P (x, -1); x, y = 0, ..., 7
... (30)

Mode 3 is a plane prediction mode and is applied only when P (x, -1) and P (-1, y); x, y = -1,0, ..., 7 are "available". In this case, the predicted pixel value Pred (x, y) of each pixel of the target macroblock A is generated as in the following Expression (31).

  As described above, the luminance signal intra prediction modes include nine types of 4 × 4 pixel and 8 × 8 pixel block units, and four types of 16 × 16 pixel macroblock unit prediction modes. This block unit mode is set for each macroblock unit. The color difference signal intra prediction modes include four types of prediction modes in units of 8 × 8 pixel blocks. This color difference signal intra prediction mode can be set independently of the luminance signal intra prediction mode.

  In addition, the 4 × 4 pixel intra prediction mode (intra 4 × 4 prediction mode) and the 8 × 8 pixel intra prediction mode (intra 8 × 8 prediction mode) of the luminance signal are 4 × 4 pixels and 8 × 8 pixels. One intra prediction mode is set for each block of luminance signals. For the 16 × 16 pixel intra prediction mode for luminance signals (intra 16 × 16 prediction mode) and the intra prediction mode for color difference signals, one prediction mode is set for one macroblock.

  Note that the types of prediction modes correspond to the directions indicated by the numbers 0, 1, 3 to 8 in FIG. Prediction mode 2 is average value prediction.

  As above. Intra prediction in the H.264 / AVC format is performed with integer pixel accuracy. On the other hand, in the image encoding device 51, intra prediction with decimal pixel accuracy is performed.

[Operation of intra prediction with decimal pixel precision]
Next, with reference to FIG. 18, an operation for realizing the intra prediction with decimal pixel accuracy will be described. In the example of FIG. 18, an example in which the target block is 4 × 4 pixels is shown.

  In the example of FIG. 18, black circles represent pixels of the target block for intra prediction, and white circles represent adjacent pixels adjacent to the target block. More specifically, among the adjacent pixels of the white circle, the upper left adjacent pixel adjacent to the upper left of the target block is A-1 and I-1, and this pixel corresponds to the pixel having the pixel value M in FIG. To do. Among the white circle adjacent pixels, the upper adjacent pixels adjacent to the upper part of the target block are A0, A1, A2,..., And these pixels correspond to the pixels having the pixel values A to H in FIG. Among the white circle adjacent pixels, the left adjacent pixels adjacent to the left portion of the target block are I0, I1, I2,..., And these pixels correspond to pixels having pixel values I to L in FIG.

  Also, a-0.5, a + 0.5,... And i-0.5, i + 0.5,... Shown between adjacent pixels represent pixels with 1/2 pixel accuracy. Further, a-0.75, a-0.25, a + 0.25, a + 0.75,... And i-0.75, i shown between the pixels of a-0.5, a + 0.5, ... and i-0.5, i + 0.5, ... -0.25, i + 0.25, i + 0.75,... Represent pixels with 1/4 pixel accuracy.

  First, as a first operation, the intra prediction unit 74 performs intra prediction for each intra prediction mode using the pixel values A to M shown in FIG. An optimal intra prediction mode is determined. When the target block is 4 × 4, this optimal intra prediction mode is one of the nine prediction modes in FIG. 10 or FIG. 11.

  For example, it is assumed that mode 0 (Vertical Prediction mode) is selected as the optimal intra prediction mode. At this time, adjacent pixels used for prediction of the target block are pixels having pixel values A to D in FIG. 12, and are pixels A0, A1, A2, and A3 in FIG.

As a second operation, in the adjacent pixel interpolation unit 75, the H.P. described above with reference to FIG. The pixels a-0.5, a + 0.5,... With 1/2 pixel accuracy in FIG. That is, the pixel a-0.5 is represented by the following equation (32).

a-0.5 = (A-2 -5 * A-1 + 20 * A0 + 20 * A1 -5 * A1 + A2 + 16) >> 5
... (32)

The same applies to the other pixels a + 0.5, a + 1.5, etc. with half-pixel accuracy.

  As a third operation, in the adjacent pixel interpolation unit 75, from the pixels A0, A1, A2, A3, the pixels a-0.5, a + 0.5, and the like, the pixels a-0.75, a having the 1/4 pixel accuracy of FIG. -0.25, a + 0.25, a + 0.75 are generated by linear interpolation. That is, the pixel a + 0.25 is expressed by the following equation (33).

a-0.5 = A0 + a + 0.5 + 1) >> 2 (33)

The same applies to other ¼ pixel precision pixels.

  As a fourth operation, in the mode 0, in the intra prediction unit 74, values of −0.75, −0.50, −0.25, +0.25, +0.50, and +0.75, which are phase differences between the integer pixels and the decimal pixel precision, are set. The optimum shift amount is determined as a candidate for the horizontal shift amount.

  For example, when the shift amount is +0.25, the pixel values of the pixels a + 0.25, a + 1.25, a + 2.25, a + 3.25 are used instead of the pixel values of the pixels A0, A1, A2, A3. Intra prediction is performed.

  Thus, the optimal shift amount is determined for the optimal intra prediction mode selected in the first operation. For example, it may be optimal when the shift amount is 0, and a pixel value of an integer pixel may be used.

  Of the nine prediction modes shown in FIG. 10 or FIG. 11, the average value processing is performed for mode 2 (DC prediction mode). Therefore, even if the shift is performed, the above-described operation is prohibited and is not performed because it is not directly related to the improvement of the coding efficiency.

  For mode 0 (Vertical Prediction mode), mode 3 (Diagonal_Down_Left Prediction mode), or mode 7 (Vertical_Left Prediction mode), only the upper adjacent pixels A0, A1, A2,.

  For mode 1 (Horizontal Prediction mode) or mode 8 (Horizontal_Up Prediction mode), only the left adjacent pixels I0, I1, I2,... In FIG.

  For the other modes (modes 4 to 6), it is necessary to consider the shift for both the upper adjacent pixel and the left adjacent pixel.

  For the upper adjacent pixel, only the horizontal shift amount is determined, and for the left adjacent pixel, only the vertical shift amount is determined.

  By performing the above first to fourth operations and determining the optimum shift amount, the choices of pixel values used in the intra prediction mode can be increased, and more optimum intra prediction can be performed. Thereby, it is possible to further improve the encoding efficiency in intra prediction.

  H. In the H.264 / AVC format, as described above with reference to FIG. 4, the 6-tap FIR filter circuit that was used only for inter motion prediction compensation can be effectively used for intra prediction. Thereby, the efficiency can be improved without increasing the circuit.

  In addition, H. It is possible to perform intra prediction at a resolution finer than 22.5 ° which is the resolution of intra prediction defined in the H.264 / AVC format.

[Example of effects of intra prediction with decimal pixel precision]
In the example of FIG. 19, the dotted line indicates the H.264 described above with reference to FIG. This indicates the direction of the prediction mode of the H.264 / AVC system intra prediction. The numbers given to the dotted lines correspond to the numbers of the nine prediction modes shown in FIG. 10 or FIG. Since mode 2 is an average value prediction, its number is not shown.

  H. In the H.264 / AVC format, intra prediction could be performed only at the resolution of 22.5 ° shown by the dotted line. On the other hand, in the image encoding device 51, by performing intra prediction with decimal pixel accuracy, it is possible to perform intra prediction with a resolution finer than 22.5 ° as indicated by a thick line. Thereby, especially the encoding efficiency with respect to the texture with a diagonal edge can be improved.

[Description of intra prediction processing]
Next, the intra prediction process as the operation described above will be described with reference to the flowchart of FIG. This intra prediction process is the intra prediction process in step S31 of FIG. 8, and in the example of FIG. 20, a case of a luminance signal will be described as an example.

  In step S41, the optimal mode determination unit 82 performs intra prediction for each of the 4 × 4 pixel, 8 × 8 pixel, and 16 × 16 pixel intra prediction modes.

  As described above, the intra 4 × 4 prediction mode and the intra 8 × 8 prediction mode have nine types of prediction modes, and one prediction mode can be defined for each block. The intra 16 × 16 prediction mode and the color difference signal intra prediction mode include four types of prediction modes, and one prediction mode can be defined for one macroblock.

  The optimum mode determination unit 82 performs intra prediction on the pixels of the processing target block with reference to the decoded adjacent image read from the adjacent image buffer 81 in all types of prediction modes of each intra prediction mode. Thereby, the prediction image in all kinds of prediction modes of each intra prediction mode is generated. Note that pixels that have not been deblocked filtered by the deblocking filter 71 are used as decoded pixels that are referred to.

  In step S42, the optimum mode determination unit 82 calculates cost function values for the 4 × 4 pixel, 8 × 8 pixel, and 16 × 16 pixel intra prediction modes. Here, the cost function value is determined based on either the High Complexity mode or the Low Complexity mode. These modes are H.264. It is defined by JM (Joint Model) which is reference software in the H.264 / AVC format.

  In other words, in the High Complexity mode, the process up to step S41 is temporarily performed up to the encoding process for all candidate prediction modes. Then, the cost function value represented by the following equation (34) is calculated for each prediction mode, and the prediction mode that gives the minimum value is selected as the optimal prediction mode.

Cost (Mode) = D + λ · R (34)

D is a difference (distortion) between the original image and the decoded image, R is a generated code amount including up to the orthogonal transform coefficient, and λ is a Lagrange multiplier given as a function of the quantization parameter QP.

  On the other hand, in the Low Complexity mode, as a process in step S41, generation of a prediction image and header bits such as motion vector information, prediction mode information, and flag information are calculated for all candidate prediction modes. The Then, the cost function value represented by the following equation (35) is calculated for each prediction mode, and the prediction mode that gives the minimum value is selected as the optimum prediction mode.

Cost (Mode) = D + QPtoQuant (QP) · Header_Bit (35)

D is a difference (distortion) between the original image and the decoded image, Header_Bit is a header bit for the prediction mode, and QPtoQuant is a function given as a function of the quantization parameter QP.

  In the Low Complexity mode, only a prediction image is generated for all prediction modes, and it is not necessary to perform encoding processing and decoding processing.

  In step S43, the optimal mode determination unit 82 determines an optimal mode for each of the 4 × 4 pixel, 8 × 8 pixel, and 16 × 16 pixel intra prediction modes. That is, as described above, in the case of the intra 4 × 4 prediction mode and the intra 8 × 8 prediction mode, there are nine types of prediction modes, and in the case of the intra 16 × 16 prediction mode, there are types of prediction modes. There are four types. Therefore, based on the cost function value calculated in step S42, the optimal mode determination unit 82 selects among them the optimal intra 4 × 4 prediction mode, optimal intra 8 × 8 prediction mode, and optimal intra 16 × 16 prediction mode. To decide.

  The optimum mode determination unit 82 is calculated in step S42 from among the optimum modes determined for the 4 × 4 pixel, 8 × 8 pixel, and 16 × 16 pixel intra prediction modes in step S44. The optimal intra prediction mode based on the cost function value is selected. That is, the mode having the minimum cost function value is selected as the optimum intra prediction mode from among the optimum modes determined for 4 × 4 pixels, 8 × 8 pixels, and 16 × 16 pixels.

  Information on the determined prediction mode is supplied to the mode determination unit 91, the optimum shift amount determination unit 83, and the predicted image generation unit 84. The cost function value corresponding to the prediction mode is also supplied to the predicted image generation unit 84.

  In step S45, the adjacent pixel interpolation unit 75 and the optimum shift amount determination unit 83 perform adjacent interpolation processing. The details of the adjacent interpolation process in step S45 will be described later with reference to FIG. 21. With this process, an optimal shift amount is determined in the shift direction corresponding to the determined optimal intra prediction mode. Information on the determined optimum shift amount is supplied to the predicted image generation unit 84.

  In step S <b> 46, the predicted image generation unit 84 generates a predicted image using adjacent pixels whose phase is shifted by the optimum shift amount.

  That is, the predicted image generation unit 84 reads adjacent pixels corresponding to the target block to be intra predicted from the adjacent image buffer 81. Then, the predicted image generation unit 84 shifts the phase of the read adjacent pixel by the optimal shift amount in the phase direction corresponding to the prediction mode, using a 6-tap FIR filter and linear interpolation. The prediction image generation unit 84 performs intra prediction in the prediction mode determined by the optimal mode determination unit 82 using the adjacent pixels whose phases are shifted, generates a prediction image of the target block, and the generated prediction image and The corresponding cost function value is supplied to the predicted image selection unit 77.

  When the optimum shift amount is 0, the pixel value of the adjacent pixel from the adjacent image buffer 81 is used.

  When a predicted image generated in the optimal intra prediction mode is selected by the predicted image selection unit 77, the information indicating the optimal intra prediction mode and the information on the shift amount are stored in the lossless encoding unit 66 by the predicted image generation unit 84. To be supplied. Then, it is encoded by the lossless encoding unit 66 and added to the header information of the compressed image (step S23 in FIG. 7 described above).

  As the encoding of the shift amount information, the difference between the determined shift amount of the target block and the shift amount in the block that gives the MostProbableMode described above with reference to FIG. 13 is encoded.

  However, for example, when MostProbableMode is mode 2 (DC prediction) and the prediction mode of the target block is mode 0 (Vertical prediction), there is no horizontal shift amount in the block giving MostProbableMode. Also, due to the fact that it is an intra macroblock in an inter slice, there is no horizontal shift amount in a block that gives MostProbableMode.

  In such a case, the differential encoding process is performed assuming that the horizontal shift amount in the block to which MostProbableMode is given is zero.

[Description of adjacent pixel interpolation processing]
Next, the adjacent pixel interpolation processing in step S45 of FIG. 20 will be described with reference to the flowchart of FIG. In the example of FIG. 21, a case where the target block is 4 × 4 will be described.

  Information on the prediction mode determined by the optimal mode determination unit 82 is supplied to the mode determination unit 91. In step S51, the mode determination unit 91 determines whether or not the optimal intra prediction mode is the DC mode. If it is determined in step S51 that the optimal intra prediction mode is not the DC mode, the process proceeds to step S52.

  In step S52, the mode determination unit 91 determines whether or not the optimal intra prediction mode is Vertical Prediction mode, Diagonal_Down_Left Prediction mode, or Vertical_Left Prediction mode.

  In Step S52, when it is determined that the optimal intra prediction mode is Vertical Prediction mode, Diagonal_Down_Left Prediction mode, or Vertical_Left Prediction mode, the process proceeds to Step S53.

  In step S <b> 53, the mode determination unit 91 outputs a control signal to the horizontal direction interpolation unit 92 to perform horizontal direction interpolation. That is, the horizontal direction interpolation unit 92 reads the upper adjacent pixel from the adjacent image buffer 81 in accordance with the control signal from the mode determination unit 91, and uses the 6-tap FIR filter and linear interpolation to read the upper adjacent pixel. On the other hand, the phase in the horizontal direction is shifted. The horizontal direction interpolation unit 92 supplies the information on the interpolated upper adjacent pixel to the optimum shift amount determination unit 83.

  In step S54, the optimal shift amount determination unit 83 determines the optimal shift amount of the upper adjacent pixel from −0.75 to +0.75 for the prediction mode determined by the optimal mode determination unit 82. For this determination, information on the target block image to be intra-predicted, upper adjacent pixels read from the adjacent image buffer 81, and interpolated upper adjacent pixels are used. At this time, the optimum shift amount for the left adjacent pixel is set to zero. Information on the determined optimum shift amount is supplied to the predicted image generation unit 84.

  If it is determined in step S52 that the optimal intra prediction mode is not Vertical Prediction mode, Diagonal_Down_Left Prediction mode, or Vertical_Left Prediction mode, the process proceeds to step S55.

  In step S55, the mode determination unit 91 determines whether or not the optimal intra prediction mode is the Horizontal Prediction mode or the Horizontal_Up Prediction mode. In Step S55, when it is determined that the optimal intra prediction mode is Horizontal Prediction mode or Horizontal_Up Prediction mode, the process proceeds to Step S56.

  In step S56, the mode determination unit 91 outputs a control signal to the vertical direction interpolation unit 93 to perform vertical direction interpolation. That is, the vertical interpolation unit 93 reads the left adjacent pixel from the adjacent image buffer 81 in accordance with the control signal from the mode determination unit 91, and reads the left adjacent pixel by the 6-tap FIR filter and linear interpolation. The phase in the vertical direction is shifted with respect to the pixel. The vertical direction interpolation unit 93 supplies the interpolated left adjacent pixel information to the optimum shift amount determination unit 83.

  In step S57, the optimal shift amount determination unit 83 determines the optimal shift amount of the left adjacent pixel from −0.75 to +0.75 for the prediction mode determined by the optimal mode determination unit 82. For this determination, information on the target block image to be intra-predicted, the left adjacent pixel read from the adjacent image buffer 81, and the interpolated left adjacent pixel are used. At this time, the optimum shift amount for the upper adjacent pixel is set to zero. Information on the determined optimum shift amount is supplied to the predicted image generation unit 84.

  In Step S55, when it is determined that the optimal intra prediction mode is not the Horizontal Prediction mode and the Horizontal_Up Prediction mode, the process proceeds to Step S58.

  In step S58, the mode determination unit 91 outputs a control signal to the horizontal interpolation unit 92, performs horizontal interpolation, outputs a control signal to the vertical interpolation unit 93, and performs vertical interpolation. To do.

  That is, the horizontal direction interpolation unit 92 reads the upper adjacent pixel from the adjacent image buffer 81 in accordance with the control signal from the mode determination unit 91, and uses the 6-tap FIR filter and linear interpolation to read the upper adjacent pixel. In contrast, the phase in the horizontal direction is shifted. The horizontal direction interpolation unit 92 supplies the information on the interpolated upper adjacent pixel to the optimum shift amount determination unit 83.

  The vertical interpolation unit 93 reads the left adjacent pixel from the adjacent image buffer 81 in accordance with the control signal from the mode determination unit 91, and reads the left adjacent pixel by a 6-tap FIR filter and linear interpolation. The phase in the vertical direction is shifted with respect to the pixel. The vertical direction interpolation unit 93 supplies the interpolated left adjacent pixel information to the optimum shift amount determination unit 83.

  In step S59, the optimal shift amount determination unit 83 determines the optimal shift amounts of the upper and left adjacent pixels from −0.75 to +0.75 for the prediction mode determined by the optimal mode determination unit 82. This determination uses the image of the target block to be intra-predicted, the upper and left adjacent pixels read from the adjacent image buffer 81, and the interpolated upper and left adjacent pixels. Information on the determined optimum shift amount is supplied to the predicted image generation unit 84.

  On the other hand, when it is determined in step S51 that the optimal intra prediction mode is the DC mode, the adjacent pixel interpolation process is ended. That is, the horizontal direction interpolation unit 82 and the vertical direction interpolation unit 83 do not operate, and the optimal shift amount determination unit 83 determines the shift amount 0 as the optimal shift amount.

[Explanation of inter motion prediction processing]
Next, the inter motion prediction process in step S32 in FIG. 8 will be described with reference to the flowchart in FIG.

  In step S61, the motion prediction / compensation unit 76 determines a motion vector and a reference image for each of eight types of inter prediction modes including 16 × 16 pixels to 4 × 4 pixels. That is, a motion vector and a reference image are determined for each block to be processed in each inter prediction mode.

  In step S62, the motion prediction / compensation unit 76 performs motion prediction on the reference image based on the motion vector determined in step S61 for each of the eight types of inter prediction modes including 16 × 16 pixels to 4 × 4 pixels. Perform compensation processing. By this motion prediction and compensation processing, a prediction image in each inter prediction mode is generated.

  In step S63, the motion prediction / compensation unit 76 adds motion vector information for adding to the compressed image the motion vectors determined for each of the eight types of inter prediction modes including 16 × 16 pixels to 4 × 4 pixels. Is generated. At this time, the motion vector generation method described above with reference to FIG. 5 is used.

  The generated motion vector information is also used when calculating the cost function value in the next step S64. When the corresponding predicted image is finally selected by the predicted image selection unit 77, the prediction mode information and reference It is output to the lossless encoding unit 66 together with the frame information.

  In step S64, the motion prediction / compensation unit 76 performs the cost function represented by the above-described equation (34) or equation (35) for each of the eight types of inter prediction modes including 16 × 16 pixels to 4 × 4 pixels. Calculate the value. The cost function value calculated here is used when determining the optimal inter prediction mode in step S34 of FIG. 8 described above.

  The operation principle in the present invention is not limited to the operation described above with reference to FIG. 18, or FIG. 20 and FIG. For example, for all intra prediction modes, prediction values of all candidate shift amounts can be calculated, and the residual can be calculated to determine the optimal intra prediction mode and the optimal shift amount. . A configuration example of the intra prediction unit and the adjacent pixel interpolation unit in the case of performing this operation is illustrated in FIG.

[Other Configuration Examples of Intra Prediction Unit and Adjacent Pixel Interpolation Unit]
FIG. 23 is a block diagram illustrating another configuration example of the intra prediction unit and the adjacent pixel interpolation unit.

  In the case of the example in FIG. 23, the intra prediction unit 74 includes an adjacent image buffer 101, an optimal mode / optimum shift amount determination unit 102, and a predicted image generation unit 103.

  The adjacent pixel interpolation unit 75 includes a horizontal direction interpolation unit 111 and a vertical direction interpolation unit 112.

  The adjacent image buffer 101 accumulates adjacent pixels of the target block for intra prediction from the frame memory 72. Also in FIG. 23, the switch 73 is not shown, but actually, the adjacent pixels are supplied from the frame memory 72 to the adjacent image buffer 101 via the switch 73.

  The optimal mode / optimum shift amount determination unit 102 receives the pixel of the target block for intra prediction from the screen rearrangement buffer 62. The optimum mode / optimum shift amount determination unit 102 reads adjacent pixels corresponding to the target block to be intra predicted from the adjacent image buffer 101.

  The optimum mode / optimum shift amount determination unit 102 supplies information on candidate intra prediction modes (hereinafter referred to as candidate modes) to the horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112. Information of adjacent pixels interpolated according to the candidate mode is input from the horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112 to the optimal mode / optimum shift amount determination unit 102.

  The optimal mode / optimum shift amount determination unit 102 uses the pixel values of the target block to be intra-predicted, the corresponding adjacent pixels, and the interpolated adjacent pixels for all candidate modes and all candidate shift amounts. Then, intra prediction is performed to generate a predicted image. Then, the optimum mode / optimum shift amount determination unit 102 calculates a cost function value, a prediction residual, and the like, and selects an optimum intra prediction mode and an optimum shift amount from all candidate modes and all shift amounts. decide. Information on the determined prediction mode and shift amount is supplied to the predicted image generation unit 103. At this time, the cost function value corresponding to the prediction mode is also supplied to the predicted image generation unit 103.

  The predicted image generation unit 103 reads adjacent pixels corresponding to the target block to be intra-predicted from the adjacent image buffer 101, and reads the adjacent pixels in the phase direction corresponding to the prediction mode by a 6-tap FIR filter and linear interpolation. Is shifted by the optimum shift amount.

  The prediction image generation unit 103 performs intra prediction in the optimal intra prediction mode determined by the optimal mode / optimum shift amount determination unit 102 using adjacent pixels whose phases are shifted, and generates a prediction image of the target block. The predicted image generation unit 103 outputs the generated predicted image and the corresponding cost function value to the predicted image selection unit 77.

  Further, when the predicted image generated in the optimal intra prediction mode is selected by the predicted image selection unit 77, the predicted image generation unit 103 converts the information indicating the optimal intra prediction mode and the shift amount information into the lossless encoding unit 66. To supply.

  The horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112 read adjacent pixels from the adjacent image buffer 101 according to the candidate mode from the optimal mode / optimum shift amount determination unit 102, respectively. The horizontal direction interpolating unit 111 and the vertical direction interpolating unit 112 shift the phase in the horizontal direction and the vertical direction, respectively, with respect to the read adjacent pixels by a 6-tap FIR filter and linear interpolation.

[Other explanation of intra prediction processing]
Next, the intra prediction process performed by the intra prediction unit 74 and the adjacent pixel interpolation unit 75 in FIG. 23 will be described with reference to the flowchart in FIG. This intra prediction process is another example of the intra prediction process in step S31 of FIG.

  The optimum mode / optimum shift amount determination unit 102 supplies information on candidate intra prediction modes to the horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112.

  In step S101, the horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112 perform adjacent pixel interpolation processing on all candidate intra prediction modes. That is, in step S101, adjacent pixel interpolation processing is executed for each intra prediction mode of 4 × 4 pixels, 8 × 8 pixels, and 16 × 16 pixels.

  The details of the adjacent interpolation process in step S101 will be described later with reference to FIG. 25. By this process, information on adjacent pixels interpolated in the shift direction corresponding to each intra prediction mode is converted into the optimum mode / optimum shift. The amount is supplied to the quantity determination unit 102.

  In step S102, the optimum mode / optimum shift amount determination unit 102 performs intra prediction for each intra prediction mode and each shift amount of 4 × 4 pixels, 8 × 8 pixels, and 16 × 16 pixels.

  That is, the optimum mode / optimum shift amount determination unit 102 uses the pixel values of the target block to be intra-predicted, the corresponding adjacent pixels, and the interpolated adjacent pixel values, for all intra prediction modes and all candidate shifts. Intra prediction is performed on the quantity. As a result, prediction images are generated for all intra prediction modes and all candidate shift amounts.

  In step S103, the optimum mode / optimum shift amount determination unit 102 described above for each intra prediction mode and each shift amount of 4 × 4 pixels, 8 × 8 pixels, and 16 × 16 pixels that generated the predicted image. The cost function value of Expression (34) or Expression (35) is calculated.

  In step S104, the optimum mode / optimum shift amount determination unit 102 compares the calculated cost function values with respect to each of the 4 × 4 pixel, 8 × 8 pixel, and 16 × 16 pixel intra prediction modes. The optimum mode and the optimum shift amount are determined respectively.

  In step S105, the optimum mode / optimum shift amount determination unit 102 determines the optimum intra prediction mode based on the cost function value calculated in step S103 from among the optimum modes and optimum shift amounts determined in step S104. Select the optimal shift amount. That is, the optimal intra prediction mode and the optimal shift amount are selected from the optimal modes and the optimal shift amounts determined for the 4 × 4 pixel, 8 × 8 pixel, and 16 × 16 pixel intra prediction modes. The The information on the selected prediction mode and shift amount is supplied to the predicted image generation unit 103 together with the corresponding cost function value.

  In step S <b> 106, the predicted image generation unit 103 generates a predicted image using adjacent pixels whose phase is shifted by the optimal shift amount.

  That is, the predicted image generation unit 103 reads adjacent pixels corresponding to the target block to be intra-predicted from the adjacent image buffer 101. Then, the predicted image generation unit 103 shifts the phase of the read adjacent pixel by the optimal shift amount in the phase direction corresponding to the determined prediction mode by the 6-tap FIR filter and linear interpolation.

  The prediction image generation unit 103 performs intra prediction in the prediction mode determined by the optimal mode / optimum shift amount determination unit 102 using adjacent pixels whose phases are shifted, and generates a prediction image of the target block. The generated predicted image is supplied to the predicted image selection unit 77 together with the corresponding cost function value.

[Description of adjacent pixel interpolation processing]
Next, the adjacent pixel interpolation processing in step S101 in FIG. 24 will be described with reference to the flowchart in FIG. The adjacent pixel interpolation process is a process performed for each candidate intra prediction mode. Also, steps S111 to S116 in FIG. 25 perform the same processing as steps S51 to S53, S55, S56, and S58 in FIG. 21, and thus detailed description thereof will be omitted as appropriate.

  Information on candidate intra prediction modes from the optimum mode / optimum shift amount determination unit 102 is supplied to the horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112. In step S111, the horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112 determine whether the candidate intra prediction mode is the DC mode. If it is determined in step S111 that the candidate intra prediction mode is not the DC mode, the process proceeds to step S112.

  In step S112, the horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112 determine whether the candidate intra prediction mode is Vertical Prediction mode, Diagonal_Down_Left Prediction mode, or Vertical_Left Prediction mode.

  If it is determined in step S112 that the candidate intra prediction mode is Vertical Prediction mode, Diagonal_Down_Left Prediction mode, or Vertical_Left Prediction mode, the process proceeds to step S113.

  In step S113, the horizontal interpolation unit 111 performs horizontal interpolation according to the candidate intra prediction mode. The horizontal direction interpolation unit 111 supplies the information on the interpolated upper adjacent pixel to the optimum mode / optimum shift amount determination unit 102. At this time, the vertical direction interpolation unit 112 does not perform vertical direction interpolation processing.

  If it is determined in step S112 that the candidate intra prediction mode is not Vertical Prediction mode, Diagonal_Down_Left Prediction mode, or Vertical_Left Prediction mode, the process proceeds to step S114.

  In step S114, the horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112 determine whether the candidate intra prediction mode is the Horizontal Prediction mode or the Horizontal_Up Prediction mode. If it is determined in step S114 that the candidate intra prediction mode is the Horizontal Prediction mode or Horizontal_Up Prediction mode, the process proceeds to Step S115.

  In step S115, the vertical direction interpolation unit 112 performs vertical direction interpolation in accordance with the candidate intra prediction mode. The vertical direction interpolation unit 112 supplies the interpolated information on the left adjacent pixel to the optimum mode / optimum shift amount determination unit 102. At this time, the horizontal interpolation unit 111 does not perform horizontal interpolation.

  If it is determined in step S114 that the candidate intra prediction mode is not the Horizontal Prediction mode and Horizontal_Up Prediction mode, the process proceeds to step S116.

  In step S116, the horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112 perform horizontal direction interpolation and vertical direction interpolation, respectively, according to the candidate intra prediction modes. The horizontal direction interpolation unit 111 and the vertical direction interpolation unit 112 supply information on the interpolated upper adjacent pixel and left adjacent pixel to the optimum mode / optimum shift amount determination unit 102, respectively.

  The encoded compressed image is transmitted via a predetermined transmission path and decoded by the image decoding device.

[Configuration Example of Image Decoding Device]
FIG. 26 shows a configuration of an embodiment of an image decoding apparatus as an image processing apparatus to which the present invention is applied.

  The image decoding device 151 includes a storage buffer 161, a lossless decoding unit 162, an inverse quantization unit 163, an inverse orthogonal transform unit 164, an operation unit 165, a deblock filter 166, a screen rearrangement buffer 167, a D / A conversion unit 168, a frame The memory 169, the switch 170, the intra prediction unit 171, the adjacent pixel interpolation unit 172, the motion prediction / compensation unit 173, and the switch 174 are configured.

  The accumulation buffer 161 accumulates the transmitted compressed image. The lossless decoding unit 162 decodes the information supplied from the accumulation buffer 161 and encoded by the lossless encoding unit 66 in FIG. 2 by a method corresponding to the encoding method of the lossless encoding unit 66. The inverse quantization unit 163 inversely quantizes the image decoded by the lossless decoding unit 162 by a method corresponding to the quantization method of the quantization unit 65 in FIG. The inverse orthogonal transform unit 164 performs inverse orthogonal transform on the output of the inverse quantization unit 163 by a method corresponding to the orthogonal transform method of the orthogonal transform unit 64 in FIG.

  The inverse orthogonal transformed output is added to the predicted image supplied from the switch 174 by the arithmetic unit 165 and decoded. The deblocking filter 166 removes block distortion of the decoded image, and then supplies the frame to the frame memory 169 to store it, and outputs it to the screen rearrangement buffer 167.

  The screen rearrangement buffer 167 rearranges images. That is, the order of frames rearranged for the encoding order by the screen rearrangement buffer 62 in FIG. 2 is rearranged in the original display order. The D / A conversion unit 168 D / A converts the image supplied from the screen rearrangement buffer 167, and outputs and displays the image on a display (not shown).

  The switch 170 reads an image to be inter-processed and a referenced image from the frame memory 169 and outputs them to the motion prediction / compensation unit 173, and also reads an image used for intra prediction from the frame memory 169 to the intra prediction unit 171. Supply.

  The intra prediction unit 171 is supplied with information indicating the intra prediction mode obtained by decoding the header information and information on the shift amount of adjacent pixels from the lossless decoding unit 162. The intra prediction unit 171 also supplies this information to the adjacent pixel interpolation unit 172.

  Based on the information, the intra prediction unit 171 shifts the phase of the adjacent pixel to the adjacent pixel interpolation unit 172 as necessary, and generates a predicted image using the adjacent pixel or the adjacent pixel whose phase is shifted. Then, the generated predicted image is output to the switch 174.

  The adjacent pixel interpolation unit 172 shifts the phase of the adjacent pixel by the shift amount supplied from the intra prediction unit 171 in the shift direction corresponding to the intra prediction mode supplied from the intra prediction unit 171. Actually, the adjacent pixel interpolation unit 172 applies a 6-tap FIR filter to the adjacent pixels in the shift direction according to the intra prediction mode, and linearly interpolates the phase of the adjacent pixels to a decimal number. Shift to pixel accuracy. The adjacent pixel interpolation unit 172 supplies the adjacent pixel whose phase has been shifted to the intra prediction unit 171.

  Information (prediction mode information, motion vector information, reference frame information) obtained by decoding the header information is supplied from the lossless decoding unit 162 to the motion prediction / compensation unit 173. When information indicating the inter prediction mode is supplied, the motion prediction / compensation unit 173 performs motion prediction and compensation processing on the image based on the motion vector information and the reference frame information, and generates a predicted image. The motion prediction / compensation unit 173 outputs the prediction image generated in the inter prediction mode to the switch 174.

  The switch 174 selects the prediction image generated by the motion prediction / compensation unit 173 or the intra prediction unit 171 and supplies the selected prediction image to the calculation unit 165.

[Configuration Example of Intra Prediction Unit and Adjacent Pixel Interpolation Unit]
FIG. 27 is a block diagram illustrating a detailed configuration example of the intra prediction unit and the adjacent pixel interpolation unit.

  In the case of the example in FIG. 27, the intra prediction unit 171 includes a prediction mode reception unit 181, a shift amount reception unit 182, and an intra prediction image generation unit 183. The adjacent pixel interpolation unit 172 includes a horizontal direction interpolation unit 191 and a vertical direction interpolation unit 192.

  The prediction mode receiving unit 181 receives the intra prediction mode information decoded by the lossless decoding unit 162. The prediction mode reception unit 181 supplies the received intra prediction mode information to the intra prediction image generation unit 183, the horizontal direction interpolation unit 191, and the vertical direction interpolation unit 192.

  The shift amount receiving unit 182 receives information on the shift amount (horizontal direction and vertical direction) decoded by the lossless decoding unit 162. Of the received shift amounts, the shift amount reception unit 182 supplies the horizontal shift amount to the horizontal interpolation unit 191 and supplies the vertical shift amount to the vertical interpolation unit 192.

  Information on the intra prediction mode received by the prediction mode reception unit 181 is input to the intra prediction image generation unit 183. In addition, the intra predicted image generation unit 183 includes information on the upper adjacent pixel or the upper adjacent pixel interpolated from the horizontal interpolation unit 191 and the left adjacent pixel or the interpolated left from the vertical interpolation unit 192. Information of adjacent pixels is input.

  The intra prediction image generation unit 183 generates and generates a prediction image by performing intra prediction using the pixel value of the adjacent pixel or the interpolated adjacent pixel in the prediction mode indicated by the input intra prediction mode information. The predicted image is output to the switch 174.

  The horizontal direction interpolation unit 191 reads the upper adjacent pixel from the frame memory 169 according to the prediction mode from the prediction mode reception unit 181. The horizontal interpolation unit 191 shifts the phase by the horizontal shift amount from the shift amount reception unit 182 with respect to the read upper adjacent pixel by a 6-tap FIR filter and linear interpolation. Information on the interpolated upper adjacent pixel or the upper adjacent pixel that has not been interpolated (that is, the adjacent pixel from the frame memory 169) is supplied to the intra-predicted image generation unit 183. In the case of FIG. 27, the switch 170 is not shown, but adjacent pixels are read out from the frame memory 169 via the switch 170.

  The vertical interpolation unit 192 reads the left adjacent pixel from the frame memory 169 in accordance with the prediction mode from the prediction mode receiving unit 181. The vertical interpolation unit 192 shifts the phase by the shift amount in the vertical direction from the shift amount reception unit 182 with respect to the read left adjacent pixel by a 6-tap FIR filter and linear interpolation. Information on the left adjacent pixel that has been linearly interpolated or the left adjacent pixel that has not been interpolated (that is, the adjacent pixel from the frame memory 169) is supplied to the intra-predicted image generation unit 183.

[Description of Decoding Process of Image Decoding Device]
Next, the decoding process executed by the image decoding device 151 will be described with reference to the flowchart of FIG.

  In step S131, the accumulation buffer 161 accumulates the transmitted image. In step S132, the lossless decoding unit 162 decodes the compressed image supplied from the accumulation buffer 161. That is, the I picture, P picture, and B picture encoded by the lossless encoding unit 66 in FIG. 2 are decoded.

  At this time, motion vector information, reference frame information, prediction mode information (information indicating an intra prediction mode or an inter prediction mode), flag information, shift amount information, and the like are also decoded.

  That is, when the prediction mode information is intra prediction mode information, the prediction mode information and the shift amount information are supplied to the intra prediction unit 171. When the prediction mode information is inter prediction mode information, motion vector information and reference frame information corresponding to the prediction mode information are supplied to the motion prediction / compensation unit 173.

  In step S133, the inverse quantization unit 163 inversely quantizes the transform coefficient decoded by the lossless decoding unit 162 with characteristics corresponding to the characteristics of the quantization unit 65 in FIG. In step S134, the inverse orthogonal transform unit 164 performs inverse orthogonal transform on the transform coefficient inversely quantized by the inverse quantization unit 163 with characteristics corresponding to the characteristics of the orthogonal transform unit 64 in FIG. As a result, the difference information corresponding to the input of the orthogonal transform unit 64 of FIG. 2 (output of the calculation unit 63) is decoded.

  In step S135, the calculation unit 165 adds the prediction image selected in the process of step S141 described later and input via the switch 174 to the difference information. As a result, the original image is decoded. In step S136, the deblocking filter 166 filters the image output from the calculation unit 165. Thereby, block distortion is removed. In step S137, the frame memory 169 stores the filtered image.

  In step S138, the intra prediction unit 171 and the motion prediction / compensation unit 173 perform image prediction processing corresponding to the prediction mode information supplied from the lossless decoding unit 162, respectively.

  That is, when intra prediction mode information is supplied from the lossless decoding unit 162, the intra prediction unit 171 performs an intra prediction process in the intra prediction mode. At this time, the intra prediction unit 171 performs an intra prediction process using adjacent pixels whose phase is shifted by the shift amount supplied from the lossless decoding unit 162 in the shift direction according to the intra prediction mode.

  The details of the prediction process in step S138 will be described later with reference to FIG. 29, and as a result, the prediction image generated by the intra prediction unit 171 or the prediction image generated by the motion prediction / compensation unit 173 is switched by the switch 174. To be supplied.

  In step S139, the switch 174 selects a predicted image. That is, a prediction image generated by the intra prediction unit 171 or a prediction image generated by the motion prediction / compensation unit 173 is supplied. Therefore, the supplied predicted image is selected and supplied to the calculation unit 165, and is added to the output of the inverse orthogonal transform unit 164 in step S134 as described above.

  In step S140, the screen rearrangement buffer 167 performs rearrangement. That is, the order of frames rearranged for encoding by the screen rearrangement buffer 62 of the image encoding device 51 is rearranged to the original display order.

  In step S141, the D / A converter 168 D / A converts the image from the screen rearrangement buffer 167. This image is output to a display (not shown), and the image is displayed.

[Explanation of prediction processing]
Next, the prediction processing in step S138 in FIG. 28 will be described with reference to the flowchart in FIG.

  In step S171, the prediction mode receiving unit 181 determines whether the target block is intra-coded. When the intra prediction mode information is supplied from the lossless decoding unit 162 to the prediction mode receiving unit 181, the prediction mode receiving unit 181 determines in step 171 that the target block is intra-encoded, and the processing is performed in step S <b> 172. Proceed to

  In step S172, the prediction mode receiving unit 181 receives and acquires intra prediction mode information from the lossless decoding unit 162. The prediction mode reception unit 181 supplies the received intra prediction mode information to the intra prediction image generation unit 183, the horizontal direction interpolation unit 191, and the vertical direction interpolation unit 192.

  In step S173, the shift amount reception unit 182 receives and acquires information on the shift amounts (horizontal direction and vertical direction) of the adjacent pixels decoded by the lossless decoding unit 162. Of the received shift amounts, the shift amount reception unit 182 supplies the horizontal shift amount to the horizontal interpolation unit 191 and supplies the vertical shift amount to the vertical interpolation unit 192.

  The horizontal direction interpolation unit 191 and the vertical direction interpolation unit 192 read adjacent pixels from the frame memory 169, and perform adjacent pixel interpolation processing in step S174. Details of the adjacent interpolation processing in step S174 are basically the same as the adjacent interpolation processing described above with reference to FIG.

  By this processing, the adjacent pixels that are interpolated in the shift direction according to the intra prediction mode from the prediction mode receiving unit 181 or the adjacent pixels that are not interpolated according to the intra prediction mode are converted into the intra prediction image generation unit. 183.

  That is, when the intra prediction mode is mode 2 (DC prediction), the horizontal direction interpolation unit 191 and the vertical direction interpolation unit 192 do not perform interpolation of adjacent pixels, and the upper and left sides read from the frame memory 169 The adjacent pixels are supplied to the intra predicted image generation unit 183.

  When the intra prediction mode is mode 0 (Vertical prediction), mode 3 (Diagonal_Down_Left prediction), or mode 7 (Vertical_Left prediction), only horizontal interpolation is performed. That is, the horizontal direction interpolation unit 191 interpolates the upper adjacent pixel read from the frame memory 169 with the horizontal shift amount from the shift amount reception unit 182, and the interpolated upper adjacent pixel is It supplies to the intra estimated image generation part 183. At this time, the vertical direction interpolation unit 192 supplies the left adjacent pixel read from the frame memory 169 to the intra predicted image generation unit 183 without performing interpolation of the left adjacent pixel.

  When the intra prediction mode is mode 1 (horizontal prediction) or mode 8 (horizontal_up prediction), interpolation is performed only in the vertical direction. That is, the vertical interpolation unit 192 interpolates the left adjacent pixel read from the frame memory 169 with the shift amount in the vertical direction from the shift amount reception unit 182, and interpolates the left adjacent pixel. Is supplied to the intra predicted image generation unit 183. At this time, the horizontal direction interpolation unit 191 supplies the upper adjacent pixel read from the frame memory 169 to the intra predicted image generation unit 183 without performing interpolation of the upper adjacent pixel.

  When the intra prediction mode is any other prediction mode, horizontal and vertical interpolation is performed. That is, the horizontal direction interpolation unit 191 interpolates the upper adjacent pixel read from the frame memory 169 with the horizontal shift amount from the shift amount reception unit 182, and the interpolated upper adjacent pixel is It supplies to the intra estimated image generation part 183. The vertical interpolation unit 192 interpolates the left adjacent pixel read from the frame memory 169 with the vertical shift amount from the shift amount receiving unit 182, and the interpolated left adjacent pixel is It supplies to the intra estimated image generation part 183.

  In step S175, the intra predicted image generation unit 183 is the prediction mode indicated by the input intra prediction mode information, and the adjacent pixels from the horizontal direction interpolation unit 191 and the vertical direction interpolation unit 192 or the interpolated adjacent pixels are displayed. Intra prediction is performed using pixel values. A prediction image is generated by the intra prediction, and the generated prediction image is output to the switch 174.

  On the other hand, if it is determined in step S171 that the intra encoding has not been performed, the process proceeds to step S176.

  When the processing target image is an inter-processed image, the inter prediction mode information, the reference frame information, and the motion vector information are supplied from the lossless decoding unit 162 to the motion prediction / compensation unit 173. In step S176, the motion prediction / compensation unit 173 acquires inter prediction mode information, reference frame information, motion vector information, and the like from the lossless decoding unit 162.

  Then, the motion prediction / compensation unit 173 performs inter motion prediction in step S177. That is, when the image to be processed is an image subjected to inter prediction processing, a necessary image is read from the frame memory 169 and supplied to the motion prediction / compensation unit 173 via the switch 170. In step S177, the motion prediction / compensation unit 173 performs motion prediction in the inter prediction mode based on the motion vector acquired in step S176, and generates a predicted image. The generated predicted image is output to the switch 174.

  As described above, in the image encoding device 51, pixels with decimal pixel precision are obtained by a 6-tap FIR filter and linear interpolation, and the optimum shift amount is determined, so that pixels used in the intra prediction mode are used. You can increase the value choices. Thereby, optimal intra prediction can be performed, and the encoding efficiency in intra prediction can be further improved.

  H. In the H.264 / AVC format, the 6-tap FIR filter circuit that has been used only for the inter motion prediction compensation described above with reference to FIG. 4 can be effectively used for intra prediction. Thereby, the efficiency can be improved without increasing the circuit.

  In addition, H. It is possible to perform intra prediction at a resolution finer than 22.5 ° which is the resolution of intra prediction defined in the H.264 / AVC format.

  Note that in the image encoding device 51, unlike the proposal described in Non-Patent Document 2, H.264 is used. Only pixels adjacent to the target block used in the intra prediction of the H.264 / AVC system at a predetermined position are used in the intra prediction. That is, only the adjacent pixels need be read out to the adjacent pixel buffer 81.

  Therefore, it is possible to avoid an increase in the number of memory accesses and an increase in processing, that is, a decrease in processing efficiency, by using pixels other than the adjacent pixels of the encoding target block in the proposal of Non-Patent Document 2 for prediction.

  In the above description, the case of the luminance signal intra 4 × 4 prediction mode has been described as an example of the adjacent pixel interpolation processing. However, the present invention is applicable to the case of the intra 8 × 8 or intra 16 × 16 prediction mode. Can also be applied. The present invention can also be applied to the case of the color difference signal intra prediction mode.

  In addition, in the case of the intra 8 × 8 prediction mode, as in the case of the intra 4 × 4 prediction mode, average value processing is performed for mode 2 (DC prediction mode). Therefore, even if the shift is performed, the above-described operation is prohibited and is not performed because it is not directly related to the improvement of the coding efficiency.

  For mode 0 (Vertical Prediction mode), mode 3 (Diagonal_Down_Left Prediction mode), or mode 7 (Vertical_Left Prediction mode), only the upper adjacent pixels A0, A1, A2,.

  For mode 1 (Horizontal Prediction mode) or mode 8 (Horizontal_Up Prediction mode), only the left adjacent pixels I0, I1, I2,... In FIG.

  For the other modes (modes 4 to 6), it is necessary to consider the shift for both the upper adjacent pixel and the left adjacent pixel.

  In addition, in the case of the intra 16 × 16 prediction mode and the color difference signal intra prediction mode, only the horizontal shift of the upper adjacent pixels is performed in the vertical prediction mode. For the horizontal prediction mode, only the vertical shift of the left adjacent pixel is performed. Shift processing is not performed for DC Prediction mode. Regarding the Plane Prediction mode, both the horizontal shift of the upper adjacent pixel and the vertical shift of the left adjacent pixel are performed.

  Furthermore, as described in Non-Patent Document 1, when interpolation processing with 1/8 pixel accuracy is performed in motion prediction, the interpolation processing with 1/8 pixel accuracy is also performed in the present invention. .

  In the above, the encoding method is H.264. The H.264 / AVC system is used, but the present invention is not limited to this, and other encoding systems / decoding systems that perform intra prediction using adjacent pixels can be applied.

  Note that the present invention is, for example, MPEG, H.264. When receiving image information (bitstream) compressed by orthogonal transformation such as discrete cosine transformation and motion compensation, such as 26x, via network media such as satellite broadcasting, cable television, the Internet, or mobile phones. The present invention can be applied to an image encoding device and an image decoding device used in the above. Further, the present invention can be applied to an image encoding device and an image decoding device used when processing on a storage medium such as an optical, magnetic disk, and flash memory. Furthermore, the present invention can also be applied to motion prediction / compensation devices included in such image encoding devices and image decoding devices.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes a computer incorporated in dedicated hardware, a general-purpose personal computer capable of executing various functions by installing various programs, and the like.

  FIG. 30 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing by a program.

  In a computer, a central processing unit (CPU) 301, a read only memory (ROM) 302, and a random access memory (RAM) 303 are connected to each other via a bus 304.

  An input / output interface 305 is further connected to the bus 304. An input unit 306, an output unit 307, a storage unit 308, a communication unit 309, and a drive 310 are connected to the input / output interface 305.

  The input unit 306 includes a keyboard, a mouse, a microphone, and the like. The output unit 307 includes a display, a speaker, and the like. The storage unit 308 includes a hard disk, a nonvolatile memory, and the like. The communication unit 309 includes a network interface and the like. The drive 310 drives a removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the computer configured as described above, for example, the CPU 301 loads the program stored in the storage unit 308 to the RAM 303 via the input / output interface 305 and the bus 304 and executes the program, thereby performing the series of processes described above. Is done.

  The program executed by the computer (CPU 301) can be provided by being recorded on a removable medium 311 as a package medium or the like, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

  In the computer, the program can be installed in the storage unit 308 via the input / output interface 305 by attaching the removable medium 311 to the drive 310. Further, the program can be received by the communication unit 309 via a wired or wireless transmission medium and installed in the storage unit 308. In addition, the program can be installed in advance in the ROM 302 or the storage unit 308.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  51 image encoding device, 66 lossless encoding unit, 74 intra prediction unit, 75 adjacent pixel interpolation unit, 76 motion prediction / compensation unit, 77 predicted image selection unit, 81 adjacent pixel buffer, 82 optimum mode determination unit, 83 optimal Shift amount determination unit, 84 predicted image generation unit, 91 mode determination unit, 92 horizontal direction interpolation unit, 93 vertical direction interpolation unit, 151 image decoding device, 162 lossless decoding unit, 171 intra prediction unit, 172 adjacent pixel interpolation Unit, 173 motion prediction / compensation unit, 174 switch, 181 prediction mode reception unit, 182 shift amount reception unit, 183 intra prediction image generation unit, 191 horizontal direction interpolation unit, 192 vertical direction interpolation unit

Claims (7)

A memory that accumulates adjacent pixels that are referred to when performing intra prediction on the pixels of the target block that is the target of the encoding process for the image to be encoded;
The phase of adjacent pixels read from the memory is shifted or the phase of adjacent pixels read from the memory is not shifted according to the prediction direction and block size when performing intra prediction on the pixels of the target block. A selection section for selecting or
An intra prediction unit that performs intra prediction on the pixels of the target block using the adjacent pixels and generates a predicted image;
An image processing apparatus comprising: an encoding unit that encodes the image using a predicted image generated by the intra prediction unit. When the selection unit selects that the phase of the adjacent pixel read from the memory is shifted, the intra prediction unit uses the adjacent pixel whose phase is shifted to perform intra prediction on the pixel of the target block. The image processing apparatus according to claim 1. When the selection unit selects that the phase of the adjacent pixel read from the memory is not shifted, the intra prediction unit uses the adjacent pixel whose phase has not been shifted to the intra of the pixel of the target block. The image processing apparatus according to claim 1, wherein prediction is performed. The image processing device
Intra-prediction is performed on pixels of the target block according to the prediction direction and block size when performing intra-prediction on the pixels of the target block that is the target of encoding processing for the image to be encoded. Select whether to shift the phase of the adjacent pixel read from the memory that stores the adjacent pixel to be referred to when performing, or not to shift the phase of the adjacent pixel read from the memory,
Using the adjacent pixels, perform intra prediction on the pixels of the target block to generate a predicted image,
An image processing method for encoding the image using the generated predicted image. The image processing according to claim 4, wherein when it is selected that the phase of an adjacent pixel read from the memory is shifted, intra prediction is performed on the pixel of the target block using the adjacent pixel whose phase is shifted. Method. The image according to claim 4, wherein when it is selected that the phase of the adjacent pixel read from the memory is not shifted, intra prediction is performed on the pixel of the target block using the adjacent pixel whose phase is not shifted. Processing method.   The recording medium which recorded the encoding stream produced | generated by the image processing apparatus described in Claim 1.

Images