JP5578974B2 - Image encoding device - Google Patents

Description

  The present invention relates to an image encoding apparatus, and more particularly to an image encoding technique for compressing video data using pixel correlation, and more particularly, intra prediction that generates a predicted image using correlation between adjacent pixels in a screen. The present invention relates to an image coding apparatus suitable for improving coding efficiency when performing the above.

  H. In the H.264 / AVC encoding technique, there is a method called intra prediction in which compression is performed using the correlation of neighboring pixels in a picture. H. H.264 / AVC encoding is defined in Non-Patent Document 1 below.

  In-screen prediction (intra prediction) creates a predicted image of a processing target image using an encoded image (reference image) in the encoding target picture, and transmits a difference between the predicted image and the processing target image. This is a prediction method for reducing the amount of information. In this way, intra prediction is a prediction method that uses only the spatial correlation of images, so that a video distorted due to a stream transmission error or the like can be restored to a normal video.

  In an encoding method using intra prediction, a prediction image is generated from neighboring pixels of a target block for a block to be encoded, and a residual component obtained by comparing the target block with the prediction image is compressed and encoded. Thus, higher encoding efficiency than that of the prior art is realized.

  H. In the intra prediction defined by the H.264 / AVC encoding method, it is possible to perform prediction processing by selecting from three types of block sizes of 4 × 4, 8 × 8, and 16 × 16 pixels as the block size for performing the prediction processing.

  Furthermore, the optimum method can be selected from a plurality of prepared prediction methods for each block size. Nine types of prediction methods are defined for 4 × 4 and 8 × 8 pixel block size intra prediction, and four types of prediction methods are defined for 16 × 16 pixel block size intra prediction.

  A method for selecting an optimum method from the plurality of defined prediction methods is not particularly defined. However, in general, it is desirable to use a prediction method with the smallest code amount in the end.

Hereinafter, a conventional intra prediction method will be described in detail with reference to FIG. 39 to FIG. 42, taking 4 × 4 pixel block size intra prediction as an example.
FIG. 39 is a diagram illustrating a positional relationship between a target 4 × 4 block 11 and a reference pixel 12 used for generating a predicted image of the block.
FIG. 40 is a conceptual diagram of nine types of prediction methods for 4 × 4 blocks.
FIG. 41 is a diagram illustrating the order in which macroblocks in a picture are processed in encoding.
FIG. 42 is a diagram illustrating an order in which 4 × 4 blocks in a macroblock are processed in encoding.

  Hereinafter, a pixel used for generating a predicted image is referred to as a “predicted reference pixel” or simply a “reference pixel”.

  In the 4 × 4 block intra prediction, as illustrated in FIG. 40, a predicted image can be generated from the reference pixel 12 described above using nine types of prediction methods (hereinafter referred to as “prediction mode”).

  For example, in mode 0, a predicted image is generated by copying the upper four pixels downward. In mode 1, a predicted image is generated by copying the left four pixels in the right direction. Thus, in intra prediction, a prediction image is generated according to a certain fixed direction from the reference pixels on the left, upper, and upper right. In mode 2, a predicted image is generated with an average value of neighboring eight pixels.

  H. In the H.264 / AVC encoding method, an input picture is divided into units called 16 × 16 pixel macroblocks (MB), and encoding processing is performed in units of MB. The order in which MBs in a picture are processed is as shown in FIG. 41, and processing is performed in the raster scan order from the upper left of the picture.

  Furthermore, the MB is divided into 4 × 4 blocks for processing, and the processing order is as shown in FIG.

  Here, the reference pixels used for predictive pixel generation are pixels of already processed blocks. For example, block 4 in FIG. 42 can use the pixels of blocks 1, 2, and 3 that have already been processed, but cannot use blocks 5, 7, 9, 10, and 13 that have not yet been processed.

Joint VideoTeam (JVT) of ISO / IEC MPEG & ITU-T VCEG: Text of ISO / IEC 14496-10 Advanced Video Coding 3rd Edition (2004).

  As described above, the conventional intra prediction method has a problem that the reference pixels such as the right and the lower cannot be used due to the restriction of the processing order of the blocks shown in FIGS.

  In the MPEG2 coding standard, the processing order of macroblocks is from left to right on the picture and from top to bottom. For this reason, the reference image that can be used for the prediction image creation process based on intra prediction is limited to the upper direction neighboring pixel and the left direction neighboring pixel 2 only in the encoding target macroblock. In order to create a prediction image using such a reference image, in the conventional intra prediction method, the prediction accuracy decreases as the lower right image of the encoding target macroblock as shown in FIG. 40, and the difference between the input image and the prediction image There is a problem that the prediction error is large.

  In contrast, the latest video compression coding standard, H.264. In H.264 / AVC, a normal 16 × 16 pixel macroblock is divided into 8 × 8 pixel or 4 × 4 pixel sub-blocks, and encoding processing is performed, so that prediction errors can be reduced by improving prediction accuracy. On the other hand, there is a problem that an increase in the amount of information accompanying sub-block division, that is, an increase in the amount of information in the macroblock header occurs.

  Thus, if reference pixels such as right and lower can be used, a prediction image can be generated bidirectionally from the upper and lower reference pixels and a more accurate prediction image can be generated. The prediction method has a problem that it cannot be realized due to processing order restrictions.

  The present invention has been made to solve the above-described problems. In the intra prediction of the image encoding apparatus according to the H.264 / AVC standard, the prediction accuracy is improved and the encoding efficiency is improved.

  The image encoding apparatus of the present invention creates N (N is a natural number of 2 or more) thinned images from an encoding target pixel block when encoding the encoding target pixel block using intra prediction. The first thinned image to the Nth thinned image are processed in order. Then, in generating the prediction pixel for the intra prediction of the pixel of the i th (i = 2,..., N) thinned image, the pixel of the i−1 th thinned image is used as the reference pixel from the pixel of the first thinned image. Use.

  At this time, the prediction pixel of the pixel of the i th thinned image is located around the prediction pixel when the pixel of the i th thinned image is around the prediction pixel from the pixel of the first thinned image. A pixel of the i-1th thinned image is generated from a pixel of a certain first thinned image as a reference pixel.

  For example, assume that the encoding target pixel block is a macroblock and N = 4.

  According to the image encoding apparatus that performs intra prediction of the present invention, it is possible to prepare nine peripheral reference pixels for a prediction target pixel, and a suitable intra prediction image can be obtained, thereby improving encoding efficiency. Can be high.

  In accordance with the present invention, H.264. In the intra prediction of the image coding apparatus according to the H.264 / AVC standard, prediction accuracy can be improved and coding efficiency can be improved.

It is the figure which compared and showed the pixel with high prediction accuracy by the intra prediction method which concerns on a prior art, and the pixel with low prediction accuracy. It is the figure which compared and showed the pixel with high prediction accuracy by the intra prediction method which concerns on 1st embodiment of this invention, and the pixel with low prediction accuracy. It is a figure explaining thinning image creation. It is a flowchart which shows the image coding process using the intra prediction method which concerns on 1st embodiment of this invention. It is a figure which shows the reconstruction image of a 1st thinning image, and the prediction image of a 2nd thinning image. It is a figure which shows the reference relationship of a 1st thinning image and a 2nd thinning image. It is a figure which shows the reconstruction image of a 1st thinning image, and the estimated image of a 3rd thinning image. It is a figure which shows the reference relationship of a 1st thinning image and a 3rd thinning image. It is a figure which shows the reconstruction image of a 1st thinning image, the reconstruction image of a 2nd thinning image, and the prediction image of a 4th thinning image. It is a figure which shows the reference relationship of a 2nd thinning image, a 3rd thinning image, and a 4th thinning image. It is a block diagram of the image coding apparatus which concerns on 1st embodiment of this invention. It is a block diagram of the bit stream obtained as an output of the image coding process in the intra prediction method according to the first embodiment of the present invention. It is a flowchart which shows the image decoding process which concerns on 1st embodiment of this invention. It is a figure explaining a mode that a combined image is created from the decoding image of a thinned image. It is a block diagram of the image decoding apparatus which concerns on 1st embodiment of this invention. It is a block diagram of the image coding apparatus which concerns on 2nd embodiment of this invention. It is a block diagram of the image decoding apparatus which concerns on 2nd embodiment of this invention. It is a figure which shows how to divide MB into 4x4 blocks and the processing order of the divided blocks. It is a figure which shows the prediction object block and reference pixel in a 1st block. FIG. 20 is a diagram showing the relationship between the first block and the reference pixel in an easy-to-understand manner by rewriting FIG. It is a figure which shows the prediction object block and reference pixel in a 2nd block. It is a figure which shows the reference pixel around the pixel p in a 2nd block. It is a conceptual diagram of each mode when a reference pixel is generated from surrounding pixels. It is a figure which shows the prediction object block and reference pixel in a 3rd block. It is a figure which shows the reference pixel around the pixel o in a 3rd block. It is a figure which shows the reference pixel around the pixel p in a 3rd block. It is a figure which shows the prediction object block and reference pixel in a 4th block. It is a figure which shows the reference pixel around the pixel p in a 4th block. It is a figure which shows the prediction object block and reference pixel in the 2nd block of MB in a slice boundary. It is a figure which shows the reference pixel around the pixel a in a 2nd block. It is a figure which shows the reference pixel around the pixel b in a 2nd block. It is a figure which shows the reference pixel around the pixel e in a 2nd block. It is a figure which shows the prediction object block and reference pixel in the 3rd block of MB in a slice boundary. It is a figure which shows the reference pixel around the pixel a in a 3rd block. It is a figure which shows the reference pixel around the pixel d in a 3rd block. It is a figure which shows the prediction object block and reference pixel in the 4th block of MB in a slice boundary. It is a figure which shows the reference pixel around the pixel a in a 4th block. It is a figure which shows the reference pixel around the pixel d in a 4th block. It is a figure which shows the positional relationship of the 4x4 block 11 used as object, and the reference pixel 12 used for the predicted image production | generation of the said block. It is a conceptual diagram of nine types of prediction methods of 4x4 block. It is a figure which shows the order which processes the macroblock in a picture in an encoding. It is a figure which shows the order which processes 4x4 block in a macroblock in encoding.

  Embodiments according to the present invention will be described below with reference to FIGS.

Embodiment 1
A first embodiment according to the present invention will be described below with reference to FIGS.
(I) Outline of Intra Prediction Method First, an outline of an intra prediction method according to the first embodiment of the present invention will be described using FIG. 1 and FIG.
FIG. 1 is a diagram comparing pixels with high prediction accuracy and pixels with low prediction accuracy according to a conventional intra prediction method.
FIG. 2 is a diagram comparing and comparing pixels with high prediction accuracy and pixels with low prediction accuracy according to the intra prediction method according to the first embodiment of the present invention.
FIG. 3 is a diagram for explaining the creation of a thinned image.

  In the intra prediction method according to the related art, the prediction image of the encoding target macroblock 201 is generated using only the reference pixels in the left direction or the upward direction. For this reason, as shown in FIG. 1, although the prediction accuracy of a pixel close to the reference pixel is high, the prediction accuracy of a pixel far from the reference pixel is low.

  The intra prediction method according to the present embodiment is to predict surrounding pixels as reference pixels in order to divide the encoding target pixel block 301 and predict some pixels of the thinned image. As shown in FIG. 2, the reference pixel and the pixel to be predicted are close in distance, and high prediction accuracy can be realized using reference images in the vertical and horizontal directions. In addition, since the macroblock header information can be greatly reduced as compared with the conventional intra prediction method, it contributes to the improvement of the coding efficiency. Furthermore, the intra prediction method according to the present embodiment can be used in combination with the conventional intra prediction method.

  In the present embodiment, as a slice unit input image, an encoding target pixel block 301 is used at the time of encoding, and divided into four to create a thinned image.

  Here, a slice is an image area that is a basic unit of encoding processing and decoding processing, and each slice on the same picture can perform encoding processing and decoding processing independently of other slices. .

In the present embodiment, as shown in FIG. 3, an example in which the slice unit input image 501 is sampled every other pixel in the horizontal direction and the vertical direction, that is, the input image is divided into four, and the first thinned image 511, An example in which the two thinned image 512, the third thinned image 513, and the fourth thinned image 514 are created will be described.
(II) Details Next the encoding process using intra prediction method, detailed processing of the image encoding process using intra prediction method according to the first embodiment of the present invention with reference to FIGS 10 explain.
FIG. 4 is a flowchart showing an image encoding process using the intra prediction method according to the first embodiment of the present invention.
FIG. 5 is a diagram illustrating a reconstructed image of the first thinned image and a predicted image of the second thinned image.
FIG. 6 is a diagram illustrating a reference relationship between the first thinned image and the second thinned image.
FIG. 7 is a diagram illustrating a reconstructed image of the first thinned image and a predicted image of the third thinned image.
FIG. 8 is a diagram illustrating a reference relationship between the first thinned image and the third thinned image.
FIG. 9 is a diagram illustrating a reconstructed image of the first thinned image, a reconstructed image of the second thinned image, and a predicted image of the fourth thinned image.
FIG. 10 is a diagram illustrating a reference relationship between the second thinned image, the third thinned image, and the fourth thinned image.

  First, an input image for each slice is acquired (step 401). This is because the intra prediction method according to the present embodiment creates a predicted image in units of slices.

  Next, a thinned-out image is created based on the image obtained in step 401 for acquiring the slice unit input image (step 402). As shown in FIG. 3, the thinned image is created by sampling pixels at equal intervals to obtain a first thinned image 511, a second thinned image 512, a third thinned image 513, and a fourth thinned image. 514 is created.

  The horizontal pixel number of the slice unit input image 501 is H (H is an even number), the vertical pixel number is V (V is an even number), and the pixel at the coordinates (x, y) of the slice unit input image 501 is Pi (x, y). In addition, the pixel at the coordinate (x, y) of the first thinned image 511 is Pi1 (x, y), the pixel at the coordinate (x, y) of the second thinned image 512 is Pi2 (x, y), and the third thinned image. If the pixel at the coordinate (x, y) of 513 is Pi3 (x, y) and the pixel at the coordinate (x, y) of the fourth thinned-out image 514 is Pi4 (x, y), the coordinate (i, j) Pi1 (i, j), Pi2 (i, j), Pi3 (i, j), Pi4 (i, j) can be expressed as follows. However, i is an integer that satisfies 0 ≦ i <(H / 2-1), and j is an integer that satisfies 0 ≦ j <(V / 2-1).

Pi1 (i, j) = P (2i, 2j) (Formula 1)
Pi2 (i, j) = P (2i + 1,2j) (Formula 2)
Pi3 (i, j) = P (2i, 2j + 1) (Formula 3)
Pi4 (i, j) = P (2i + 1,2j + 1) (Formula 4)
Next, in order to encode the first thinned image 511, the first thinned image 511 created by the thinned image creation processing 402 is subjected to coding processing by the conventional intra prediction method (step 403). The conventional intra prediction method is H.264. H.264 / AVC or the like is used to refer to pixels that are adjacent in the left direction or the upper direction for each macroblock (16 × 16 pixels) in the first thinned-out image 511, and in units of 16 × 16 pixels. This is a method of adaptively creating a predicted image in units of 8 × 8 pixels or 4 × 4 pixels.

  In the encoding process, the following process is performed.

  A prediction image is created in the intra prediction method or the like (step 411).

  Next, the difference (prediction error) between the predicted image created in step 411 and the input image is calculated (step 412).

  Next, orthogonal transformation processing is performed on the prediction error calculated in step 412 to obtain data after orthogonal transformation (step 413).

  Next, quantization processing is performed on the orthogonally transformed data obtained in step 413 to obtain quantized data (step 414).

  Next, entropy encoding processing is performed on the quantized data obtained in step 414 (step 415).

  In the intra prediction method according to the present embodiment, since the encoding process of the first thinned image 511 requires particularly high image quality, it is preferable to set the quantization step in the quantization process 414 small.

  Next, following step 403 of the coding process of the first thinned image 511, a prediction image is created for the second thinned image 512 created by the thinned image creating process 402, and the coding process is performed (step 404).

  As shown in FIG. 5, a reconstructed image 601 of the first thinned image that has already been encoded is used as a reference image for creating a predicted image of the second thinned image 512. Here, the reconstructed image is created by performing inverse quantization processing and inverse orthogonal transform processing on the quantized data obtained by the quantization processing 414 and adding the predicted image created by the predicted image creation processing 411. This is the same image as the image obtained by the decoding process.

  The second thinned image 512 can be considered as an interpolation pixel in the horizontal direction with respect to the first thinned image 511, as shown in FIG. Therefore, it is effective to apply an image with interpolation pixels created by applying a horizontal filter to the reconstructed image 601 of the first thinned image as the predicted image of the second thinned image 512. Here, it is assumed that a predicted image of the second thinned image 512 (hereinafter, a predicted image 611 of the second thinned image) is created by creating an interpolation pixel using a horizontal 2-tap filter.

  The pixel of the coordinate (x, y) of the predicted image 611 of the second thinned image is set as Pp2 (x, y), and the pixel of the coordinate (x, y) of the reconstructed image 601 of the first thinned image is Pr1 (x, y). When y), the pixel Pp2 (n, m) 612 at the coordinates (n, m) of the predicted image 611 of the second thinned image can be expressed by the following (formula 5). That is, the average of the reconstructed image 601 of the first thinned image on both the left and right sides of the pixel of the predicted image 611 of the second thinned image is taken. Here, the reason why the numerator is +1 is to round off the last bit.

However, n is an integer that satisfies 0 ≦ n <(H / 2-1), and m is an integer that satisfies 0 ≦ m <(V / 2-1). H and V are respectively the number of horizontal pixels and the number of vertical pixels of the slice unit input image 501, both of which are even numbers. In (Expression 5), when n = (H / 2-1), a pixel of Pr1 (H / 2, y), that is, a pixel outside the screen of the reconstructed image 601 of the first thinned image is required. When n ≧ (H / 2-1), Pr1 (n, y) = Pr1 (H / 2-1, y) is set, that is, the screen edge pixel is copied to be an off-screen pixel, etc. That's fine.
Pp2 (n, m) = (Pr1 (n, m) + Pr1 (n + 1, m) +1) / 2
... (Formula 5)
For simplicity, the pixel interpolation using the horizontal 2-tap filter has been described. However, the prediction image 611 of the second thinned image may be created by pixel interpolation using a multi-tap filter or a two-dimensional filter that can express higher frequency components. .

  Next, following step 404 of the coding process of the second thinned image 512, a prediction image is created for the third thinned image 513 created by the thinned image creating process 402, and the coding process is performed (step 405).

  In creating the predicted image of the third thinned image 513, as shown in FIG. 7, a reconstructed image 601 of the first thinned image that has already been encoded is used as a reference image.

  As shown in FIG. 8, the third thinned image 513 can be considered as an interpolation pixel in the horizontal direction with respect to the first thinned image 511. Therefore, it is effective to apply an image with interpolation pixels created by applying a vertical filter to the reconstructed image 601 of the first thinned image as the predicted image of the third thinned image 513. Here, it is assumed that a predicted image of the third thinned image 513 (hereinafter, a predicted image 811 of the third thinned image) is created by interpolation pixel creation using a vertical 2-tap filter. That is, the average of the reconstructed image 601 of the first thinned image on both the upper and lower sides of the pixel of the predicted image 811 of the third thinned image is taken.

  The pixel of the coordinate (x, y) of the predicted image 811 of the third thinned image is set as Pp3 (x, y), and the pixel of the coordinate (x, y) of the reconstructed image 601 of the first thinned image is Pr1 (x, y). If y), the pixel Pp3 (n, m) 812 at the coordinates (n, m) of the predicted image 811 of the third thinned image can be expressed by the following (formula 6).

However, n is an integer that satisfies 0 ≦ n <(H / 2-1), and m is an integer that satisfies 0 ≦ m <(V / 2-1). H and V are respectively the number of horizontal pixels and the number of vertical pixels of the slice unit input image 501, both of which are even numbers. In Expression (6), when m = (V / 2-1), a pixel of Pr1 (x, V / 2), that is, a pixel outside the screen of the reconstructed image 601 of the first thinned image is necessary. When m ≧ (V / 2-1), if Pr1 (x, m) = Pr1 (x, V / 2-1), that is, if the screen edge pixel is copied to be an off-screen pixel, etc. Good.
Pp3 (n, m) = (Pr1 (n, m) + Pr1 (n, m + 1) +1) / 2
... (Formula 6)
For the sake of simplicity, the method of creating the predicted image 811 of the third thinned image using only the reconstructed image 601 of the first thinned image as a reference image has been described, but the reconstructed image of the second thinned image may also be used as the reference image. Good. Further, although the pixel interpolation using the vertical 2-tap filter has been described, the predicted image 811 of the third thinned-out image may be generated by the multi-tap filter that can express higher frequency components or the pixel interpolation using the two-dimensional filter.

  Next, following step 405 of the coding process of the third thinned image 513, a predicted image is created for the fourth thinned image 514 created by the thinned image creating process 402, and the coding process is performed (step 406).

  As shown in FIG. 9, the predicted image of the fourth thinned image 514 is generated using the reconstructed image 1001 of the second thinned image and the reconstructed image 1002 of the third thinned image that have already been encoded as reference images. Use.

  As shown in FIG. 10, the fourth thinned image 514 can be considered as an interpolation pixel in the vertical direction with respect to the second thinned image 512, and is considered as an interpolation pixel in the horizontal direction with respect to the third thinned image 513. be able to. Therefore, it is possible to apply to the predicted image of the fourth thinned image 514 an image with interpolation pixels created by applying a two-dimensional filter to the reconstructed image 1001 of the second thinned image and the reconstructed image 1002 of the third thinned image. It is effective. Here, it is assumed that a predicted image of the fourth thinned image 514 (hereinafter, a predicted image 1011 of the second thinned image) is created by creating an interpolated pixel using a two-dimensional filter with two taps each in the horizontal and vertical directions.

  The pixel of the coordinate (x, y) of the predicted image 1011 of the fourth thinned image is set as Pp4 (x, y), and the pixel of the coordinate (x, y) of the reconstructed image 1001 of the second thinned image is Pr2 (x, y). y) When the pixel of the coordinate (x, y) of the reconstructed image 1002 of the third thinned image is set to Pr3 (x, y), the pixel Pp4 of the coordinate (n, m) of the predicted image 1011 of the fourth thinned image (N, m) 1012 can be expressed by the following (formula 7). That is, the average of the reconstructed image 1011 of the second thinned image on the upper and lower sides of the pixel of the predicted image 1011 of the fourth thinned image and the reconstructed image 1002 of the third thinned image on both the left and right sides are taken. Here, the reason why +2 is used in the numerator is to round off the last bit.

However, n is an integer that satisfies 0 ≦ n <(H / 2-1), and m is an integer that satisfies 0 ≦ m <(V / 2-1). H and V are respectively the number of horizontal pixels and the number of vertical pixels of the slice unit input image 501, both of which are even numbers. In (Expression 7), when n = (H / 2-1) or m = (V / 2-1), the pixel of Pr3 (H / 2, y) or Pr2 (x, V / 2) Pixels are required. That is, there may be a case where the out-of-screen pixels of the reconstructed image 1001 of the second decimation image and the reconstructed image 1002 of the third decimation image are necessary, which is Pr3 (n, y) = Pr3 (H / 2-1, y), when m ≧ (V / 2-1), Pr2 (x, m) = Pr2 (x, V / 2-1). What is necessary is just to copy and make it an off-screen pixel.
Pp4 (n, m) = (Pr2 (n, m) + Pr2 (n, m + 1) + Pr3 (n, m) + Pr2 (n + 1, m) +2) / 4
... (Formula 7)
For the sake of simplicity, the method of creating the predicted image 1011 of the fourth decimation image using only the reconstructed image 1001 of the second decimation image and the reconstructed image 1002 of the third decimation image as the reference image has been shown. The configuration image 601 may also be used as a reference image. In addition, the pixel interpolation using the two-dimensional filter with two taps in each of the horizontal and vertical directions has been described, but the predicted image 1011 of the fourth thinned image is created by the pixel interpolation using the multi-tap filter or the one-dimensional filter that can express higher frequency components. May be.
(III) Configuration of Image Encoding Device Next, the configuration of the image encoding device according to the first embodiment of the present invention will be described with reference to FIG.
FIG. 11 is a configuration diagram of an image encoding device according to the first embodiment of the present invention.

  As shown in FIG. 11, the image coding apparatus 100 according to the present embodiment includes a thinned image creation unit 101, a thinned image buffer 102, a first intra predicted image creation unit 103, a prediction error calculation processing unit 104, and an orthogonal transformation process. Unit 105, quantization processing unit 106, inverse quantization processing unit 107, inverse orthogonal transform processing unit 108, reconstructed image creation unit 109, reconstructed image buffer 110, second intra predicted image creation unit 111, entropy encoding process The unit 112 is provided.

  The thinned image creating unit 101 is a processing unit that performs a thinned image creating process 402, and creates four thinned images from a first thinned image 511 to a fourth thinned image 514 based on the input image 151.

  The thinned image buffer 102 temporarily stores the first thinned image 511, the second thinned image 512, the third thinned image 513, and the fourth thinned image 514, and a buffer for adjusting the output timing of each thinned image. The first thinned image 511, the second thinned image 512, the third thinned image 513, and the fourth thinned image 514 are output in this order. As described in the prediction process in step 403 of the first decimation image encoding process, the first intra prediction processing unit 103 is a processing unit that creates a predicted image by the conventional intra prediction method. The prediction error calculation processing unit 104 is a processing unit that calculates a difference (that is, a prediction error) between the thinned image of the input image and the predicted image, and is a processing unit that executes the prediction error calculation processing 412. The orthogonal transformation processing unit 105 is a processing unit that executes the orthogonal transformation processing 413. The quantization processing unit 106 is a processing unit that executes the quantization processing 414. The inverse quantization processing unit 107 is a processing unit that performs inverse quantization processing on the quantized data output from the quantization processing unit 106. The inverse orthogonal transform processing unit 108 is a processing unit that performs an inverse orthogonal transform process on the dequantized data output from the inverse quantization processing unit 107. The reconstructed image creation unit 109 is a processing unit that creates the reconstructed image by adding the data after inverse orthogonal transform output from the inverse orthogonal transform processing unit 108 and the predicted image input to the prediction error calculation processing unit 104. The reconstructed image buffer 110 is a buffer for storing the reconstructed image created by the reconstructed image creating unit 109. The second intra prediction processing unit 111 acquires a reconstructed image of a necessary thinned image from the reconstructed image buffer 110, and predicts the second thinned image 512, the third thinned image 513, and the fourth thinned image 514. Create. That is, the second intra prediction processing unit 111 performs the predicted image generation process according to step 404 of the second decimation image encoding process, step 405 of the third decimation image encoding process, and step 406 of the fourth decimation image encoding process. It is a processing unit that performs. The entropy encoding processing unit 112 is a processing unit that performs entropy encoding processing 415 and outputs a bit stream 152.

The image coding apparatus 100 is greatly different from the conventional image coding apparatus in that it includes the above-described thinned image creating unit 101, the thinned image buffer 102, the reconstructed image buffer 110, and the second intra prediction processing unit 111. .
(IV) Configuration of Bitstream Next, the configuration of a bitstream obtained as an output of image encoding processing in the intra prediction method of the present embodiment will be described using FIG.
FIG. 12 is a configuration diagram of a bitstream obtained as an output of an image encoding process in the intra prediction method according to the first embodiment of the present invention.

  Since the intra prediction method according to the present embodiment is encoded in units of slices, only the configuration below the slice header is shown in FIG.

  As shown in FIG. 12, the bit stream obtained as the output of the image encoding process includes slice header information 1201, code information 1211 regarding the first decimation image, code information 1212 regarding the second decimation image, and code regarding the third decimation image. It consists of information 1213 and code information 1214 regarding the fourth thinned image. These pieces of information are output in the order of slice header information 1201, code information 1211 related to the first thinned image, code information 1212 related to the second thinned image, code information 1213 related to the third thinned image, and code information 1214 related to the fourth thinned image. The code information 1214 related to the fourth thinned image is followed by the next slice header information 1202. Since the first decimation image is encoded by the same method as the conventional intra prediction method, the code information 1211 related to the first decimation image includes macroblock header information.

Since the intra prediction method according to the present embodiment performs intra prediction in units of slices, slice header information in a bitstream is used in combination with a conventional intra prediction method that performs intra prediction in units of macroblocks. It is only necessary to add 1-bit data for identifying which prediction method, and the code amount overhead is extremely small.
(V) Details of Image Decoding Processing Next, detailed processing of image decoding processing of an image encoded by the intra prediction method according to the first embodiment of the present invention will be described using FIG. 13 and FIG. .
FIG. 13 is a flowchart showing an image decoding process according to the first embodiment of the present invention.
FIG. 14 is a diagram for explaining how a combined image is created from a decoded image of a thinned image.

  First, the decoding process of the first thinned image is performed (step 1301).

  The decryption process is performed according to the following procedure.

  First, as entropy decoding processing, post-quantization data is created from a bit stream (step 1311).

  Next, as inverse quantization processing, data after orthogonal transformation is created from the quantized data (step 1312).

  Next, as an inverse orthogonal transform process, a prediction error is generated from the data after orthogonal transform (step 1313).

  Next, a predicted image is created (step 1314).

  Next, the prediction error and the prediction image are added to create a decoded image (step 1315).

  In particular, in the predicted image creation process in step 1314 in the decoding process of the first thinned image, a predicted image is created by the conventional intra prediction method.

  Next, after the decoding process of the first thinned image, the decoding process of the second thinned image is performed (step 1302).

  In particular, in the predicted image creation process in step 1314 in the decoding process of the second thinned image, the same predicted image creation process as the predicted image creation process in step 411 in the second thinned image encoding process in step 404 shown in FIG. To do.

  Next, after the decoding process of the second thinned image, the decoding process of the third thinned image is performed (step 1303).

  In particular, in the predicted image creation process in step 1314 in the decoding process of the third thinned image, the same predicted image creation process as the predicted image creation process in step 411 in the third thinned image encoding process in step 405 is performed.

  Next, after the decoding process of the third thinned image, the decoding process of the fourth thinned image is performed (step 1304).

  In particular, in the predicted image creation process in step 1314 in the decoding process of the fourth thinned image, the same predicted image creation process as the predicted image creation process in step 411 in the fourth thinned image encoding process in step 406 is performed.

  When the decoding process of the first decimation image or the decoding process of the fourth decimation image is finished, as shown in FIG. 14, the decoded image 1401 and the second decimation of the first decimation image decoded by the respective decoding processes are performed. The decoded image 1402, the decoded image 1403 of the third thinned image, and the decoded image 1404 of the fourth thinned image are combined to create a combined image 1411 (step 1305). This process is equivalent to performing the reverse process to the thinned image creation process 402. Since the intra prediction method according to the present invention uses slices as the basic unit, the combined image 1411 is an image in units of slices.

The difference between the first decimation image decoding process 1301 in step 1301 and the second decimation image decoding process to the fourth decimation image decoding process 1304 in step 1302 is that the predicted image generation method of the decoding process and the first decimation image decoding Only the entropy decoding process of the macroblock header information is necessary only in the conversion process 1301.
(VI) Configuration of Image Decoding Device Next, the configuration of the image decoding device according to the first embodiment of the present invention will be described with reference to FIG.
FIG. 15 is a block diagram of an image decoding apparatus according to the first embodiment of the present invention.

  As shown in FIG. 15, the image decoding apparatus 1500 includes an entropy decoding processing unit 1501, an inverse quantization processing unit 107, an inverse orthogonal transform processing unit 108, a reconstructed image creating unit 109, a reconstructed image buffer 110, a 1 intra prediction processing unit 103, second intra prediction processing unit 111, thinned image buffer 1502, and combined image creation unit 1503.

  The inverse quantization processing unit 107, the inverse orthogonal transform processing unit 108, the reconstructed image creation unit 109, the reconstructed image buffer 110, and the first intra prediction processing unit 103 are the same as those described for the image encoding device 100 in FIG. is there.

  The entropy decoding processing unit 1501 is a processing unit that decodes a bit stream and creates post-quantization data and the like, and performs processing corresponding to the entropy decoding processing in step 1311. The thinned image buffer 1502 is a buffer that temporarily stores a decoded image of the thinned image. The combined image unit 1503 is a processing unit that performs the combined image creation shown in the combined image creation processing in step 1305.

  For the input bitstream 152, the image decoding apparatus 1500 first converts the code information 1211 related to the first decimation image shown in FIG. 12 into quantized data or the like by the entropy decoding processing unit 1501, and performs inverse quantization. Decoding of the first decimation image is performed by performing inverse quantization processing and inverse orthogonal transformation processing in the quantization processing unit 107 and the inverse orthogonal transform processing unit 108, and adding the predicted image created by the first predicted image creation processing unit 103 The image is acquired and stored in the thinned image buffer 1502.

  Next, the code information 1212 related to the second thinned image shown in FIG. 12 is converted into quantized data by the entropy decoding processing unit 1501, and the inverse quantization processing unit 107 and the inverse orthogonal transform processing unit 108 respectively perform inverse processing. Quantization processing and inverse orthogonal transform processing are performed, and a decoded image of the second thinned image is acquired by adding the predicted image created by the second predicted image creation processing unit 111 and stored in the thinned image buffer 1502.

  Thereafter, the code information 1213 related to the third thinned image and the code information 1213 related to the fourth thinned image shown in FIG. 12 are processed in the same manner as the code information 1212 related to the second thinned image, and the acquired third thinned image is decoded. The image and the decoded image of the fourth thinned image are stored in the thinned image buffer 1502.

  After these processes, the combined image creation unit 1503 reads out the decoded images of all the thinned images from the thinned image buffer 1502, creates a combined image, and outputs it as a decoded image 1551.

  The image decoding device 1500 is greatly different from the conventional image decoding device in that it includes the reconstructed image buffer 110, the second intra prediction processing unit 111, the thinned image buffer 1502, and the combined image creation unit 1503 described above. is there.

[Embodiment 2]
Hereinafter, a second embodiment according to the present invention will be described with reference to FIGS. 16 and 17.

  In the first embodiment, the image encoding process has been described in which the encoding target pixel block is divided to create some thinned images and use them as predicted images in intra prediction.

In the present embodiment, an image coding apparatus and an image decoding apparatus that can improve coding efficiency by using the intra prediction method according to the present invention and the conventional intra prediction method will be described.
FIG. 16 is a configuration diagram of an image encoding device according to the second embodiment of the present invention.
FIG. 17 is a configuration diagram of an image decoding apparatus according to the second embodiment of the present invention.

  As shown in FIG. 16, the image encoding device 1600 includes a first image encoding device 1601, a second image encoding device 1602, an encoding efficiency statistical processing unit 1606, a stream buffer 1603, a stream buffer 1604, and an output selection circuit. 1605.

  The first form image encoding apparatus 100 is an image encoding apparatus using the intra prediction method according to the first embodiment described in FIG. The second image encoding device 1602 is H.264. This is an image coding apparatus using a conventional intra prediction method such as H.264 / AVC. The coding efficiency statistical processing unit 1606 uses a bit stream obtained from the first image coding device 1601 and a bit stream obtained from the second image coding device based on statistical data that is an index of code amount and image quality in units of slices. Among these, it is a processing unit that outputs a control signal for selecting a bitstream that further improves the encoding efficiency. The stream buffer 1603 is a buffer that temporarily stores the bit stream output from the first image encoding device 1601, and has a function of managing and outputting the bit stream data for each slice unit. The stream buffer 1604 is a buffer for temporarily storing the bit stream output from the second image encoding device 1602 and has a function of managing and outputting the bit stream data for each slice unit. The output selection processing unit 1605 switches and outputs one of the bit streams output from the stream buffer 1603 and the stream buffer 1604 in units of slices in accordance with the control signal output from the coding efficiency statistical processing unit 1606. Furthermore, it has a function of assigning 1 bit of code for identifying the intra prediction method according to the first embodiment and the conventional intra prediction method as slice header information of the bitstream.

  Hereinafter, the operation of the image encoding device 1600 will be described. The image encoding device 1600 performs image encoding processing on the input image 1651 by the first image encoding device 1601 and the second image encoding device 1602, and outputs the obtained bit streams to the stream buffer 1603 and the stream buffer 1604, respectively. To do. Further, the first image encoding device 1601 outputs statistical data 1631 serving as a generated code amount and image quality index in units of slices to the encoding efficiency statistical processing unit 1606. The second image encoding device 1602 also outputs statistical data 1632 serving as a generated code amount and image quality index in units of slices to the encoding efficiency statistical processing unit 1606. The coding efficiency statistical processing unit 1606 outputs control information 1633 for selecting a bitstream that further improves the coding efficiency to the output selection processing 1605 based on the input statistical data, and the output selection processing 1605 Based on the control information 1633, one of the bit streams output from the stream buffer 1603 and the stream buffer 1604 is switched in units of slices and output as a bit stream 1652. At that time, 1 bit of code for identifying the first intra prediction method and the conventional intra prediction method is added to the slice header of the adopted bitstream.

  Next, as shown in FIG. 17, the image decoding apparatus 1700 includes an entropy decoding processing unit 1701, an inverse quantization processing unit 107, an inverse orthogonal transform processing unit 108, a reconstructed image creating unit 109, a reconstructed image buffer. 110, a first intra prediction processing unit 103, a second intra prediction processing unit 111, a thinned image buffer 1502, a combined image creation unit 1503, an output selection processing unit 1702, and a delay buffer 1703.

  The inverse quantization processing unit 107, the inverse orthogonal transform processing unit 108, the reconstructed image creation unit 109, the reconstructed image buffer 110, the first intra prediction processing unit 103, and the second intra prediction processing unit 111 are the first implementation. The thinned-out image buffer 1502 and the combined image creating unit 1503 are the same as those described with reference to FIG.

  The entropy decoding processing unit 1701 is the intra prediction method according to the first embodiment or the bit stream below the slice header information to be processed from the slice header information of the input bit stream 1652 or the conventional intra prediction method. This is a processing unit that has a function of outputting the determination result to the output selection processing unit 1702 as the determination result information 1731 and decoding the bit stream and outputting the quantized data and the like.

  The output selection processing unit 1702 selects the output of the delay buffer 1703 and the output of the combined image creation unit 1503 based on the determination result information 1731, and outputs the selected image 1751.

  The delay buffer 1703 calculates a time difference between a processing delay time when obtaining a decoded image from the bit stream of the intra prediction method according to the first embodiment and a processing delay time when obtaining a decoded image from the bit stream of the conventional intra prediction method. It is a buffer for absorption, and requires a capacity capable of storing decoded image data for about one slice.

  Hereinafter, the operation of the image decoding apparatus 1700 will be described. When the entropy decoding processing unit 1701 of the image decoding device 1700 determines that the bit stream is the bit stream of the intra prediction method according to the first embodiment, the dequantized data obtained by decoding is subjected to the inverse quantization processing unit 107, An image is obtained by the inverse orthogonal transform processing unit 108, the reconstructed image creating unit 109, the reconstructed image buffer 110, the first intra prediction processing unit 103, the second intra prediction processing unit 111, the thinned image buffer 1502, and the combined image creating unit 1503. Processing similar to that of the decoding device 1500 is performed to obtain a decoded image of the intra prediction method according to the first embodiment. On the other hand, when the entropy decoding processing unit 1701 determines that the bit stream of the conventional intra prediction method is used, the quantized data obtained by decoding is converted into an inverse quantization processing unit 107, an inverse orthogonal transform processing unit 108, and a reconstructed image. The creation unit 109 and the first intra prediction processing unit 103 create a decoded image of the conventional intra prediction method and store it in the delay buffer 1703. If the determination result information 1731 is a result determined to be the intra prediction method according to the first embodiment, the output selection processing unit 1702 outputs the decoded image of the combined image creation unit 1503 and determines the conventional intra prediction method. If so, the decoded image of the buffer 1703 is output.

[Embodiment 3]
The third embodiment of the present invention will be described below with reference to FIGS.

  In the first embodiment, a method has been described in which an image is input in units of slices, then decomposed into several thinned images, and a predicted image is created from other thinned images in order to perform intra prediction of a thinned image.

  In the present embodiment, the MB is divided into a plurality of basic blocks in a manner different from the first embodiment, and for intra prediction of the pixels of a certain basic block, the pixels of the other basic block are referred to. A method for creating a prediction pixel will be described.

First, MB basic block division will be described with reference to FIG.
FIG. 18 is a diagram illustrating a method of dividing an MB into 4 × 4 blocks and a processing order of the divided blocks.

In the present embodiment, as shown in FIG. 18, the input 16 × 16 pixel MB is divided into 8 × 8 pixel blocks, and the corresponding pixels are extracted as follows, and the extracted pixel group is configured as a 4 × 4 block. To do. Each of these blocks is a basic block for performing prediction from the first block to the fourth block.
First block: p (2x, 2y) ... x, y = 1, 2, 3, 4
Second block: p (2x-1,2y-1) x, y = 1,2,3,4
Third block: p (2x, 2y-1) ... x, y = 1, 2, 3, 4
Fourth block: p (2x-1, 2y) ... x, y = 1, 2, 3, 4
Note that p (x, y) indicates a pixel position in the 8x8 block, and can take positions 1 to 8 in the x direction and 1 to 8 in the y direction.

  The processing order in the 8 × 8 block is performed in the order of the first block, the second block, the third block, and the fourth block.

Next, a predicted image generation method in the first block will be described with reference to FIGS. 19 and 20.
FIG. 19 is a diagram illustrating prediction target blocks and reference pixels in the first block.
FIG. 20 is a diagram showing the relationship between the first block and the reference pixel in an easy-to-understand manner by rewriting FIG.

  The prediction target pixels in the first block are ap pixels in FIG. In addition, pixels A to H, I to L, M, N to U, and V to Y can be used as reference pixels in the first block. At this time, the white pixel in FIG. 19 cannot be used as a reference pixel. Of the pixels A to H, I to L, M, N to U, and V to Y that can be used as reference pixels, every other pixel is taken as shown in FIG. To do. This is based on the idea of using a reference pixel closer to the prediction target pixel. The conventional H.264 method is used. Intra prediction is performed on a 4 × 4 pixel block in nine modes used in the H.264 / AVC encoding scheme.

Next, a predicted image generation method in the second block will be described with reference to FIGS.
FIG. 21 is a diagram illustrating prediction target blocks and reference pixels in the second block.
FIG. 22 is a diagram illustrating reference pixels around the pixel p in the second block.
FIG. 23 is a conceptual diagram of each mode when a reference pixel is generated from surrounding pixels.

  In the processing of the second block, the pixel corresponding to the first block can be used as a reference pixel.

  Here, the prediction pixel generation method will be described with reference to FIG.

  In the vicinity of the prediction target pixel p, A, C, F, and H in FIG. 22 are pixels of the first block and can be used as prediction reference pixels.

  In the intra prediction method according to the present embodiment, B, D, E, and G portions that cannot be used as reference pixels are predicted from the A, C, F, and H pixels of the first block before the pixel p is predicted. Generated as a reference pixel.

Here, each prediction reference pixel is an average of surrounding reference pixels, and is calculated by the following equation.
B = (A + C + 1) >> 1
D = (A + F + 1) >> 1
E = (C + H + 1) >> 1
G = (F + H + 1) >> 1
Here, the letters A, B, C... Representing each pixel also represent a pixel value. “+” Indicates addition of pixel values, “>>” indicates a right shift of bits, and “>> 1” indicates division by two. In parentheses, +1 means rounding.

  By calculating the prediction reference pixels B, D, E, and G using the above formula, all nine pixels around the prediction target pixel can be used as reference pixels.

  In the present invention, the prediction target pixel is predicted using nine reference pixels around the prediction target pixel.

  The prediction reference pixels around a to o are similarly generated according to the relative position to the target pixel.

  Next, a predicted pixel generation method using nine peripheral pixels will be described.

There are nine types of prediction pixel generation modes, and as shown in FIG.
Mode 0: p = (B + G + 1) >> 1
Mode 1: p = (D + E + 1) >> 1
Mode 2: p = (A + C + F + H + 2) >> 2
Mode 3: p = (C + F + 1) >> 1
Mode 4: p = (A + H + 1) >> 1
Mode 5: p = (A + B + G + H + 2) >> 2
Mode 6: p = (A + D + E + H + 2) >> 2
Mode 7: p = (B + C + F + G + 2) >> 2
Mode 8: p = (D + F + C + E + 2) >> 2
“>> 2” represents dividing by 4. In parentheses, +2 means rounding.

  Here, in consideration of the ease of realization in hardware, calculation is possible by addition and bit shift. In consideration of expansion of the conventional method, the number of modes is set to nine.

  In the third and fourth blocks to be described later, first, nine reference pixels around the pixel are generated, and then the prediction target pixel is generated. Therefore, the prediction target pixel generation method is the same as that of the third and fourth blocks. It will be the same.

Next, a predicted image generation method in the third block will be described with reference to FIGS.
FIG. 24 is a diagram illustrating prediction target blocks and reference pixels in the third block.
FIG. 25 is a diagram illustrating reference pixels around the pixel o in the third block.
FIG. 26 is a diagram illustrating reference pixels around the pixel p in the third block.

  In the processing of the third block, the pixels corresponding to the first and second blocks can be used as reference pixels.

  Here, the prediction pixel generation method will be described with reference to FIG. 25 and FIG.

  In the vicinity of the prediction target pixel o, K, B, L, M, G, and H are pixels of the first block, and I, J, D, and E are pixels of the second block.

As in the second block, A, C, F, and H portions that cannot be used as reference pixels are generated as predicted reference pixels. Each prediction reference pixel is an average of surrounding reference pixels.
A = (I + K + B + D + 2) >> 2
C = (J + B + L + E + 2) >> 2
F = (2D + M + G + 2) >> 2
H = (2E + G + N + 2) >> 2
Here, the formula for F is obtained by adding D and dividing by 4 using symmetry. The same applies to the formula for H.

Similarly, peripheral pixels C, E, and H that cannot be used as reference pixels are generated for the periphery of the pixel p (A and F are shown in the generation method for the periphery of the pixel o).
C = (A + 3B + 2) >> 2
H = (F + 3G + 2) >> 2
E = (C + H + 1) >> 1
In the expression of C, a weight of 3 is assigned to the closer B, and a weight of A is set to 1, so that the closer pixels are reflected. The same applies to the formula of H.

  If nine reference pixels around each prediction target pixel are generated, the prediction pixel generation method using the nine peripheral reference pixels is the same as in the second block.

Next, a predicted image generation method in the fourth block will be described with reference to FIGS. 27 and 28.
FIG. 27 is a diagram illustrating prediction target blocks and reference pixels in the fourth block.
FIG. 28 is a diagram illustrating reference pixels around the pixel p in the fourth block.

  In the process of the fourth block, the pixels corresponding to the first, second, and third blocks can be used as reference pixels.

  Here, the prediction pixel generation method will be described with reference to FIG.

In the vicinity of the prediction target pixel p, D and E are pixels of the first block, B is a pixel of the second block, and A and C are pixels of the third block .
In the vicinity of the pixel p of the fourth block, it is necessary to generate reference pixels at positions corresponding to F, G, and H. Based on the same concept as before, the reference pixel is generated by the following equation.
F = (A + 3D + 2) >> 2
H = (C + 3E + 2) >> 2
G = (F + H + 1) >> 1
Also for the pixels m, n, and o, reference pixels of pixels whose relative positions correspond to F, G, and H are generated in the same manner as p.

  In the above, the intra prediction processing method of the 1st block-the 4th block when the reference pixel of MB boundary is usable was explained.

  Next, for example, when the target block is located at the upper end of the picture, there is no upper MB, so the upper, upper right, and upper left reference pixels cannot be used. Also, when the target block is located at the left end of the picture, the left and upper left reference pixels cannot be used because there is no left MB.

  A prediction processing method in a case where such MB boundary reference pixels cannot be used will be described.

  In the first block, H. The same process as H.264 / AVC and the usable mode is only mode 2 (DC), and the DC prediction value is predicted as an intermediate value (128 if the number of bits of the input pixel is 8 bits).

Next, a predicted image generation method in the second block of the MB at the slice boundary will be described with reference to FIGS.
FIG. 29 is a diagram illustrating prediction target blocks and reference pixels in the second block of the MB at the slice boundary.
FIG. 30 is a diagram illustrating reference pixels around the pixel a in the second block.
FIG. 31 is a diagram illustrating reference pixels around the pixel b in the second block.
FIG. 32 is a diagram illustrating reference pixels around the pixel e in the second block.

  When the target MB is on the slice boundary, the reference pixels outside the thick frame in FIG. 29 cannot be used. A method of predicting these reference pixels will be described with reference to FIGS. 30, 31, and 32 by taking the vicinity of the prediction target pixels a, b, and e as an example.

  The reference pixels A to G around the prediction target pixel a are all set to the same value as H.

The prediction method of the reference pixels A, B, and C around the prediction target pixel b is performed as follows. For D, E, and G, the average of the pixels at both ends may be taken as described above.
A = F
C = H
B = (A + C + 1) >> 1
Here, A copies the pixel value of A using symmetry. The same applies to C.

The prediction method of the reference pixels A, D, and F around the prediction target pixel e is performed as follows. For B, E, and G, the average of the pixels at both ends described above may be taken.
A = C
F = H
D = (A + F + 1) >> 1
Next, a predicted image generation method in the third block of the MB at the slice boundary will be described with reference to FIGS.
FIG. 33 is a diagram illustrating prediction target blocks and reference pixels in the third block of the MB at the slice boundary.
FIG. 34 is a diagram illustrating reference pixels around the pixel a in the third block.
FIG. 35 is a diagram illustrating reference pixels around the pixel d in the third block.

  When the target MB is on the slice boundary, the reference pixels outside the thick frame in FIG. 33 cannot be used. A method of predicting these reference pixels will be described with reference to FIGS. 34 and 35, taking the vicinity of the prediction target pixels a and d as an example.

The prediction method of the reference pixels A, B, and C around the prediction target pixel a is performed as follows. As described above, F and H may be an average of neighboring pixels (for example, F = (D + G + 1) >> 1).
A = D
C = E
B = (A + C + 1) >> 1
The prediction method of the reference pixels A, B, C, E, and H around the prediction target pixel d is performed as follows. F may be an average of the peripheral pixels described above.
A = D
B = A
C = (A + 3B + 2) >> 2
H = G
E = H
Next, a predicted image generation method in the fourth block of the MB at the slice boundary will be described with reference to FIGS.
FIG. 36 is a diagram illustrating prediction target blocks and reference pixels in the fourth block of the MB at the slice boundary.
FIG. 37 is a diagram illustrating reference pixels around the pixel a in the fourth block.
FIG. 38 is a diagram illustrating reference pixels around the pixel d in the fourth block.

  At the slice boundary, reference pixels outside the thick frame in FIG. 36 cannot be used. A method of predicting these reference pixels will be described with reference to FIGS. 37 and 38 by taking the vicinity of the prediction target pixels a and m as an example.

The prediction method of the reference pixels A, D, and F around the prediction target pixel a is performed as follows.
A = (3B + C + 2) >> 2
F = (3G + H + 2) >> 2
D = (A + F + 1) >> 1
The prediction method of the reference pixels A and D around the prediction target pixel m is performed as follows. F may be an average of surrounding pixels as described above.
A = (3B + C + 2) >> 2
D = A
H = (3E + C + 2) >> 2
G = H

DESCRIPTION OF SYMBOLS 11 ... 4x4 block 12 ... Reference pixel 100 ... Video coding apparatus 101 ... Thinned image creation part 102 ... Thinned image buffer 103 ... First intra prediction processing part 104 ... Prediction error calculation processing part 105 ... Orthogonal transformation processing part 106 ... Quantum Processing unit 107 ... Inverse quantization processing unit 108 ... Inverse orthogonal transform processing unit 109 ... Reconstructed image creation unit 110 ... Reconstructed image buffer 111 ... Second intra prediction image creation 112 ... Entropy encoding processing unit 151 ... Input image 152: Bit stream 201 ... Encoding target macroblock 211 ... Upward proximity pixel 212 ... Left proximity pixel 301 ... Encoding target pixel block 501 ... Slice unit input image 511 ... First decimation image 512 ... Second decimation image 513 ... Third thinned image 514... Fourth thinned image 601... Reconstructed image 6 of the first thinned image 1 ... predicted image 612 ... pixel Pp2 of the second thinned image (n, m)
811 ... Predicted image of third thinned image 812 ... Pixel Pp3 (n, m)
1001 ... Reconstructed image of the second thinned image 1002 ... Reconstructed image of the third thinned image 1011 ... Predicted image 1012 of the fourth thinned image ... Pixel Pp4 (n, m)
1201 ... Slice header information 1202 ... Slice header information 1211 ... Code information 1212 related to the first thinned image ... Code information 1213 related to the second thinned image ... Code information 1214 related to the third thinned image ... Code information 1401 related to the fourth thinned image ... First Decoded image 1402 of the decimation image ... Decoded image 1403 of the second decimation image ... Decoded image 1403 of the third decimation image ... Decoded image 1411 of the fourth decimation image ... Combined image 1500 ... Image decoding device 1501 ... Entropy decoding processing unit 1502 ... Thinned-out image buffer 1503 ... Combined image creation unit 1551 ... Decoded image 1600 ... Image encoding process 1601 ... First image encoding device 1602 ... Second image encoding device 1603 ... Stream buffer 1604 ... Stream buffer 1605 ... Output selection circuit 160 ... Coding efficiency statistical processing unit 1631 ... Statistical data 1632 serving as an index of generated code amount and image quality in slice units ... Statistical data 1633 serving as an index of generated code amount and image quality in slice units ... Control information 1651 ... Input image 1652 ... bits Stream 1700 ... image decoding device 1701 ... entropy decoding processing unit 1702 ... output selection processing unit 1703 ... delay buffer 1731 ... determination result information 1751 ... decoded image.

Claims (4)

In encoding an image, in an image encoding device that encodes a pixel block to be encoded using intra prediction,
A thinned-out image generating unit that generates N thinned images of substantially the same size (N is a natural number of 2 or more) by sampling the partial images at equal intervals in units of partial images larger than the encoding target pixel block; ,
The Nth thinned image from the first thinned image that is sequentially input is encoded using the predicted image of the Nth thinned image that is input from the predicted image of the first thinned image that is input correspondingly. A processing unit that obtains the same image as the decoded image of the converted image;
A re-synthesized image creating unit that creates a re-synthesized image by synthesizing the decoded image and the predicted image of the first to Nth thinned images;
For the encoding target pixel block set in the first thinned image, a pixel adjacent to the set encoding target pixel block is acquired from the recombined image creation unit, and the first thinned image is obtained. A first in-screen prediction processing unit for generating a predicted image of
The predicted image of the i th (i = 2,..., N) thinned image is used as the reference image from the first thinned image to the i−1 th thinned image of the same partial image acquired from the re-synthesized image creating unit. A second in-screen prediction processing unit to be generated using the
The thinned-out image generating unit causes the pixels of the second thinned-out image to be adjacent to the pixels of the first thinned-out image in either the horizontal or vertical direction, and the pixels of the third thinned-out image are The pixels of the thinned image are adjacent to each other in the vertical or horizontal direction, and the second thinned image pixels are adjacent to each other in a direction different from the adjacent direction, and the pixels of the fourth thinned image are vertically and horizontally aligned. Adjacent to the second thinned image pixel or the third thinned image pixel in the direction,
The second intra-screen prediction processing unit generates a predicted image of the second thinned image by applying a horizontal or vertical filter to the acquired first thinned image, and the third By applying a filter in a direction different from the filter that generated the predicted image of the second thinned image to at least one of the acquired first thinned image and the second thinned image, the predicted image of the thinned image The predicted image of the fourth thinned image is included in the acquired first thinned image to third thinned image, and is adjacent to the pixels in the predicted image of the fourth thinned image in the vertical and horizontal directions. An image encoding device characterized by generating by interpolating four pixels.   In the header of each partial image, the processing unit uses a method using only the first intra-screen prediction processing unit or a method using the second intra-screen prediction processing unit. If a flag indicating a method using the second in-screen prediction processing unit is added and a flag indicating a method using the second intra-screen prediction processing unit is added, the quantization is performed when the predicted image of the first thinned image is encoded. The encoding step is set smaller than the other predicted images, and when encoding the predicted image of the i-th (i = 2,..., N) thinned image, the header of the encoding target pixel block unit is not added. The image coding apparatus according to claim 1. The image encoding device according to claim 1, wherein the first image encoding device outputs first statistical data serving as an index of generated code amount and image quality in units of slices;
In encoding an image, an image encoding apparatus that encodes a pixel block to be encoded using intra prediction, using an adjacent reference pixel on the upper or left side, and the generated code amount and image quality in slice units. A second image encoding device that outputs second statistical data serving as an index;
Based on the first statistical data and the second statistical data, the coding efficiency of the bit stream obtained from the first image coding device and the bit stream obtained from the second image coding device is more efficient. A coding efficiency statistical processing unit that outputs a control signal for selecting a bitstream that improves
A first buffer that temporarily stores a bitstream output by the first image encoding device, manages bitstream data for each slice unit, and outputs the first stream;
A second buffer for temporarily storing a bit stream output by the second image encoding device, managing bit stream data for each slice unit, and outputting the data;
An image coding apparatus comprising: an output selection unit that switches and outputs one of bit streams output from the first buffer and the second buffer in units of slices according to the control signal. In encoding an image, in an image encoding device that encodes a pixel block to be encoded using intra prediction,
In a unit of a partial image larger than the encoding target pixel block, the partial image is sampled at equal intervals, divided into 8 × 8 pixel blocks, and a first thinned image of coordinates (2x, 2y), coordinates (2x− 4 of substantially the same size of the second thinned image of 1, 2y-1), the third thinned image of coordinates (2x, 2y-1), and the fourth thinned image of coordinates (2x-1, 2y) A thinned-out image generator for creating a thinned-out image (x and y are integers of 1 to 8);
Every other pixel in the adjacent pixel on the 8 × 8 pixel block and the adjacent pixel on the left is taken, and the upper left, upper, upper right, left, right, lower left, lower and lower right of the first thinned image Mode 0 (pixels calculated by adding the pixel values of the upper reference pixel and the lower reference pixel and rounding them to 2) and mode 1 (the left reference pixel and the right reference pixel) Pixel calculated by adding and rounding the pixel value of the reference pixels of 2) and mode 2 (the upper left reference pixel, the upper right reference pixel, the lower left reference pixel, and the lower right reference pixel) Pixel value calculated by dividing the pixel value by adding and rounding by 4), mode 3 (by adding the pixel values of the upper right reference pixel and the lower left reference pixel and rounding by 2 and calculating) Mode 4), mode 4 (the upper left reference pixel and the above Pixel calculated by adding the pixel value of the lower reference pixel and rounding it to 2), mode 5 (the upper left reference pixel, the upper reference pixel, the lower reference pixel, and the lower right reference) Pixel calculated by adding the pixel values of the pixels and rounding the result and dividing by 4), mode 6 (the pixels of the upper left reference pixel, the left reference pixel, the right reference pixel, and the lower right reference pixel) Pixel calculated by dividing the value and rounding it by 4), mode 7 (the pixel values of the upper right reference pixel, the upper reference pixel, the lower reference pixel, and the lower left reference pixel are added) And a value calculated by dividing the rounded value by 4) and mode 8 (the pixel values of the upper right reference pixel, the left reference pixel, the right reference pixel, and the lower left reference pixel are added and rounded off) Pixel calculated by dividing the value by 4 A first in-screen prediction processing unit that performs intra prediction of a 4 × 4 pixel block according to 9 modes) and generates a predicted image of the first thinned image;
As the reference pixel of the second thinned image, the pixel of the generated first thinned image is used as the upper left pixel, the upper right pixel, the lower left pixel, and the lower right pixel of the pixel. the pixel above using the pixels calculated by dividing the value obtained by rounding off by adding the pixel value in the upper right of the pixel of the corresponding pixel and the pixel value of the pixel on the left of the corresponding pixel in the 2, the left of the corresponding pixel The pixel obtained by dividing the pixel value of the upper left pixel of the pixel and the pixel value of the lower left pixel of the pixel by adding and rounding the pixel value by 2 is used as the pixel of the pixel. It is used at the top right pixel of the pixel value and a value obtained by rounding off by adding the pixel value of the pixel at the lower right of the corresponding pixel calculated by dividing by 2 pixels of the corresponding pixel in the pixel under the corresponding pixel the right of the corresponding pixel and the pixel values of the lower left pixel of the corresponding pixel Using the pixel calculated by adding the pixel values of the two pixels and dividing the result by dividing by 2, the intra prediction of the 4 × 4 pixel block in the 9 mode is performed to generate the predicted image of the second thinned image A second in-screen prediction processing unit;
As reference pixels of the third thinned image, the pixels of the generated first thinned image are used for the upper and lower pixels of the pixel, and the generated pixels are used for the left and right pixels of the pixel. using pixels of the second thinned image, the upper left pixel of the corresponding pixel, the upper right pixel, a lower left pixel and the lower right pixel, the pixel on the corresponding pixel, pixels beneath the corresponding pixel, wherein Using the pixel calculated by adding the pixel value of the pixel to the left of the pixel and the pixel to the right of the pixel and dividing the result by four, the images referred to at the lower left, lower and lower right of the pixel are If not, the pixel at the lower right of the lower left pixel and the corresponding pixel of the corresponding pixel, the pixel value of the left pixel of the corresponding pixel and 2 times the pixel value of the pixel on the corresponding pixel the value rounded off by adding the pixel value of the right pixel of the pixel Using the pixels calculated by dividing the upper right of the corresponding pixel, when there is no image to be referenced to the right and lower right, the upper right pixel of the corresponding pixel is three times the pixel on the corresponding pixel The pixel value is calculated by adding the pixel value and the pixel value of the upper left pixel of the pixel and dividing the result by dividing the result by 4, and the lower right pixel of the pixel is the pixel below the pixel. 3 times the pixel value and a value obtained by rounding off by adding the pixel values of the lower left pixel of the corresponding pixel using the pixels calculated by dividing by 4, to the right of the pixel of the corresponding pixel, the upper right of the pixel the calculated Using the pixel calculated by adding the pixel value of the pixel and the pixel value of the calculated lower-left pixel and dividing the result by dividing by 2, the intra prediction of the 4 × 4 pixel block in the 9 mode is performed, and the third Third in-screen prediction processing for generating a predicted image of a thinned image And
As a reference image of the fourth thinned image, the upper left pixel, the upper pixel, the upper right pixel, the left pixel, the right pixel, the lower left pixel, the lower pixel, and the lower right pixel of the pixel are pixel of the generated first decimated image, the pixels of the second thinned-out image the generated or using pixels of the third thinned image said generating, the lower left of the corresponding pixel, an image to be referred to below, and lower right Otherwise, the pixel value at the lower left of the pixel is divided by 4 by adding the pixel value three times that of the pixel to the left of the pixel and the pixel value of the pixel at the upper left of the pixel and rounding off. using the pixels calculated Te, the pixel at the lower right of the concerning picture element, the three times the pixel value of the right pixel of the pixel and the pixel values of the upper right pixel of the corresponding pixel is rounded by adding a value using the pixels calculated by dividing by 4, the pixel under the concerning picture element, The value rounded off by adding the pixel value of the pixel at the lower right with the calculated pixel values of the lower left pixels serial calculated using the pixels calculated by dividing by 2, the intra prediction of 4 × 4 pixel block according to the 9 mode And a fourth intra-screen prediction processing unit for generating a predicted image of the fourth thinned image.

Images