JP2011259205A - Image decoding device, image encoding device, and method and program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve an encoding efficiency.SOLUTION: An encoded bit stream is processed by a reversible decoding part 52, an inverse quantization part 53 and an inverse orthogonal transformation part 54 in this order to obtain coefficient data and encoding parameter information after orthogonal transformation. The inverse orthogonal transformation part 54 inverse-transforms the coefficient data by means of a preset basis according to a position of a transformation block in a macro block indicated by the encoding parameter information to obtain predictive error data. An intra-prediction part 62 generates predictive image data. An addition part 55 adds the predictive image data to the predictive error data to obtain image data. By use of the basis set according to the position of the transformation block, the optimum inverse orthogonal transformation can be performed to improve an encoding efficiency.

Description

  The present invention relates to an image decoding device, an image encoding device, a method thereof, and a program. Specifically, an image decoding apparatus, an image encoding apparatus, a method thereof, and a program that can perform efficient decoding and encoding are provided.

  In recent years, image information has been handled as digital data. At that time, the purpose is to transmit and store information efficiently, and it is based on a method such as MPEG that compresses by orthogonal transform and motion compensation using redundancy unique to image information. Devices are becoming popular for both information distribution in broadcasting stations and information reception in general households.

  In particular, MPEG2 (ISO / IEC 13818-2) is defined as a general-purpose image coding system. MPEG2 compression is a standard that covers both interlaced and progressively scanned images, as well as standard resolution and high definition images, and is currently widely used in a wide range of professional and consumer applications. By using the MPEG2 compression method, a high compression rate and good image quality can be realized by assigning a code amount (bit rate) of 18 to 22 Mbps for, for example, a high-resolution interlaced scanned image having 1920 × 1088 pixels. It is.

  MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, but did not support encoding methods with a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. With the widespread use of mobile terminals, the need for such an encoding system is expected to increase in the future, and the MPEG4 encoding system has been standardized accordingly. Regarding the image coding system, the standard was approved as an international standard as ISO / IEC 14496-2 in December 1998.

  Further, in recent years, compared with encoding methods such as MPEG2 and MPEG4, although a large amount of calculation is required for encoding and decoding, H.264 can realize higher encoding efficiency. H.264 and MPEG-4 Part 10 (Advanced Video Coding, hereinafter referred to as H.264 / AVC) are international standards. This H. H.264 / AVC is H.264. Based on 26L, It also incorporates functions not supported by 26L.

  H. Patent Document 1 discloses that image data is more efficiently encoded using H.264 / AVC.

JP 2008-4984 A

  By the way, in intra prediction, a method called MDDT (Mode dependent directional transform) has been proposed in which conversion methods are switched in accordance with the direction of intra prediction. When such an MDDT method is used, it is difficult to improve coding efficiency unless conversion performed in accordance with the direction of intra prediction is optimized.

  Therefore, an object of the present invention is to provide an image decoding apparatus, an image encoding apparatus, a method thereof, and a program that can improve encoding efficiency.

  According to a first aspect of the present invention, encoding is performed by orthogonally transforming prediction error data, which is an error between image data and predicted image data, for each transform block and processing the coefficient data after the orthogonal transform. In an image decoding device that decodes the image data from the bit stream, a data processing unit that processes the encoded bit stream to obtain coefficient data and encoding parameter information after the orthogonal transformation, and the encoding parameter information An inverse orthogonal transform unit that obtains a prediction error by performing inverse orthogonal transform of the coefficient data using a base set in advance according to the position of the transform block in the indicated macroblock, and generates the predicted image data Add the predicted image data generated by the predicted image data generation unit to the prediction error obtained by the predicted image data generation unit and the inverse orthogonal transform unit In the image decoding apparatus and an addition unit for decoding the image data Te.

  In the image decoding apparatus according to the present invention, when inverse orthogonal transformation is performed on coefficient data after orthogonal transformation obtained by processing the coded bit stream, the image data included in the coded bit stream is decoded. Inverse orthogonal transform using the block position of the transform block in the macroblock indicated by the encoding parameter information and the base set in advance according to the block position and the prediction mode indicated by the encoding parameter information For example, the Karhunen-Loeve inverse transformation is performed. In addition, when there are a plurality of transform blocks included in the macroblock, the coefficient data after the orthogonal transform of the block using the coefficient of the lowest frequency component after the orthogonal transform of each transform block is previously stored according to the prediction mode. The Karoonen-Labe inverse transform is performed using the set basis. Further, the base used in the inverse orthogonal transform unit is a reverse example of the base used when the prediction error data is orthogonally transformed for each transform block. Such a base is provided in advance, and the prediction error data before the orthogonal transformation is generated by performing the inverse orthogonal transformation by selecting and using the basis corresponding to the block position or the like.

  According to a second aspect of the present invention, there is provided an encoding generated by orthogonally transforming prediction error data, which is an error between image data and predicted image data, for each transform block and processing the coefficient data after the orthogonal transform In an image decoding method for decoding the image data from a bit stream, a data processing step of processing the encoded bit stream to obtain coefficient data and encoding parameter information after the orthogonal transformation, and the encoding parameter information An inverse orthogonal transform step for obtaining a prediction error by performing inverse orthogonal transform of the coefficient data using a preset basis according to the position of the transform block in the indicated macroblock, and generating the predicted image data A predicted image data generating step, and adding the generated predicted image data to the prediction error obtained by the inverse orthogonal transform unit to generate the image data. In a picture decoding method is provided an addition step of decoding the data.

  According to a third aspect of the present invention, there is provided an encoding generated by orthogonally transforming prediction error data, which is an error between image data and predicted image data, for each transform block, and processing the coefficient data after the orthogonal transform. A program for causing a computer to execute image coding for decoding the image data from a bitstream, and processing the coded bitstream to obtain coefficient data and coding parameter information after the orthogonal transformation; An inverse orthogonal transform procedure for obtaining a prediction error by performing inverse orthogonal transform of the coefficient data using a preset basis according to the position of the transform block in the macroblock indicated by the encoding parameter information; A predicted image data generation procedure for generating the predicted image data, and the prediction error generated by the prediction error obtained by the inverse orthogonal transform unit. There adds the image data and the addition procedure for decoding the image data in the program to be executed by the computer.

  According to a fourth aspect of the present invention, in an image encoding device that encodes image data, a prediction unit that generates predicted image data of the image data, and prediction that is an error between the image data and the predicted image data An orthogonal unit that performs orthogonal transform of the prediction error for each transform block and performs orthogonal transform using a base set in advance according to the position of the transform block in a macro block. The image coding apparatus includes a transform unit and a data processing unit that processes output data of the orthogonal transform unit to generate a coded bitstream.

  In the image coding apparatus according to the present invention, when orthogonally transforming prediction error data indicating an error between image data and predicted image data for each transform block, the block position and block position of the transform block in the macro block and the predicted image data are obtained. A base set in advance according to the prediction mode at the time of generation is used, and orthogonal transformation, for example, Karhunen-Loeve transformation, is performed. In addition, when there are a plurality of transform blocks included in the macroblock, Karoonen-Label transform is performed on the block configured with the coefficient of the lowest frequency component after orthogonal transform in each transform block. In this Karoonen-Loeve transform, a base set in advance according to the prediction mode is used. This base is obtained by using a plurality of images prepared for base learning in advance, in each transform block for each macroblock size, each transform block size, each transform block position in the macroblock, and each prediction mode. Is an eigenvector corresponding to the eigenvalue of the matrix calculated from the prediction error data. The bases are grouped according to the distance between the bases or the distance from the reference pixel. Such a base is provided in advance, and an orthogonal transform is performed by selecting and using a base corresponding to a block position or the like. Further, the coefficient data after the orthogonal transform is subjected to processing such as quantization and lossless encoding, and an encoded bit stream is generated.

  According to a fifth aspect of the present invention, in an image encoding method for encoding image data, a predicted image data generation step for generating predicted image data of the image data, and an error between the image data and the predicted image data A subtraction process for generating prediction error data, and orthogonal transformation of the prediction error is performed for each transformation block, and the orthogonal transformation is performed using a base set in advance according to the position of the transformation block in a macroblock. The image encoding method includes an orthogonal transform step for performing the above.

  According to a sixth aspect of the present invention, there is provided a program for causing a computer to execute encoding of image data, a predicted image data generation procedure for generating predicted image data of the image data, the image data, the predicted image data, A subtraction procedure for generating prediction error data that is an error of the above, and orthogonal transformation of the prediction error for each transform block, using a base set in advance according to the position of the transform block in a macroblock, There is a program for causing the computer to execute an orthogonal transformation procedure for performing orthogonal transformation.

  The program of the present invention is, for example, a storage medium or communication medium provided in a computer-readable format to a general-purpose computer system capable of executing various program codes, such as an optical disk, a magnetic disk, a semiconductor memory, etc. Or a program that can be provided by a communication medium such as a network. By providing such a program in a computer-readable format, processing corresponding to the program is realized on the computer system.

  According to the present invention, in the orthogonal transformation performed at the time of encoding image data, orthogonal transformation is performed using a base set in advance according to the block position of the transformation block in the macroblock. In addition, in decoding of an encoded bitstream generated by processing coefficient data obtained by performing orthogonal transformation using a base set in advance according to a block position, it is included in the encoded bitstream. Inverse orthogonal transformation is performed using a base set in advance according to the block position in the macroblock indicated by the encoding parameter information, so that coefficient data after orthogonal transformation is predicted before orthogonal transformation. Error data can be restored. As described above, since orthogonal transform and inverse orthogonal transform are performed using a base corresponding to the block position in the macroblock, it is possible to perform a transform optimized according to the block position and improve coding efficiency. be able to.

It is the figure which showed the structure of the image coding apparatus. It is a figure which shows the intra prediction mode about a 4x4 pixel block. It is the figure which showed the relationship between prediction mode and a prediction error. It is a figure which shows KL conversion in an orthogonal transformation part. It is a figure which shows the structure of an orthogonal transformation part. It is a flowchart which shows an image coding process operation. It is a flowchart which shows a prediction process. It is a flowchart which shows an intra prediction process. It is a flowchart which shows the inter prediction process. It is a flowchart which shows an encoding parameter production | generation process. It is a flowchart which shows an orthogonal transformation process. It is a figure for demonstrating orthogonal transformation operation | movement. It is the figure which showed the structure of the image decoding apparatus. It is a figure which shows the structure of an inverse orthogonal transformation part. It is a flowchart which shows an image decoding process operation. It is a flowchart which shows an inverse orthogonal transformation process. It is a figure for demonstrating an inverse orthogonal transformation process. It is a flowchart which shows a prediction process. It is a flowchart which shows the learning operation | movement of a basis. It is a figure for demonstrating grouping of a basis. It is the figure which illustrated schematic structure of the television apparatus. It is the figure which illustrated schematic structure of the mobile phone. It is the figure which illustrated schematic structure of the recording / reproducing apparatus. It is the figure which illustrated schematic structure of the imaging device.

Hereinafter, modes for carrying out the invention will be described. The description will be given in the following order.
1. 1. Configuration of image encoding device 2. Configuration of orthogonal transform unit 3. Operation of image encoding device 4. Configuration of image decoding device Configuration of inverse orthogonal transform unit 6. 6. Operation of image decoding device Base learning operation 8. 8. Software processing When applied to electronic equipment

<1. Configuration of Image Encoding Device>
FIG. 1 shows the configuration of an image encoding device. The image encoding device 10 includes an analog / digital conversion unit (A / D conversion unit) 11, a screen rearrangement buffer 12, a subtraction unit 13, an orthogonal transformation unit 14, a quantization unit 15, a lossless encoding unit 16, and a storage buffer 17. The rate control unit 18 is provided. Further, the image encoding device 10 includes an inverse quantization unit 21, an inverse orthogonal transform unit 22, an addition unit 23, a deblocking filter 24, a frame memory 27, an intra prediction unit 31, a motion prediction / compensation unit 32, a predicted image / optimum A mode selection unit 33 is provided.

  The A / D converter 11 converts an analog image signal into digital image data and outputs the digital image data to the screen rearrangement buffer 12.

  The screen rearrangement buffer 12 rearranges the frames of the image data output from the A / D conversion unit 11. The screen rearrangement buffer 12 rearranges frames according to a GOP (Group of Pictures) structure related to encoding processing, and subtracts the image data after the rearrangement, the intra prediction unit 31, and the motion prediction / compensation unit. 32.

  The subtraction unit 13 is supplied with the image data output from the screen rearrangement buffer 12 and the prediction image data selected by the prediction image / optimum mode selection unit 33 described later. The subtraction unit 13 calculates prediction error data that is a difference between the image data output from the screen rearrangement buffer 12 and the prediction image data supplied from the prediction image / optimum mode selection unit 33, and sends the prediction error data to the orthogonal transformation unit 14. Output.

  The orthogonal transform unit 14 performs orthogonal transform processing on the prediction error data output from the subtraction unit 13. Moreover, the orthogonal transformation part 14 performs the orthogonal transformation process according to prediction mode, when performing intra prediction. The orthogonal transform unit 14 outputs coefficient data obtained by performing the orthogonal transform process to the quantization unit 15.

  The quantization unit 15 is supplied with coefficient data output from the orthogonal transform unit 14 and a rate control signal from a rate control unit 18 described later. The quantization unit 15 quantizes the coefficient data and outputs the quantized data to the lossless encoding unit 16 and the inverse quantization unit 21. Further, the quantization unit 15 changes the bit rate of the quantized data by switching the quantization parameter (quantization scale) based on the rate control signal from the rate control unit 18.

  The lossless encoding unit 16 is supplied with quantized data output from the quantization unit 15 and encoding parameter information from an intra prediction unit 31, a motion prediction / compensation unit 32, and a predicted image / optimum mode selection unit 33, which will be described later. Is done. The coding parameter information includes information indicating whether the prediction is intra prediction or inter prediction, macroblock information indicating the macroblock size, information regarding intra prediction, information regarding inter prediction, and the like. The lossless encoding unit 16 performs lossless encoding processing on the quantized data by, for example, variable length encoding or arithmetic encoding, generates an encoded bit stream, and outputs the encoded bit stream to the accumulation buffer 17. In addition, the lossless encoding unit 16 performs lossless encoding on the encoding parameter information and adds it to, for example, header information of the encoded bit stream. Note that the quantization unit 15 and the lossless encoding unit 16 correspond to a data processing unit that processes output data of the orthogonal transform unit 14 to generate an encoded bit stream.

  The accumulation buffer 17 accumulates the encoded bit stream from the lossless encoding unit 16. The accumulation buffer 17 outputs the accumulated encoded bit stream at a transmission rate corresponding to the transmission path.

  The rate control unit 18 monitors the free capacity of the accumulation buffer 17, generates a rate control signal according to the free capacity, and outputs the rate control signal to the quantization unit 15. The rate control unit 18 acquires information indicating the free capacity from the accumulation buffer 17, for example. The rate control unit 18 reduces the bit rate of the quantized data by the rate control signal when the free space is low. In addition, when the free capacity of the storage buffer 17 is sufficiently large, the rate control unit 18 increases the bit rate of the quantized data by the rate control signal.

  The inverse quantization unit 21 performs an inverse quantization process on the quantized data supplied from the quantization unit 15. The inverse quantization unit 21 outputs coefficient data obtained by performing the inverse quantization process to the inverse orthogonal transform unit 22.

  The inverse orthogonal transform unit 22 outputs data obtained by performing an inverse orthogonal transform process on the coefficient data supplied from the inverse quantization unit 21 to the addition unit 23.

  The adding unit 23 adds the data supplied from the inverse orthogonal transform unit 22 and the predicted image data supplied from the predicted image / optimum mode selection unit 33 to generate reference image data, and the deblocking filter 24 and the intra prediction. To the unit 31.

  The deblocking filter 24 performs a filter process for reducing block distortion that occurs when an image is encoded. The deblocking filter 24 performs a filtering process to remove block distortion from the reference image data supplied from the adding unit 23, and outputs the filtered reference image data to the frame memory 27.

  The frame memory 27 holds the filtered reference image data supplied from the deblocking filter 24.

  The intra prediction unit 31 performs an intra prediction process using the image data of the encoding target image output from the screen rearrangement buffer 12 and the reference image data supplied from the addition unit 23. The intra prediction unit 31 performs an intra prediction process for each transform block size in orthogonal transform and for each prediction mode of intra prediction. The intra prediction unit 31 outputs the generated predicted image data to the predicted image / optimum mode selection unit 33. Further, the intra prediction unit 31 generates coding parameter information related to the intra prediction process, and outputs the coding parameter information to the lossless coding unit 16 and the predicted image / optimum mode selection unit 33. The intra prediction unit 31 includes, for example, the macro block size, the transform block size, the position of the transform block in the macro block, the prediction mode, and the like in the encoding parameter information.

  The intra prediction unit 31 calculates a cost function value in each intra prediction process, and selects an intra prediction process that minimizes the calculated cost function value, that is, an optimal intra prediction process that maximizes the coding efficiency. The intra prediction unit 31 outputs the encoding parameter information and the cost value in the optimal intra prediction process, and the predicted image data generated in the optimal intra prediction process to the predicted image / optimum mode selection unit 33.

  The motion prediction / compensation unit 32 performs inter prediction processing with all the motion compensation block sizes for the macroblock, generates predicted image data, and outputs the predicted image data to the predicted image / optimum mode selection unit 33. The motion prediction / compensation unit 32 uses the filtered reference image data read from the frame memory 27 for each image of each motion compensation block size in the encoding target image read from the screen rearrangement buffer 12. To detect a motion vector. Further, the motion prediction / compensation unit 32 performs motion compensation processing on the reference image based on the detected motion vector to generate predicted image data. In addition, the motion prediction / compensation unit 32 generates encoding parameter information related to inter prediction processing, for example, encoding parameter information indicating a macroblock size, a motion compensation block size, a motion vector, and the like, and performs prediction with the lossless encoding unit 16. Output to the image / optimum mode selection unit 33.

  In addition, the motion prediction / compensation unit 32 calculates a cost function value for each motion compensation block size, and performs an inter prediction process in which the calculated cost function value is the minimum, that is, an inter prediction process in which the encoding efficiency is the highest. Select. The motion prediction / compensation unit 32 outputs the encoding parameter information and cost value in the optimal inter prediction process, and the predicted image data generated in the optimal inter prediction process to the predicted image / optimum mode selection unit 33.

  When the intra prediction unit 31 performs intra prediction processing for each transform block size or prediction mode and selects the optimal intra prediction processing, the predicted image / optimum mode selection unit 33 performs lossless encoding on the encoding parameter information with the orthogonal transform unit 14. Unit 16 outputs the predicted image data to the subtraction unit 13. When the motion prediction / compensation unit 32 performs inter prediction processing for each prediction block and selects the optimal inter prediction processing, the prediction image / optimum mode selection unit 33 converts the encoding parameter information into the orthogonal transform unit 14 and the lossless code. Output to the conversion unit 16, and output the predicted image data to the subtraction unit 13. Further, when the predicted image / optimum mode selection unit 33 selects either the optimal intra prediction process or the optimal inter prediction process to obtain the optimal mode, the cost function value of the optimal intra prediction process and the cost function of the optimal inter prediction process are selected. Compare values. The predicted image / optimum mode selection unit 33 selects a prediction process with a small cost function value, that is, a prediction process with high encoding efficiency, as the optimal mode based on the comparison result, and predicted image data generated in the selected optimal mode. Is output to the subtraction unit 13. Further, the predicted image / optimum mode selection unit 33 outputs encoding parameter information indicating the prediction process of the optimal mode to the orthogonal transform unit 14 and the lossless encoding unit 16.

<2. Configuration of orthogonal transform unit>
In the intra prediction process, prediction is performed using pixels of encoded adjacent blocks, and an optimal prediction direction is selected from a plurality of prediction directions. For example, H.M. In H.264 / AVC, four modes of prediction mode 0 to prediction mode 3 are set as prediction modes for a block of 16 × 16 pixels. In addition, nine prediction modes of prediction mode 0 to prediction mode 8 are set as prediction modes for an 8 × 8 pixel block. Further, nine prediction modes of prediction mode 0 to prediction mode 8 are set as prediction modes for a 4 × 4 pixel block.

  FIG. 2 shows a prediction mode for a block of 4 × 4 pixels, for example. Hereinafter, each prediction mode in FIG. 2 will be briefly described. In FIG. 2, the arrow indicates the prediction direction.

  FIG. 2A shows prediction mode 0 (vertical). The prediction mode 0 is a mode for generating more prediction values for reference pixels A to D adjacent in the vertical direction. FIG. 2B shows prediction mode 1 (horizontal). Prediction mode 1 is a mode in which a prediction value is generated from reference pixels I to L adjacent in the horizontal direction as indicated by arrows. FIG. 2C shows prediction mode 2 (DC). The prediction mode 2 is a mode in which a prediction value is generated from the reference pixels A to D and I to L adjacent in the vertical direction and the horizontal direction of the block among the 13 reference pixels A to M.

  FIG. 2D shows prediction mode 3 (diagonal down-left). The prediction mode 3 is a mode in which a prediction value is generated from the reference pixels A to H that are continuous in the horizontal direction among the 13 reference pixels A to M. FIG. 2E shows a prediction mode 4 (diagonal down-right). The prediction mode 4 is a mode in which a prediction value is generated by the reference pixels A to D and I to M adjacent to the block among the 13 reference pixels A to M. FIG. 2F shows prediction mode 5 (vertical-right). The prediction mode 5 is a mode in which a prediction value is generated by the reference pixels A to D and I to M adjacent to the block among the 13 reference pixels A to M.

  FIG. 2G shows prediction mode 6 (horizontal-down). The prediction mode 6 is a mode in which, as in the prediction modes 4 and 5, the prediction value is generated by the reference pixels A to D and I to M adjacent to the block among the 13 reference pixels A to M. . (H) of FIG. 2 has shown prediction mode 7 (vertical-left). In the prediction mode 7, among the 13 reference pixels A to M, four reference pixels A to D adjacent above the block, and four reference pixels E following the four reference pixels A to D are used. This is a mode for generating a predicted value by ~ G. (I) of FIG. 2 shows prediction mode 8 (horizontal-up). The prediction mode 8 is a mode in which a prediction value is generated by four reference pixels I to L adjacent to the left of the block among the 13 reference pixels A to M.

  When a prediction value is generated in this way, an error from a prediction value (prediction error) among pixels in a block often decreases as a pixel closer to a pixel used for prediction. Therefore, for example, when the prediction mode 0 (vertical) is selected as the optimum mode as shown in FIG. 3A, the pixels P0 to P3 have fewer prediction errors than the pixels P12 to P15. Also, as shown in FIG. 3B, when the prediction mode 1 (horizontal) is selected, the pixels P0, P4, P8, and P12 have fewer prediction errors than the pixels P3, P7, P11, and P15. Also, as shown in FIG. 3C, when the prediction mode 4 (diagonal down-right) is selected, the pixel P0 has less prediction error than the pixel P15. Thus, the prediction error depends on the prediction mode. Also, with respect to block positions within a macroblock, the prediction error often decreases as the block is closer to the encoded adjacent macroblock, and the prediction error also depends on the block position in the macroblock. Therefore, the orthogonal transform unit 14 optimizes the orthogonal transform of the prediction error by setting an optimum base for each position of the block that performs the orthogonal transform in the prediction mode and the macroblock.

  In the orthogonal transform, the Karoonen-Loeve transform (hereinafter referred to as “KL (Karhunen-Loeve) transform”) is a transform method that transforms the coefficients after the transform so as to be uncorrelated with each other, that is, obtains the maximum coding efficiency. It is known that this is the optimum conversion method. However, in order to know the basis of the KL transformation, it is necessary to generate a matrix based on the prediction error and to calculate an eigenvector corresponding to the eigenvalue of the generated matrix. Here, if the base is calculated each time by the image encoding device, the amount of calculation in the image encoding device becomes large. Moreover, if the calculated base is added to the encoded bitstream, the encoding efficiency is deteriorated. Therefore, an optimal base is calculated in advance for each block position and prediction mode for performing orthogonal transformation in the macroblock. If the calculated base is used in the image coding apparatus and the image decoding apparatus, it is not necessary to calculate the base in the image coding apparatus and the image decoding apparatus, and the image coding apparatus and the image decoding apparatus The configuration is simple compared to the case of calculating the base. Furthermore, since it is not necessary to transmit the base, it is possible to increase the coding efficiency using the KL transform. The basis learning will be described later.

  In the intra prediction, when the macroblock is 16 × 16 pixels, the transform block size that is the block size of the encoding target image is, for example, any block size of 16 × 16 pixels, 8 × 8 pixels, or 4 × 4 pixels. It is said. In addition, when the macro block is 8 × 8 pixels, the conversion block size is, for example, any block size of 8 × 8 pixels and 4 × 4 pixels. Therefore, as shown in FIG. 4, when the macro block is 16 × 16 pixels, the orthogonal transform unit 14 has a block size of 16 × 16 pixels, 8 × 8 pixels, and 4 × 4 pixels according to the prediction mode. It is configured so that conversion can be performed. The orthogonal transform unit 14 is configured to perform KL transform according to the prediction mode with a block size of 8 × 8 pixels and 4 × 4 pixels when the macroblock is 8 × 8 pixels. Further, when a plurality of transform blocks are provided in the macro block, the orthogonal transform unit 14 performs KL transform corresponding to the block position loc in the macro block.

  FIG. 5 illustrates the configuration of the orthogonal transform unit 14 using KL transform. The orthogonal transform unit 14 includes a 16 × 16 KL transform unit 141, an 8 × 8KL transform unit 142, a 2 × 2KL transform unit 143, 146, a 4 × 4KL transform unit 144, 145, a DCT unit 147, and a coefficient selection unit 148. Yes.

  The 16 × 16 KL conversion unit 141 performs KL conversion of prediction error data in block units of 16 × 16 pixels using an optimal base learned in advance for each prediction mode, and the obtained coefficient is used as a coefficient selection unit 148. Output to.

  The 8 × 8 KL conversion unit 142 performs KL conversion of prediction error data in units of blocks of 8 × 8 pixels, using an optimal base previously learned for each block position in the prediction mode and macroblock. When the prediction error data is data corresponding to a block size of 16 × 16 pixels, the block of 16 × 16 pixels includes four blocks of 8 × 8 pixels. Accordingly, the 8 × 8KL conversion unit 142 outputs the coefficient of the lowest frequency component in each block of 8 × 8 pixels (hereinafter referred to as “lowest frequency component coefficient”) to the 2 × 2 KL conversion unit 143 and uses the other coefficients as coefficients. The data is output to the selection unit 148. In addition, when the prediction error data is data corresponding to a block size of 8 × 8 pixels, the 8 × 8KL conversion unit 142 uses an optimal base learned in advance for each prediction mode, and uses 8 × 8 pixels. KL conversion of prediction error data is performed in block units. The 8 × 8 KL conversion unit 142 outputs the coefficient obtained by the KL conversion to the coefficient selection unit 148.

  The 2 × 2 KL conversion unit 143 uses the optimal base learned in advance for each prediction mode, and performs the KL conversion of the coefficients of 2 × 2 blocks supplied from the 8 × 8 KL conversion unit 142 to the prediction mode. The basis is used, and the obtained coefficient is output to the coefficient selection unit 148.

  The 4 × 4 KL conversion unit 144 performs KL conversion of prediction error data in units of 4 × 4 pixel blocks, using an optimal base learned in advance for each block position in the prediction mode and macroblock. When the prediction error data is data corresponding to a block size of 16 × 16 pixels, the 16 × 16 pixel block includes 16 4 × 4 pixel blocks. Therefore, the 4 × 4KL conversion unit 144 outputs the lowest frequency component coefficient in each block of 4 × 4 pixels to the 4 × 4KL conversion unit 145 and outputs the other coefficients to the coefficient selection unit 148. When the prediction error data is data corresponding to a block size of 8 × 8 pixels, the block of 8 × 8 pixels includes four 4 × 4 pixel blocks. Therefore, the 4 × 4 KL conversion unit 144 outputs the lowest frequency component coefficient in each block of 4 × 4 pixels to the 2 × 2 KL conversion unit 146 and outputs the other coefficients to the coefficient selection unit 148.

  The 4 × 4 KL conversion unit 145 performs KL conversion on the block of the lowest frequency component coefficient for 4 × 4 blocks supplied from the 4 × 4 KL conversion unit 144 using the optimal base learned in advance for each prediction mode. This is performed using a base corresponding to the prediction mode indicated by the 4 × 4KL conversion unit 144. The 4 × 4 KL conversion unit 145 outputs the coefficient obtained by the KL conversion to the coefficient selection unit 148.

  The 2 × 2 KL conversion unit 146 performs KL conversion on the block of the lowest frequency component coefficient for 2 × 2 blocks supplied from the 4 × 4 KL conversion unit 144 using the optimum base learned in advance for each prediction mode. This is done using the basis corresponding to the prediction mode. The 2 × 2 KL conversion unit 146 outputs the coefficient obtained by the KL conversion to the coefficient selection unit 148.

  The DCT unit 147 performs discrete cosine transform on the prediction error data and outputs the obtained coefficient to the coefficient selection unit 148.

  The coefficient selection unit 148 selects a coefficient according to the macroblock size and the block size corresponding to the transform block size, that is, the prediction error data. When the macro block size is 16 × 16 pixels, the coefficient selection unit 148 is the coefficient output from the 16 × 16 KL conversion unit 141, the coefficient output from the 8 × 8 KL conversion unit 142 and the 2 × 2 KL conversion unit 143, 4 One of the coefficients output from the × 4KL conversion unit 144 and the 4 × 4KL conversion unit 145 is selected based on the conversion block size. The coefficient selection unit 148 outputs the selected coefficient to the quantization unit 15.

  In addition, when the macro block size is 8 × 8 pixels, the coefficient selection unit 148 is a coefficient output from the 8 × 8KL conversion unit 142, and a coefficient output from the 4 × 4KL conversion unit 144 and the 2 × 2KL conversion unit 146. Is selected based on the transform block size. The coefficient selection unit 148 outputs the selected coefficient to the quantization unit 15. Note that the coefficient selection unit 148, when the encoding parameter information supplied from the predicted image / optimum mode selection unit 33 indicates the inter prediction mode, the coefficient output from the DCT unit 147 is quantized by the quantization unit 15 Output to.

<3. Operation of Image Encoding Device>
Next, the image encoding processing operation will be described. FIG. 6 is a flowchart showing the image encoding processing operation. In step ST11, the A / D converter 11 performs A / D conversion on the input image signal.

  In step ST12, the screen rearrangement buffer 12 performs image rearrangement. The screen rearrangement buffer 12 stores the image data supplied from the A / D conversion unit 11, and rearranges from the display order of each picture to the encoding order.

  In step ST13, the subtraction unit 13 generates prediction error data. The subtraction unit 13 calculates a difference between the image data of the images rearranged in step ST12 and the predicted image data selected by the predicted image / optimum mode selection unit 33, and generates prediction error data. The prediction error data has a smaller data amount than the original image data. Therefore, the data amount can be compressed as compared with the case where the image is encoded as it is.

  In step ST14, the orthogonal transform unit 14 performs an orthogonal transform process. The orthogonal transformation unit 14 performs orthogonal transformation on the prediction error data supplied from the subtraction unit 13. For example, the orthogonal transform unit 14 performs orthogonal transform such as Karhunen-Loeve transform or discrete cosine transform on the prediction error data, and outputs coefficient data. Details of the operation of the orthogonal transform unit 14 will be described later.

  In step ST15, the quantization unit 15 performs a quantization process. The quantization unit 15 quantizes the coefficient data. At the time of quantization, rate control is performed as described in the process of step ST26 described later.

  In step ST16, the inverse quantization unit 21 performs an inverse quantization process. The inverse quantization unit 21 inversely quantizes the coefficient data quantized by the quantization unit 15 with characteristics corresponding to the characteristics of the quantization unit 15.

  In step ST17, the inverse orthogonal transform unit 22 performs an inverse orthogonal transform process. The inverse orthogonal transform unit 22 performs inverse orthogonal transform on the coefficient data inversely quantized by the inverse quantization unit 21 with characteristics corresponding to the characteristics of the orthogonal transform unit 14.

  In step ST18, the adding unit 23 generates reference image data. The adder 23 adds the predicted image data supplied from the predicted image / optimum mode selection unit 33 and the data after inverse orthogonal transformation of the block position corresponding to the predicted image data to generate reference image data.

  In step ST19, the deblocking filter 24 performs a filter process. The deblocking filter 24 filters the reference image data output from the addition unit 23 to remove block distortion.

  In step ST20, the frame memory 27 stores reference image data. The frame memory 27 stores the reference image data after the filter processing.

  In step ST21, the intra prediction unit 31 and the motion prediction / compensation unit 32 each perform a prediction process. That is, the intra prediction unit 31 performs intra prediction processing in the intra prediction mode, and the motion prediction / compensation unit 32 performs motion prediction / compensation processing in the inter prediction mode. The details of the prediction process will be described later with reference to FIG. 7. With this process, the prediction process is performed in all candidate prediction modes, and the cost function values are calculated in all candidate prediction modes. The Then, based on the calculated cost function value, the optimal intra prediction process and the optimal inter prediction process are selected, and the predicted image data generated by the selected prediction process and its cost function and coding parameter information are predicted image The optimum mode selection unit 33 is supplied.

  In step ST22, the predicted image / optimum mode selection unit 33 selects predicted image data. The predicted image / optimum mode selection unit 33 determines the optimal mode with the best coding efficiency based on the cost function values output from the intra prediction unit 31 and the motion prediction / compensation unit 32. Further, the predicted image / optimum mode selection unit 33 selects the predicted image data of the determined optimal mode and supplies it to the subtraction unit 13 and the addition unit 23. As described above, the predicted image data is used for the calculations in steps ST13 and ST18.

  In step ST23, the predicted image / optimum mode selection unit 33 performs an encoding parameter information generation process. The prediction image / optimum mode selection unit 33 outputs the encoding parameter information regarding the selected prediction image data to the orthogonal transform unit 14 and the lossless encoding unit 16 as the encoding parameter information of the optimal mode.

  In step ST24, the lossless encoding unit 16 performs a lossless encoding process. The lossless encoding unit 16 performs lossless encoding on the quantized data output from the quantization unit 15. That is, lossless encoding such as variable length encoding or arithmetic encoding is performed on the quantized data, and the data is compressed. At this time, the encoding parameter information supplied to the lossless encoding unit 16 in step ST23 described above is also losslessly encoded. Further, lossless encoded data such as encoding parameter information is added to the header information of the encoded bitstream generated by lossless encoding of the quantized data.

  In step ST25, the accumulation buffer 17 performs an accumulation process. The accumulation buffer 17 accumulates the encoded bit stream output from the lossless encoding unit 16. The encoded bit stream stored in the storage buffer 17 is appropriately read and transmitted to the decoding side via the transmission path.

  In step ST26, the rate control unit 18 performs rate control. The rate control unit 18 controls the rate of the quantization operation of the quantization unit 15 so that overflow or underflow does not occur in the accumulation buffer 17 when the encoded bit stream is accumulated in the accumulation buffer 17.

  Next, the prediction process in step ST21 in FIG. 6 will be described with reference to the flowchart in FIG.

  In step ST31, the intra prediction unit 31 performs an intra prediction process. The intra prediction unit 31 performs an intra prediction process on the image of the block to be processed in all candidate prediction modes. In the intra prediction process, the reference image data supplied from the adding unit 23 is used. Intra prediction, as will be described later, intra prediction processing is performed in each prediction mode, and a cost function value in each prediction mode is calculated. Then, based on the calculated cost function value, the intra prediction process with the highest coding efficiency is selected.

  In step ST32, the motion prediction / compensation unit 32 performs inter prediction. The motion prediction / compensation unit 32 uses the reference image data after filter processing stored in the frame memory 27 to perform inter prediction processing with each motion compensation block size. In inter prediction, inter prediction processing is performed with each motion compensation block size, and a cost function value in each prediction block is calculated. Then, based on the calculated cost function value, the inter prediction process with the highest coding efficiency is selected.

  Next, the intra prediction process in step ST31 in FIG. 7 will be described with reference to the flowchart in FIG.

  In step ST41, the intra prediction unit 31 temporarily performs an intra prediction process with each prediction mode and transform block size. The intra prediction unit 31 performs processing from generation of prediction image data and generation of prediction error data to lossless encoding using reference image data supplied from the addition unit 23 in each prediction mode and transform block size. . Note that the intra prediction unit 31 outputs coding parameter information related to the intra prediction process to the orthogonal transform unit 14 and the lossless encoding unit 16 in each intra prediction process.

  In step ST42, the intra prediction unit 31 calculates a cost function value for each prediction mode and each transform block size. The cost function value is H.264. As defined by JM (Joint Model), which is reference software in the H.264 / AVC format, this is performed based on either the High Complexity mode or the Low Complexity mode.

That is, in the High Complexity mode, as the process of step ST41, the process up to the lossless encoding process is performed for each prediction mode and transform block size, and the cost function value represented by the following equation (1) is calculated for each prediction. Calculate for mode and transform block size.
Cost (Mode∈Ω) = D + λ · R (1)

  Ω indicates the entire set of prediction modes and transform block sizes that are candidates for encoding the block or macroblock. D indicates the difference energy (distortion) between the reference image and the input image when encoding is performed in the prediction mode and the transform block size. R is a generated code amount including orthogonal transform coefficients and coding parameter information, and λ is a Lagrange multiplier given as a function of the quantization parameter QP.

  That is, in order to perform encoding in High Complexity Mode, in order to calculate the parameters D and R, it is necessary to perform temporary encoding processing once with all candidate prediction modes and transform block sizes. Computation amount is required.

On the other hand, in the Low Complexity mode, as a process of Step ST41, for all prediction modes and transform block sizes that are candidates, prediction image generation and header bits such as encoding parameter information are calculated. The cost function value represented by the equation (2) is calculated for each prediction mode.
Cost (Mode∈Ω) = D + QPtoQuant (QP) · Header_Bit (2)

  Ω indicates the entire set of prediction modes and transform block sizes that are candidates for encoding the block or macroblock. D indicates the difference energy (distortion) between the reference image and the input image when encoding is performed in the prediction mode and the transform block size. Header_Bit is a header bit for the prediction mode and transform block size, and QPtoQuant is a function given as a function of the quantization parameter QP.

  That is, in Low Complexity Mode, it is necessary to perform prediction processing for each prediction mode and transform block size, but since it is not necessary to obtain a decoded image, it is possible to realize with a calculation amount lower than that in High Complexity Mode. .

  In step ST43, the intra prediction unit 31 determines an optimal intra prediction process. Based on the cost function value calculated in step ST42, the intra prediction unit 31 selects one intra prediction process whose cost function value is the minimum value, and determines the optimum intra prediction process.

  Next, the inter prediction process in step ST32 in FIG. 7 will be described with reference to the flowchart in FIG.

  In step ST51, the motion prediction / compensation unit 32 temporarily performs inter prediction processing with each motion compensation block size. The motion prediction / compensation unit 32 performs motion prediction using the image data and reference image data of the block to be encoded at each motion compensation block size. The motion prediction / compensation unit 32 performs motion compensation of the reference image data based on the detected motion vector, and generates predicted image data. Note that the motion prediction / compensation unit 32 outputs the encoding parameter information related to the inter prediction process to the orthogonal transform unit 14 and the lossless encoding unit 16 in each inter prediction process.

  In step ST52, the motion prediction / compensation unit 32 calculates a cost function value for each motion compensation block size. The motion prediction / compensation unit 32 calculates the cost function value using the above-described equation (1) or equation (2). In calculating the cost function value, a generated code amount including coding parameter information and the like is used. Note that the cost function value for the inter prediction mode is calculated using the H.264 standard. Evaluation of Skip Mode and Direct Mode cost function values defined in the H.264 / AVC format is also included.

  In step ST53, the motion prediction / compensation unit 32 determines an optimal inter prediction process. Based on the cost function value calculated in step ST54, the motion prediction / compensation unit 32 selects one inter prediction process having the minimum cost function value from them, and determines the optimum inter prediction process. .

  Next, with reference to the flowchart of FIG. 10, the case of an intra prediction process is demonstrated about the encoding parameter information generation process of step ST23 in FIG. The encoding parameter information is generated by the intra prediction unit 31 as described above. Further, when the optimum mode is selected by the predicted image / optimum mode selection unit 33, the prediction image / optimum mode selection unit 33 may generate encoding parameter information corresponding to the selected prediction process.

  In step ST61, the intra prediction unit 31 determines whether or not the macroblock size is 16 × 16 pixels. The intra prediction unit 31 proceeds to step ST62 when the macroblock size is 16 × 16 pixels, and proceeds to step ST63 when the macroblock size is not 16 × 16 pixels.

  In step ST62, the intra prediction unit 31 sets transform block size information for 16 × 16 pixels, and proceeds to step ST65. For example, when the transform block size when the KL transform is performed by the orthogonal transform unit 14 is 4 × 4 pixels, the intra prediction unit 31 sets transform block size information indicating the transform block size to “0”. The intra prediction unit 31 sets the transform block size when the orthogonal transform unit 14 performs KL transform to 8 × 8 pixels, sets the transform block size information to “1”, and sets the transform block size information to “2”. Set to.

  In step ST63, the intra prediction unit 31 determines whether or not the macroblock size is 8 × 8 pixels. The intra prediction unit 31 proceeds to step ST64 when the macroblock size is 8 × 8 pixels, and proceeds to step ST65 when the macroblock size is not 8 × 8 pixels.

  In step ST64, the intra prediction unit 31 sets transform block size information for 8 × 8 pixels, and proceeds to step ST65. For example, when the transform block size when the KL transform is performed by the orthogonal transform unit 14 is 4 × 4 pixels, the intra prediction unit 31 sets transform block size information to “0”. The intra prediction unit 31 sets the transform block size information to “1” when the transform block size when the orthogonal transform unit 14 performs the KL transform is 8 × 8 pixels.

  In step ST65, the intra prediction unit 31 generates encoding parameter information. The intra prediction unit 31 configures encoding parameter information using information indicating that the prediction is intra prediction, macroblock size, transformed block size information, prediction mode, block position in the macroblock, and the like.

  Next, orthogonal transform processing will be described with reference to the flowchart of FIG. In step ST71, the orthogonal transform unit 14 determines whether or not intra prediction is performed. The orthogonal transform unit 14 proceeds to step ST72 when the encoding parameter information indicates that it is intra prediction, and proceeds to step ST81 when it is not indicated that it is intra prediction.

  In step ST72, the orthogonal transform unit 14 determines whether or not the macroblock size is 16 × 16 pixels. The orthogonal transform unit 14 proceeds to step ST73 when the encoding parameter information indicates that the macroblock size is 16 × 16 pixels, and does not indicate that it is 16 × 16 pixels, that is, 8 × 8 pixels. If so, the process proceeds to step ST78.

  In step ST73, the orthogonal transform unit 14 determines whether the transform block size is 4 × 4 pixels. The orthogonal transform unit 14 proceeds to step ST74 when the coding parameter information indicates that the transform block size is 4 × 4 pixels, and proceeds to step ST75 when it does not indicate that it is 4 × 4 pixels.

  In step ST74, the orthogonal transform unit 14 performs 4 × 4 orthogonal transform processing. The orthogonal transform unit 14 performs KL transform for each block of 4 × 4 pixels using a base learned in advance according to the prediction mode and the block position. Here, since the 16 × 16 pixel block includes 16 4 × 4 pixel blocks, KL conversion is performed 16 times. Further, the orthogonal transform unit 14 selects the lowest frequency component coefficient from the coefficients obtained by performing the KL transform on the 4 × 4 pixel block, and according to the prediction mode for the selected 4 × 4 coefficient. KL conversion is performed using the basis. The orthogonal transform unit 14 outputs the coefficient obtained by performing the KL transform on the lowest frequency component coefficient and other coefficients excluding the lowest frequency component coefficient to the quantization unit 15. That is, the coefficient selection unit 148 of the orthogonal transform unit 14 illustrated in FIG. 5 selects the coefficients output from the 4 × 4KL transform units 144 and 146 and outputs the selected coefficients to the quantization unit 15.

  In step ST75, the orthogonal transform unit 14 determines whether the transform block size is 8 × 8 pixels. The orthogonal transform unit 14 proceeds to step ST76 when the coding parameter information indicates that the transform block size is 8 × 8 pixels, and proceeds to step ST77 when it does not indicate that it is 8 × 8 pixels.

  In step ST76, the orthogonal transform unit 14 performs 8 × 8 orthogonal transform processing. The orthogonal transform unit 14 performs KL transform for each block of 8 × 8 pixels using a base learned in advance according to the prediction mode and the block position. Here, since a block of 16 × 16 pixels includes four blocks of 8 × 8 pixels, KL conversion is performed four times. Further, the orthogonal transform unit 14 selects the lowest frequency component coefficient from the coefficients obtained by performing the KL transform on the 8 × 8 pixel block, and according to the prediction mode for the selected 2 × 2 coefficient. KL conversion is performed using the basis. The orthogonal transform unit 14 outputs the coefficient obtained by performing the KL transform on the lowest frequency component coefficient and other coefficients excluding the lowest frequency component coefficient to the quantization unit 15. That is, the coefficient selection unit 148 of the orthogonal transformation unit 14 illustrated in FIG. 5 selects the coefficients output from the 8 × 8 KL conversion unit 142 and the 2 × 2 KL conversion unit 143 and outputs the selected coefficients to the quantization unit 15.

  In step ST77, the orthogonal transform unit 14 performs 16 × 16 orthogonal transform processing. The orthogonal transform unit 14 performs KL transform of a block of 16 × 16 pixels using a base learned in advance according to the prediction mode, and outputs the obtained coefficient to the quantization unit 15. That is, the coefficient selection unit 148 of the orthogonal transformation unit 14 illustrated in FIG. 5 selects the coefficient output from the 16 × 16 KL conversion unit 141 and outputs the selected coefficient to the quantization unit 15.

  When proceeding from step ST72 to step ST78, the orthogonal transform unit 14 determines whether or not the transform block size is 4 × 4 pixels. The orthogonal transform unit 14 proceeds to step ST79 when the coding parameter information indicates that the transform block size is 4 × 4 pixels, and proceeds to step ST80 when it does not indicate that it is 4 × 4 pixels.

  In step ST79, the orthogonal transform unit 14 performs 4 × 4 orthogonal transform processing. The orthogonal transform unit 14 performs KL transform for each block of 4 × 4 pixels using a base learned in advance according to the prediction mode and the block position. Here, since an 8 × 8 pixel block includes four 4 × 4 pixel blocks, KL conversion is performed four times. Further, the lowest frequency component coefficient is selected from the coefficients obtained by performing the KL conversion on the 4 × 4 pixel block, and the KL conversion is performed on the selected 2 × 2 coefficient using the basis corresponding to the prediction mode. I do. The orthogonal transform unit 14 outputs the coefficient obtained by performing the KL transform on the lowest frequency component coefficient and other coefficients excluding the lowest frequency component coefficient to the quantization unit 15. That is, the coefficient selection unit 148 of the orthogonal transform unit 14 illustrated in FIG. 5 selects the coefficients output from the 4 × 4KL conversion unit 144 and the 2 × 2KL conversion unit 146 and outputs the selected coefficients to the quantization unit 15.

  In step ST80, the orthogonal transform unit 14 performs orthogonal transform in units of 8 × 8 pixel blocks. The orthogonal transform unit 14 performs KL transform of an 8 × 8 pixel block using a base learned in advance according to the prediction mode, and outputs the obtained coefficient to the quantization unit 15. That is, the coefficient selection unit 148 of the orthogonal transformation unit 14 illustrated in FIG. 5 selects the coefficient output from the 8 × 8 KL conversion unit 142 and outputs the selected coefficient to the quantization unit 15.

  In step ST81, the orthogonal transform unit 14 performs discrete cosine transform (DCT). The orthogonal transform unit 14 outputs the coefficient obtained by performing the discrete cosine transform to the quantization unit 15. That is, the coefficient selection unit 148 of the orthogonal transform unit 14 illustrated in FIG. 5 selects the coefficient output from the DCT unit 147 and outputs the selected coefficient to the quantization unit 15.

  FIG. 12 is a diagram for explaining the orthogonal transform operation. When the macroblock size is 16 × 16 pixels as shown in FIG. 12A, the transform block size is 4 × 4 pixels. As shown in FIG. 12B, the 16 macroblocks are included in the macroblock. The number in the block indicates the block position loc. The 4 × 4 KL transform unit 144 of the orthogonal transform unit 14 performs KL transform on each transform block using a base optimized for the prediction mode and block position of each block, and is shown in FIG. Thus, a coefficient for each block is generated. Further, the 4 × 4 KL conversion unit 145 forms a 4 × 4 block as shown in FIG. 12D using the lowest frequency component coefficient (indicated by hatching) in each block. The 4 × 4 KL conversion unit 145 performs KL conversion on the block using a base optimized in accordance with the prediction mode, and generates a coefficient for each block as illustrated in FIG. The orthogonal transform unit 14 outputs the coefficient shown in (E) of FIG. 12 and other coefficients excluding the lowest frequency component coefficient in (C) of FIG. 12 to the quantization unit 15.

  When the macroblock size is 8 × 8 pixels as shown in FIG. 12F and the transform block size is 4 × 4 pixels, 4 macroblocks are included in the macroblock as shown in FIG. Contains transformation blocks. The number in the block indicates the block position loc. The 4 × 4 KL transform unit 144 of the orthogonal transform unit 14 performs KL transform on each transform block using a base optimized for the prediction mode and block position of each block, and is shown in FIG. Thus, a coefficient for each block is generated. Further, the 2 × 2 KL conversion unit 146 forms a 2 × 2 block as shown in (I) of FIG. 12 using the lowest frequency component coefficient (indicated by hatching) in each block. The 2 × 2 KL conversion unit 146 performs KL conversion on the block using a base optimized in accordance with the prediction mode, and generates a coefficient for each block as illustrated in FIG. The orthogonal transform unit 14 outputs the coefficient shown in (J) of FIG. 12 and other coefficients excluding the lowest frequency component coefficient in (H) of FIG.

  As described above, according to the image coding apparatus and method of the present invention, in the orthogonal transform performed at the time of image data coding, a base set in advance according to the block position of the transform block in the macro block is used. Orthogonal transformation is performed. Therefore, it is possible to perform the conversion optimized according to the block position, and the encoding efficiency can be improved. Furthermore, by performing orthogonal transformation using a base set in advance according to not only the block position but also the prediction mode, it is possible to perform further optimized orthogonal transformation and further improve the coding efficiency. it can. Further, by improving the encoding efficiency, for example, the image quality can be improved without increasing the data amount of the encoded bit stream.

<4. Configuration of Image Decoding Device>
An encoded bit stream generated by encoding an input image is supplied to an image decoding apparatus via a predetermined transmission path, a recording medium, or the like and decoded.

  FIG. 13 shows the configuration of the image decoding apparatus. The image decoding device 50 includes a storage buffer 51, a lossless decoding unit 52, an inverse quantization unit 53, an inverse orthogonal transform unit 54, an addition unit 55, a deblocking filter 56, a screen rearrangement buffer 57, a digital / analog conversion unit ( D / A converter 58). Furthermore, the image decoding device 50 includes a frame memory 61, an intra prediction unit 62, a motion compensation unit 63, and a selector 64.

  The accumulation buffer 51 accumulates the transmitted encoded bit stream. The lossless decoding unit 52 decodes the encoded bit stream supplied from the accumulation buffer 51 by a method corresponding to the encoding method of the lossless encoding unit 16 of FIG.

  The lossless decoding unit 52 outputs the encoding parameter information obtained by decoding the header information of the encoded bitstream to the intra prediction unit 62, the motion compensation unit 63, and the deblocking filter 56. Further, the lossless decoding unit 52 sets prediction motion vector candidates using the motion vectors of the decoding target block and the decoded adjacent block. The lossless decoding unit 52 selects a motion vector from prediction motion vector candidates based on prediction motion vector selection information obtained by lossless decoding of the encoded bitstream, and uses the selected motion vector as a prediction motion vector. . Further, the lossless decoding unit 52 adds the predicted motion vector to the difference motion vector obtained by lossless decoding of the encoded bitstream to calculate the motion vector of the block to be decoded, and the motion compensation unit 63 Output.

  The inverse quantization unit 53 inversely quantizes the quantized data decoded by the lossless decoding unit 52 by a method corresponding to the quantization method of the quantization unit 15 of FIG. The inverse orthogonal transform unit 54 performs inverse orthogonal transform on the output of the inverse quantization unit 53 by a method corresponding to the orthogonal transform method of the orthogonal transform unit 14 of FIG.

  The adding unit 55 adds the data after inverse orthogonal transformation and the predicted image data supplied from the selector 64 to generate decoded image data, and outputs the decoded image data to the deblocking filter 56 and the intra prediction unit 62.

  The deblocking filter 56 performs a filtering process on the decoded image data supplied from the adding unit 55, removes block distortion, supplies the frame memory 61 for accumulation, and outputs the frame memory 61 to the screen rearrangement buffer 57.

  The screen rearrangement buffer 57 rearranges images. That is, the order of frames rearranged for the encoding order by the screen rearrangement buffer 12 in FIG. 1 is rearranged in the original display order and output to the D / A conversion unit 58.

  The D / A conversion unit 58 performs D / A conversion on the image data supplied from the screen rearrangement buffer 57 and outputs it to a display (not shown) to display an image.

  The frame memory 61 holds the decoded image data after the filtering process supplied from the deblocking filter 24.

  The intra prediction unit 62 generates a predicted image based on the encoding parameter information supplied from the lossless decoding unit 52, and outputs the generated predicted image data to the selector 64.

  The motion compensation unit 63 performs motion compensation based on the encoding parameter information and the motion vector supplied from the lossless decoding unit 52, generates predicted image data, and outputs the prediction image data to the selector 64. That is, the motion compensation unit 63 performs motion compensation on the basis of the motion vector for the reference image indicated by the reference frame information based on the motion vector and the reference frame information supplied from the lossless decoding unit 52, Prediction image data having a compensation block size is generated.

  The selector 64 supplies the predicted image data generated by the intra prediction unit 62 to the adding unit 55. Further, the selector 64 supplies the predicted image data generated by the motion compensation unit 63 to the addition unit 55.

<5. Configuration of Inverse Orthogonal Transformer>
FIG. 14 shows the configuration of the inverse orthogonal transform unit 54. The inverse orthogonal transform unit 54 includes a 16 × 16KL inverse transform unit 541, a 2 × 2KL inverse transform unit 542, 545, an 8 × 8KL inverse transform unit 543, a 4 × 4KL inverse transform unit 544, 546, an IDCT unit 547, and a data selection unit. 548.

  The 16 × 16 KL inverse conversion unit 541 performs KL inverse conversion corresponding to the KL conversion performed by the 16 × 16 KL conversion unit 141 illustrated in FIG. 5. The 16 × 16KL inverse transform unit 541 is output from the inverse quantization unit 53 using a basis corresponding to the prediction mode (optimum prediction mode) indicated by the coding parameter information of the optimum mode supplied from the lossless decoding unit 52. KL inverse transformation of the dequantized data is performed. The 16 × 16 KL inverse transform unit 541 outputs the image data obtained by performing the KL inverse transform to the data selection unit 548.

  The 2 × 2 KL inverse conversion unit 542 performs KL inverse conversion corresponding to the KL conversion performed by the 2 × 2 KL conversion unit 143 illustrated in FIG. 5. The 2 × 2 KL inverse transform unit 542 performs KL inverse transform of the dequantized data output from the inverse quantization unit 53 using a basis corresponding to the prediction mode indicated by the coding parameter information of the optimal mode. The 2 × 2 KL inverse transform unit 542 outputs the lowest frequency component coefficient obtained by performing the KL inverse transform to the 8 × 8 KL inverse transform unit 543.

  The 8 × 8KL reverse conversion unit 543 performs KL reverse conversion corresponding to the KL conversion performed by the 8 × 8KL conversion unit 143 illustrated in FIG. 5. The 8 × 8 KL inverse transform unit 543 performs KL inverse transform based on the coding parameter information of the optimal mode supplied from the lossless decoding unit 52. For example, when the macro block size is 16 × 16 pixels, the 8 × 8 KL inverse transform unit 543 uses the prediction mode indicated by the coding parameter information of the optimal mode and the basis corresponding to the block position to perform the 2 × 2 KL inverse transform. The KL inverse transform between the lowest frequency component coefficient output from the unit 542 and the post-inverse quantization data output from the inverse quantization unit 53 is performed. The 8 × 8 KL inverse transform unit 543 outputs the image data obtained by performing the KL inverse transform to the data selection unit 548. Further, the 8 × 8KL inverse transform unit 543 uses the basis corresponding to the prediction mode and the block position when the macroblock size is 8 × 8 pixels, and the dequantized data output from the inverse quantization unit 53 KL inverse transform is performed, and the obtained image data is output to the data selection unit 548.

  The 4 × 4KL inverse conversion unit 544 performs KL inverse conversion corresponding to the KL conversion performed by the 4 × 4KL conversion unit 145 illustrated in FIG. The 4 × 4KL inverse transform unit 544 performs KL inverse transform on the dequantized data output from the inverse quantization unit 53 using a basis corresponding to the prediction mode indicated by the coding parameter information of the optimal mode. The 4 × 4KL inverse transform unit 544 outputs the lowest frequency component coefficient obtained by performing the KL inverse transform to the 4 × 4KL inverse transform unit 546.

  The 2 × 2 KL inverse conversion unit 545 performs KL inverse conversion corresponding to the KL conversion performed by the 2 × 2 KL conversion unit 146 illustrated in FIG. 5. The 2 × 2 KL inverse transform unit 545 performs KL inverse transform on the dequantized data output from the inverse quantization unit 53 using a basis corresponding to the prediction mode indicated by the coding parameter information of the optimal mode. The 2 × 2 KL inverse transform unit 545 outputs the lowest frequency component coefficient obtained by performing the KL inverse transform to the 4 × 4 KL inverse transform unit 546.

  The 4 × 4KL inverse conversion unit 546 performs KL inverse conversion corresponding to the KL conversion performed by the 4 × 4KL conversion unit 144 illustrated in FIG. 5. The 4 × 4KL inverse transform unit 546 performs KL inverse transform based on the coding parameter information of the optimal mode supplied from the lossless decoding unit 52. For example, when the macro block size is 16 × 16 pixels, the 4 × 4 KL inverse transform unit 546 uses the prediction mode indicated by the coding parameter information of the optimal mode and the basis corresponding to the block position to perform the 4 × 4 KL inverse transform. The KL inverse transform between the lowest frequency component coefficient output from the unit 544 and the dequantized data output from the inverse quantization unit 53 is performed. The 4 × 4 KL inverse transform unit 546 outputs the image data obtained by performing the KL inverse transform to the data selection unit 548. In addition, when the macro block size is 8 × 8 pixels, the 4 × 4 KL inverse transform unit 546 uses the base corresponding to the prediction mode and the block position, and the lowest frequency component output from the 2 × 2 KL inverse transform unit 545. KL inverse transform is performed between the coefficient and the data after inverse quantization output from the inverse quantization unit 53. The 4 × 4 KL inverse transform unit 546 outputs the image data obtained by performing the KL inverse transform to the data selection unit 548.

  The IDCT unit 547 performs inverse discrete cosine transform using the dequantized data output from the inverse quantization unit 53 and outputs the obtained image data to the data selection unit 548.

  The data selection unit 548 selects the image data output from the 16 × 16KL inverse transform unit 541, the 8 × 8KL inverse transform unit 543, the 4 × 4KL inverse transform unit 546, and the IDCT unit 547 based on the encoding parameter information. Do. The data selection unit 548 outputs the selected image data to the addition unit 55 as prediction error data.

<6. Operation of Image Decoding Device>
Next, the image decoding processing operation performed by the image decoding device 50 will be described with reference to the flowchart of FIG.

  In step ST91, the accumulation buffer 51 accumulates the transmitted encoded bit stream. In step ST92, the lossless decoding unit 52 performs lossless decoding processing. The lossless decoding unit 52 decodes the encoded bit stream supplied from the accumulation buffer 51. That is, quantized data of each picture encoded by the lossless encoding unit 16 in FIG. 1 is obtained. In addition, the lossless decoding unit 52 performs lossless decoding of the encoding parameter information included in the header information of the encoded bitstream, and supplies the obtained encoding parameter information to the deblocking filter 56 and the selector 64. Furthermore, the lossless decoding unit 52 outputs the encoding parameter information to the intra prediction unit 62 when the encoding parameter information is information related to the intra prediction mode. Also, the lossless decoding unit 52 outputs the encoding parameter information to the motion compensation unit 63 when the encoding parameter information is information related to the inter prediction mode.

  In step ST93, the inverse quantization unit 53 performs an inverse quantization process. The inverse quantization unit 53 inversely quantizes the quantized data decoded by the lossless decoding unit 52 with characteristics corresponding to the characteristics of the quantization unit 15 in FIG.

  In step ST94, the inverse orthogonal transform unit 54 performs an inverse orthogonal transform process. The inverse orthogonal transform unit 54 performs inverse orthogonal transform corresponding to the orthogonal transform of the orthogonal transform unit 14 of FIG. 1 on the data after inverse quantization from the inverse quantization unit 53.

  In step ST95, the adding unit 55 generates decoded image data. The adder 55 adds the prediction error data obtained by performing the inverse orthogonal transform process and the prediction image data selected in step ST99 described later to generate decoded image data. As a result, the original image is decoded.

  In step ST96, the deblocking filter 56 performs a filtering process. The deblocking filter 56 performs a filtering process on the decoded image data output from the adding unit 55 to remove block distortion included in the decoded image.

  In step ST97, the frame memory 61 performs storage processing of decoded image data.

  In step ST98, the intra prediction unit 62 and the motion compensation unit 63 perform prediction processing. The intra prediction unit 62 and the motion compensation unit 63 perform prediction processing corresponding to the encoding parameter information supplied from the lossless decoding unit 52, respectively.

  That is, when the encoding parameter information supplied from the lossless decoding unit 52 indicates intra prediction, the intra prediction unit 62 performs intra prediction processing based on the encoding parameter information, and obtains predicted image data. Generate. Also, when the encoding parameter information supplied from the lossless decoding unit 52 indicates inter prediction, the motion compensation unit 63 performs motion compensation based on the encoding parameter information and generates predicted image data. .

  In step ST99, the selector 64 selects predicted image data. That is, the selector 64 selects the prediction image data supplied from the intra prediction unit 62 and the prediction image data generated by the motion compensation unit 63 and supplies the selection image data to the addition unit 55. As described above, the selector 64 performs the reverse operation in step ST95. It is added to the output of the orthogonal transformation unit 54.

  In step ST100, the screen rearrangement buffer 57 performs image rearrangement. That is, the screen rearrangement buffer 57 rearranges the order of frames rearranged for encoding by the screen rearrangement buffer 12 of the image encoding device 10 of FIG. 1 to the original display order.

  In step ST101, the D / A converter 58 D / A converts the image data from the screen rearrangement buffer 57. This image is output to a display (not shown), and the image is displayed.

  Next, inverse orthogonal transform processing will be described using the flowchart shown in FIG. In step ST111, the inverse orthogonal transform unit 54 determines whether or not intra prediction is performed. For example, the inverse orthogonal transform unit 54 determines whether the block to be decoded is intra prediction based on the encoding parameter information extracted from the encoded bitstream by the lossless decoding unit 52. The inverse orthogonal transform unit 54 proceeds to step ST112 when the encoding parameter information indicates intra prediction, and proceeds to step ST121 when it does not indicate intra prediction, that is, when it is inter prediction.

  In step ST112, the inverse orthogonal transform unit 54 determines whether or not the macroblock size is 16 × 16 pixels. The inverse orthogonal transform unit 54 proceeds to step ST113 when the encoding parameter information indicates that the macroblock size is 16 × 16 pixels, and proceeds to step ST118 when it does not indicate that it is 16 × 16 pixels.

  In step ST113, the inverse orthogonal transform unit 54 determines whether the transform block size is 4 × 4 pixels. The inverse orthogonal transform unit 54 proceeds to step ST114 when the transform block size information in the coding parameter information is “0”, and proceeds to step ST115 when it is not “0”.

  In step ST114, the inverse orthogonal transform unit 54 performs 4 × 4 inverse orthogonal transform processing. The inverse orthogonal transform unit 54 performs 4 × 4 KL inverse transform using a base learned in advance according to the prediction mode and the block position. When the macroblock size is 16 × 16 pixels, in coding, the KL conversion is performed by selecting the lowest frequency component coefficient from the coefficients obtained by performing KL conversion and KL conversion 16 times. Therefore, the inverse orthogonal transform unit 54 performs KL inverse transform on the data after inverse quantization of the lowest frequency component coefficient, using the basis corresponding to the prediction mode. Further, the inverse orthogonal transform unit 54 uses the basis corresponding to the prediction mode and the block position for 16 blocks including the lowest frequency component coefficient obtained by the KL inverse transform and the coefficients of the other components. Perform inverse transformation. The inverse orthogonal transform unit 54 outputs prediction error data obtained by performing the KL inverse transform to the addition unit 55. That is, the data selection unit 548 of the inverse orthogonal transform unit 54 shown in FIG. 14 uses the output of the 4 × 4KL inverse transform unit 544 to perform the data obtained by performing the KL inverse transform in the 4 × 4KL inverse transform unit 546. Select and output to the adder 55.

  In step ST115, the inverse orthogonal transform unit 54 determines whether the transform block size is 8 × 8 pixels. The inverse orthogonal transform unit 54 proceeds to step ST116 when the transform block size information in the coding parameter information is “1”, and proceeds to step ST117 when it is not “1”.

  In step ST116, the inverse orthogonal transform unit 54 performs 8 × 8 inverse orthogonal transform processing. The inverse orthogonal transform unit 54 performs 8 × 8 KL inverse transform using a base learned in advance according to the prediction mode and the block position. When the macroblock size is 16 × 16 pixels, in coding, the KL conversion is performed by selecting the lowest frequency component coefficient from the coefficients obtained by performing KL conversion and KL conversion four times. Therefore, the inverse orthogonal transform unit 54 performs KL inverse transform on the data after inverse quantization of the lowest frequency component coefficient, using the basis corresponding to the prediction mode. Further, the inverse orthogonal transform unit 54 uses the basis corresponding to the prediction mode and the block position for the four blocks including the lowest frequency component coefficient obtained by the KL inverse transform and the coefficients of the other components. Perform inverse transformation. The inverse orthogonal transform unit 54 outputs prediction error data obtained by performing the KL inverse transform to the addition unit 55. That is, the data selection unit 548 of the inverse orthogonal transform unit 54 illustrated in FIG. 14 uses the output of the 2 × 2KL inverse transform unit 542 to perform the KL inverse transform in the 8 × 8KL inverse transform unit 543. Select and output to the adder 55.

  In step ST117, the inverse orthogonal transform unit 54 performs 16 × 16 inverse orthogonal transform processing. The inverse orthogonal transform unit 54 performs 16 × 16 KL inverse transform using a base learned in advance according to the prediction mode. The inverse orthogonal transform unit 54 outputs prediction error data obtained by performing the KL inverse transform to the addition unit 55. That is, the data selection unit 548 of the inverse orthogonal transform unit 54 illustrated in FIG. 14 selects the data obtained by performing the KL inverse transform in the 16 × 16 KL inverse transform unit 541 and outputs the data to the adder 55.

  When the process proceeds from step ST112 to step ST118, the inverse orthogonal transform unit 54 determines whether the transform block size is 4 × 4 pixels. The inverse orthogonal transform unit 54 proceeds to step ST119 when the transform block size information in the coding parameter information is “0”, and proceeds to step ST120 when it is not “0”.

  In step ST119, the inverse orthogonal transform unit 54 performs 4 × 4 inverse orthogonal transform processing. The inverse orthogonal transform unit 54 performs 4 × 4 KL inverse transform processing using a base learned in advance according to the prediction mode and the block position. When the macroblock size is 8 × 8 pixels, in coding, the KL conversion is performed by selecting the lowest frequency component coefficient from the coefficients obtained by performing KL conversion and KL conversion four times. Therefore, the inverse orthogonal transform unit 54 performs KL inverse transform on the data after inverse quantization of the lowest frequency component coefficient, using the basis corresponding to the prediction mode. Further, the inverse orthogonal transform unit 54 uses the basis corresponding to the prediction mode and the block position for the four blocks including the lowest frequency component coefficient obtained by the KL inverse transform and the coefficients of the other components. Perform inverse transformation. The inverse orthogonal transform unit 54 outputs prediction error data obtained by performing the KL inverse transform to the addition unit 55. That is, the data selection unit 548 of the inverse orthogonal transform unit 54 illustrated in FIG. 14 uses the output of the 2 × 2KL inverse transform unit 545 to perform the data obtained by performing the KL inverse transform in the 4 × 4KL inverse transform unit 546. Select and output to the adder 55.

  In step ST120, the inverse orthogonal transform unit 54 performs 8 × 8 inverse orthogonal transform processing. The inverse orthogonal transform unit 54 performs 8 × 8 KL inverse transform using a base learned in advance according to the prediction mode. The inverse orthogonal transform unit 54 outputs prediction error data obtained by performing the KL inverse transform to the addition unit 55. That is, the data selection unit 548 of the inverse orthogonal transform unit 54 illustrated in FIG. 14 selects the data obtained by performing the KL inverse transform in the 8 × 8KL inverse transform unit 543 and outputs the selected data to the adder 55.

  In step ST121, the inverse orthogonal transform unit 54 performs inverse discrete cosine transform (IDCT). The inverse orthogonal transform unit 54 outputs the coefficient obtained by performing the inverse discrete cosine transform to the addition unit 55. That is, the data selection unit 548 of the inverse orthogonal transform unit 54 illustrated in FIG. 14 selects the data output from the IDCT unit 547 and outputs the data to the addition unit 55.

  FIG. 17 is a diagram for explaining the inverse orthogonal transform operation, and illustrates the inverse orthogonal transform of the transform coefficient generated by the orthogonal transform operation of FIG.

  For example, the macroblock size is 16 × 16 pixels and the conversion block size is 4 × 4 pixels. In this case, the 4 × 4KL inverse transform unit 544 uses the basis corresponding to the prediction mode indicated by the coding parameter information of the optimal mode, and performs the KL-transformed data (inverse quantum) of the lowest frequency component coefficient shown in FIG. KL inverse transform of the data). The 4 × 4KL inverse transform unit 544 generates the coefficient of the lowest frequency component shown in FIG. 17B by the KL inverse transform. As shown in FIG. 17C, the 4 × 4KL inverse transform unit 546 returns the lowest frequency component coefficient and other KL-transformed data (inverse quantized data) to coefficients for each block. Further, as shown in FIG. 17D, the 4 × 4 KL inverse transform unit 546 uses the prediction mode indicated by the encoding parameter information and the basis corresponding to the block position to perform KL every 16 4 × 4 blocks. Inverse transformation is performed to generate prediction error data shown in FIG. The data selection unit 548 selects the generated prediction error data and outputs it to the addition unit 55.

  Also assume that the macroblock size is 8 × 8 pixels and the transform block size is 4 × 4 pixels. In this case, the 2 × 2 KL inverse transform unit 545 uses the basis corresponding to the prediction mode indicated by the coding parameter information of the optimum mode, and performs the KL-transformed data (inverse of the lowest frequency component coefficient shown in FIG. 17F). KL inverse transform of (quantized data) is performed. The 2 × 2 KL inverse transform unit 545 generates the lowest frequency component coefficient shown in FIG. 17G by the KL inverse transform. As shown in FIG. 17H, the 4 × 4KL inverse transform unit 546 returns the lowest frequency component coefficient and other KL-transformed data (inverse quantized data) to coefficients for each block. Further, as shown in (I) of FIG. 17, the 4 × 4KL inverse transform unit 546 uses the prediction mode indicated by the encoding parameter information and the base corresponding to the block position to perform KL for every 4 × 4 blocks. Inverse transformation is performed to generate prediction error data shown in FIG. The data selection unit 548 selects the generated prediction error data and outputs it to the addition unit 55.

  Next, the prediction process in step ST98 in FIG. 15 will be described with reference to the flowchart in FIG.

  In step ST131, the lossless decoding unit 52 determines whether or not the target block is intra-coded. When the encoding parameter information obtained by performing lossless decoding is intra prediction information, the lossless decoding unit 52 supplies the encoding parameter information to the intra prediction unit 62, and proceeds to step ST132. Also, when the encoding parameter information is not intra prediction information, the lossless decoding unit 52 supplies the encoding parameter information to the motion compensation unit 63 and proceeds to step ST133.

  In step ST132, the intra prediction unit 62 performs an intra prediction process. The intra prediction unit 62 performs intra prediction using the decoded image data and the encoding parameter information supplied from the addition unit 55, and generates predicted image data.

  In step ST133, the motion compensation unit 63 performs an inter prediction process. The motion compensation unit 63 performs motion compensation on the decoded image data supplied from the frame memory 61 based on the encoding parameter information and the motion vector from the lossless decoding unit 52. Further, the motion compensation unit 63 outputs predicted image data generated by motion compensation to the selector 64.

  As described above, in the image decoding apparatus and method of the present invention, coded bits generated by processing coefficient data obtained by performing orthogonal transformation using a base set in advance according to a block position. In decoding a stream, inverse orthogonal transform is performed using a base set in advance according to the block position in the macroblock indicated by the encoding parameter information included in the encoded bitstream. Therefore, the coefficient data after orthogonal transformation can be restored to the prediction error data before orthogonal transformation, so even if orthogonal transformation is performed using the basis corresponding to the block position in the macroblock, the prediction error before orthogonal transformation is performed. You can return to data. In addition, even if encoding is performed using a basis corresponding to the prediction mode, coefficient data after orthogonal transformation is obtained by using a basis set in advance according to the prediction mode indicated by the encoding parameter information. The prediction error data before the orthogonal transformation can be restored.

<7. Base learning action>
Next, a base generating unit that generates bases used in the orthogonal transform unit 14 and the inverse orthogonal transform unit 54 in advance by a learning operation will be described. FIG. 19 is a flowchart showing a base learning operation, and the base generation unit generates a base by performing the processing shown in FIG. 19 using an image prepared for learning. As learning images, as many different images as possible are used so that learning is not biased depending on the contents of the images.

  In step ST <b> 141, the base generation unit determines whether an image that is not used for learning remains. The base generation unit proceeds to step ST142 when an image not used for learning remains, and proceeds to step ST152 when learning is performed using all images.

  In step ST142, the base generation unit determines whether there is a macroblock that is not used for learning. The base generation unit proceeds to step ST143 when macroblocks not used for learning remain in the image used for learning, and returns to step ST141 when learning is performed using all macroblocks.

  In step ST143, the base generation unit determines whether the macroblock size is 16 × 16 pixels. The base generation unit proceeds to step ST144 when the macroblock size is 16 × 16 pixels, and proceeds to step ST148 when the macroblock size is not 16 × 16 pixels.

  In step ST144, the base generation unit generates 16 × 16 prediction error data. The base generation unit performs intra prediction and generates prediction error data of 16 × 16 pixels.

In step ST145, the base generation unit calculates a symmetric matrix of 4 × 4 orthogonal transformation. The base generation unit divides the 16 × 16 prediction error data into 16 transform blocks each having 4 × 4 pixels, and calculates a symmetric matrix M for each block position of the transform block in the prediction mode and the macroblock. The base generation unit calculates the difference between the average of the 16th-order vectors and each vector as 16th-order vectors by arranging the prediction error data of the 4 × 4 pixel conversion blocks. The base generation unit calculates the symmetric matrix M by performing the calculation of Expression (3) using this difference as “q”.

  In Expression (3), “mdt” is conversion mode information that makes it possible to determine the macroblock size and the conversion block size. “Mid” is a prediction mode of intra prediction. “Loc” is the block position of the transform block within the macroblock. “Num” is the number of learning times. “T” indicates a transposed matrix.

  In step ST146, the base generation unit calculates a symmetric matrix of 8 × 8 orthogonal transformation. The base generation unit divides the 16 × 16 prediction error data into four transform blocks each having 8 × 8 pixels, and calculates a symmetric matrix M for each block position of the transform block in the prediction mode and the macroblock. The base generation unit calculates the difference between the average of the 64th order vector and each vector as a 64th order vector by arranging the prediction error data of the 8 × 8 pixel transform block. The base generation unit calculates the symmetric matrix M by performing the calculation of Expression (3) using this difference as “q”.

  In step ST147, the base generation unit calculates a 16 × 16 orthogonal transformation symmetric matrix. The base generation unit calculates the difference between the average of the 256th order vectors and each vector as a 256th order vector by arranging the prediction error data of the 16 × 16 pixel transform blocks for each prediction mode. The base generation unit calculates the symmetric matrix M for each prediction mode by performing the calculation of Expression (3) using this difference as “q”.

  When the process proceeds from step ST143 to step ST148, the base generation unit determines whether the macroblock size is 8 × 8 pixels. The base generation unit proceeds to step ST149 when the macroblock size is 8 × 8 pixels, and returns to step ST142 when the macroblock size is not 8 × 8 pixels.

  In step ST149, the base generation unit generates 8 × 8 prediction error data. The base generation unit performs intra prediction and generates prediction error data of 8 × 8 pixels.

  In step ST150, the base generation unit calculates a symmetric matrix of 4 × 4 orthogonal transformation. The base generation unit divides the 8 × 8 prediction error data into four transform blocks each having 4 × 4 pixels, and calculates a symmetric matrix M for each block position of the transform block in the prediction mode and the macroblock. The base generation unit calculates the difference between the average of the 16th-order vectors and each vector as 16th-order vectors by arranging the prediction error data of the 4 × 4 pixel conversion blocks. The base generation unit calculates the symmetric matrix M by performing the calculation of Expression (3) using this difference as “q”.

  In step ST151, the base generation unit calculates a symmetric matrix of 8 × 8 orthogonal transformation. The base generation unit calculates the difference between the average of the 64th order vectors and each vector as a 64th order vector by arranging the prediction error data of the 8 × 8 pixel transform block for each prediction mode. The base generation unit calculates the symmetric matrix M for each prediction mode by performing the calculation of Expression (3) using this difference as “q”.

  In step ST152, the base generation unit calculates the base of the KL conversion. The base generation unit obtains eigenvectors corresponding to the eigenvalues of each symmetric matrix M, arranges the eigenvectors in the order of the eigenvalue magnitudes, and uses them as the basis of the KL transformation.

  By performing such processing, a base for performing KL conversion by the 16 × 16 KL conversion unit 141, the 8 × 8KL conversion unit 142, the 2 × 2KL conversion units 143 and 146, and the 4 × 4KL conversion units 144 and 145 can be generated. . Also, by calculating the inverse matrix of each base, the 16 × 16KL inverse transform unit 541, 2 × 2KL inverse transform unit 542, 545, 8 × 8KL inverse transform unit 543, 4 × 4KL inverse transform unit 544, 546 A base for performing KL inverse transformation can be generated.

  Furthermore, when the base for performing KL transformation or KL inverse transformation of each block is stored in each of the image encoding device and the image decoding device for each macroblock size, each prediction mode, and each block position in the macroblock, The number of bases to memorize increases. That is, a memory with a large capacity is required. Therefore, base grouping is performed to reduce the bases to be stored.

  Next, two methods will be exemplified for the grouping method. The first method calculates the Euclidean distance between the bases obtained by learning, groups those having a small distance, and replaces them with one base representing a plurality of bases in the group. By performing grouping in this way, the number of bases can be reduced.

  The second method is a method of grouping according to the distance of the reference pixel. As shown in FIG. 20, in the prediction mode 0 (Vertical), for example, the blocks of Group1 = {P4, P5, P6, P7} have the same distance from the reference pixel. In such a case, the prediction errors of the pixels P4, P5, P6, and P7 often have similar characteristics. Therefore, all the Groups 1 adopt the same base. Similarly, the groups 0, 2, and 3 can be reduced from 16 types to 4 types by adopting the same base.

  Similarly, in the prediction mode 1 (horizontal), for example, the block of Group1 = {P1, P5, P9, P13} has the same positional relationship (or distance) from the reference pixel. In such a case, the prediction errors of the pixels P1, P5, P9, and P13 often have similar characteristics. Therefore, all the Groups 1 adopt the same base. Similarly, the groups 0, 2, and 3 can be reduced from 16 types to 4 types by adopting the same base.

  In prediction mode 4 (diagonal down-right), the positional relationship between the reference pixel and each block is not the same. However, by rotating 90 degrees, P3 and P12 have the same positional relationship with the reference pixel. Therefore, {P1, P4}}, {P2, P8}, {P6, P9}, {P7, P13}, {P11, P14}, which have the same positional relationship with the reference pixel by rotating 90 degrees, respectively Group and adopt the same basis.

  Further, since the positional relationship between the reference pixel and each block when the prediction mode 0 (Vertical) is rotated 90 degrees is equal to the prediction mode 1 (horizontal), the prediction mode 0 (Vertical) and the prediction mode 1 (horizontal) If the groups are grouped, the base can be further reduced.

<8. For software processing>
The series of processes described in the specification can be executed by hardware, software, or a combined configuration of both. When processing by software is executed, a program in which a processing sequence is recorded is installed and executed in a memory in a computer incorporated in dedicated hardware. Alternatively, the program can be installed and executed on a general-purpose computer capable of executing various processes.

  For example, the program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium. Alternatively, the program is temporarily or permanently stored on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored (recorded). Such a removable recording medium can be provided as so-called package software.

  The program is installed on the computer from the removable recording medium as described above, or is wirelessly transferred from the download site to the computer, or is wired to the computer via a network such as a LAN (Local Area Network) or the Internet. The computer can receive the program transferred in this manner and install it on a recording medium such as a built-in hard disk.

  The step of describing the program includes not only the processing that is performed in time series in the order described, but also the processing that is not necessarily performed in time series but is executed in parallel or individually.

<9. When applied to electronic devices>
In the above, H.264 is used as the encoding method / decoding method. Although the H.264 / AVC format is used, the present invention can also be applied to an image encoding device / image decoding device using other encoding / decoding methods.

  Furthermore, the present invention relates to MPEG, H.264, for example. The image information (encoded bit stream) compressed by orthogonal transformation such as discrete cosine transformation and motion compensation, such as 26x, etc., and network media such as satellite broadcasting, cable TV (television), the Internet, and cellular phones. The present invention can be applied to an image encoding device and an image decoding device that are used when receiving via the storage medium or processing on a storage medium such as an optical, magnetic disk, and flash memory.

  The image encoding device 10 and the image decoding device 50 described above can be applied to any electronic device. Examples thereof will be described below.

  FIG. 21 illustrates a schematic configuration of a television device to which the present invention is applied. The television apparatus 90 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, a display unit 906, an audio signal processing unit 907, a speaker 908, and an external interface unit 909. Furthermore, the television apparatus 90 includes a control unit 910, a user interface unit 911, and the like.

  The tuner 902 selects and demodulates a desired channel from the broadcast wave signal received by the antenna 901, and outputs the obtained encoded bit stream to the demultiplexer 903.

  The demultiplexer 903 extracts video and audio packets of the program to be viewed from the encoded bit stream, and outputs the extracted packet data to the decoder 904. Further, the demultiplexer 903 supplies a packet of data such as EPG (Electronic Program Guide) to the control unit 910. If scrambling is being performed, descrambling is performed by a demultiplexer or the like.

  The decoder 904 performs a packet decoding process, and outputs video data generated by the decoding process to the video signal processing unit 905 and audio data to the audio signal processing unit 907.

  The video signal processing unit 905 performs noise removal, video processing according to user settings, and the like on the video data. The video signal processing unit 905 generates video data of a program to be displayed on the display unit 906, image data by processing based on an application supplied via a network, and the like. The video signal processing unit 905 generates video data for displaying a menu screen for selecting an item and the like, and superimposes the video data on the video data of the program. The video signal processing unit 905 generates a drive signal based on the video data generated in this way, and drives the display unit 906.

  The display unit 906 drives a display device (for example, a liquid crystal display element or the like) based on a drive signal from the video signal processing unit 905 to display a program video or the like.

  The audio signal processing unit 907 performs predetermined processing such as noise removal on the audio data, performs D / A conversion processing and amplification processing on the audio data after processing, and outputs the audio data to the speaker 908.

  The external interface unit 909 is an interface for connecting to an external device or a network, and performs data transmission / reception such as video data and audio data.

  A user interface unit 911 is connected to the control unit 910. The user interface unit 911 includes an operation switch, a remote control signal receiving unit, and the like, and supplies an operation signal corresponding to a user operation to the control unit 910.

  The control unit 910 is configured using a CPU (Central Processing Unit), a memory, and the like. The memory stores a program executed by the CPU, various data necessary for the CPU to perform processing, EPG data, data acquired via a network, and the like. The program stored in the memory is read and executed by the CPU at a predetermined timing such as when the television device 90 is activated. The CPU controls each unit so that the television device 90 operates according to the user operation by executing the program.

  Note that the television device 90 is provided with a bus 912 for connecting the tuner 902, the demultiplexer 903, the video signal processing unit 905, the audio signal processing unit 907, the external interface unit 909, and the control unit 910.

  In the television apparatus configured as described above, the decoder 904 is provided with the function of the image decoding apparatus (image decoding method) of the present application. Therefore, by using the function of the image encoding device of the present application on the broadcasting station side, even if the encoding bit stream is generated with the improvement of the encoding efficiency and the image quality, the encoding is performed by the television device. Bitstream decoding can be performed correctly.

  FIG. 22 illustrates a schematic configuration of a mobile phone to which the present invention is applied. The cellular phone 92 includes a communication unit 922, an audio codec 923, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording / reproducing unit 929, a display unit 930, and a control unit 931. These are connected to each other via a bus 933.

  An antenna 921 is connected to the communication unit 922, and a speaker 924 and a microphone 925 are connected to the audio codec 923. Further, an operation unit 932 is connected to the control unit 931.

  The mobile phone 92 performs various operations such as transmission / reception of voice signals, transmission / reception of e-mail and image data, image shooting, and data recording in various modes such as a voice call mode and a data communication mode.

  In the voice call mode, the voice signal generated by the microphone 925 is converted into voice data and compressed by the voice codec 923 and supplied to the communication unit 922. The communication unit 922 performs transmission processing of audio data, frequency conversion processing, and the like to generate a transmission signal. The communication unit 922 supplies a transmission signal to the antenna 921 and transmits it to a base station (not shown). In addition, the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and supplies the obtained audio data to the audio codec 923. The audio codec 923 performs data expansion of the audio data and conversion to an analog audio signal and outputs the result to the speaker 924.

  In addition, when mail transmission is performed in the data communication mode, the control unit 931 accepts character data input by operating the operation unit 932 and displays the input characters on the display unit 930. In addition, the control unit 931 generates mail data based on a user instruction or the like in the operation unit 932 and supplies the mail data to the communication unit 922. The communication unit 922 performs mail data modulation processing, frequency conversion processing, and the like, and transmits the obtained transmission signal from the antenna 921. In addition, the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and restores mail data. This mail data is supplied to the display unit 930 to display the mail contents.

  Note that the mobile phone 92 can also store the received mail data in a storage medium by the recording / playback unit 929. The storage medium is any rewritable storage medium. For example, the storage medium is a removable medium such as a semiconductor memory such as a RAM or a built-in flash memory, a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card.

  When transmitting image data in the data communication mode, the image data generated by the camera unit 926 is supplied to the image processing unit 927. The image processing unit 927 performs encoding processing of image data and generates encoded data.

  The demultiplexing unit 928 multiplexes the encoded data generated by the image processing unit 927 and the audio data supplied from the audio codec 923 by a predetermined method, and supplies the multiplexed data to the communication unit 922. The communication unit 922 performs modulation processing and frequency conversion processing of multiplexed data, and transmits the obtained transmission signal from the antenna 921. In addition, the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and restores multiplexed data. This multiplexed data is supplied to the demultiplexing unit 928. The demultiplexing unit 928 performs demultiplexing of the multiplexed data, and supplies the encoded data to the image processing unit 927 and the audio data to the audio codec 923. The image processing unit 927 performs a decoding process on the encoded data to generate image data. The image data is supplied to the display unit 930 and the received image is displayed. The audio codec 923 converts the audio data into an analog audio signal, supplies the analog audio signal to the speaker 924, and outputs the received audio.

  In the cellular phone device configured as described above, the image processing unit 927 is provided with the functions of the image encoding device (image encoding method) and the image decoding device (image decoding method) of the present application. Therefore, encoding efficiency and image quality can be improved when communicating image data.

  FIG. 23 illustrates a schematic configuration of a recording / reproducing apparatus to which the present invention is applied. The recording / reproducing apparatus 94 records, for example, audio data and video data of a received broadcast program on a recording medium, and provides the recorded data to the user at a timing according to a user instruction. The recording / reproducing device 94 can also acquire audio data and video data from another device, for example, and record them on a recording medium. Furthermore, the recording / reproducing device 94 decodes and outputs the audio data and video data recorded on the recording medium, thereby enabling image display and audio output on the monitor device or the like.

  The recording / reproducing apparatus 94 includes a tuner 941, an external interface unit 942, an encoder 943, an HDD (Hard Disk Drive) unit 944, a disk drive 945, a selector 946, a decoder 947, an OSD (On-Screen Display) unit 948, a control unit 949, A user interface unit 950 is included.

  The tuner 941 selects a desired channel from a broadcast signal received by an antenna (not shown). The tuner 941 outputs an encoded bit stream obtained by demodulating the received signal of a desired channel to the selector 946.

  The external interface unit 942 includes at least one of an IEEE 1394 interface, a network interface unit, a USB interface, a flash memory interface, and the like. The external interface unit 942 is an interface for connecting to an external device, a network, a memory card, and the like, and receives data such as video data and audio data to be recorded.

  The encoder 943 performs encoding by a predetermined method when the video data and audio data supplied from the external interface unit 942 are not encoded, and outputs an encoded bit stream to the selector 946.

  The HDD unit 944 records content data such as video and audio, various programs, and other data on a built-in hard disk, and reads them from the hard disk at the time of reproduction or the like.

  The disk drive 945 records and reproduces signals with respect to the mounted optical disk. An optical disk such as a DVD disk (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.), a Blu-ray disk, or the like.

  The selector 946 selects one of the encoded bit streams from the tuner 941 or the encoder 943 and supplies it to either the HDD unit 944 or the disk drive 945 when recording video or audio. Further, the selector 946 supplies the encoded bit stream output from the HDD unit 944 or the disk drive 945 to the decoder 947 at the time of reproduction of video and audio.

  The decoder 947 performs a decoding process on the encoded bitstream. The decoder 947 supplies the video data generated by performing the decoding process to the OSD unit 948. The decoder 947 outputs audio data generated by performing the decoding process.

  The OSD unit 948 generates video data for displaying a menu screen for selecting an item and the like, and superimposes the video data on the video data output from the decoder 947 and outputs the video data.

  A user interface unit 950 is connected to the control unit 949. The user interface unit 950 includes an operation switch, a remote control signal receiving unit, and the like, and supplies an operation signal corresponding to a user operation to the control unit 949.

  The control unit 949 is configured using a CPU, a memory, and the like. The memory stores programs executed by the CPU and various data necessary for the CPU to perform processing. The program stored in the memory is read and executed by the CPU at a predetermined timing such as when the recording / reproducing apparatus 94 is activated. The CPU executes the program to control each unit so that the recording / reproducing device 94 operates in accordance with the user operation.

  In the recording / reproducing apparatus configured as described above, the encoder 943 is provided with the function of the image encoding apparatus (image encoding method) of the present application, and the decoder 947 is provided with the function of the image decoding apparatus (image decoding method). Video recording and reproduction can be performed efficiently by improving the efficiency and image quality.

  FIG. 24 illustrates a schematic configuration of an imaging apparatus to which the present invention is applied. The imaging device 96 images a subject and displays an image of the subject on a display unit, or records it on a recording medium as image data.

  The imaging device 96 includes an optical block 961, an imaging unit 962, a camera signal processing unit 963, an image data processing unit 964, a display unit 965, an external interface unit 966, a memory unit 967, a media drive 968, an OSD unit 969, and a control unit 970. Have. In addition, a user interface unit 971 is connected to the control unit 970. Furthermore, the image data processing unit 964, the external interface unit 966, the memory unit 967, the media drive 968, the OSD unit 969, the control unit 970, and the like are connected via a bus 972.

  The optical block 961 is configured using a focus lens, a diaphragm mechanism, and the like. The optical block 961 forms an optical image of the subject on the imaging surface of the imaging unit 962. The imaging unit 962 is configured using a CCD or CMOS image sensor, generates an electrical signal corresponding to the optical image by photoelectric conversion, and supplies the electrical signal to the camera signal processing unit 963.

  The camera signal processing unit 963 performs various camera signal processes such as knee correction, gamma correction, and color correction on the electrical signal supplied from the imaging unit 962. The camera signal processing unit 963 supplies the image data after the camera signal processing to the image data processing unit 964.

  The image data processing unit 964 performs an encoding process on the image data supplied from the camera signal processing unit 963. The image data processing unit 964 supplies the encoded data generated by performing the encoding process to the external interface unit 966 and the media drive 968. Further, the image data processing unit 964 performs a decoding process on the encoded data supplied from the external interface unit 966 and the media drive 968. The image data processing unit 964 supplies the image data generated by performing the decoding process to the display unit 965. Further, the image data processing unit 964 superimposes the processing for supplying the image data supplied from the camera signal processing unit 963 to the display unit 965 and the display data acquired from the OSD unit 969 on the image data. To supply.

  The OSD unit 969 generates display data such as a menu screen or an icon made up of symbols, characters, or graphics and outputs it to the image data processing unit 964.

  The external interface unit 966 includes, for example, a USB input / output terminal, and is connected to a printer when printing an image. In addition, a drive is connected to the external interface unit 966 as necessary, a removable medium such as a magnetic disk or an optical disk is appropriately mounted, and a computer program read from them is installed as necessary. Furthermore, the external interface unit 966 has a network interface connected to a predetermined network such as a LAN or the Internet. For example, the control unit 970 reads the encoded data from the memory unit 967 in accordance with an instruction from the user interface unit 971, and supplies the encoded data to the other device connected via the network from the external interface unit 966. it can. Also, the control unit 970 may acquire encoded data and image data supplied from another device via the network via the external interface unit 966 and supply the acquired data to the image data processing unit 964. it can.

  As a recording medium driven by the media drive 968, any readable / writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory is used. The recording medium may be any type of removable medium, and may be a tape device, a disk, or a memory card. Of course, a non-contact IC card or the like may be used.

  Further, the media drive 968 and the recording medium may be integrated and configured by a non-portable storage medium such as a built-in hard disk drive or an SSD (Solid State Drive).

  The control unit 970 is configured using a CPU, a memory, and the like. The memory stores programs executed by the CPU, various data necessary for the CPU to perform processing, and the like. The program stored in the memory is read and executed by the CPU at a predetermined timing such as when the imaging device 96 is activated. The CPU executes the program to control each unit so that the imaging device 96 operates according to the user operation.

  In the imaging device configured as described above, the image data processing unit 964 is provided with the functions of the image encoding device (image encoding method) and the image decoding device (image decoding method) of the present application. Therefore, when the captured image is recorded in the memory unit 967, a recording medium, or the like, it is possible to improve the encoding efficiency and the image quality and efficiently record and reproduce the captured image.

  Furthermore, the present invention should not be construed as being limited to the above-described embodiments. For example, it should not be limited to the above-described macroblock size, transform block size, and prediction mode. The embodiments of the present invention disclose the present invention in the form of examples, and it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. That is, in order to determine the gist of the present invention, the claims should be taken into consideration.

  In the image decoding apparatus, the image encoding apparatus, the method and the program according to the present invention, in the orthogonal transform performed at the time of encoding image data, a base set in advance according to the block position of the transform block in the macro block is set. To perform orthogonal transformation. In addition, in decoding of an encoded bitstream generated by processing coefficient data obtained by performing orthogonal transformation using a base set in advance according to a block position, it is included in the encoded bitstream. The base set in advance according to the block position in the macroblock indicated by the encoding parameter information is used, inverse orthogonal transformation is performed, and the coefficient data after orthogonal transformation is predicted before orthogonal transformation. Returned to error data. As described above, since orthogonal transform and inverse orthogonal transform are performed using a base corresponding to the block position in the macroblock, it is possible to perform a transform optimized according to the block position and improve coding efficiency. be able to. Therefore, image information (encoded bitstream) obtained by performing encoding in block units, such as MPEG and H.26x, is transmitted via network media such as satellite broadcasting, cable TV, the Internet, and cellular phones. Therefore, the present invention is suitable for an image decoding device, an image encoding device, or the like used when transmitting / receiving data or processing on a storage medium such as an optical, magnetic disk, or flash memory.

DESCRIPTION OF SYMBOLS 10 ... Image encoding apparatus, 11 ... A / D conversion part, 12, 57 ... Screen rearrangement buffer, 13 ... Subtraction part, 14 ... Orthogonal transformation part, 15 ... Quantization , 16... Lossless encoding unit, 17, 51... Accumulation buffer, 18... Rate control unit, 21, 53, inverse quantization unit, 22, 54, inverse orthogonal transform unit, 23, 55 ... addition unit, 24, 56 ... deblocking filter, 27, 61 ... frame memory, 31, 62 ... intra prediction unit, 32, 63 ... motion prediction / compensation unit, 33 ... predicted image / optimum mode selection unit, 50 ... image decoding device, 52 ... lossless decoding unit, 58 ... D / A conversion unit, 64, 946 ... selector, 90 ..Television device, 92... Mobile phone, 94. Imaging device, 141... 16 × 16 KL converter, 142... 8 × 8 KL converter, 143, 146... 2 × 2 KL converter, 144, 145... 4 × 4 KL converter, 147. DCT section, 148... Coefficient selection section, 541... 16 × 16 KL inverse transform section, 542, 545... 2 × 2 KL inverse transform section, 543... 8 × 8 KL inverse transform section, 544, 546 KL inverse transform unit, 547 ... IDCT unit, 548 ... data selection unit, 901, 921 ... antenna, 902, 941 ... tuner, 903 ... demultiplexer, 904, 947 ..Decoder, 905 ... Video signal processing unit, 906 ... Display unit, 907 ... Audio signal processing unit, 908 ... Speaker, 909, 942, 966 ... External interface unit, 910,931 , 949, 970 ... control unit, 911, 932, 971 ... user interface unit, 912, 933, 972 ... bus, 922 ... communication unit, 923 ... voice codec, 924 ... Speaker, 925 ... Microphone, 926 ... Camera part, 927 ... Image processing part, 928 ... Demultiplexing part, 929 ... Recording / reproducing part, 930 ... Display part, 943 ... Encoder, 944... HDD section, 945... Disk drive, 948, 969... OSD section, 961... Optical block, 962. ..Image data processing unit, 965... Display unit, 967... Memory unit, 968.

Claims (16)

Prediction error data, which is an error between image data and predicted image data, is orthogonally transformed for each transform block, and the image data is decoded from an encoded bitstream generated by processing the coefficient data after the orthogonal transformation. In the image decoding device,
A data processing unit that processes the encoded bitstream to obtain coefficient data and encoding parameter information after the orthogonal transformation;
An inverse orthogonal transform unit that obtains prediction error data by performing inverse orthogonal transform of the coefficient data using a base set in advance according to the position of the transform block in the macroblock indicated by the encoding parameter information ,
A predicted image data generation unit that generates the predicted image data;
An image decoding apparatus comprising: an addition unit configured to add the prediction image data generated by the prediction image data generation unit to the prediction error data obtained by the inverse orthogonal transform unit and decode the image data.   The image decoding according to claim 1, wherein the inverse orthogonal transform unit performs the inverse orthogonal transform using a base set in advance according to a position of the transform block and a prediction mode indicated by the encoding parameter information. Device.   The inverse orthogonal transform unit, when a plurality of transform blocks are included in a macroblock based on the coding parameter information, performs orthogonal transform on coefficient data of the lowest frequency component after orthogonal transform of each transform block included in the macroblock The image decoding apparatus according to claim 2, wherein the inverse orthogonal transformation is performed on the subsequent coefficient data using a base set in advance according to a prediction mode.   The image decoding apparatus according to claim 2, wherein the base used in the inverse orthogonal transform unit is an inverse example of a base used when the prediction error data is orthogonally transformed for each transform block.   The image decoding device according to claim 1, wherein the inverse orthogonal transform unit performs Karoonen-Labe inverse transform using the base. Prediction error data, which is an error between image data and predicted image data, is orthogonally transformed for each transform block, and the image data is decoded from an encoded bitstream generated by processing the coefficient data after the orthogonal transformation. In the image decoding method,
A data processing step of processing the encoded bitstream to obtain coefficient data and encoding parameter information after the orthogonal transformation;
An inverse orthogonal transform step for obtaining a prediction error by performing inverse orthogonal transform of the coefficient data using a preset basis according to the position of the transform block in the macroblock indicated by the encoding parameter information;
A predicted image data generation step of generating the predicted image data;
An image decoding method comprising: an addition step of decoding the image data by adding the generated predicted image data to the prediction error obtained by the inverse orthogonal transform unit. Prediction error data, which is an error between image data and predicted image data, is orthogonally transformed for each transform block, and the image data is decoded from an encoded bitstream generated by processing the coefficient data after the orthogonal transformation. A program for causing a computer to execute image encoding,
A data processing procedure for processing the encoded bit stream to obtain coefficient data and encoding parameter information after the orthogonal transformation;
An inverse orthogonal transform procedure for obtaining a prediction error by performing an inverse orthogonal transform of the coefficient data using a preset basis according to the position of the transform block in the macroblock indicated by the encoding parameter information;
A predicted image data generation procedure for generating the predicted image data;
A program for causing the computer to execute an addition procedure for adding the generated predicted image data to the prediction error obtained by the inverse orthogonal transform unit and decoding the image data. In an image encoding device that encodes image data,
A prediction unit that generates predicted image data of the image data;
A subtraction unit that generates prediction error data that is an error between the image data and the predicted image data;
An orthogonal transform unit that performs orthogonal transform of the prediction error for each transform block, and performs the orthogonal transform using a preset base according to the position of the transform block in a macroblock;
An image encoding apparatus comprising: a data processing unit that processes output data of the orthogonal transform unit to generate an encoded bit stream.   The said orthogonal transformation part performs the said orthogonal transformation using the base currently preset according to the position of the said transformation | conversion block, and the prediction mode when the said prediction part produces | generates the said prediction image data. The image encoding device described.   The orthogonal transform unit, when there are a plurality of transform blocks included in the macro block, according to the prediction mode for a block using the coefficient of the lowest frequency component after orthogonal transform of each transform block included in the macro block The image encoding device according to claim 9, wherein orthogonal transformation is performed using a preset basis.   The base used in the orthogonal transform unit uses a plurality of images prepared in advance, the macroblock size, the transform block size, the position of the transform block in the macroblock, and each transform for each prediction mode The image encoding device according to claim 9, wherein the image encoding device is an eigenvector corresponding to an eigenvalue of a matrix calculated from prediction error data in a block.   The image encoding device according to claim 11, wherein the bases used in the orthogonal transform unit are grouped according to a distance between the bases.   The image encoding device according to claim 11, wherein bases used in the orthogonal transform unit are grouped according to a distance from a reference pixel.   The image encoding device according to claim 8, wherein the orthogonal transform unit performs Karoonen-Loeve transform using the base. In an image encoding method for encoding image data,
A predicted image data generation step of generating predicted image data of the image data;
A subtraction step of generating prediction error data that is an error between the image data and the predicted image data;
An image encoding provided with an orthogonal transform step for performing orthogonal transform of the prediction error for each transform block, and performing the orthogonal transform using a preset base according to the position of the transform block in a macro block Method.

A program for causing a computer to execute encoding of image data,
A predicted image data generation procedure for generating predicted image data of the image data;
A subtraction procedure for generating prediction error data that is an error between the image data and the predicted image data;
Performing orthogonal transform of the prediction error for each transform block, and causing the computer to execute an orthogonal transform procedure for performing the orthogonal transform using a preset base according to the position of the transform block in a macro block. program.

Images