JP2007288473A - Coding unit, and image processing unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a coding unit capable of reducing a load upon an arithmetic unit and providing a high quality moving image with a more compact and simpler structure. <P>SOLUTION: The coding unit includes a motion estimator 212 and a motion compensator 211, for generating an inter-predicted image by comparing the correlation values of image data between frames; a prediction image generator 215 for generating the inter-predicted image, based on the prediction result; after the start of comparing the correlation values in the motion estimator 212 and the motion compensator 211, and before the start of generating the inter-predicted image in the prediction image generator 215; an intra-prediction section 213 for determining an arithmetic mode for generating an intra-predicted image; and the prediction image generator 215 for generating the intra-predicted image, based on the above prediction result. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an encoding apparatus, and more particularly to H.264. The present invention relates to an encoding device and an image processing device that encode moving image data using H.264 / AVC.

Currently, a technology for encoding and decoding a moving image is widely used in a small terminal device such as a mobile phone or a TV phone. As a matter of course, it is desirable that such a small terminal device be as small and light as possible in order to carry it. Further, in order to spread the apparatus, it is necessary to reduce the manufacturing cost of the apparatus and reduce the price.
By the way, one of the methods for encoding moving images is H.264. H.264 / AVC. H. In H.264 / AVC, inter-prediction image generation processing that generates a predicted image by comparing correlation values of image data between frames, and a correlation value is compared between block images obtained by dividing image data for one frame. Then, intra prediction image generation processing for generating a prediction image is performed. H. In the H.264 / AVC processing, the generated inter prediction image or intra prediction image is selected to obtain a higher compression rate, and the difference between the selected image and the original image to be encoded is encoded. To do.

  By the way, intra prediction is performed for each block image composed of a predetermined number of pixels obtained by dividing one frame. The block image is configured by arranging the same number of pixels in the vertical and horizontal directions, such as 16 × 16, 8 × 8, and 4 × 4. When image data is encoded by the intra prediction method, a specific arrangement direction such as vertical or horizontal, for example, of pixels arranged in a block image to be encoded is determined, and encoded in advance along this direction. Pixel values are sequentially subtracted from pixel values of pixels included in other block images.

  In the above-described processing, the direction with the smallest sum of the prediction error values obtained as a result of subtraction along the arrangement direction among the plurality of arrangement directions is set as the subtraction direction (prediction mode) when generating the predicted image. Is done. The reason that the direction in which the sum of the prediction error values of each pixel is the smallest is set as the subtraction direction is that the amount of data after encoding is smaller as the sum of the prediction error values is smaller.

Such intra prediction increases the amount of calculation compared to conventional coding techniques, increases the processing time, and increases the load on the arithmetic unit. For this reason, the prior art aiming at reducing the load concerning the production | generation process of an intra estimated image has been proposed conventionally.
As a prior art regarding intra prediction, for example, Patent Literature 1 can be cited. Japanese Patent Application Laid-Open No. 2004-228561 reduces the load on processing for determining the calculation direction by considering the characteristics of an image for calculation or thinning out data used for calculation in such intra prediction processing. Is.
JP 2004-304724 A

However, Patent Document 1 described above may reduce the accuracy of intra prediction by thinning out data that should originally be used for intra prediction. In recent years, small-sized terminal devices have been increasingly required to provide high-quality images, and the technique described in Patent Document 1 is considered to go against such demands in order to increase the processing speed. It is done.
The present invention has been made in view of the above points, and by improving the efficiency of processing while maintaining the accuracy of intra prediction, a high-quality moving image can be provided at a higher speed or with a smaller and simple configuration. An object is to provide an encoding device and an image processing device.

  In order to solve the above problems, an encoding apparatus according to the present invention includes inter prediction image generation means for generating an inter prediction image by comparing correlation values of image data between frames, and the correlation value of image data for one frame. An intra-predicted image generating unit that generates an intra-predicted image by comparing block images obtained by dividing the image data, and an operation mode for generating the intra-predicted image by the intra-predicted image generating unit Mode determination means, and the mode determination means performs an operation for generating an intra prediction image after the start of comparison of correlation values by the inter prediction image generation means and before the start of generation of the inter prediction image. The mode is determined.

  According to such an invention, the inter prediction image generated by comparing the correlation value of the image data between the frames, and the correlation value of the image data are compared between the block images obtained by dividing the image data for one frame. In the configuration capable of generating the intra-predicted image generated in this way, the calculation for generating the intra-predicted image is performed after the start of the comparison of the correlation value by the inter-predicted image generating unit and before the generation of the inter-predicted image Mode can be determined.

For this reason, it is possible to provide an encoding device capable of encoding the image data at a higher speed or with a smaller and simpler configuration, making processing more efficient. In addition, since the present invention does not use a method such as thinning out data to be processed in order to increase the processing efficiency, the accuracy of intra prediction can be maintained and a high-quality moving image can be provided.
An image processing apparatus according to the present invention is an image processing apparatus including an encoding device and a decoding device that decodes image data encoded by the encoding device. Inter prediction image generation means for comparing data correlation values between frames to generate an inter prediction image, and comparing the correlation values of image data between block images obtained by dividing image data for one frame. Intra prediction image generation means for generating a prediction image, and after the start of comparison of correlation values by the inter prediction image generation means and before the start of generation of the inter prediction image, the intra prediction image generation means of the intra prediction image generation means Mode determining means for determining a mode of operation for generation, and the decoding device receives the image data encoded by the encoding device. And Gosuru decoding means, characterized in that it comprises, the predicted image generating means for generating a prediction image to be added with the decoded image data by said decoding means.

  According to such an invention, in the encoding device, an inter prediction image generated by comparing the correlation value of image data between frames, and the correlation value of image data can be obtained by dividing image data for one frame. In the configuration capable of generating an intra prediction image generated by comparing between block images, the intra prediction image is generated after the start of the comparison of the correlation value by the inter prediction image generation means and before the generation of the inter prediction image. The mode of operation for generation can be determined.

And the image data used as object can be encoded using the produced | generated intra estimated image, and it can decode on the decoding apparatus side.
For this reason, it is possible to provide an image processing apparatus that can process efficiently and encode and decode image data at a higher speed or with a small and simple configuration. In addition, since the present invention does not use a method such as thinning out data to be processed in order to increase the processing efficiency, the accuracy of intra prediction can be maintained and a high-quality moving image can be provided.

Embodiment 1 of an encoding apparatus and an image processing apparatus according to the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram for explaining the configuration of the encoding apparatus according to the present embodiment. The encoding apparatus sequentially inputs image data (input image data) to be encoded, which is input from the outside, one macroblock at a time. The encoding apparatus encodes a prediction error that is a difference between input image data and predicted image data obtained by predicting the input image data.

The encoding device has a configuration in which software and hardware operate together, and a configuration for processing software is referred to as a processor 1 and a hardware configuration is referred to as an accelerator 2 in the drawing.
The processor 1 quantizes the data processed by the subtractor 202 that subtracts the input image data from the predicted image, the direct transform unit 203 that performs Hadamard transform or DCT (Discrete Cosine Transform) processing on the subtraction value, and the direct transform unit 203. The quantization unit 204 includes a lossless encoding unit 205 that encodes the quantized image data.

  In intra prediction and inter prediction, in order to predict a prediction image more accurately, inter prediction and intra prediction are performed using image data in a decoded state. For this reason, the accelerator 2 includes an inverse quantization unit 206 that inversely quantizes image data that has been once quantized, an inverse orthogonal transform unit 207 that performs an inverse process to the Hadamard transform or DCT process performed earlier, and an inverse orthogonal transform. An adder 208 that adds the predicted image to the data processed by the unit 207 is provided.

The accelerator 2 includes a deblocking filter (Deblocking Filter) unit 209 that suppresses the occurrence of block noise in the image data added by the adder 208. The deblocking filter unit 209 smoothes only the block boundary of the orthogonal transform.
The accelerator 2 includes a frame storage unit 210 that stores the image data added by the adder 208. The frame storage unit 210 is a frame memory that stores data for generating an inter predicted image (including inter prediction). Furthermore, the encoding apparatus according to the present embodiment includes a motion compensation unit 211 that performs inter prediction using image data stored in the frame storage unit 210 and a motion estimation unit 212 that detects a motion vector necessary for inter prediction processing. ing. The motion compensation unit 211 generates data relating to the minimum value (also referred to as cost) of the absolute value difference sum from the input image data based on the motion vector detected by the motion estimation unit 212.

In addition, the encoding apparatus according to the present embodiment includes an intra prediction unit 213 that performs intra prediction. The intra prediction unit 213 according to the present embodiment inputs input image data in units of macroblocks and calculates a compression rate of the predicted image data without thinning out the data.
The intra-prediction unit 213 performs intra-prediction that provides the highest compression ratio among a plurality of types of operations in units of macroblocks (denoted as intra-prediction modes. Currently, there are eight intra-prediction modes). Select a mode. And the information regarding the mode of selected intra prediction is output outside.

  Inter prediction is a process for obtaining a predicted image based on the correlation between frames of image data. The correlation between frames is performed by detecting a motion vector. Intra prediction is a process of obtaining a predicted image based on the correlation between block images (macroblocks) obtained by dividing input image data from image data for one frame. Since both inter prediction and intra prediction are known techniques, further explanation is omitted here.

  Further, the accelerator 2 includes a mode determination unit 214 and a predicted image generation unit 215. The mode determination unit 214 receives the cost and motion vector output from the motion compensation unit 211 and the intra prediction mode output from the intra prediction unit 213, and selects inter prediction or intra prediction. The selection of inter prediction or intra prediction is performed by selecting the inter prediction or intra prediction that has a higher compression ratio of the encoded image data, that is, the prediction image having a smaller prediction error with respect to the input image. .

The prediction image generation unit 215 generates the inter prediction image or the intra prediction image selected by the mode determination unit 214. Although not shown, image data input to the motion compensation unit 211 and image data input to the intra prediction unit 213 are input to the predicted image generation unit 215.
In the configuration of FIG. 1, the intra prediction unit 213 branches and inputs the image data added by the adder 208 immediately before the deblocking filter unit 209. In such a configuration, a memory for storing image data for at least one macroblock is provided inside or immediately before the intra prediction unit 213. Moreover, this embodiment can also consider using input image data for intra prediction.

Such an encoding apparatus of this embodiment generates both an inter prediction image and an intra prediction image, and either one is selected and used for encoding. Since inter prediction compares the correlation of image data between frames, it cannot be executed until image data for one frame is stored in the frame storage unit 210.
On the other hand, since intra prediction processes input image data in units of macroblocks, one intra prediction is completed within a shorter time than inter prediction. The present invention has been made paying attention to this point, and generates intra prediction images in units of macroblocks based on intra prediction and intra prediction during inter prediction processing.

According to the present embodiment as described above, each function of the encoding device can be effectively used by generating an intra predicted image while waiting for processing of an inter predicted image. Effective use of each functional unit can provide an encoding device capable of encoding image data at a higher speed or with a small and simple configuration.
Moreover, since this embodiment does not use a technique such as reducing the number of data used for intra prediction, the processing can be made more efficient while maintaining the accuracy of intra prediction. Therefore, it is possible to provide a high-quality moving image while improving the processing efficiency.

  FIG. 2 is a diagram for explaining the relationship between each configuration and function shown in FIG. The accelerator 2 compares the input image data and the correlation value of the input image data between frames, and generates an inter prediction image based on the prediction result of the inter prediction cost calculation unit 105 and the inter prediction cost calculation unit 105. Part 106 is provided. The correlation value is compared by detecting a motion vector. The Inter prediction cost calculation unit 105 and the Inter prediction image generation unit 106 function as an inter prediction image generation unit of the present embodiment.

  Further, the accelerator 2 uses the intra prediction image generation unit after the start of the process for comparing the correlation value by the inter prediction cost calculation unit 105 and before the generation of the inter prediction image by the inter prediction image generation unit 106. An intra prediction cost calculation unit 107 that determines a calculation mode for generating an intra prediction image, and an intra prediction image generation unit 108 that generates an intra prediction image according to the determined intra prediction mode are provided. The intra prediction cost calculation unit 107 functions as a mode determination unit of the present embodiment.

  In the configuration of the accelerator 2 shown in FIG. 2, the processor 101 is a smaller processor than the processor 1 for comprehensively controlling the functions of the accelerator 2 described above. The processor 101 may function as the mode determination unit 214. The local buffer 103 is a memory used as a work area or the like when each function executes processing. An external memory 102 connected to the accelerator 2 via the bus B is a memory that receives a predicted image generated by the accelerator.

In the present embodiment, the motion compensation unit 211 and the motion estimation unit 212 illustrated in FIG. 1 correspond to the Inter prediction cost calculation unit 105. The intra prediction unit 213 corresponds to the intra prediction cost calculation unit 107. Further, the predicted image generation unit 215 corresponds to the Intra predicted image generation unit 108 and the Inter predicted image generation unit 106.
Next, the operation of the encoding apparatus of the present embodiment described above will be described using the flowchart of FIG. The encoding apparatus according to the present embodiment starts when a command for predicting whether image data should be encoded by inter prediction or intra prediction is issued from the processor 1 to the accelerator 2.

  After receiving the command, the accelerator 2 inputs the input image data and the image data of the dequantized difference. And the parallel processing A which determines the mode of an estimated image is performed (SA). In the parallel processing A, inter prediction (S301) by the motion compensation unit 211, intra prediction (S302) by the intra prediction unit 213, and generation of an intra prediction image by the prediction image generation unit 215 (S303) are executed in parallel. .

Inter-prediction, intra-prediction, and intra-prediction image parallel processing can be easily performed by separately providing a frame storage unit 211 and a memory for storing the image data added by the adder 208, as described in FIG. realizable. It is also possible to use input image data for intra prediction and generation of an intra predicted image.
The mode determination unit 214 calculates the minimum value of the prediction error when using the inter prediction image predicted in step S301 and the absolute value difference sum of the intra prediction image generated in the intra prediction mode predicted in step S302. Compare. Then, one of inter prediction and intra prediction is selected as a method for increasing the compression rate of the encoded image data. Furthermore, the mode of the selected inter prediction or intra prediction is determined as a mode applied to generation of a prediction image (S304).

  In the present embodiment, when the mode determination unit 214 determines the mode to inter prediction (S inter), the prediction image generation unit 215 displays the motion vector between frames, the size of the block image used for processing, and the frame to be referred to An inter prediction image is generated according to parameters such as an image number (S305). The generated inter prediction image is transferred to the subtracter 202 and subtracted from the input image data.

Further, in the present embodiment, information including a motion vector between frames, a prediction mode, and a frame image number to be referred to is output as a parameter together with the inter prediction image. This information is information necessary for decoding the encoded image data on the decoding side, and is added to the encoded prediction error data output from the encoding device.
In addition, when the mode determination unit 214 determines the mode to intra prediction (S intra), the predicted image generation unit 215 uses the intra predicted image generated in step S303 according to the intra prediction mode determined in step S302. The data is transferred to the subtracter 202 (S306). In the present embodiment, parameters including the intra prediction mode used to generate the intra prediction image are output together with the intra prediction image. The parameter is information necessary for decoding the encoded image data on the decoding side, and is added to the encoded prediction error data output from the encoding device.

  In the present embodiment described above, intra prediction and generation of an intra predicted image can be completed before one frame of image data is stored in the frame storage unit 210 in order to form an inter predicted image. For this reason, when the mode determination unit 215 determines the mode to intra prediction, the intra prediction image generated without generating the intra prediction image again can be transferred to the subtracter 202.

  In addition, when the mode determination unit 215 determines the mode to inter prediction, the generated intra prediction image may be discarded and an inter prediction image may be generated again. Even in such a case, the intra prediction image to be discarded is generated during the inter prediction. Therefore, the encoding processing time for generating an intra predicted image does not increase or the processing efficiency does not decrease.

  In the present embodiment described above, the inter prediction image is generated after the mode is determined in step S304. However, the present embodiment is not limited to such a configuration. That is, when the inter prediction image is generated before mode determination as in the past and one is used for encoding as compared with the previously generated intra prediction image, the present embodiment generates the intra prediction image again. There is no need to do. Therefore, the present invention has an effect of increasing the processing efficiency of the encoding device.

  FIG. 4 is a flowchart showing the operation of the conventional configuration shown for comparison with the characteristics of the encoding apparatus of the present embodiment described above. In the conventional configuration, after inter prediction (S401), an inter prediction image is generated based on the prediction result (S402). Subsequently, intra prediction is executed (S403), and the mode of intra prediction is determined as a result of the execution. Then, an intra-predicted image is generated according to the determined mode (S404).

  The generated inter prediction image and intra prediction image are input to the mode determination unit. Then, the mode determination unit determines one of the inter predicted image and the intra predicted image as a mode (S405). The mode determination unit transfers the inter prediction image or the intra prediction image to the subtracter according to the selected mode (S406, S407). The encoding apparatus obtains the transferred predicted image as subtracted from the input image data (S408).

In such a conventional example, it is necessary to generate an intra prediction image again after generating an inter prediction image. Compared to such a conventional example, the encoding apparatus according to the present embodiment that generates an intra-prediction image while generating an inter-prediction image can reduce the time required for processing to generate the prediction image and the load on the functional unit. it can.
The encoding apparatus of the present embodiment described above can be provided in a device or the like by the encoding apparatus alone. It is also possible to configure an image processing device in combination with an encoding device. FIG. 5 shows an example of a decoding device. Note that, in the configuration illustrated in FIG. 5, the same configuration as the configuration described in FIG.

  The illustrated decoding apparatus includes a decoding unit 501 that decodes prediction error data encoded by the encoding apparatus of the present embodiment described above, and a prediction image that is added to the image data decoded by the decoding unit 501. An inverse quantization unit 206, an inverse orthogonal return unit 207, a deblock filter unit 209, a frame storage unit 210, a predicted image generation unit 215, and a mode determination unit 214 are provided.

  Note that generally, the results of inter prediction and intra prediction are input to the decoding apparatus together with the encoded data. For this reason, it is not necessary to perform inter prediction and intra prediction independently, and a configuration for performing intra prediction during inter prediction as in the case of an encoding device is not required.

It is a figure for demonstrating the structure of the encoding apparatus of one Embodiment of this invention. It is a figure for demonstrating the relationship between each structure and function shown in FIG. It is a flowchart for demonstrating operation | movement of the encoding apparatus of one Embodiment of this invention. It is a flowchart which shows operation | movement of the conventional structure shown in order to compare with the characteristic of the encoding apparatus of one Embodiment of this invention. It is a figure for demonstrating the structure of the decoding apparatus of one Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,101 Processor, 2 Accelerator, 102 Memory, 103 Local buffer, 105 Inter prediction cost calculation part, 106 Inter prediction image generation part, 107 Intra prediction cost calculation part, 108 Intra prediction image generation part, 201 Coding control part, 202 Subtractor, 203 Direct transformation unit, 204 Quantization unit, 205 Lossless encoding unit, 206 Lossless quantization unit, 207 Inverse transformation unit, 208 Adder, 209 Deblock filter unit, 210 Frame accumulation unit, 211 Motion compensation unit 212 motion estimation unit, 213 intra prediction unit, 214 mode determination unit, 215 predicted image generation unit, 501 decoding unit

Claims (2)

An inter prediction image generating means for generating an inter prediction image by comparing correlation values of image data between frames;
An intra-predicted image generating means for generating an intra-predicted image by comparing the correlation values of image data between block images obtained by dividing image data for one frame;
Mode determining means for determining a mode of calculation for generating an intra predicted image by the intra predicted image generating means;
With
The mode determination unit determines a calculation mode for generating an intra prediction image after the start of comparison of correlation values by the inter prediction image generation unit and before the start of generation of an inter prediction image. An encoding device. An image processing apparatus comprising: an encoding device; and a decoding device that decodes image data encoded by the encoding device,
The encoding device includes:
An inter prediction image generating means for generating an inter prediction image by comparing correlation values of image data between frames;
An intra-predicted image generating means for generating an intra-predicted image by comparing the correlation values of image data between block images obtained by dividing image data for one frame;
Mode determination for determining an operation mode for generating an intra-predicted image by the intra-predicted image generating unit after the start of comparison of correlation values by the inter-predicted image generating unit and before starting the generation of the inter-predicted image Means, and
The decoding device
An image comprising: decoding means for decoding image data encoded by the encoding device; and prediction image generation means for generating a prediction image to be added to the image data decoded by the decoding means. Processing equipment.

Images