JP4768011B2 - Moving picture encoding apparatus and moving picture decoding apparatus - Google Patents

Description

  The present invention relates to a moving image encoding apparatus and a moving image decoding apparatus that divide and encode a quantization target image into a plurality of blocks.

  In order to efficiently transmit and record a moving image having an enormous amount of information, a technique of dividing image data into a plurality of blocks and quantizing and encoding each block is widely used. In a moving image quantized and encoded for each block, block distortion (block noise) in which pixel values such as luminance levels discontinuously change occurs at the boundary of each block. As a technique for reducing the block distortion, the H.P. A deblocking filter in the H.264 / AVC video coding system is known.

  Hereinafter, H.C. FIG. 11 to FIG. 15 regarding a moving picture encoding apparatus 100 that encodes a moving picture using the H.264 / AVC moving picture encoding scheme and a moving picture decoding apparatus 200 that decodes a moving picture encoded using the same scheme. While explaining.

[Configuration of Conventional Video Encoding Device]
FIG. 1 is a functional block diagram illustrating a schematic configuration of a moving image encoding apparatus 100 that encodes a moving image using an H.264 / AVC moving image encoding method. As shown in FIG. 11, the moving picture coding apparatus 100 includes a DCT unit 1, a quantization unit 2, a variable length coding unit 3, an inverse quantization unit 4, an IDCT unit 5, a deblocking filter processing unit 6, a frame A memory 7, an intra prediction unit 8, an inter prediction unit 9, and an encoding control unit 10 that controls each of the above units are provided.

  The DCT unit 1 divides a difference image obtained by subtracting a prediction image described later from an original image into blocks each having 4 × 4 pixels or 8 × 8 pixels, and orthogonally transforms the image signals of each block (integer precision DCT). To do. A transform coefficient (corresponding to a DCT coefficient in discrete cosine transform) obtained by the orthogonal transform is sent to the quantization unit 2. The quantization unit 2 quantizes the transform coefficient of each block according to the quantization parameter supplied from the encoding control unit 10. The quantized representative value obtained as a result of the quantization is sent to the variable length coding unit 3 and the inverse quantization unit 4.

  The variable length encoding unit 3 converts the various encoding parameters supplied from the encoding control unit 10 and the quantized representative value (transformed coefficient of each quantized block) supplied from the quantization unit 3 into a variable length code. Turn into. The encoded data obtained as a result of encoding by the variable length encoding unit 3 is sent to the moving picture decoding apparatus 200 described later.

  The inverse quantization unit 4 inversely quantizes the quantization representative value (transformed coefficient of each quantized block) supplied from the quantization unit 3 according to the quantization parameter supplied from the encoding control unit 10. That is, the inverse quantization unit 4 restores the transform coefficient of each block from the quantized representative value by an operation reverse to the quantization operation by the quantization unit 2. The transform coefficient of each block restored by inverse quantization is sent to the IDCT unit 5. The IDCT unit 5 converts the transform coefficient of each block obtained by inverse quantization into an image signal in the spatial domain, and restores the difference image. Here, the inverse orthogonal transform applied to the transform coefficient by the IDCT unit 5 is the inverse transform (integer accuracy IDCT) of the orthogonal transform applied by the DCT unit 1. The locally decoded image obtained by adding the difference image restored by the IDCT unit 5 and the predicted image is sent to the deblocking filter processing unit 6.

  The deblocking filter processing unit 6 performs adaptive filter processing on the local decoded image in order to remove block distortion in the local decoded image obtained by adding the predicted image and the difference image. Details of adaptive filter processing by the deblocking filter processing unit 6 will be described in detail later.

  The locally decoded image from which block distortion has been removed by the deblocking filter processing unit 6 is temporarily stored in the frame memory 7. The frame memory 7 can store a plurality of locally decoded images. The locally decoded image stored in the frame memory 7 is referred to by the intra prediction unit 8 or the inter prediction unit 9 as a reference image.

  The intra prediction unit 8 generates a predicted image from the reference image recorded in the frame memory 7 by performing intra-frame prediction. The intra-prediction unit 8 It is possible to perform intra-frame prediction using a plurality of prediction modes (prediction algorithms) defined in the H.264 / AVC video coding standard, and execute intra-frame prediction using a prediction mode specified by the encoding control unit 10. To do.

  The inter prediction unit 9 generates a prediction image by inter-frame prediction (motion compensation prediction) based on the motion vector determined by the encoding control unit 10 and the reference image stored in the frame memory 7. The inter prediction unit 9 performs inter-frame prediction using a block having a size specified by the encoding control unit 10 and using a plurality of reference images specified by the encoding control unit 10.

  The encoding control unit 10 determines which prediction method is used to generate a prediction image among intra-frame prediction and inter-frame prediction, and determines various encoding parameters according to the determined prediction method. The encoding parameter when performing intra-frame prediction includes information specifying a prediction mode in intra-frame prediction. In addition, the encoding parameters when performing inter-frame prediction include information specifying a motion vector, a block size, and a reference image. Furthermore, the encoding control unit 10 specifies a quantization parameter for the quantization unit 2 and the inverse quantization unit 4.

  Next, the encoding operation of the moving image encoding device 100 shown in FIG. 11 will be described. Schematically speaking, the moving image encoding apparatus 100 encodes a moving image by repeating the following steps 1 to 6.

  (Step 1) The encoding control unit 10 determines whether to perform intra-frame prediction or inter-frame prediction, and determines an encoding parameter and a quantization parameter necessary for encoding.

  (Step 2) In accordance with the determination result in Step 1, the intra prediction unit 8 or the inter prediction unit 9 generates a prediction image based on the reference image stored in the frame memory 7.

  (Step 3) A difference image between the predicted image generated in Step 2 and the input original image is generated and supplied to the DCT unit 1.

  (Step 4) The DCT unit 1 and the quantization unit 2 orthogonally transform the image signal of the difference image obtained in step 3 for each block, and quantize the obtained transform coefficient. The obtained quantized representative value is variable-length encoded by the variable-length encoding unit 3 and output as encoded data.

  (Step 5) The inverse quantization unit 4 and the IDCT unit 5 dequantize the quantized representative value obtained in step 4 to restore the difference image.

  (Step 6) The difference image restored in step 5 and the predicted image generated in step 2 are added, and the obtained locally decoded image is supplied to the deblocking filter processing unit 6.

  (Step 7) The deblocking filter processing unit 6 removes block distortion from the local decoded image obtained in step 6, and the local decoded image with reduced block distortion is stored in the frame memory 7 as a reference image.

[Configuration of Conventional Video Decoding Device]
Next, H.I. A moving picture decoding apparatus 200 that decodes a moving picture encoded by the H.264 / AVC moving picture encoding method will be described with reference to FIG. As shown in FIG. 12, the moving picture decoding apparatus 200 includes an inverse quantization unit 4, an IDCT unit 5, a deblocking filter processing unit 6, a frame memory 7, an intra prediction unit 8, an inter prediction unit 9, and a variable length. A decoding unit 20 is provided.

  Here, among the functional blocks constituting the moving image decoding apparatus 200, the only functional block not included in the above-described moving image encoding apparatus 100 (FIG. 11) is the variable length decoding unit 20. About the block which has the same function as the moving image encoder 100, a name and code | symbol are the same and description is abbreviate | omitted. The variable length decoding unit 20 has a function of variable length decoding the encoding parameter and the quantized representative value (quantized transform coefficient).

  Generally speaking, the moving picture decoding apparatus 200 shown in FIG. 12 decodes encoded data by repeating the following steps 1 to 6.

  (Step 1) The variable length decoding unit 20 performs variable length decoding on the encoded data to obtain an encoding parameter and a quantized representative value (quantized transform coefficient).

  (Step 2) In accordance with the decoded encoding parameter, the intra prediction unit 8 or the inter prediction unit 9 generates a prediction image based on the reference image stored in the frame memory 7.

  (Step 3) The inverse quantization unit 4 and the IDCT unit 5 dequantize the quantized representative value obtained in step 1 to restore the difference image.

  (Step 4) The difference image restored in step 3 and the predicted image generated in step 2 are added, and the obtained decoded image is supplied to the deblocking filter processing unit 6.

  (Step 5) The deblocking filter processing unit 6 removes block distortion from the decoded image obtained in step 4, and stores the decoded image with reduced block distortion in the frame memory 7. The decoded image stored in the frame memory 7 can be read at an arbitrary timing and used as a reference image or a display image.

(Step 6) The decoded image stored in the frame memory is output to an image projection means such as a display at an appropriate timing as a display image.
[Deblocking filter]
As described above, H.C. In the H.264 / AVC moving picture coding system, the common deblocking filter processing unit 6 is used in both the moving picture coding apparatus 100 and the moving picture decoding apparatus 200, so that the moving picture is quantized and dequantized. The generated block distortion is reduced. Hereinafter, the deblocking filter processing unit 6 will be described in more detail with reference to FIGS. 13 to 15.

FIG. 2 is an explanatory diagram illustrating a block division pattern of a quantization target image (a difference image between an original image and a predicted image) in a moving image encoding process using the H.264 / AVC moving image encoding method. As shown in FIG. 13, the quantization target image is divided into W × H rectangular blocks. The W × H blocks are assigned B _{0 to} B _{W × H−1} in order from the upper left end of the quantization target image.

FIG. 14 is a diagram showing two adjacent blocks, block _Bn and block _{Bn + 1} , in the quantization target image shown in FIG. As shown in FIG. 14, each of the block B _n and the block B _{n + 1} is composed of a total of 16 pixels arranged in 4 rows and 4 columns. As with the block _Bn , all the blocks B _{0 to} B _{W × H−1} constituting the encoding target image in FIG. 13 are composed of 16 pixels arranged in 4 rows and 4 columns.

The pixel constituting the quantization target image is designated by a combination (n, u, v) of a variable n for specifying a block including the pixel and variables u and v for specifying the position of the pixel in the block. be able to. That is, the pixel (n, u, v) is a pixel in the u-th column and the v-th row of the block B _n . In the following description, the attribute value of the pixel (n, u, v) is expressed as X (n, u, v). For example, the pixel value of the pixel (n, u, v) in the encoding target image is expressed as P (n, u, v).

The blocks B _{0 to} B _{W × H−1} shown in FIG. 13 and FIG. 14 are processing units of a series of processes of orthogonal transform, quantization, inverse quantization, and inverse orthogonal transform in moving image coding. That is, the image data of the quantization target image is converted into a conversion coefficient for each block B _{0 to} B _{W × H−1} and quantized.

  However, since quantization is an irreversible process, even if the image to be quantized is restored by inverse quantization / inverse orthogonal transformation, the restored image does not match the image to be encoded and has been restored. Block distortion occurs at the boundary between adjacent blocks in the image. The deblocking filter processing unit 6 reduces this block distortion.

H. The deblocking filter processing unit 6 used in the H.264 / AVC moving image coding system performs filtering between blocks (B _n , B _{n + 1} ) adjacent in the horizontal direction, and blocks (B _n , B adjacent in the vertical direction). _{n + W} ) are independently filtered. Further, the filter strength of the filter processing performed by the deblocking filter processing unit 6 is adaptively set according to conditions such as a prediction mode applied to each block.

As an example of the filter processing performed by the deblocking filter processing unit 6, a filter processing calculation with the strongest filter strength for the block _Bn and the block _{Bn + 1} adjacent in the horizontal direction is shown below. In each of the following expressions, P (n, u, v) is a pixel in the processing target image (local decoded image in the moving image encoding device 100 or decoded image in the moving image decoding device 200) of the deblocking filter process. The pixel value (n, u, v) and P ′ (n, u, v) represent the pixel value of the pixel (n, u, v) in the filter output image output by the deblocking filter processing unit 6.

  Although a detailed description is omitted, the same filtering process is applied to adjacent blocks in the vertical direction.

The effect | action with respect to the process target image of the said filter process is shown to Fig.15 (a)-(c). FIG. 15A is a graph showing pixel values p (n, u, v) of the original image in the block B _n and the block B _{n + 1} . FIG. 15B is a graph showing the pixel value P (n, u, v) of the decoded image to be processed by the deblocking filter processing unit 6 in the block B _n and the block B _{n + 1} . As shown in FIG. 15B, in the decoded image, discontinuous changes in pixel values at block boundaries, that is, block distortion occurs. FIG. 15C is a graph showing pixel values P ′ (n, u, v) in the block B _n and the block B _{n + 1} of the image after the filtering processing by the deblocking filter 6 is performed on the decoded image. . Since the deblocking filter 6 performs convolution of pixel values across adjacent blocks, discontinuous changes in pixel values in the restored encoding target image are smoothed, and block distortion is reduced.
ITU-T Recommendation H.264: "Advanced Video Coding for generic audiovisual services" (2003)

  However, in the above-described conventional video encoding apparatus, as a side effect of reducing the block distortion by the deblocking filter, image quality deterioration different from the block distortion such as image blurring or image flickering at the time of moving image reproduction occurs. There was a problem that.

The side effects of the deblocking filter will be described in more detail below. As is clear from the above-described equations (1) to (8), when the pixel value of the image to be filtered is P (n, u, v), the average pixel value <P _n > of the block B _n before the filtering process And the average pixel value <P ′ _n > of the block B _n after the filter processing is given by the following equations, respectively.

  That is, the average pixel value in each block is different before and after the filtering process. Therefore, when the conventional deblocking filter is used, the average pixel value in each block is not stored before and after the filtering process, and thus the difference in the average pixel value between the original image and the decoded image can be expanded by the filtering process.

  Then, in the moving image encoding device, a predicted image is generated using a locally decoded image having a difference in average pixel value from the original image as a reference image. In the moving image decoding apparatus, a decoded image having an average pixel value difference with respect to the original image is generated as a reference image or a display image. For this reason, blurring or flickering of the image in the moving image occurs.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to reduce a moving image code that can reduce block distortion in a decoded moving image and does not cause side effects such as image blurring and flickering. It is to implement | achieve a conversion apparatus and a moving image decoding apparatus.

  In order to solve the above-described problem, the video encoding apparatus according to the present invention divides an image into a plurality of blocks, quantizes the image data of each block, and encodes a quantized representative value obtained by the quantization. The moving picture encoding apparatus is characterized by comprising filter processing means for performing filter processing for removing frequency components that generate block distortion on the image data before quantization.

  The filter processing means removes a frequency component that causes block distortion from the image data before quantization. For this reason, the quantization target image to be quantized is obtained by removing frequency components that generate block distortion in advance. Therefore, a decoded image obtained by quantizing / inverse-quantizing the quantization target image that has been subjected to the filter processing by the filter processing means has reduced block distortion. In addition, the frequency component previously removed by the filter processing means can be restored by an inverse filter process corresponding to the inverse transform of the filter process of the filter process means.

  That is, the moving picture coding apparatus having the above configuration has an effect that it can generate coded data that can restore a decoded picture with reduced block distortion without missing a specific frequency component.

  In the moving image encoding apparatus of the present invention, it is preferable that the filter processing performed by the filter processing means is to keep the average pixel value of each block unchanged before and after the processing.

  According to said structure, the filter process by the said filter process means is performed so that the average pixel value (for example, average luminance level) of each said block may be kept unchanged. For this reason, it is possible to effectively prevent blurring and flickering of the screen caused by the fact that the average pixel value for each block is not stored before and after the filtering process, which has occurred in a conventional video encoding apparatus using a deblocking filter. There is a further effect of being able to.

  The moving image encoding apparatus of the present invention further includes a predicted image generating unit that generates a predicted image, and an inverse filter processing unit that performs an inverse filter process corresponding to the inverse transform of the filter process on the local decoded image. Preferably, the predicted image generation means generates the predicted image using the locally decoded image subjected to the inverse filter processing as a reference image.

  According to said structure, the estimated image production | generation means can produce | generate an estimated image by using the local decoded image to which the said reverse filter process was performed as a reference image. That is, the predicted image generation means can generate a predicted image based on a reference image in which block distortion is reduced and a specific frequency component is not lost, that is, a reference image that approximates the original image. There is an effect.

  As a method for generating a predicted image by the predicted image generation unit, intra-frame prediction, inter-frame prediction, or the like can be used. Further, the predicted image generation means may switch between these prediction methods as appropriate according to the state of the image to be encoded.

  Note that the inverse filter processing performed by the inverse filter processing means may correspond to a strict inverse transform or an approximate inverse transform with respect to the filter processing of the filter processing means. May be. That is, the inverse filter processing is sufficient if it approximates a strict inverse transform within a range that does not adversely affect the quality of the final moving image. For example, the calculation accuracy of the filter processing calculation (eg, integer accuracy) A degree of error is acceptable.

  In the video encoding device of the present invention, the quantization target image to be quantized is a difference image between the predicted image and the original image subjected to the filtering process, and the local decoded image is It is preferable that the image is obtained by adding the image data obtained by dequantizing the quantized representative value and the image data of the predicted image in the moving image coding apparatus.

  According to the above configuration, the original image becomes the processing target image of the filtering process, and the difference image between the original image subjected to the filtering process and the predicted image becomes the quantization target image. The local decoded image obtained by adding the image data obtained by dequantizing the quantized representative value obtained by quantizing the quantization target image and the image data of the predicted image is the result of the inverse filter processing. The local decoded image that is the target and has been subjected to the inverse filter processing is used as a reference image for generating a predicted image. Therefore, according to the above configuration, the predicted image generation unit can generate a predicted image based on the reference image that better approximates the original image.

  In the video encoding device of the present invention, the quantization target image to be quantized is obtained by performing the filtering process on the difference image between the predicted image and the original image, and the locally decoded image. Is preferably an image obtained by adding the image data obtained by dequantizing the quantized representative value and the image data of the predicted image in the moving picture coding apparatus.

  According to the above configuration, the difference image between the predicted image and the original image is the target of the filtering process, and the difference image subjected to the filtering process is the target of quantization. On the other hand, the local decoded image obtained by adding the image data obtained by dequantizing the quantized representative value obtained by quantizing the quantization target image and the image data of the predicted image is the result of the inverse filter processing. The local decoded image that is the target and has been subjected to the inverse filter processing is used as a reference image for generating a predicted image. Therefore, according to the above configuration, the predicted image generation unit can generate a predicted image based on the reference image that better approximates the original image.

  In the moving image encoding apparatus of the present invention, the filter processing means calculates a pixel value to be subtracted from the image data of the processing target image to be subjected to the filtering process from the image data of the predicted image, The inverse filter processing means calculates a pixel value to be added to the image data of the processing target image to be subjected to the inverse filter processing from the image data of the predicted image in the same manner as the filter processing means. It is preferable.

  According to said structure, the pixel value which the said filter process means subtracts from the image data of a process target image, and the pixel value which the said reverse filter process means adds to the image data of a process target image are from the same estimated image. It is calculated by the same method. Therefore, the inverse filter processing unit can more completely restore the frequency component that generates the block distortion removed from the processing target image by the filter processing unit.

  In order to solve the above problems, a moving image decoding apparatus according to the present invention is a moving image decoding apparatus that decodes encoded data obtained by encoding image data from which frequency components that generate block distortion are removed. The image processing apparatus includes an inverse filter processing unit that performs inverse filter processing for restoring the removed frequency component on the image data of the decoded image obtained by decoding.

  The moving picture decoding apparatus decodes encoded data obtained by encoding image data from which a frequency component causing block distortion has been removed in advance. Here, since the encoded data is obtained by removing frequency components that generate block distortion in advance, block distortion in a decoded image obtained by decoding is suppressed. And the decoded image obtained by decoding restores the frequency component removed in the process of encoding by the inverse filter processing means. Therefore, the decoded image obtained by performing the inverse filter process is an image in which block distortion is reduced and a specific frequency component is not lost, that is, an image that better approximates the original image.

  That is, the moving picture coding apparatus having the above configuration has an effect that a decoded picture with reduced block distortion can be restored without missing a specific frequency component.

  Other objects, features, and advantages of the present invention will be fully understood from the following description. The advantages of the present invention will become apparent from the following description with reference to the accompanying drawings.

It is a functional block diagram which shows the structure of the moving image encoder which concerns on one embodiment of this invention. It is a functional block diagram which shows the structure of the moving image decoding apparatus which concerns on one embodiment of this invention. 3 is a flowchart showing an outline of a filter processing calculation by a filter processing unit of the moving image encoding device shown in FIG. 1. It is a graph which shows the effect | action of the filter process calculation shown in FIG. 3, and is a graph showing the pixel value of the process target image used as the object of a filter process. It is a graph which shows the effect | action of the filter processing calculation shown in FIG. 3, and is a graph showing the average pixel value for every block of a process target image. FIG. 4 is a graph showing an operation of the filter processing calculation shown in FIG. 3, and is a graph showing a predicted value obtained by linear interpolation using the average pixel value shown in FIG. 4 (b). It is a graph which shows the effect | action of the filter process calculation shown in FIG. 3, and is a graph showing the pixel value of a filter output image. 3 is a flowchart showing an overview of inverse filter processing calculation by an inverse filter processing unit included in the video encoding device shown in FIG. 1 and the video decoding device shown in FIG. 2. 6 is a graph showing the operation of the inverse filter processing calculation shown in FIG. 5, and is a graph showing the pixel value of the processing target image that is the target of the inverse filter processing. It is a graph which shows the effect | action of the inverse filter process calculation shown in FIG. 5, and is a graph showing the average pixel value for every block of a process target image. FIG. 6 is a graph showing an operation of the inverse filter processing calculation shown in FIG. 5, and is a graph showing a prediction value obtained by linear interpolation using the average pixel value shown in FIG. It is a graph which shows the effect | action of the reverse filter process calculation shown in FIG. It is a functional block diagram which shows the other structural example of the moving image encoder which concerns on one embodiment of this invention. FIG. 9 is a functional block diagram illustrating another configuration example of the video decoding device according to the embodiment of the present invention and illustrating a configuration of the video decoding device corresponding to the video encoding device in FIG. 7. It is a functional block diagram which shows the other structural example of the moving image encoder which concerns on one embodiment of this invention. FIG. 10 is a functional block diagram illustrating another configuration example of a video decoding device according to an embodiment of the present invention and illustrating a configuration of a video decoding device corresponding to the video encoding device in FIG. 9. It is a functional block diagram which shows the prior art and shows the structure of the moving image encoder provided with the deblocking filter. FIG. 12 is a functional block diagram illustrating a conventional technique and illustrating a configuration of a video decoding device corresponding to the video encoding device illustrated in FIG. 11. It is explanatory drawing which shows the division pattern of the image in the quantization object image used as the object of quantization, or the process object image used as the object of a filter process. It is the enlarged view which expanded and showed two adjacent blocks in the image divided | segmented into the several block with the division | segmentation pattern shown in FIG. It is a graph which shows a prior art and shows the effect | action of a deblocking filter. In particular, it is a graph showing pixel values of an original image to be encoded. It is a graph which shows a prior art and shows the effect | action of a deblocking filter. In particular, it is a graph showing pixel values of a locally decoded image including block distortion. It is a graph which shows a prior art and shows the effect | action of a deblocking filter. It is a graph which especially shows the pixel value of the output image of a deblocking filter.

Explanation of symbols

DESCRIPTION OF SYMBOLS 300 Moving image encoder 400 Moving image decoder 1 DCT part 2 Quantization part 3 Variable length encoding part 4 Inverse quantization part 5 IDCT part 7 Frame memory 8 Intra prediction part 9 Inter prediction part 10 Coding control part 11 Filter Processing unit (filter processing means)
12 Inverse filter processing unit (inverse filter processing means)
300a, 300b moving image encoding device 400a, 400b moving image decoding device

  An embodiment of the moving picture coding apparatus of the present invention will be described below with reference to FIGS.

[Configuration of Video Encoding Device]
First, the configuration of the video encoding apparatus according to the present embodiment will be described. FIG. 1 is a functional block diagram showing a schematic configuration of a moving image encoding apparatus 300 according to the present embodiment. As shown in FIG. 1, the moving picture coding apparatus 300 includes a DCT unit 1, a quantization unit 2, a variable length coding unit 3, an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, and an intra prediction unit 8. , An inter prediction unit 9, an encoding control unit 10, a filter processing unit 11, and an inverse filter processing unit 12.

  The difference between the moving picture coding apparatus 300 of FIG. 1 and the conventional moving picture coding apparatus 100 (FIG. 11) is that the moving picture coding apparatus 300 uses the filter processing unit 11 instead of the conventional deblocking filter 6. And an inverse filter processing unit 12. In FIG. 1, blocks having the same functions as those of the conventional moving image encoding apparatus 100 are denoted by the same names and reference numerals as those in FIG. 11, and description thereof is omitted.

  A characteristic configuration of the moving image encoding apparatus 300 is a filter processing unit 11 and an inverse filter processing unit 12. The filter processing unit 11 uses the original image to be encoded as a processing target image, and performs a filtering process to remove a frequency component that generates block distortion from the processing target image. The DCT unit 1 is supplied with a difference image obtained by subtracting the prediction image generated by the intra prediction unit 8 or the inter prediction unit 9 from the image subjected to the filter processing by the filter processing unit 11.

  On the other hand, the inverse filter processing unit 12 is obtained by adding the difference image restored by the inverse quantization unit 4 and the IDCT unit 5 and the prediction image generated by the intra prediction unit 8 or the inter prediction unit 9. A locally decoded image is supplied. The inverse filter processing unit 12 sets the local decoded image as a processing target image, and performs an inverse filtering process corresponding to the inverse transformation of the filter process performed by the filter processing unit 11 on the processing target image. The locally decoded image that has been subjected to the inverse filter processing by the inverse filter processing unit 12 is stored in the frame memory 7 as a reference image, and is used for generation of a predicted image by the intra prediction unit 8 or the inter prediction unit 9.

  The DCT unit 1 and the quantization unit 2 divide the quantization target image into a plurality of blocks and quantize the image data of each block. Therefore, a block image can be included in a decoded image obtained by dequantizing a quantized image. That is, block distortion may be included in a locally decoded image generated inside the moving image encoding device 300 to generate a predicted image and a decoded image generated by a moving image decoding device described later. However, in the moving image encoding device 300, the quantization target image supplied to the DCT unit 1 is a difference image between the image subjected to the filter processing by the filter processing unit 11 and the predicted image. Since the filter processing unit 11 acts on the processing target image so as to remove in advance the frequency component that generates block distortion, it is possible to effectively reduce block distortion that occurs in the process of quantization / inverse quantization.

  Further, the reference image that the intra prediction unit 8 or the inter prediction unit 9 refers to in order to generate a predicted image is a locally decoded image that has been subjected to the inverse filter processing by the inverse filter processing unit 12. Therefore, the frequency component removed from the original image is restored in the reference image. That is, the intra prediction unit 8 and the inter prediction unit 9 can generate a prediction image based on a reference image in which a specific frequency component is not lost while the block distortion is reduced.

  In other words, the spatial frequency component lost by the quantization processing in the conventional video encoding device 100 is temporarily removed by the filter processing unit 11 in the video encoding device 300, and is then sent to the inverse filter processing unit 12. Thus, the influence of the quantization process can be avoided and the occurrence of block distortion can be suppressed.

Also, the filter processing unit 11 and the inverse filter processing unit 12 are different from the deblocking filter processing unit 6 in the conventional moving image encoding device 100, as described later, average pixel values before and after processing in each block (for example, average) (Luminance level) can be maintained. Therefore, it is possible to avoid problems such as blurring of images and flickering during reproduction of moving images due to the deblocking filter processing.
[Configuration of video decoding apparatus]
FIG. 2 is a functional block diagram showing a schematic configuration of a video decoding device 400 corresponding to the video encoding device 300 shown in FIG. As illustrated in FIG. 2, the moving picture decoding apparatus 400 includes an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, an intra prediction unit 8, an inter prediction unit 9, a variable length decoding unit 20, and an inverse filter process. Part 12 is provided.

  The difference between the moving picture decoding apparatus 400 of FIG. 2 and the conventional moving picture decoding apparatus 200 (FIG. 12) is that the moving picture decoding apparatus 400 includes an inverse filter processing unit 12 instead of the conventional deblocking filter 6. It is a point. In FIG. 2, blocks having the same functions as those of the conventional moving image decoding apparatus 200 are denoted by the same names and symbols as those in FIG. 12, and description thereof is omitted.

  The encoded data decoded by the moving image decoding apparatus 400 is generated based on the original image from which the frequency component that generates block distortion is removed in the moving image encoding apparatus 300. The variable length decoding unit 20 of the video decoding device 400 decodes the encoded data, and the inverse quantization unit 4 and the IDCT unit 5 inversely quantize the values obtained by the decoding. The image restored by inverse quantization corresponds to a difference image obtained by subtracting the predicted image from the original image. The moving picture decoding apparatus 400 adds the predicted image generated by the intra prediction unit 8 or the inter prediction unit 9 to the restored difference image to generate a decoded image.

  The inverse filter processing unit 12 included in the video decoding device 400 performs the same inverse filter processing as the inverse filter processing unit 12 included in the video encoding device 300. That is, the inverse filter processing unit 12 operates to restore the frequency component removed at the time of encoding on the decoded image lacking the frequency component that generates block distortion. The decoded image on which the inverse filter processing by the inverse filter processing unit 12 is performed is referred to by the intra prediction unit 8 or the inter prediction unit 9 as a reference image for generating a prediction image, and is output as a display image.

  As described above, the frequency component removed from the original image by the operation of the inverse filter processing unit 12 is restored in the display image or the reference image. That is, the moving picture decoding apparatus 400 can output a display image in which a specific frequency component is not lost while the block distortion is reduced. In addition, the prediction image can be generated based on the reference image in which the specific frequency component is not lost at the same time that the block distortion is reduced.

[Details of filtering]
Next, filter processing performed by the filter processing unit 11 included in the moving image encoding apparatus 300 will be described in detail.

  The filter processing unit 11 performs horizontal filter processing and vertical filter processing by dividing a processing target image to be filtered into a plurality of blocks. Here, the division pattern in which the filter processing unit 11 divides the processing target image is as shown in FIGS. 13 and 14, and the DCT unit 1 and the quantization unit 2 perform the quantization target image for quantization. Is the same as the division pattern for dividing.

Filter processing unit 11, the image data of the block B _n of the filter output image in which the filter processing unit 11 outputs the image data of the blocks adjacent to the image data and the block B _n of the block B _n of the processing target image Calculate from In the filter processing in the horizontal direction, the image data of the block B _n is calculated with reference to the block B _n +1 image data adjacent to the right side of the block B _n, the vertical direction in the filtering, the block B _n The image data is calculated with reference to the image data of the block B _{n + W} adjacent below the block.

Hereinafter, the filter processing calculation executed by the filter processing unit 11 in the horizontal filter processing will be described with reference to FIGS. 3 and 4. In addition, the filter processing calculation for calculating the pixel value (for example, luminance level) of one block _Bn is demonstrated. The filter processing unit 11 completes the horizontal filter processing by repeating the filter processing calculation described below for all adjacent two blocks. Note that the above repetition is performed by pairing adjacent even-numbered and odd-numbered blocks such as (B ₁ , B ₂ ), (B ₃ , B ₄ ), (B ₅ , B ₆ ). The above-mentioned filter processing calculation may be executed for all pairs, or (B ₁ , B ₂ ), (B ₂ , B ₃ ), (B ₃ , B ₄ ), etc. With respect to this block, the filtering processing calculation may be executed while referring to the next block. However, in these repetitions, a pair of non-adjacent blocks, for example, a pair consisting of a block B _{k × W−1} (k is an integer) at the right end of the image and a next block B _{k × W} located at the lower left end of the block. For the above, the above filter processing is not performed.

FIG. 3 is a flowchart schematically illustrating the filter processing calculation executed by the filter processing unit 11. As shown in FIG. 3, the filter processing calculation by the filter processing unit 11 includes a step S1 for calculating an average pixel value, a step S2 for calculating a predicted value, and a step S3 for calculating a filter output image. . Steps S1 to S3 will be described in more detail as follows. In the following description, a filtering process operation for calculating a pixel value of 4 pixels arranged in the v-th row among 16 pixels arranged in 4 rows and 4 columns belonging to the block _Bn will be described. The filter processing unit 11 performs the filter processing operation described below for the first to fourth rows of the block B _n sequentially or in parallel, so that the pixel values of all the pixels belonging to the block B _n are obtained. Is calculated.

(Step S1) filter processing unit 11, the average pixel values of four pixels arranged in v-th row of the block _{B n} of the processing target image _{<p n, v>} and, v-th row of the block _{B n + 1} in the processing target image The average pixel value <p _{n + 1, v} > of the four pixels arranged in ( ₁ ) is calculated. The calculation formula for the filter processing unit 11 to calculate these average pixel values is as follows.

Here, p (n, u, v) is a pixel value in the pixel (n, u, v) of the processing target image. FIG. 4A is a graph showing pixel values of the processing target image. FIG. 4B is a graph showing the average pixel value obtained in step S1.

(Step S2) The filter processing unit 11 performs linear interpolation using the average pixel values <p _{n, v} > and <p _{n + 1, v} > obtained in step S1, for each row of the block B _{n in} the v row. A predicted value p _pred (n, u, v) for the pixel and a predicted value p _pred (n + 1, u, v) for each pixel in the v-th row of the block B _{n + 1} are calculated. The calculation formula for the filter processing unit 11 to calculate the predicted values p _pred (n, u, v) and p _pred (n + 1, u, v) is as follows.

  FIG. 4C is a graph showing the predicted value obtained in step S2.

(Step S3) The filter processing unit 11 calculates the predicted value p _pred (n, u, v) obtained in step S2 and the average pixel value <p _{n, v} > of the block B _n obtained in step S1. Is a removal component to be removed from the processing target image, and subtracting the removal component from the pixel value p (n, u, v) of the processing target image, thereby obtaining a pixel value p ′ (n, u, v) is calculated. The calculation formula used to calculate the pixel value p ′ (n, u, v) of the filter processing unit 11a filter output image is as follows.

  FIG. 4D is a graph showing the pixel value of the filter output image obtained in step S3.

Here, the average pixel value of the block B _n in the filter output image matches the average pixel value of the block B _n in the processing target image. That is, the filter processing unit 11 calculates the average pixel value of each block before and after the processing. Note that it remains constant. For this reason, it is possible to prevent blurring and flickering of moving images due to the filter processing.

The filter processing unit 11 performs the filtering process in the vertical direction after performing the filtering process in the horizontal direction as described above. Filtering operation in the vertical direction, the image data of the block B _n, but is intended to calculate with reference to the block B _{n + W} adjacent to the lower side of the block B _n, the filter processing in the horizontal direction for its calculation method Since this is the same, the description thereof is omitted. In addition, since the horizontal filter processing and the vertical filter processing are independent of each other, the filter processing unit 11 performs the horizontal filter processing after performing the vertical filter processing. Also good.

[Details of inverse filter processing]
Next, the inverse filter processing performed by the inverse filter processing unit 12 included in the video encoding device 300 and the video decoding device 400 will be described.

  Similar to the filter processing unit 11, the inverse filter processing unit 12 executes the horizontal reverse filter processing and the vertical reverse filter processing by dividing the processing target image into a plurality of blocks. Here, the division pattern in which the inverse filter processing unit 12 divides the processing target image is the same as the division pattern in which the filter processing unit 11 divides the processing target image.

Inverse filter processing unit 12, the image data of the block B _n, the block B _n image data of the processing target image, and is calculated from the image data of the block adjacent to the block B _n. Specifically, in the filter processing in the horizontal direction, the image data of the block B _n, calculated with reference to the block B _{n + 1} image data adjacent to the right side of the block B _n in the process target image, the vertical filter in the processing, the image data of the block B _n, is calculated by referring to the image data of the block B _{n + W} adjacent the lower side of the block B _n.

Hereinafter, the inverse filter processing calculation performed by the inverse filter processing unit in the horizontal inverse filter processing will be described with reference to FIGS. 5 and 6. In the following, an inverse filter processing calculation for calculating a pixel value (for example, a luminance level) of one block _Bn will be described. The inverse filter processing unit 12 repeats the filter processing operation described below for all adjacent two blocks, thereby completing the horizontal filter processing.

FIG. 5 is a flowchart schematically illustrating the inverse filter processing calculation executed by the inverse filter processing unit 12. As shown in FIG. 5, the filter processing calculation by the inverse filter processing unit 12 includes a step T1 for calculating an average pixel value, a step T2 for calculating a predicted value, and a step T3 for calculating an inverse filter output image. It is out. Steps T1 to T3 will be described in more detail as follows. In the following description, an inverse filter processing operation for calculating a pixel value of 4 pixels arranged in the v-th row among 16 pixels arranged in 4 rows and 4 columns belonging to the block _Bn will be described. The inverse filter processing unit 12 performs the inverse filter processing operation described below for the first to fourth rows of the block B _n sequentially or in parallel, thereby calculating all the pixels belonging to the block B _n . The pixel value is calculated.

(Step T1) inverse filter processing unit 12, processing the average pixel values of four pixels arranged in v-th row of the block _{B n} of the target image _{<P n, v>} and, v rows of the block _{B n + 1} in the processing target image The average pixel value <P _{n + 1, v} > of the four pixels arranged in the eye is calculated. The calculation formula used by the inverse filter processing unit 12 to calculate the average pixel values <P _{n, v} > and <P _{n + 1, v} > is as follows.

  Here, P (n, u, v) is a pixel value in the pixel (n, u, v) of the processing target image. FIG. 6A is a graph showing the pixel values of the processing target image, and FIG. 6B is a graph showing the average pixel values obtained in step T1.

(Step T2) The inverse filter processing unit 12 performs linear interpolation using the average pixel values <P _{n, v} > and <P _{n + 1, v} > obtained in step T1 _, and the v th row of the block B _n A predicted value P _pred (n, u, v) for each pixel and a predicted value P _pred (n + 1, u, v) for each pixel in the v-th row of the block B _{n + 1} are calculated. The calculation formula used by the inverse filter processing unit 12 to calculate the predicted values P _pred (n, u, v) and P _pred (n + 1, u, v) is as follows.

  FIG. 6C is a graph showing the predicted value obtained in step T2.

(Step T3)
The inverse filter processing unit 12 determines the difference between the predicted value P _pred (n, u, v) obtained in step T2 and the average pixel value <P _{n, v} > of the block B _n obtained in step T1. Is added to the processing target image, and the additional component is added to the pixel value P (n, u, v) of the processing target image, thereby obtaining the pixel value P ′ (n, u, v) of the filter output image. ) Is calculated. The calculation formula used by the inverse filter processing unit 12 to calculate the pixel value P ′ (n, u, v) of the inverse filter output image is as follows.

FIG. 6D is a graph showing the pixel value of the inverse filter output image obtained in step T3. Here, it should be noted that the average pixel value of the block _Bn in the inverse filter output image matches the average pixel value of the block _Bn in the processing target image. For this reason, the occurrence of blurring and flickering of moving images is effectively prevented.

The inverse filter processing unit 12 performs the inverse filter process in the horizontal direction as described above, and then performs the inverse filter process in the vertical direction. The inverse filtering operation in the vertical direction, the image data of the block B _n, but is intended to calculate with reference to the block B _{n + W} adjacent to the lower side of the block B _n, opposite horizontal direction about the calculation method Since this is the same as the filtering process, the description thereof is omitted. In addition, since the horizontal reverse filter processing and the vertical reverse filter processing are independent from each other, the reverse filter processing unit 12 performs the horizontal reverse filter processing after performing the vertical reverse filter processing. It is good also as a structure which performs a process.

  The inverse filter processing performed by the inverse filter processing unit 12 defined as described above corresponds to the inverse transformation of the filter processing performed by the filter processing unit 11. That is, p ′ (n, u, v) = P (n) between the output p ′ (n, u, v) of the filter processing unit 11 and the input P (n, u, v) of the inverse filter processing unit 12. , U, v), the relationship between the input p (n, u, v) of the filter processor 11 and the output P ′ (n, u, v) of the inverse filter processor 12 is p (n, The relationship u, v) = P ′ (n, u, v) is established. Therefore, the frequency component that generates the block distortion removed from the processing target image by the filter processing of the filter processing unit 11 is restored by the inverse filter processing of the inverse filter processing unit 12. Therefore, a specific frequency component is not lost in the restored image.

[Additional notes on filtering and inverse filtering]
In the above description, the predicted value calculated by the filter processing unit 11 is calculated by linear interpolation using the average pixel value of the adjacent block, but the present invention is not limited to this. That is, the prediction value used by the filter processing unit 11 to calculate the pixel value of the filter output image may be, for example, a prediction value calculated based on an average pixel value in units of blocks, or is adjacent. It may be a value calculated by cubic interpolation using the average pixel values <P _{n−1, v} >, <P _{n, v} >, <P _{n + 1, v} > of the three blocks. Although a detailed description is omitted, the inverse filter processing unit 12 corresponding to the filter processing unit 11 that performs these filter processes is easily configured to perform an inverse filter process corresponding to the inverse transformation of these filter processes. Can be done.

  In addition, when using the prediction in which the average pixel value of the prediction value for each block is different from the average pixel value of the input image for each block, the difference between the average pixel value of the prediction value and the average pixel value of the input image is used as the filter output. It is preferable to make a correction so that the average pixel value of the filter input and the filter output is maintained.

  In the above description, the filter processing unit 11 performs one-dimensional filter processing independently in the horizontal direction and the vertical direction. However, the present invention is not limited to this. A configuration that performs dimensional filtering may be used. An example of a two-dimensional filter process performed by the filter processing unit 11 will be described as follows.

When performing two-dimensional filter processing, the filter processing unit 11 calculates the pixel value of the block of interest with reference to the pixel values of four blocks that are adjacent vertically and horizontally. The filter processing calculation performed by the filter processing unit 11 to calculate the pixel value of the target block _Bn includes the following steps 1 to 3.

(Step 1) filter processing unit 11, the average pixel value of the block of interest _{B n,} and the block _{B n-1} are vertically and horizontally adjacent to each of the target block _{B n, B n + 1,} B n-W, average pixel of _{B n + W} Calculate the value. The calculation formula used by the filter processing unit 11 to calculate the average pixel value of each block is as follows.

(Step 2) The filter processing unit 11 performs the prediction value p _pred (n, u, v) for each pixel of the block B _n by linear interpolation using the average pixel value of each block obtained in Step 1 above. ) Is calculated. The calculation formula used for the filter processing unit 11 to calculate the predicted value p _pred (n, u, v) is as follows.

(Step 3) The filter processing unit 11 calculates the predicted value p _pred (n, u, v) obtained in Step 2 and the average pixel value <p _{n, v} > of the block B _n obtained in Step 1. Is a removal component to be removed from the processing target image, and subtracting the removal component from the pixel value p (n, u, v) of the processing target image, thereby obtaining a pixel value p ′ (n, u, v) is calculated. The calculation formula for the filter processing unit 11 to calculate the pixel value p ′ (n, u, v) of the block B _n in the filter output image is as follows.

When removing a frequency component that generates block distortion by two-dimensional filter processing, the filter processing unit 11 calculates the pixel value of the entire filter output image by repeating steps 1 to 3 for each block. Although a detailed description is omitted, the corresponding inverse filter processing unit 12 may be configured to perform an inverse filter process corresponding to the inverse transformation of the filter process of the filter processing unit 11.
[Modification 1]
A modification of the moving image encoding apparatus 300 will be described with reference to FIGS.

  FIG. 7 is a functional block diagram illustrating a schematic configuration of a video encoding device 300a which is a modification of the video encoding device 300. As illustrated in FIG. 7, the moving image coding apparatus 300a includes a DCT unit 1, a quantization unit 2, a variable length coding unit 3, an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, and an intra prediction unit 8. , An inter prediction unit 9, an encoding control unit 10, a filter processing unit 11a, and an inverse filter processing unit 12a.

  The difference between the moving image encoding device 300a and the moving image encoding device 300 (FIG. 1) is that the moving image encoding device 300a performs a filter process based on a predicted image instead of the filter processing unit 11. The processing unit 11a is provided, and instead of the inverse filter processing unit 12, an inverse filter processing unit 12a that performs inverse filter processing based on the predicted image is provided. Further, as shown in FIG. 7, the intra prediction unit 8 and the inter prediction unit 9 supply the generated predicted image to the filter processing unit 11a and the inverse filter processing unit 12a. 7, blocks having the same functions as those of the moving picture coding apparatus 300 in FIG. 1 are denoted by the same names and symbols as those in FIG. 1, and description thereof is omitted.

  FIG. 8 is a functional block diagram showing a schematic configuration of a moving picture decoding apparatus 400a corresponding to the moving picture encoding apparatus 300a shown in FIG. As illustrated in FIG. 8, the video decoding device 400a includes an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, an intra prediction unit 8, an inter prediction unit 9, a variable length decoding unit 20, and an inverse filter process. A portion 12a is provided.

  The difference between the moving picture decoding apparatus 400a and the moving picture decoding apparatus 400 (FIG. 2) in FIG. 8 is that the moving picture decoding apparatus 400a uses the same inverse filter as the moving picture encoding apparatus 300a instead of the inverse filter processing unit 12. It is the point provided with the process part 12a. Further, as shown in FIG. 8, the intra prediction unit 8 and the inter prediction unit 9 supply the generated predicted image to the inverse filter processing unit 12a.

  The filter processing unit 11a and the inverse filter processing unit 12a that can be suitably used in the moving image encoding device 300a and the moving image decoding device 400a will be described as follows.

The filter processing unit 11a and the inverse filter processing unit 12a calculate the image data of each block in the filter output image from the image data of the prediction image supplied from the intra prediction unit 8 or the inter prediction unit 9. Specifically, in the filter processing in the horizontal direction, the image data of the block B _n, calculated by referring to the image data of the block B _n and the block B _{n + 1} in the prediction image, the filtering process in the vertical direction, the block the image data of B _n, is calculated by referring to the image data of the block B _n and the block B _{n + W} in the prediction image. Hereinafter, the filter processing calculation for calculating the pixel value of one block _Bn will be described. However, the filter processing unit 11a and the inverse filter processing unit 12a repeat the filter calculation processing described below to output the filter output. Complete the horizontal filtering of the entire image. Further, the entire filtering process is completed by executing the filtering process in the vertical direction in the same manner.

  The filter processing calculation executed by the filter processing unit 11a in the horizontal filter processing includes the following steps S1a to S3a.

(Step S1a) The filter processing unit 11a and the average pixel value <q _{n, v} > of the four pixels arranged in the v-th row of the block B _n in the predicted image and the v-th row of the block B _{n + 1} in the processing target image. The average pixel value <q _{n + 1, v} > of the arranged four pixels is calculated. The calculation formula for the filter processing unit 11a to calculate these average pixel values is as follows.

  Here, q (n, u, v) is a pixel value in the pixel (n, u, v) of the predicted image.

(Step S2a) The filter processing unit 11a performs linear interpolation using the average pixel values <qn _{, v} > and <qn _{+ 1, v} > obtained in step S1a, for each v-th row of the block _Bn. A predicted value q _pred (n, u, v) for the pixel and a predicted value q _pred (n + 1, u, v) for each pixel in the v-th row of the block B _{n + 1} are calculated. The calculation formula for the filter processing unit 11a to calculate the predicted values q _pred (n, u, v) and q _pred (n + 1, u, v) is as follows.

(Step S3a) The filter processing unit 11a calculates the predicted value q _pred (n, u, v) obtained in step S2a and the average pixel value <q _{n, v} > of the block B _n obtained in step S1a. Is a removal component to be removed from the processing target image, and subtracting the removal component from the pixel value p (n, u, v) of the processing target image, thereby obtaining a pixel value p ′ (n, u, v) is calculated. The calculation formula used by the filter processing unit 11a to calculate the pixel value p ′ (n, u, v) of the filter output image is as follows.

In addition, although said step S1a-S3a is a filter process calculation which calculates the pixel value of 4 pixels arranged in the v row among 16 pixels arranged in 4 rows and 4 columns which belong to block _Bn , step S1a~S3a from the first row of the block B _n for the fourth line, sequentially, or by executing in parallel, the pixel values of all pixels belonging to the block B _n is calculated.

  The inverse filter processing calculation executed by the filter processing unit 12a in the vertical filter processing includes the following steps T1a to T3a.

(Step T1a) inverse filtering unit 12a, the prediction average pixel values of four pixels arranged in v-th row of the block _{B n} of the image _{<q n, v>} and, in v-th row of the block _{B n + 1} in the prediction image The average pixel value <q _{n + 1, v} > of the arranged four pixels is calculated. The calculation formula used by the inverse filter processing unit 12a to calculate the average pixel values <qn _{, v} > and <qn _{+ 1, v} > is the same as [Equation 22].

(Step T2a) The inverse filter processing unit 12a performs linear interpolation using the average pixel values <qn _{, v} > and <qn _{+ 1, v} > obtained in step T1a, so that the vth row of the block _Bn A predicted value q _{pred (n, u, v)} for each pixel and a predicted value q _pred (n + 1, u, v) for each pixel in the v-th row of the block B _{n + 1} are calculated. The calculation formula used by the inverse filter processing unit 12a to calculate the predicted values q _{pred (} n, u, v) and q _pred (n + 1, u, v) is the same as [Formula 23] and [Formula 24]. is there.

(Step T3a)
The inverse filter processing unit 12a calculates the difference between the predicted value q _pred (n, u, v) obtained in step T2a and the average pixel value <q _{n, v} > of the block B _n obtained in step T1a. Is added to the processing target image, and the additional component is added to the pixel value P (n, u, v) of the processing target image, thereby obtaining the pixel value P ′ (n, u, v) of the filter output image. ) Is calculated. The calculation formula used by the inverse filter processing unit 12a to calculate the pixel value P ′ (n, u, v) of the inverse filter output image is as follows.

The steps T1a to T3a are filter processing operations for calculating the pixel values of 4 pixels arranged in the v-th row among 16 pixels arranged in 4 rows and 4 columns belonging to the block _Bn. step T1a~T3a from the first row of the block B _n for the fourth line, sequentially, or by executing in parallel, the pixel values of all pixels belonging to the block B _n is calculated.

As described above, the filter processing unit 11a calculates the pixel value to be subtracted from the image data of the target image to be filtered from the image data of the predicted image. Further, the inverse filter processing unit 12a calculates a pixel value to be added to the image data of the processing target image to be subjected to the inverse filtering process from the image data of the predicted image by the same method as the filter processing unit. That is, the pixel value that the filter processing unit 11a subtracts from the processing target image and the pixel value that the inverse filter processing unit 12a adds to the processing target image are the same image value calculated from the same predicted image. Therefore, according to the filter processing unit 11a and the inverse filter processing unit 12a, compared with the moving image encoding apparatus 300 that performs the filter processing / inverse filter processing based on the processing target image having a difference in quantization error level, the filter processing is performed. The frequency component that generates block distortion removed from the processing target image by the unit 11a can be more completely restored by the inverse filter processing unit 12a.
[Modification 2]
Another modification of the moving picture coding apparatus 300 will be described with reference to FIGS. 9 and 10.

  FIG. 9 is a functional block diagram illustrating a schematic configuration of a video encoding device 300b which is another modification of the video encoding device 300. As illustrated in FIG. 9, the moving picture coding apparatus 300a includes a DCT unit 1, a quantization unit 2, a variable length coding unit 3, an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, and an intra prediction unit 8. , An inter prediction unit 9, an encoding control unit 10, a filter processing unit 11b, and an inverse filter processing unit 12b.

  The difference between the moving image encoding device 300b and the moving image encoding device 300 (FIG. 1) is that the moving image encoding device 300a performs a filter process based on a predicted image instead of the filter processing unit 11. It is the point provided with the process part 11b and having the reverse filter process part 12a which performs the reverse filter process based on a predicted image instead of the reverse filter process part 12. A further difference is that the filter processing unit 11b is provided immediately before the DCT unit 1, and uses a difference image between the original image and the predicted image as a processing target image of the filter processing. Note that the prediction images used for the filter processing by the filter processing unit 11b and the inverse filter processing unit 12b are both supplied from the intra prediction unit 8 or the inter prediction unit 9.

  As shown in FIG. 9, the functional blocks of the moving image encoding device 300b excluding the filter processing unit 11b and the inverse filter processing unit 12b are the same as those of the moving image encoding device 300 of FIG. Therefore, in FIG. 9, blocks having the same functions as those of the moving picture coding apparatus 300 of FIG.

  FIG. 10 is a functional block diagram showing a schematic configuration of a moving picture decoding apparatus 400b corresponding to the moving picture encoding apparatus 300b shown in FIG. As illustrated in FIG. 10, the video decoding device 400b includes an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, an intra prediction unit 8, an inter prediction unit 9, a variable length decoding unit 20, and an inverse filter process. A portion 12b is provided.

  The difference between the moving image decoding apparatus 400b and the moving image decoding apparatus 400 (FIG. 2) in FIG. 10 is that the moving image decoding apparatus 400b is the same as the moving image encoding apparatus 300b, instead of the inverse filter processing unit 12. The filter processing unit 12b is provided. Further, as shown in FIG. 10, the prediction image generated by the intra prediction unit 8 and the inter prediction unit 9 is supplied to the inverse filter processing unit 12b.

  Hereinafter, the filtering process in the filter processing unit 11b suitable for the moving image coding apparatus 300b and the inverse filtering process in the inverse filter processing unit 12b suitable for the moving image coding apparatus 300 and the moving image decoding apparatus 400b will be described.

The filter processing unit 11b and the inverse filter processing unit 12b calculate the image data of each block in the filter output image from the image data of the prediction image supplied from the intra prediction unit 8 or the inter prediction unit 9. Specifically, in the filter processing in the horizontal direction, the image data of the block B _n, calculated by referring to the image data of the block B _n and the block B _{n + 1} in the prediction image, the filtering process in the vertical direction, the block the image data of B _n, is calculated by referring to the image data of the block B _n and the block B _{n + W} in the prediction image. Hereinafter, the filter processing calculation for calculating the pixel value of one block _Bn will be described. However, the filter processing unit 11b and the inverse filter processing unit 12b repeat the filter calculation processing described below to output the filter output. Complete the horizontal filtering of the entire image. Further, the entire filtering process is completed by executing the filtering process in the vertical direction in the same manner.

  The filter processing calculation executed by the filter processing unit 11b in the horizontal filter processing includes the following steps S1b to S3b.

(Step S1b) The filter processing unit 11b includes the average pixel value <q _{n, v} > of four pixels arranged in the v-th row of the block B _n in the predicted image and the v-th row of the block B _{n + 1} in the processing target image. The average pixel value <q _{n + 1, v} > of the arranged four pixels is calculated. The calculation formula for the filter processing unit 11b to calculate these average pixel values is the same as [Equation 22].

(Step S2b) The filter processing unit 11b uses the average pixel values <qn _{, v} > and <qn _{+ 1, v} > obtained in step S1b, for each pixel in the vth row of the block _Bn . A predicted value q ′ _pred (n, u, v) and a predicted value q ′ _pred (n + 1, u, v) for each pixel in the v-th row of the block B _{n + 1} are calculated. Calculation formulas for the filter processing unit 11b to calculate the predicted values q ′ _pred (n, u, v) and q ′ _pred (n + 1, u, v) are as follows.

(Step S3b) The filter processing unit 11b obtains the predicted value q ′ _pred obtained in step S2b from the pixel value p (n, u, v) of the filter processing target image (difference image between the original image and the predicted image). The pixel value p ′ (n, u, v) of the filter output image is calculated by subtracting (n, u, v). The calculation formula used by the filter processing unit 11b to calculate the pixel value p ′ (n, u, v) of the filter output image is as follows.

The steps S1b to S3b are filter processing operations for calculating the pixel values of the four pixels arranged in the vth row among the 16 pixels arranged in the 4th row and the 4th column belonging to the block _Bn. step S1b~S3b from the first row of the block B _n for the fourth line, sequentially, or by executing in parallel, the pixel values of all pixels belonging to the block B _n is calculated.

  The inverse filter processing calculation executed by the inverse filter processing unit 12b in the vertical filter processing includes the following steps T1b to T3b.

(Step T1b) inverse filtering unit 12b, predicted average pixel values of four pixels arranged in v-th row of the block _{B n} of the image _{<q n, v>} and, in v-th row of the block _{B n + 1} in the prediction image The average pixel value <q _{n + 1, v} > of the arranged four pixels is calculated. The calculation formula used by the inverse filter processing unit 12b to calculate these average pixel values is the same as [Equation 22].

(Step T2b) The inverse filter processing unit 12b uses the average pixel values <qn _{, v} > and <qn _{+ 1, v} > obtained in step T1b for each pixel in the vth row of the block _Bn. Predicted value q _pred (n, u, v) and a predicted value q _pred (n + 1, u, v) for each pixel in the v-th row of the block B _{n + 1} are calculated. The calculation formula used by the inverse filter processing unit 12b to calculate the predicted values q _pred (n, u, v) and q _pred (n + 1, u, v) is the same as [Expression 27] and [Expression 28]. .

(Step T3b)
The inverse filter processing unit 12b adds the predicted value q _pred (n, u, v) obtained in step T2b to the pixel value P (n, u, v) of the inverse filter processing target image, thereby obtaining a filter. A pixel value P ′ (n, u, v) of the output image is calculated. The calculation formula used by the inverse filter processing unit 12b to calculate the pixel value P ′ (n, u, v) of the filter output image is as follows.

The above steps T1b to T3b are filter processing operations for calculating pixel values of 4 pixels arranged in the v-th row among 16 pixels arranged in 4 rows and 4 columns belonging to the block _Bn. step T1b~T3b from the first row of the block B _n for the fourth line, sequentially, or by executing in parallel, the pixel values of all pixels belonging to the block B _n is calculated.

As described above, the filter processing unit 11b calculates the pixel value to be subtracted from the image data of the target image to be filtered from the image data of the predicted image. Further, the inverse filter processing unit 12b calculates a pixel value to be added to the image data of the processing target image to be subjected to the inverse filtering process from the image data of the predicted image by the same method as the filter processing unit. That is, the pixel value that the filter processing unit 11b subtracts from the processing target image and the pixel value that the inverse filter processing unit 12b adds to the processing target image are the same image value calculated from the same predicted image. Therefore, according to the filter processing unit 11b and the inverse filter processing unit 12b, compared with the moving image encoding apparatus 300 that performs the filter processing / inverse filter processing based on the processing target image having a difference in the degree of quantization error, the filter processing is performed. The frequency component that generates block distortion removed from the processing target image by the unit 11b can be restored more completely by the inverse filter processing unit 12b.
[Additional Notes]
In addition, this invention is not limited to embodiment mentioned above, A various change is possible in the range shown to the claim. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention. For example, the present invention can be configured as follows.

  A moving image encoding apparatus according to the present invention is a moving image encoding apparatus that divides an image into a plurality of blocks and encodes the image, and a filter processing unit that performs a predetermined filter process on the image in block units; You may comprise so that the reverse filter process means which performs the reverse filter process which is the reverse transformation of the said filter process means may be provided.

  Moreover, in the moving image encoding device according to the present invention, the filter processing unit performs a predetermined filter process on the encoding target image of the block based on the encoding target images of a plurality of adjacent blocks. It may be configured.

  The video encoding apparatus according to the present invention further includes prediction means for performing intra prediction or inter prediction for each block, and the filter processing means predicts a plurality of adjacent blocks output by the prediction means. Based on the image, a predetermined filtering process may be performed on the encoding target image of the block.

  The video encoding apparatus according to the present invention further includes prediction means for performing intra prediction or inter prediction for each block, and the filter processing means predicts a plurality of adjacent blocks output by the prediction means. Based on the image, a predetermined filter process may be performed on the difference image between the encoding target image and the predicted image of the block.

  The moving picture decoding apparatus according to the present invention is a moving picture decoding apparatus for decoding moving picture data encoded for each block, and may be configured to include the inverse filter processing means.

  Finally, the above-described blocks of the moving image encoding devices 300, 300a, 300b of the present invention and the moving image decoding devices 400, 400a, 400b of the present invention are particularly adapted to the filter processing means 11, 11a, 11b and vice versa. The filter processing means 12, 12a, and 12b may be configured by hardware logic, or may be realized by software using a CPU as follows.

  That is, the moving image encoding device / moving image decoding device develops a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, and the program. A random access memory (RAM), and a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is to read a program code (execution format program, intermediate code program, source program) of a control program of the above-described moving image encoding apparatus / moving image decoding apparatus, which is software that realizes the above-described functions, by a computer. It is also achieved by supplying a recording medium recorded as possible to the moving picture encoding apparatus / moving picture decoding apparatus, and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU). Is possible.

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the moving image encoding device / moving image decoding device may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  As described above, the moving image encoding apparatus according to the present invention divides an image into a plurality of blocks, quantizes image data of each block, and encodes a quantized representative value obtained by the quantization. The image encoding device includes filter processing means for performing filter processing for removing frequency components that generate block distortion on the image data before quantization.

  Therefore, it is possible to generate encoded data that can restore a decoded image with reduced block distortion without losing a specific frequency component.

  The moving picture decoding apparatus according to the present invention is a moving picture decoding apparatus that decodes encoded data obtained by encoding image data from which frequency components that generate block distortion are removed as described above. The image processing apparatus includes an inverse filter processing unit that performs inverse filter processing for restoring the removed frequency component on the image data of the decoded image obtained by decoding.

Accordingly, a decoded image with reduced block distortion can be decoded without losing a specific frequency component. The specific embodiments or examples made in the detailed description section of the present invention are not limited to the present invention. The technical contents of the present invention should be clarified, and should not be construed in a narrow sense by limiting only to such specific examples, but various modifications may be made within the spirit of the present invention and the following claims. Can be implemented.

  The present invention is suitably used as a moving image storage device that encodes and stores a moving image, a moving image transmission device that encodes and transmits a moving image, or a moving image reproducing device that decodes and reproduces a moving image. be able to. Specifically, for example, it can be suitably used for a hard disk recorder or a mobile phone terminal.

Claims (9)

In a video encoding device that divides an image into a plurality of blocks, quantizes the image data of each block, and encodes the quantized representative value obtained by the quantization,
Filtering means for applying a filtering process to remove frequency components that generate block distortion is applied to the image data before quantization.
The filtering means is
An average pixel value calculation process for calculating an average pixel value in each block of the processing target image to be subjected to the filtering process;
The pixel value of each pixel of the processing target image is interpolated from the average pixel value calculated in the average pixel value calculation process in the block including the pixel and the average pixel value in the adjacent block adjacent to the block. A prediction process predicted by
For each pixel of the processing target image, the difference between the predicted pixel value of the pixel predicted by the prediction process and the average pixel value of the block to which the pixel calculated by the average pixel value calculation process belongs, A removal component to be removed from the processing target image, and a subtraction process for subtracting the removal component from a pixel value of the pixel of the processing target image.
A moving picture coding apparatus characterized by the above. The filter processing performed by the filter processing means is to keep the average pixel value of each block unchanged before and after processing.
The moving picture coding apparatus according to claim 1, wherein: A predicted image generating means for generating a predicted image;
An inverse filter processing means for performing an inverse filter process corresponding to the inverse transform of the filter process on the locally decoded image;
The predicted image generation means generates the predicted image using the local decoded image subjected to the inverse filter processing as a reference image,
The quantization target image to be quantized is a difference image between the predicted image and the original image subjected to the filtering process.
The locally decoded image is an image obtained by adding the image data obtained by dequantizing the quantized representative value and the image data of the predicted image in the moving image coding apparatus.
The moving picture coding apparatus according to claim 1 or 2 , wherein A predicted image generating means for generating a predicted image;
An inverse filter processing means for performing an inverse filter process corresponding to the inverse transform of the filter process on the locally decoded image;
The predicted image generation means generates the predicted image using the local decoded image subjected to the inverse filter processing as a reference image,
The quantization target image to be quantized is obtained by performing the filtering process on the difference image between the predicted image and the original image,
The locally decoded image is an image obtained by adding the image data obtained by dequantizing the quantized representative value and the image data of the predicted image in the moving image coding apparatus.
The moving picture coding apparatus according to claim 1 or 2 , wherein Video coding that divides the difference image obtained by subtracting the predicted image from the original image into a plurality of blocks, quantizes the image data of each block, and encodes the quantized representative value obtained by the quantization In the device
  Filter processing means for applying a filter process for removing a frequency component that generates block distortion to the original image or the difference image,
  The filtering means is
    An average pixel value calculation process for calculating an average pixel value in each block of the predicted image;
    The pixel value of each pixel of the processing target image to be filtered is calculated by the average pixel value calculation process, the average pixel value in the block including the pixel, and the adjacent block adjacent to the block A prediction process for predicting by interpolation from the average pixel value;
    For each pixel of the processing target image, the difference between the predicted pixel value of the pixel predicted by the prediction process and the average pixel value of the block to which the pixel calculated by the average pixel value calculation process belongs, A removal component to be removed from the processing target image, and a subtraction process for subtracting the removal component from a pixel value of the pixel of the processing target image.
A moving picture coding apparatus characterized by the above. A video decoding device that decodes encoded data obtained by encoding image data from which frequency components that generate block distortion are removed,
An inverse filter process for restoring the removed frequency component is performed on the difference image obtained by decoding the encoded data or the image data of the decoded image obtained by adding the prediction image to the difference image. An inverse filter processing means for applying,
The inverse filter processing means includes:
An average pixel value calculation process for calculating an average pixel value in each block of the processing target image to be subjected to the inverse filtering process ;
The pixel value of each pixel of the processing target image is interpolated from the average pixel value calculated in the average pixel value calculation process in the block including the pixel and the average pixel value in the adjacent block adjacent to the block. A prediction process predicted by
For each pixel of the processing target image, the difference between the predicted pixel value of the pixel predicted by the prediction process and the average pixel value of the block to which the pixel calculated by the average pixel value calculation process belongs, An additional component to be added to the processing target image, and adding the additional component to a pixel value of the pixel of the processing target image.
A moving picture decoding apparatus characterized by the above.   A video decoding device that decodes encoded data obtained by encoding image data from which frequency components that generate block distortion are removed,
  An inverse filter process for restoring the removed frequency component is performed on the difference image obtained by decoding the encoded data or the image data of the decoded image obtained by adding the prediction image to the difference image. An inverse filter processing means for applying,
  The inverse filter processing means includes:
    An average pixel value calculation process for calculating an average pixel value in each block of the predicted image;
    The pixel value of each pixel of the processing target image to be filtered is calculated by the average pixel value calculation process, the average pixel value in the block including the pixel, and the adjacent block adjacent to the block A prediction process for predicting by interpolation from the average pixel value;
    For each pixel of the processing target image, the difference between the predicted pixel value of the pixel predicted by the prediction process and the average pixel value of the block to which the pixel calculated by the average pixel value calculation process belongs, An additional component to be added to the processing target image, and adding the additional component to a pixel value of the pixel of the processing target image.
A moving picture decoding apparatus characterized by the above. A moving image recording apparatus comprising the moving image encoding device according to any one of claims 1 to 5 , wherein the moving image recording records a moving image encoded by the moving image encoding device. apparatus. A moving image reproducing apparatus comprising the moving image decoding apparatus according to claim 6 , wherein the moving image reproducing apparatus reproduces a moving image decoded by the moving image decoding apparatus.

Images