WO2007108254A1

WO2007108254A1 - Moving picture coding device and moving picture decoding device

Info

Publication number: WO2007108254A1
Application number: PCT/JP2007/052575
Authority: WO
Inventors: Maki Takahashi; Tomoko Aono
Original assignee: Sharp Kabushiki Kaisha
Priority date: 2006-03-17
Filing date: 2007-02-14
Publication date: 2007-09-27
Also published as: JP2011166832A; JP4768011B2; JPWO2007108254A1

Abstract

Since a moving picture coding device (300) is provided with a filter processing unit (11) for eliminating frequency components which cause block distortions to picture data prior to quantizing, the moving picture coding device can generate coding data to restore a decoded picture in which block distortions are reduced without elimination of specific frequency components. Thus, a moving picture coding device is put into practice which uses no deblocking filter which causes a picture a blur or flicker but can reduce block distortions in a decoded moving picture.

Description

Specification

Moving picture encoding apparatus and moving picture decoding apparatus

Technical field

The present invention relates to a moving image encoding device and a moving image decoding device that divide and encode a quantization target image into a plurality of blocks.

Background art

[0002] In order to efficiently transmit and record a moving image having a huge amount of information, a technique of dividing image data into a plurality of blocks and quantizing and coding each block is widely used. It has been. In a moving image quantized and encoded for each block, block distortion (block noise) in which pixel values such as luminance levels discontinuously change occurs at the boundary of each block. As a technique for reducing the block distortion, a deblocking filter in the H.264 / AVC moving image code system described in Non-Patent Document 1 is known.

[0003] Hereinafter, with respect to the moving image encoding device 100 that encodes moving images by the H.264ZAVC moving image encoding method and the moving image decoding device 200 that decodes moving images encoded by the same method, FIG. Description will be made with reference to FIG.

[Configuration of Conventional Video Encoding Device]

FIG. 11 is a functional block diagram showing a schematic configuration of a moving picture encoding apparatus 100 that encodes a moving picture using the H.264ZAVC moving picture coding method. As shown in FIG. 11, the moving image coding apparatus 100 includes a DCT unit 1, a quantization unit 2, a variable length coding unit 3, an inverse quantization unit 4, an I DCT unit 5, a deblocking filter processing unit 6, A frame memory 7, an intra prediction unit 8, an inter prediction unit 9, and a code key control unit 10 that controls each of the above units are provided.

[0005] The DCT unit 1 divides a difference image obtained by subtracting a prediction image, which will be described later, from an original image into blocks each consisting of 4 × 4 pixels or 8 × 8 pixels, and orthogonally transforms the image signal of each block ( Integer precision DCT). The transform coefficient obtained by the orthogonal transform (corresponding to the DCT coefficient in the discrete cosine transform) is sent to the quantization unit 2. The quantization unit 2 quantizes the transform coefficient of each block according to the quantization parameter supplied from the encoding control unit 10. The quantized representative value obtained as a result of quantization is sent to the variable length coding unit 3 and the inverse quantization unit 4. The

[0006] The variable-length encoding unit 3 receives various encoding parameters supplied from the encoding control unit 10 and quantization representative values (transform coefficients of each quantized block) supplied from the quantization unit 3. Variable length coding. The code key data obtained as a result of the code key by the variable length code key unit 3 is sent to the moving picture decoding apparatus 200 described later.

[0007] The inverse quantization unit 4 reverses the quantized representative value (transform coefficient of each quantized block) supplied from the quantization unit 3 according to the quantization parameter supplied from the encoding control unit 10. Quantify. That is, the inverse quantization unit 4 restores the transform coefficient of each block from the quantized representative value by the reverse operation of the quantization operation by the quantization unit 2. The transform coefficient of each block restored by inverse quantization is sent to the IDCT unit 5. The IDCT unit 5 converts the transform coefficient of each block obtained by inverse quantization into an image signal in the spatial domain, and restores the difference image. Here, the inverse orthogonal transform applied to the transform coefficient by the IDCT unit 5 is the inverse transform (integer precision IDCT) of the orthogonal transform applied by the DCT unit 1. The local decoded image obtained by adding the difference image restored by the IDCT unit 5 and the predicted image is sent to the deblocking filter processing unit 6.

[0008] The deblocking filter processing unit 6 performs adaptive filtering on the local decoded image in order to remove block distortion in the local decoded image obtained by adding the prediction image and the difference image. Details of adaptive filter processing by the deblocking filter processing unit 6 will be described in detail later.

The locally decoded image from which block distortion has been removed by the deblocking filter processing unit 6 is temporarily stored in the frame memory 7. The frame memory 7 can store a plurality of locally decoded images. The locally decoded image stored in the frame memory 7 is referred to by the intra prediction unit 8 or the inter prediction unit 9 as a reference image.

[0010] The intra prediction unit 8 generates a predicted image from the reference image recorded in the frame memory 7 by performing intra-frame prediction. The intra-prediction unit 8 is capable of performing intra-frame prediction using a plurality of prediction modes (prediction algorithms) defined in accordance with the H.264ZAVC video coding standard. Intraframe prediction is performed according to the prediction mode specified by Section 10. [0011] Based on the motion vector determined by the sign key control unit 10 and the reference image stored in the frame memory 7, the inter prediction unit 9 generates a prediction image by inter-frame prediction (motion compensation prediction). Generate. The inter prediction unit 9 performs inter-frame prediction using a block having a size specified by the sign key control unit 10 and using a plurality of reference images specified by the sign key control unit 10.

[0012] The code control unit 10 determines which prediction method to generate a prediction image of intra-frame prediction and inter-frame prediction, and various encoding parameters according to the determined prediction method. To decide. The code parameter when performing intra-frame prediction includes information specifying a prediction mode in intra-frame prediction. In addition, the code parameter when performing inter-frame prediction includes information specifying a motion vector, a block size, and a reference image. Further, the sign key control unit 10 designates quantization parameters for the quantization unit 2 and the inverse quantization unit 4.

Next, the code key operation of moving picture code key device 100 shown in FIG. 11 will be described. Generally speaking, the moving image coding apparatus 100 encodes a moving image by repeating the following steps 1 to 6.

(Step 1) Code Key Control Unit 10 Determines whether intra-frame prediction or inter-frame prediction is performed, and determines the code key parameters and quantization parameters necessary for the code key.

(Step 2) In accordance with the determination result in Step 1, the intra prediction unit 8 or the inter prediction unit 9 generates a predicted image based on the reference image stored in the frame memory 7.

(Step 3) A difference image between the predicted image generated in Step 2 and the input original image is generated and supplied to the DCT unit 1.

(Step 4) The DCT unit 1 and the quantization unit 2 orthogonally transform the image signal of the difference image obtained in step 3 for each block, and quantize the obtained transform coefficients. The obtained quantization representative value is variable-length encoded by the variable-length encoding unit 3 and output as encoded data.

(Step 5) The inverse quantization unit 4 and the IDCT unit 5 dequantize the quantized representative value obtained in step 4 to restore the difference image.

[0019] (Step 6) The difference image restored in Step 5 and the prediction generated in Step 2 The images are added to each other, and the obtained locally decoded image is supplied to the deblocking filter processing unit 6.

[0020] (Step 7) The local decoding image power block distortion obtained in Step 6 is removed by the deblocking filter processing unit 6 and the local decoded image with reduced block distortion is stored in the frame memory 7 as a reference image. Accumulated.

[Configuration of Conventional Video Decoding Device]

Next, a video decoding device 200 that decodes a video encoded by the H.264ZAVC video encoding method will be described with reference to FIG. As shown in FIG. 12, the moving image decoding apparatus 200 includes an inverse quantization unit 4, an IDCT unit 5, a deblocking filter processing unit 6, a frame memory 7, an intra prediction unit 8, an inter prediction unit 9, and a variable A long decoding unit 20 is provided.

[0022] Here, among the functional blocks constituting the moving picture decoding apparatus 200, the variable length decoding section 20 is the only functional block that does not exist in the moving picture encoding apparatus 100 (FIG. 11). Blocks having the same functions as those of the moving image code display device 100 have the same names and reference numerals and description thereof is omitted. The variable length decoding unit 20 has a function of variable length decoding the encoding parameter and the quantized representative value (quantized transform coefficient).

[0023] Generally speaking, the moving picture decoding apparatus 200 shown in Fig. 12 decodes encoded data by repeating the following steps 1 to 6.

(Step 1) The variable length decoding unit 20 performs variable length decoding on the encoded data to obtain an encoding parameter and a quantized representative value (quantized transform coefficient).

(Step 2) In accordance with the decoded code parameter, the intra prediction unit 8 or the inter prediction unit 9 generates a prediction image based on the reference image stored in the frame memory 7.

(Step 3) The inverse quantization unit 4 and the IDCT unit 5 dequantize the quantized representative value obtained in step 1 to restore the difference image.

(Step 4) The difference image restored in Step 3 and the predicted image generated in Step 2 are added, and the obtained decoded image is supplied to the deblocking filter processing unit 6.

[0028] (Step 5) The decoded image obtained in step 4 by the deblocking filter processing unit 6 Image force Block distortion is removed, and a decoded image with reduced block distortion is stored in the frame memory 7. The decoded image stored in the frame memory 7 can be read at an arbitrary timing and used as a display image.

[0029] (Step 6) The decoded image stored in the frame memory is output to an image display means such as a display at an appropriate timing as a display image.

[Deblocking filter]

As described above, in the H.264ZAVC moving image encoding method, by using the common deblocking filter processing unit 6 in both the moving image encoding device 100 and the moving image decoding device 200, the moving image is encoded. Quantization · Block distortion generated in the inverse quantization process is reduced. Hereinafter, the deblocking filter processing unit 6 will be described in more detail with reference to FIGS.

FIG. 13 is an explanatory diagram showing a block division pattern of a quantization target image (difference image between an original image and a predicted image) in a moving image encoding process using the H.264ZAVC moving image encoding method. As shown in FIG. 13, the quantization target image is divided into WX H rectangular blocks. WX H blocks include B ~ in order from the upper left corner of the image to be quantized.

0

B's

WXH-1 code is attached.

FIG. 14 is a diagram showing two adjacent blocks, block B and block B, in the quantization target image shown in FIG. As shown in Figure 14, Block B and Block B

n + l n n + 1 is composed of a total of 16 pixel arrays arranged in 4 rows and 4 columns. As with block B, all blocks B to B that make up the image to be encoded in Fig. 13 are arranged in 4 rows and 4 columns.

0 WXH- 1 n

It consists of an array of 16 pixels.

[0032] A pixel constituting the quantization target image is a combination of a variable n for specifying a block including the pixel and variables _u and V for specifying the position of the pixel in the block ( _n , _u , v ) Can be specified. That is, the pixel (n, u, v) is the pixel in the u-th column and the V-th row of the block B. In the following description, the attribute value of the pixel (n, u, v) is expressed as X (n, u, v). For example, the pixel value of the pixel (n, u, v) in the encoding target image is expressed as P (n, u, v).

[0033] Blocks B to B shown in FIG. 13 and FIG. Transformation 'Quantization' Dequantization 'Inverse orthogonal transformation is a processing unit of a series of processing. In other words, the image data of the quantization target image is converted into a conversion coefficient for each of the blocks B to B, and the amount is converted.

0 WXH- 1

It becomes a child.

[0034] However, since quantization is an irreversible process, even if the image to be quantized is restored by inverse quantization and inverse orthogonal transformation, the restored image matches the image to be encoded. Instead, block distortion occurs at the boundary between adjacent blocks in the restored image. The deblocking filter processing unit 6 reduces this block distortion.

[0035] The deblocking filter processing unit 6 used in the H.264ZAVC moving image encoding method uses filtering between adjacent blocks (B, B) in the horizontal direction and adjacent n n + 1 in the vertical direction.

Filter processing between adjacent blocks (B, B) is performed independently. Deblocky n n + W

The filter strength of the filter processing performed by the filtering filter processing unit 6 is adaptively set according to conditions such as the prediction mode applied to each block.

[0036] As an example of the filter processing performed by the deblocking filter processing unit 6, the filter processing performance with the strongest filter strength for the blocks B and B adjacent in the horizontal direction n n + 1

The calculation is shown below. In the following equations, P (n, u, v) is a pixel in the processing target image (local decoded image in the moving image encoding device 100 or decoded image in the moving image decoding device 200) of the deblocking filter processing. The pixel value (n, u, v) and P ′ (n, u, v) represent the pixel value of the pixel (n, u, v) in the filter output image output from the deblocking filter processing unit 6.

[0037] [Equation 1]

P '(n, 0, v) = P (n, 0, v)

[0038] [Equation 2]

P '0,1, V) = 丄 ί2Ρ (η, 0, ν) + 3Ρ (, 1, ν) + Ρ {η, 2, ν) + Ρ (η, 3, ν) + Ρ (η + 1 , 0, ν)}

[0039] [Equation 3] P '(n, 2, v) =-{P (n, v) + P (n, 2, v) + («, 3, v) + P (n + 1,0, v)}

[0040] [Equation 4]

P '(", 3, v) = 丄 (P (n, l, v) + 2P (n, 2, v) + 2P (n, 3, v) + 2P (n + 1,0, v) + P n + 1,1,)}

8

[0041] [Equation 5]

P '(n + 1,0, v) = 丄 (P (n, 2, v) + 2P (n, 3, v) + 2P (n + 1,0, v) + 2P (n + 1,1 , v) + P (n + l, 2, v)}-8

[0042] Garden '(n + 1,1, v) =-(P (n, 3, v) + P (n + 1,0, v) + P (n + 1,1, v) + P ( n + 1,2, v)}

[0043] [Equation 7]

P '(n + l, 2, v) =-(P (n, 3, v) + P (n + l, 0, v) + P (n + 1,1, v) + 3P (n + 1 , 2, v) + 2P (n + 1,3, v)}

8

[0044] [Equation 8]

P '(n + l, 3, v) = P (n + l, 3, v)

[0045] Although detailed description is omitted, the same filter processing is applied to adjacent blocks in the vertical direction.

[0046] The effects of the filter processing on the processing target image are shown in FIGS. Figure 15 (a) is a graph n n + 1 representing the pixel values p (n, u, v) of the original image in block B and block B.

It is. Figure 15 (b) shows the deblocking filter processing n n + 1 for block B and block B.

7 is a graph showing pixel values P (n, u, v) of a decoded image to be processed by the processing unit 6. As shown in FIG. 15 (b), in the decoded image, discontinuous changes in pixel values at the block boundary, that is, block distortion occurs. Figure 15 (c) shows the deblocking pattern on the decoded image. Block B and Block B of the image after filtering by filter 6

It is a graph showing pixel values ( _n , _u , v) in n n + 1. Since the deblocking filter 6 convolves pixel values between adjacent blocks, discontinuous changes in pixel values in the restored target image are smoothed, and block distortion is reduced. The

Non-Patent Literature 1: ITU-T Recommendation H.2b4: Advanced Video Coding for generic audiovisual services (2003)

Disclosure of the invention

However, in the above-described conventional video encoding device, as a side effect of reducing the block distortion by the deblocking filter, it is different from the block distortion such as blurring of the image or flickering of the image at the time of moving image reproduction. There is a problem that image quality degradation occurs.

[0048] The side effects of the deblocking filter will be described in more detail below. As is clear from the equations (1) to (8) described above, if the pixel value of the image to be filtered is P (n, u, v), the average pixel value of the block B before filtering is P> The average pixel value of block B after filtering is given by the following equations.

[0049] [Equation 9]

[0050] [Equation 10]

10P («, 0, v) + 6P (n, l, v) + 6P (n, 2, v) + 10 尸 (<<, 3, v)

+ 10 O + 1,0, v) + 6P (n + 1,1, v) + 6P (n + 1,2, v) + 10P (n + 1,3, v)

That is, the average pixel value in each block is different before and after the filter processing. Therefore, when the conventional deblocking filter is used, the average pixel value in each block is not saved before and after the filtering process, so that the difference in the average pixel value between the original image and the decoded image is enlarged by the filtering process. obtain.

[0052] Then, the moving image encoding apparatus has a difference in average pixel values from the original image. A predicted image is generated using the locally decoded image to be used as a reference image. In the moving image decoding apparatus, a decoded image having an average pixel value difference with respect to the original image is generated as a reference image or a display image. For this reason, blurring or flickering of the moving image occurs.

[0053] The present invention has been made in view of the above-described problems, and an object of the present invention is to reduce block distortion in a decoded moving image and to prevent side effects such as blurring and flickering of the image. An object is to realize a moving image encoding device and a moving image decoding device.

[0054] In order to solve the above-described problem, the video encoding apparatus according to the present invention divides an image into a plurality of blocks, quantizes the image data of each block, and obtains a quantum obtained by the quantization. In the moving picture encoding apparatus for encoding the representative representative value, the apparatus includes a filter processing means for performing a filter process for removing a frequency component that generates block distortion on the image data before quantization. The

[0055] The filter processing means removes a frequency component that generates block distortion from the image data before quantization. For this reason, the quantization target image to be quantized is one in which frequency components that generate block distortion have been removed in advance. Therefore, a decoded image obtained by quantizing / inverse-quantizing the quantization target image subjected to the filtering process by the filter processing unit has a reduced block distortion. However, the frequency component previously removed by the filter processing means can be restored by inverse filter processing corresponding to the inverse transformation of the filter processing of the filter processing means.

That is, the moving picture coding apparatus having the above-described configuration can generate coded data that can restore a decoded picture with reduced block distortion without missing a specific frequency component. And! /, Has the effect.

[0057] In the video image encoding apparatus of the present invention, it is preferable that the filter processing performed by the filter processing means is to keep the average pixel value of each block unchanged before and after the processing. .

[0058] According to the above configuration, the filter processing by the filter processing means is performed so as to keep the average pixel value (for example, average luminance level) of each block unchanged. For this reason, the conventional video coding apparatus using a deblocking filter has been There is an additional effect that it is possible to effectively prevent screen blurring caused by the fact that the average pixel value for each block is not stored before and after the filter processing.

[0059] In the moving picture coding apparatus of the present invention, a predicted image generating means for generating a predicted image and an inverse filter process corresponding to the inverse transform of the filter process are performed on the locally decoded image. It is preferable that the prediction image generation unit further includes a reverse filter processing unit, and the prediction image generation unit generates the prediction image using the locally decoded image subjected to the reverse filter processing as a reference image.

[0060] According to the above configuration, the predicted image generation means can generate a predicted image using the local decoded image subjected to the inverse filter process as a reference image. In other words, the predicted image generation means may generate a predicted image based on a reference image in which block distortion is reduced and a specific frequency component is not lost, that is, a reference image that approximates the original image. There is an effect that can be done.

[0061] As a method for the predicted image generating means to generate a predicted image, intra-frame prediction, inter-frame prediction, or the like can be used. Further, the predicted image generation means may use these prediction methods by appropriately switching these prediction methods according to the state of the image to be encoded.

[0062] Note that the inverse filter processing performed by the inverse filter processing means may correspond to a strict inverse transformation or an approximate inverse transformation with respect to the filter processing of the filter processing means. It may be what you do. In other words, the inverse filtering process is sufficient if it approximates a strict inverse transform within a range that does not adversely affect the quality of the final moving image. For example, the calculation accuracy of the filtering process (for example, integer precision) A degree of error is acceptable.

[0063] In the video encoding device of the present invention, the quantization target image to be quantized is a difference image between the predicted image and the original image subjected to the filtering process. The local decoded image is an image obtained by adding the image data obtained by inversely quantizing the quantized representative value to the moving image coding apparatus and the image data of the predicted image. Is preferred.

[0064] According to the above configuration, the original image becomes a processing target image of the filter processing, and the filter The difference image between the original image subjected to the data processing and the predicted image becomes the quantization target image. Then, the local decoded image obtained by adding the image data obtained by dequantizing the quantized representative value obtained by quantizing the quantization target image and the image data of the predicted image is the result of the inverse filter processing. The local decoded image subjected to the above inverse filter processing is used as a reference image for generating a predicted image. Therefore, according to the above configuration, the predicted image generating means can generate a predicted image based on the reference image that better approximates the original image.

[0065] In the video encoding device of the present invention, the quantization target image to be quantized is obtained by performing the filtering process on a difference image between the predicted image and the original image. The locally decoded image is preferably an image obtained by adding the image data obtained by dequantizing the quantized representative value and the image data of the predicted image in the moving image coding apparatus. .

[0066] According to the above configuration, the difference image between the predicted image and the original image is the target of the filtering process, and the difference image subjected to the filtering process is the target of quantization. On the other hand, a local decoded image obtained by adding the image data obtained by dequantizing the quantization representative value obtained by quantizing the quantization target image and the image data of the predicted image is the inverse filter. The locally decoded image that has been subjected to the processing and subjected to the inverse filter processing is used as a reference image for generating a predicted image. Therefore, according to the above configuration, the predicted image generation means can generate a predicted image based on the reference image that better approximates the original image.

[0067] In the video encoding device of the present invention, the filter processing means calculates a pixel value to be subtracted from the image data of the processing target image that is the target of the filtering process, as an image data force of the predicted image. The inverse filter processing means calculates a pixel value to be added to the image data of the processing target image to be subjected to the inverse filtering process by calculating the image data force of the predicted image by the same method as the filter processing means. It is preferable that

[0068] According to the above configuration, the pixel value that the filter processing unit subtracts from the image data of the processing target image and the pixel value that the inverse filter processing unit adds to the image data of the processing target image are the same. Is calculated from the predicted image by the same method. Therefore, the above fill The inverse filter processing means can restore the frequency component that generates block distortion removed from the processing target image by the data processing means more completely.

[0069] In order to solve the above-described problem, the moving image decoding apparatus according to the present invention decodes encoded data obtained by encoding image data from which frequency components that generate block distortion are removed. An image decoding apparatus, comprising: inverse filter processing means for performing inverse filter processing for restoring the removed frequency component on the image data of a decoded image obtained by decoding! / Speak.

[0070] The moving image decoding apparatus decodes encoded data obtained by encoding image data from which a frequency component causing block distortion has been removed in advance. Here, since the encoded data is obtained by removing frequency components that generate block distortion in advance, block distortion in a decoded image obtained by decoding is suppressed. The decoded image obtained by decoding restores the frequency component removed in the encoding process by the inverse filter processing means. Therefore, a decoded image obtained by performing the inverse filter processing is an image in which block distortion is reduced and a specific frequency component is not lost, that is, an image that better approximates the original image.

That is, the moving picture coding apparatus having the above configuration has an effect of being able to restore a decoded picture with reduced block distortion without losing a specific frequency component.

[0072] Other objects, features, and advantages of the present invention will be sufficiently understood from the following description. The advantages of the present invention will be apparent from the following description with reference to the accompanying drawings.

Brief Description of Drawings

FIG. 1 is a functional block diagram showing a configuration of a moving picture encoding apparatus according to an embodiment of the present invention.

FIG. 2 is a functional block diagram showing a configuration of a moving picture decoding apparatus according to an embodiment of the present invention.

3 is a flowchart showing an outline of filter processing calculation by the filter processing unit of the video encoding device shown in FIG. 1. FIG. FIG. 4 (a) is a graph showing the effect of the filter processing operation shown in FIG. 3, and is a graph showing the pixel value of the processing target image to be subjected to the filter processing.

[FIG. 4 (b)] is a graph showing the operation of the filter processing calculation shown in FIG.

[FIG. 4 (c)] is a graph showing the operation of the filter processing operation shown in FIG. 3, and is a graph showing a predicted value obtained by linear interpolation using the average pixel value shown in FIG. 4 (b). .

[FIG. 4 (d)] is a graph showing the effect of the filter processing calculation shown in FIG. 3, and is a graph showing the pixel value of the filter output image.

FIG. 5 is a flowchart showing an overview of the inverse filter processing calculation by the inverse filter processing unit included in the video encoding device shown in FIG. 1 and the video decoding device shown in FIG.

FIG. 6 (a) is a graph showing the operation of the inverse filter processing operation shown in FIG. 5, and is a graph showing the pixel value of the processing target image to be subjected to the inverse filter processing.

[FIG. 6 (b)] is a graph showing the operation of the inverse filter processing operation shown in FIG.

[FIG. 6 (c)] is a graph showing the effect of the inverse filter processing operation shown in FIG. 5, and is a graph showing the predicted value obtained by linear interpolation using the average pixel value shown in FIG. 6 (b). is there.

[FIG. 6 (d)] is a graph showing the operation of the inverse filter processing calculation shown in FIG. 5, and is a graph showing the pixel value of the inverse filter output image.

[7] FIG. 7 is a functional block diagram showing another configuration example of the video encoding device according to the embodiment of the present invention.

8] Another example of the configuration of the video decoding device according to the embodiment of the present invention is shown, and is a functional block diagram showing the configuration of the video decoding device corresponding to the video encoding device of FIG. is there.

[9] FIG. 9 is a functional block diagram showing another configuration example of the video encoding device according to the embodiment of the present invention.

圆 10] Another example of the configuration of the video decoding device according to an embodiment of the present invention is shown, and a functional block showing the configuration of the video decoding device corresponding to the video encoding device of FIG. FIG.

[Fig. 11] Fig. 11 is a functional block diagram showing a conventional technique and showing a configuration of a moving picture coding apparatus provided with a deblocking filter.

FIG. 12 is a functional block diagram showing a conventional technique and showing a configuration of a video decoding device corresponding to the video encoding device shown in FIG. 11.

FIG. 13 is an explanatory diagram showing an image division pattern in a quantization target image to be quantized or a processing target image to be filtered.

14 is an enlarged view showing two adjacent blocks in an image divided into a plurality of blocks by the division pattern shown in FIG. 13. FIG.

FIG. 15 (a) is a graph showing the prior art and showing the action of the deblocking filter. In particular, it is a graph showing pixel values of an original image to be encoded.

FIG. 15 (b) is a graph showing the prior art and showing the action of the deblocking filter. In particular, it is a graph showing pixel values of a locally decoded image including block distortion.

FIG. 15 (c) is a graph showing the prior art and showing the action of the deblocking filter. It is a graph which especially shows the pixel value of the output image of a deblocking filter.

Explanation of symbols

300 Video encoder

400 video decoding device

1 DCT 咅

2 Quantization part

3 Variable length coding unit

4 Inverse quantization section

5 IDCT 咅

7 frame memory

8 Intra prediction section

9 Inter prediction section

10 Coding control unit

11 Filter processing section (filter processing means) 12 Inverse filter processing unit (inverse filter processing means)

300a, 300b video encoding device

400a, 400b video decoding device

BEST MODE FOR CARRYING OUT THE INVENTION

One embodiment of the moving picture encoding apparatus of the present invention will be described below with reference to FIGS. 1 to 10.

[Configuration of Moving Picture Encoding Device]

First, the configuration of the moving picture coding apparatus according to the present embodiment will be described. FIG. 1 is a functional block diagram showing a schematic configuration of a video encoding device 300 according to the present embodiment. As shown in FIG. 1, the moving picture coding apparatus 300 includes a DCT unit 1, a quantization unit 2, a variable length coding unit 3, an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, and an intra prediction unit. 8. Inter prediction unit 9, encoding control unit 10, filter processing unit 11, and inverse filter processing unit 12 are provided.

[0077] The difference between the moving picture coding apparatus 300 in FIG. 1 and the conventional moving picture coding apparatus 100 (FIG. 11) is that the moving picture coding apparatus 300 uses a filter instead of the conventional deblocking filter 6. The processing unit 11 and the inverse filter processing unit 12 are provided. In FIG. 1, blocks having the same functions as those of the conventional moving image coding apparatus 100 are shown using the same names and symbols as those in FIG. 11, and the description thereof is omitted.

A characteristic configuration of the moving image encoder 300 is a filter processing unit 11 and an inverse filter processing unit 12. The filter processing unit 11 uses the original image to be encoded as a processing target image, and performs a filtering process to remove a frequency component that causes block distortion from the processing target image. The DCT unit 1 is supplied with a differential image obtained by subtracting the prediction image generated by the intra prediction unit 8 or the inter prediction unit 9 from the image subjected to the filter processing by the filter processing unit 11.

[0079] On the other hand, the inverse filter processing unit 12 calculates the difference image restored by the inverse quantization unit 4 and the IDCT unit 5 and the prediction image generated by the intra prediction unit 8 or the inter prediction unit 9 from the calorie calculation. Thus, a locally decoded image obtained is supplied. The inverse filter processing unit 12 sets this locally decoded image as a processing target image, and performs filtering on the processing target image by the filter processing unit 11. Inverse filter processing corresponding to inverse transformation of data processing is performed. The locally decoded image that has been subjected to the inverse filter processing by the inverse filter processing unit 12 is stored in the frame memory 7 as a reference image, and is used for generation of a predicted image by the intra prediction unit 8 or the inter prediction unit 9.

[0080] The DCT unit 1 and the quantization unit 2 divide the quantization target image into a plurality of blocks and quantize the image data of each block. Therefore, a block image can be included in a decoded image obtained by inverse quantization of a quantized image. That is, block distortion may be included in a local decoded image generated inside the moving image encoding apparatus 300 to generate a predicted image and a decoded image generated by a moving image decoding device described later. However, in the moving image coding apparatus 300, the quantization target image supplied to the DCT unit 1 is a difference image between the image subjected to the filter processing by the filter processing unit 11 and the predicted image. Since the filter processing unit 11 acts on the processing target image so as to remove in advance the frequency component that generates block distortion, the block distortion generated in the quantization / inverse quantization process can be effectively reduced.

[0081] Further, the reference image that the intra prediction unit 8 or the inter prediction unit 9 refers to in order to generate a predicted image is a locally decoded image that has been subjected to the inverse filter processing by the inverse filter processing unit 12. Therefore, the frequency component removed from the original image is restored in the reference image. In other words, the intra prediction unit 8 and the inter prediction unit 9 can generate a prediction image based on a reference image in which a specific frequency component is not lost while the block distortion is reduced.

In other words, the spatial frequency component lost due to the quantization process in the conventional video encoding device 100 is removed and removed by the filter processing unit 11 in the video encoding device 300, and the reverse operation is performed. Restoration by the filter processing unit 12 can avoid the influence of quantization processing and suppress the occurrence of block distortion.

[0083] Also, unlike the deblocking filter processing unit 6 in the conventional video encoding device 100, the filter processing unit 11 and the inverse filter processing unit 12 are, as will be described later, average pixel values before and after processing in each block ( For example, an average luminance level) can be maintained. Therefore, it is possible to avoid problems such as image blurring and flickering during moving image playback caused by the deblocking filter processing. [Configuration of video decoding apparatus]

FIG. 2 is a functional block diagram showing a schematic configuration of a video decoding device 400 corresponding to the video encoding device 300 shown in FIG. As shown in FIG. 2, the moving picture decoding apparatus 400 includes an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, an intra prediction unit 8, an inter prediction unit 9, a variable length decoding unit 20, and an inverse filter. A processing unit 12 is provided.

The difference between the moving picture decoding apparatus 400 in FIG. 2 and the conventional moving picture decoding apparatus 200 (FIG. 12) is that the moving picture decoding apparatus 400 uses an inverse filter processing unit instead of the conventional deblocking filter 6. It is a point with 12. In FIG. 2, blocks having the same functions as those of the conventional video decoding device 200 are denoted by the same names and symbols as those in FIG. 12, and description thereof is omitted.

[0085] The encoded data decoded by the moving image decoding apparatus 400 is generated based on the original image from which the frequency component causing block distortion is removed in the moving image encoding apparatus 300. The variable length decoding unit 20 of the video decoding device 400 decodes the encoded data, and the inverse quantization unit 4 and the IDCT unit 5 inversely quantize the value obtained by the decoding. The image restored by inverse quantization corresponds to the difference image obtained by subtracting the predicted image from the original image. The video decoding device 400 adds the prediction image generated by the intra prediction unit 8 or the inter prediction unit 9 to the restored difference image, and generates a decoded image.

[0086] The inverse filter processing unit 12 included in the video decoding device 400 performs the same inverse filter processing as the inverse filter processing unit 12 included in the video encoding device 300. In other words, the inverse filter processing unit 12 acts on the decoded image lacking the frequency component that generates block distortion so as to restore the frequency component removed at the time of encoding. The decoded image that has been subjected to the inverse filter processing by the inverse filter processing unit 12 is referred to by the intra prediction unit 8 or the inter prediction unit 9 as a reference image for generating a prediction image, and is output as a display image. .

As described above, the frequency component removed from the original image is restored in the display image or the reference image by the operation of the inverse filter processing unit 12. That is, the moving picture decoding apparatus 400 has a specific frequency component missing at the same time that the block distortion is reduced. It is possible to output a display image that never occurs. Further, at the same time that the block distortion is reduced, a prediction image can be generated based on a reference image in which a specific frequency component is not lost.

[Details of Filter Processing]

Next, filter processing performed by the filter processing unit 11 included in the moving image encoding device 300 will be described in detail.

The filter processing unit 11 executes horizontal filter processing and vertical filter processing by dividing a processing target image to be filtered into a plurality of blocks. Here, the division pattern in which the filter processing unit 11 divides the processing target image is as shown in FIGS. 13 and 14, and the DCT unit 1 and the quantization unit 2 perform the quantization target image for quantization. This is the same as the division pattern for dividing.

[0090] The filter processing unit 11 converts the image data of the block B in the filter output image output by the filter processing unit 11 into the image data of the block B in the processing target image and the image of the block adjacent to the block B. Calculate from the data. In the horizontal filter processing, the image data of block B is calculated by referring to the block B + 1 image data adjacent to the right side of the block B, and in the vertical filter processing, the image of block B Data is the image data of block B adjacent to the lower side of the block n n + W

Calculated with reference to the data.

Hereinafter, the filter processing calculation executed by the filter processing unit 11 in the horizontal filter processing will be described with reference to FIG. 3 and FIG. A filter processing calculation for calculating a pixel value (for example, a luminance level) of one block B will be described. The filter processing unit 11 completes the horizontal filter processing by repeating the filter processing calculation described below for all two adjacent blocks. In addition, the above repetition is performed by (B, B), (B, B), (B, B)...

1 2 3 4 5 6

The above filter processing operations may be executed for all pairs, and (B, B), (B, B), (B ,: B). So for all blocks,

1 2 2 3 3 4

The filter processing calculation may be executed while referring to the next block. However, in these repetitions, a pair of non-adjacent blocks, for example, the right edge of the image It is assumed that the above filter processing is not performed for a pair consisting of block B (k is an integer) and the next block B located at the lower left end of the block or kXW-1 kXW.

FIG. 3 is a flowchart schematically illustrating the filter processing calculation executed by the filter processing unit 11. As shown in FIG. 3, the filter processing calculation by the filter processing unit 11 includes step S1 for calculating the average pixel value, step S2 for calculating the predicted value, and step S3 for calculating the filter output image. It is out. Steps S1 to S3 will be described in more detail as follows. In the following description, a filter processing operation for calculating the pixel value of 4 pixels arranged in the V-th row among the 16 pixels arranged in 4 rows and 4 columns belonging to the block B will be described. The filter processing unit 11 executes the following filter processing operation on the first row of block B as well as on the fourth row, either sequentially or in parallel, so that the pixel values of all the pixels belonging to block B are obtained. Is calculated.

(Step S1) The filter processing unit 11 calculates the average pixel value of the four pixels arranged in the V-th row of the block B in the processing target image, and the V π and ν η of the block B in the processing target image. + Calculate the average pixel value ρ> of the four pixels arranged in the first row. Filter processing unit 11 η + 1,

However, the calculation formula for calculating these average pixel values is as follows.

[0094] [Equation 11]〉 ∑ "'",

Here, p (n, u, v) is a pixel value in the pixel (n, u, v) of the processing target image. FIG. 4 (a) is a graph showing the pixel values of the processing target image. FIG. 4 (b) is a graph showing average pixel values obtained in step S1 above.

[0096] (Step S2) The filter processing unit 11 calculates the average pixel value obtained in step S1 as p

By linear interpolation using> and <>, n + l, v n for each pixel in the V row of block Β

Predicted value P (n, u, v) and predicted value p (n + 1 pred n + 1 pred) for each pixel in the v-th row of block B

, U, v). The filter processing unit 11 predicts the predicted values p (n, u, v) and (n + 1, u, v) pred pred

The calculation formula for calculating is as follows.

[0097] [Equation 12] ― 3 〈;》

[0098] [Equation 13] "+ ¹ '"' ⁼ — 《Re> One〉) ^M + ( ³ 〈〉 + ⁵ 〈P «+ i,)

[0099] FIG. 4 (c) is a graph showing the predicted values obtained in step S2.

[0100] (Step S3) The filter processing unit 11 uses the predicted value p (n, u, v) obtained in Step S2.

pred

) And the average pixel value of block B obtained in step SI.

By subtracting the removal component from the pixel value p (n, u, v) of the processing target image as a removal component to be removed from the image, the pixel value p ′ (n, u, v) of the filter output image is obtained. calculate. Filter processing unit 11a The calculation formula used to calculate the pixel value p ′ (n, u, v) of the filter output image is as follows.

[0101] [Equation 14] p 'in, u, v) = p (n, u, v)-{p _pred (n, u, v) ―,,, _v 〉}

[0102] FIG. 4 (d) is a graph showing pixel values of the filter output image obtained in step S3.

[0103] Here, the average pixel value of block B in the filter output image matches the average pixel value of block B in the processing target image, that is, the filter processing unit 11 processes the average pixel value of each block. Please note that it is something that remains unchanged before and after. For this reason, it is possible to prevent blurring and flickering of moving images due to the filter processing.

[0104] The filter processing unit 11 performs the filtering process in the horizontal direction as described above, and then performs the filtering process in the vertical direction. The vertical filtering processing is a force that calculates the image data of block B with reference to block B adjacent to the lower side of block B n n + W

The calculation method is the same as the horizontal filtering process, so the explanation is omitted. The Further, since the horizontal filter processing and the vertical filter processing are independent from each other, the filter processing unit 11 performs the horizontal filter processing after performing the vertical filter processing. Also good.

[Details of inverse filter processing]

Next, the inverse filter processing performed by the inverse filter processing unit 12 included in the moving image encoding device 300 and the moving image decoding device 400 will be described.

[0106] Similar to the filter processing unit 11, the inverse filter processing unit 12 executes the horizontal direction reverse filter processing and the vertical direction reverse filter processing by dividing the processing target image into a plurality of blocks. Here, the division pattern in which the inverse filter processing unit 12 divides the processing target image is the same as the division pattern in which the filter processing unit 11 divides the processing target image.

[0107] The inverse filter processing unit 12 calculates the image data of the block B from the image data of the block B in the processing target image and the image data of the block adjacent to the block B. Specifically, in the horizontal filter processing, the image data of block B is calculated by referring to the block B image data n n + 1 adjacent to the right side of the block B in the processing target image, and the vertical In the direction filtering process, the image data of block B is calculated by referring to the image data of block B adjacent to the lower side of block B n n + W

To do.

Hereinafter, the inverse filter processing calculation executed by the inverse filter processing unit in the horizontal inverse filter processing will be described with reference to FIG. 5 and FIG. In the following, the inverse filter processing calculation for calculating the pixel value (for example, luminance level) of one block B will be described. The inverse filter processing unit 12 repeats the filter processing calculation described below for all adjacent two blocks, thereby completing the horizontal filter processing.

FIG. 5 is a flowchart schematically illustrating the inverse filter processing calculation performed by the inverse filter processing unit 12. As shown in FIG. 5, the filtering processing by the inverse filter processing unit 12 includes step T1 for calculating an average pixel value, step T2 for calculating a predicted value, and step T3 for calculating an inverse filter output image. Including. The steps Tl to T3 will be described in more detail as follows. In the following description, out of 16 pixels arranged in 4 rows and 4 columns belonging to block Β, the inverse of calculating the pixel values of 4 pixels arranged in the V row The filter processing operation will be described. The inverse filter processing unit 12 performs the inverse filter processing operation described below for the first row force of the block B and the fourth row, sequentially or in parallel, thereby obtaining the pixel values of all the pixels belonging to the block B. Is calculated.

[0110] (Step T1) The inverse filter processing unit 12 calculates the average pixel value of four pixels arranged in the Vth row of the block B in the processing target image, and the blocks B n and η + 1 in the processing target image. Calculate the average pixel value Ρ> of the four pixels arranged in the ν row of. Inverse filter processing η + 1,

The calculation formula used by the unit 12 to calculate the average pixel values Ρ> and <> is as follows.

[0111] [Equation 15]

Here, P (n, u, v) is a pixel value in the pixel (n, u, v) of the processing target image. Fig 6

(a) is a graph showing the pixel values of the processing target image, and FIG. 6 (b) is a graph showing the average pixel values obtained in step T1.

[0113] (Step T2) The inverse filter processing unit 12 determines that the average pixel value obtained in Step T1 <Pπ,

By linear interpolation using> and <>, the predicted value P (n, u, V) for each pixel in the V row of block Β and the predicted value P (for each pixel in the V row of block ( n + pred n + 1 pred

Calculate 1, u, v). The calculation formula used by the inverse filter processing unit 12 to calculate the predicted values P (n, u, v) and P (n + 1, u pred pred, V) is as follows.

[0114] [number 16], ", = - (, - <,)" ten ^{^{11,, v> - 3 <}} )

[0115] [Equation 17]

[0116] Fig. 6 (c) is a graph showing the predicted values obtained in step T2 above. [0117] (Step T3)

The inverse filter processing unit 12 calculates the predicted value P (n, u, v) obtained in step Τ2 and the step

pred

Add the difference from the average pixel value of block B obtained in T1 to the image to be processed η η

A pixel value (n, u, v) of the filter output image is calculated by adding the additional component to the pixel value P (n, u, v) of the processing target image as a power additional component. The calculation formula used by the inverse filter processing unit 12 to calculate the pixel value (n, u, v) of the inverse filter output image is as follows.

[0118] [Equation 18]

P '= P (n, u, V) + \ P _p ,, _d (n, u, v) ―,, _v

FIG. 6 (d) is a graph showing the pixel values of the inverse filter output image obtained in step T3. Note that the average pixel value of block B in the inverse filter output image matches the average pixel value of block B in the processing target image. For this reason, the occurrence of blurring and flickering of moving images is effectively prevented.

[0120] The inverse filter processing unit 12 performs the inverse filter process in the horizontal direction as described above, and then performs the inverse filter process in the vertical direction. The vertical inverse filtering operation is to calculate the image data of block B with reference to block B adjacent to the lower side of block B n n + W

Since the calculation method is the same as that of the horizontal inverse filter processing, the description thereof is omitted. In addition, since the horizontal reverse filter processing and the vertical reverse filter processing are independent from each other, the reverse filter processing unit 12 performs the horizontal reverse filter processing after performing the vertical reverse filter processing. A configuration in which inverse filter processing is performed may be employed.

[0121] The inverse filter processing performed by the inverse filter processing unit 12 defined as described above corresponds to the inverse transformation of the filter processing performed by the filter processing unit 11. That is, (n, u, v) = P (n, u, v) between the output iT (n, u, v) of the filter processing unit 11 and the input P (n, u, v) of the inverse filter processing unit 12 When the relationship v) holds, p (n, u, ν) is between the input p (n, u, v) of the filter processing unit 11 and the output (n, u, v) of the inverse filter processing unit 12. = Ρ '(n, u, v). Accordingly, the block removed from the processing target image by the filtering process of the filter processing unit 11 is performed. The frequency component that generates the noise distortion is restored by the inverse filter processing of the inverse filter processing unit 12. Therefore, a specific frequency component is not lost in the restored image.

[0122] [Filter processing and inverse filter processing!

In the above description, the predicted value calculated by the filter processing unit 11 is calculated by linear interpolation using the average pixel value of the adjacent block, but the present invention is not limited to this. That is, the predicted value used by the filter processing unit 11 to calculate the pixel value of the filter output image may be a predicted value calculated based on an average pixel value in block units, for example. The average pixel value of three adjacent blocks may be a value calculated by cubic interpolation using P_>, , and . Details η η + 1, ν

Although a detailed description is omitted, the inverse filter processing unit 12 corresponding to the filter processing unit 11 that performs these filter processes is easily configured to perform an inverse filter process corresponding to the inverse transformation of these filter processes. obtain.

[0123] In addition, when using a prediction in which the average pixel value of the prediction value for each block is different from the average pixel value of the input image for each block, the average pixel value of the prediction value and the average pixel value of the input image are used for the filter output It is preferable to make a correction so that the average pixel value of the filter input and filter output is maintained.

[0124] In the above description, the filter processing unit 11 performs one-dimensional filter processing independently in the horizontal direction and the vertical direction, but the present invention is not limited to this. It may be configured to perform two-dimensional filter processing. An example of the two-dimensional filter processing performed by the filter processing unit 11 is as follows.

When performing two-dimensional filter processing, the filter processing unit 11 calculates the pixel value of the block of interest with reference to the pixel values of four blocks that are adjacent vertically and horizontally. The filter processing calculation performed by the filter processing unit 11 to calculate the pixel value of the target block Β includes the following steps 1 to 3.

[0126] (Step 1) The filter processing unit 11 calculates the average pixel value of the target block Β and the average pixel value of blocks Β, 、, Β, and 隣接 adjacent to the target block Β vertically and horizontally n n -1 n + 1 nW n + W

To do. The calculation used by the filter processing unit 11 to calculate the average pixel value of each block. The formula is as follows.

[0127] [Equation 19] »<4 ι '<4

1 _{)) / =} η _{υ =} η

(Step 2) The filter processing unit 11 performs the prediction value ρ (η for each pixel of the block により by linear interpolation using the average pixel value of each block obtained in Step 1 above.

n pred

, U, v). Used by the filter processor 11 to calculate the predicted value p (n, u, v)

pred

The calculation formula is as follows.

[0129] [Equation 20]

、, η, =

+ ^ (5 „—!> + 3 〈+ 5„ — „,〉 + 3 〈,, _{+ w} 》

[0130] (Step 3) The filter processing unit 11 calculates the predicted value p (n, u, v) obtained in Step 2,

pred

The difference from the average pixel value of block B obtained in step 1 is calculated from the image to be processed.

η η

The pixel value (n, u, v) of the filter output image is calculated by subtracting the removal component from the pixel value p (n, u, v) of the processing target image as the removal component to be removed. . The calculation formula for the filter processing unit 11 to calculate the pixel value iT (n, u, v) of the block B in the filter output image is as follows.

[0131] [Equation 21]

P 'in, u, v) = p (n, u, v) +-(p "-,) + (p„ ₊₁ ) + (p „. _W ) +??„ _{+ W} 》 ― P _pred ( ", u, v)

[0132] When removing a frequency component that generates block distortion by two-dimensional filtering, the filter processing unit 11 repeats the above steps 1 to 3 for each block, thereby obtaining a pixel value of the entire filter output image. Is calculated. Although a detailed description is omitted, the corresponding inverse filter processing unit 12 may be configured to perform an inverse filter process corresponding to the inverse transformation of the filter process of the filter processing unit 11. (Modification 1)

A modification of the moving image encoding device 300 will be described with reference to FIGS.

FIG. 7 is a functional block diagram showing a schematic configuration of a video encoding device 300a that is a modification of the video encoding device 300. As shown in FIG. As shown in FIG. 7, the moving picture coding apparatus 300a includes a DCT unit 1, a quantization unit 2, a variable length coding unit 3, an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, and an intra prediction unit. 8, an inter prediction unit 9, an encoding control unit 10, a filter processing unit lla, and an inverse filter processing unit 12a.

[0134] The difference between the moving image encoding device 300a and the moving image encoding device 300 (Fig. 1) is that the moving image encoding device 300a is based on a predicted image instead of the filter processing unit 11. A filter processing unit 11a that performs filter processing is provided, and an inverse filter processing unit 12a that performs inverse filter processing based on a predicted image is provided instead of the inverse filter processing unit 12. Further, as shown in FIG. 7, the intra prediction unit 8 and the inter prediction unit 9 supply the generated prediction image to the filter processing unit 1 la and the inverse filter processing unit 12a. In FIG. 7, blocks having the same functions as those of the moving picture coding apparatus 300 in FIG. 1 are denoted by the same names and symbols as those in FIG. 1, and description thereof is omitted.

FIG. 8 is a functional block diagram showing a schematic configuration of a video decoding device 400a corresponding to the video encoding device 300a shown in FIG. As shown in FIG. 8, the video decoding device 400a includes an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, an intra prediction unit 8, an inter prediction unit 9, a variable length decoding unit 20, and an inverse filter process. Part 12a is provided.

[0136] The difference between the video decoding device 400a and the video decoding device 400 (Fig. 2) in Fig. 8 is that the video decoding device 400a is different from the video encoding device 300a in place of the inverse filter processing unit 12. The same inverse filter processing unit 12a is provided. Also, as shown in FIG. 8, the intra prediction unit 8 and the inter prediction unit 9 supply the generated predicted image to the inverse filter processing unit 12a.

[0137] The filter processing unit 11a and the inverse filter processing unit 12a that can be suitably used in the moving image coding apparatus 300a and the moving image decoding device 400a will be described as follows.

[0138] The filter processing unit 11a and the inverse filter processing unit 12a The image data of the block is calculated from the image data of the predicted image supplied from the intra prediction unit 8 or the inter prediction unit 9. Specifically, in horizontal filtering, the image data of block B is calculated by referring to the nnn + 1 image data of block B and block B in the predicted image, and in the vertical filtering, Refer to the image data of block B and block B in the predicted image for the image data of B n n + W

To calculate. In the following, the force filter processing unit 11a and the inverse filter processing unit 12a that describe the filter processing calculation for calculating the pixel value of one block B repeat the filter calculation processing described below so that the filter output image Complete the entire horizontal filtering process. In addition, the entire filtering process is completed by performing the filtering process in the vertical direction in the same manner.

[0139] The filter processing calculation executed by the filter processing unit 11a in the horizontal filter processing includes the following steps Sla to S3a.

[0140] (Step Sla) The filter processing unit 11a calculates the average pixel value <q> of the four pixels arranged in the v-th row of the block B in the predicted image and the V-rows π and ν of the block B in the processing target image. Calculate the average pixel value q> of the four pixels arranged in the n + 1st eye. Filter processing unit 11a is n + l,

The calculation formula for calculating these average pixel values is as follows.

[0141] [Equation 22]

1 "<4

<"> = 7∑,",

Here, q (n, u, v) is a pixel value in the pixel (n, u, v) of the predicted image.

[0143] (Step S2a) The filter processing unit 11a calculates the average pixel value obtained in Step Sla q

> And <> for each pixel in the Vth row of block B by linear interpolation

, V n + l, v n

Predicted value q (n, u, v) and predicted value q (n + 1 pred n + 1 pred) for each pixel in the vth row of block B

, U, v). The filter processing unit 11a predicts q (n, u, v) and q (n + 1, u, v) pred pred

The calculation formula for calculating is as follows.

[0144] [Equation 23]

[0145] [Equation 24] r _e "+, v) =-^ ((¾ _{> v} ) ― (q _{n + v} )) + i (3 (¾ _{> v} ) + 5 {¾ _{+1; V} )) [ [0146] (Step S3a) The filter processing unit 11a obtains the predicted value q (n,

The difference between pred u, v) and the average pixel value q> of block B obtained in step S la is the removal component to be removed from the processing target image, and the pixel value p (n, The pixel value p ′ (n, u, v) of the filter output image is calculated by subtracting the removed component from u, v). The calculation formula used by the filter processing unit 11a to calculate the pixel value p ′ (n, u, v) of the filter output image is as follows.

[0147] [Equation 25]

P '(", u, v) = p (n, u, v)-\ q _pred (n, u, v)-(¾ _{> v} )}

Note that the above steps Sla to S3a are filter processing operations that calculate the pixel values of the four pixels arranged in the Vth row among the 16 pixels arranged in the fourth row and the fourth column belonging to the block B. However, the pixel values of all the pixels belonging to the block B are calculated by executing the above steps Sla to S3a sequentially or in parallel with respect to the fourth row from the first row force of the block B.

[0149] The inverse filter processing operation executed by the filter processing unit 12a in the vertical filter processing includes the following steps Tla to T3a.

[0150] (Step Tla) The inverse filter processing unit 12a includes the average pixel value <q> of four pixels arranged in the Vth row of the block B in the predicted image, and the Vth row of the block B in the predicted image.

Calculate the average pixel value q> of the four pixels arranged in. The inverse filter processing unit 12a

The calculation formula used to calculate the average pixel value q> and q> is Are the same.

[0151] (Step T2a) The inverse filter processing unit 12a performs linear interpolation using the average pixel values <q> and <> obtained in Step Tla for each pixel in the Vth row of the block B. And the predicted value q for each pixel in the Vth row of block B (n

pred ι, η ti, ν) η + 1 pred

+ 1, u, ν). The inverse filter processing unit 12a performs prediction values q n, u, v) and q (n + 1,

The calculation formula used to calculate pred (pred u, v) is the same as [Equation 23] and [Equation 24].

[0152] (Step T3a)

The inverse filter processing unit 12a calculates the predicted value q (n, u, v) obtained in step T2a and the step

pred

The difference from the average pixel value q> of block B obtained by Tla is added to the image to be processed

η η

A pixel value (n, u, v) of the filter output image is calculated by adding the additional component to the pixel value P (n, u, v) of the processing target image as an additional component to be processed. The calculation formula used by the inverse filter processing unit 12a to calculate the pixel value (n, u, v) of the inverse filter output image is as follows.

[0153] [Equation 26]

P 'u, v) = P (n, u, v) + (q _pKd («, u, v)-(q _nv )

Note that the above steps Tla to T3a are filter processing operations for calculating the pixel values of the four pixels arranged in the V-th row among the 16 pixels arranged in the fourth row and the fourth column belonging to the block B. However, the pixel values of all the pixels belonging to the block B are calculated by executing the above steps Tla to T3a sequentially or in parallel for the fourth row from the first row force of the block B.

[0155] As described above, the filter processing unit 11a calculates, from the image data of the predicted image, a pixel value that also subtracts the image data of the target image to be filtered. Further, the inverse filter processing unit 12a calculates the image data force of the predicted image by the same method as the filter processing means, with respect to the pixel value to be added to the image data of the processing target image to be subjected to the inverse filter processing. That is, the pixel value that the filter processing unit 11a subtracts from the processing target image and the pixel value that the inverse filter processing unit 12a adds to the processing target image are calculated from the same predicted image. The same image value. Therefore, according to the filter processing unit 11a and the inverse filter processing unit 12a, compared with the moving image coding apparatus 300 that performs the filter processing 'inverse filter processing based on the processing target image having a difference in quantization error level, The frequency component that generates block distortion from which the processing target image force has also been removed by the filter processing unit 1 la can be completely restored by the inverse filter processing unit 12a.

(Modification 2)

Another modification of the moving image encoding device 300 will be described with reference to FIG. 9 and FIG.

FIG. 9 is a functional block diagram showing a schematic configuration of a video encoding device 300b which is another modification of the video encoding device 300. As shown in FIG. As shown in FIG. 9, the moving picture coding apparatus 300a includes a DCT unit 1, a quantization unit 2, a variable length coding unit 3, an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, and an intra prediction unit. 8, an inter prediction unit 9, an encoding control unit 10, a filter processing unit l ib, and an inverse filter processing unit 12b.

[0157] The difference between the moving image encoding device 300b and the moving image encoding device 300 (Fig. 1) is that the moving image encoding device 300a is based on a predicted image instead of the filter processing unit 11. The filter processing unit l ib for executing the filter processing is provided, and instead of the inverse filter processing unit 12, an inverse filter processing unit 12a for executing the inverse filter processing based on the predicted image is provided. Further, the difference is that the filter processing unit ib is provided immediately before the DCT unit 1 and uses the difference image between the original image and the predicted image as the processing target image of the filter processing. Note that the prediction images used by the filter processing unit 1 lb and the inverse filter processing unit 12b for the filter processing are both supplied from the intra prediction unit 8 or the inter prediction unit 9.

[0158] As shown in Fig. 9, the functional blocks of the video encoding device 300b excluding the filter processing unit ib and the inverse filter processing unit 12b are the same as those of the video encoding unit 300 in Fig. 1. They are identical. Therefore, in FIG. 9, blocks having the same functions as those of the moving picture coding apparatus 300 in FIG. 1 are denoted by the same names and symbols as those in FIG. 1, and description thereof is omitted.

FIG. 10 is a functional block diagram showing a schematic configuration of a video decoding device 400b corresponding to the video encoding device 300b shown in FIG. As shown in FIG. 10, the video decoding device 400b includes an inverse quantization unit 4, an IDCT unit 5, a frame memory 7, an intra prediction unit 8, and an inter prediction. Unit 9, a variable length decoding unit 20, and an inverse filter processing unit 12b.

[0160] The difference between the video decoding device 400b and the video decoding device 400 (Fig. 2) in Fig. 10 is that the video decoding device 400b is different from the video encoding device 300b in place of the inverse filter processing unit 12. The same inverse filter processing unit 12b is provided. Further, as shown in FIG. 10, it is supplied to the predicted image power inverse filter processing unit 12b generated by the intra prediction unit 8 and the inter prediction unit 9.

[0161] In the following, filter processing in the filter processing unit l ib suitable for the moving image encoding device 300b, and inverse filtering processing in the inverse filter processing unit 12b suitable for the moving image encoding device 300 and the moving image decoding device 400b Will be described.

[0162] The filter processing unit l ib and the inverse filter processing unit 12b calculate the image data of each block in the filter output image from the image data of the prediction image supplied from the intra prediction unit 8 or the inter prediction unit 9. Specifically, in the horizontal filter processing, the image data of block B is used as the block B and block B of the predicted image.

n n n + 1 Calculated with reference to the image data, and in the vertical filter processing, the image data of block B is referred to the image data of block B and block B in the predicted image.

n n + W

To calculate. In the following, the force filter processing unit l ib and the inverse filter processing unit 12b, which explain the filter processing calculation for calculating the pixel value of one block B, repeat the filter calculation processing described below, thereby outputting the filter output. Complete the horizontal filtering of the entire image. In addition, the entire filtering process is completed by performing the filtering process in the vertical direction in the same manner.

[0163] The filter processing calculation performed by the filter processing unit l ib in the horizontal filter processing includes the following steps Slb to S3b.

[0164] (Step Sib) The filter processing unit l ib calculates the average pixel value <q> of the four pixels arranged in the vth row of the block B in the predicted image and the Vth row of the block B in the processing target image.

Calculate the average pixel value q> of the four pixels arranged in the π and ν n + 1 eyes. Filter processing section l ib

n + l,

The calculation formula for calculating these average pixel values is the same as [Equation 22].

[0165] (Step S2b) The filter processing unit l ib calculates the average pixel value obtained in step Sib q

> And <> and the predicted value for each pixel in the Vth row of block B ( n, u, v) and the predicted value (η + 1, u, v) for each pixel in the vth row of block B is n + 1 pred

calculate. Filter processing unit l ib calculates predicted values (n, u, v) and (n + 1, u, v) pred pred

The calculation formula for doing this is as follows.

[0166] [Equation 27] 1 _d (n, u, v)-+-((¾, _ν >-(¾ ₊ ι, _ν ))

[0167] [Equation 28]

¾ ("+ 1, = 《-〈+ i.) ^W 《-,)

[0168] (Step S3b) The filter processing unit l ib calculates the prediction obtained in step S2b from the pixel value p (n, u, v) of the image to be filtered (difference image between the original image and the predicted image). The pixel value p ′ (n, u, v) of the filter output image is calculated by subtracting the values (η, pred u, v). The calculation formula used by the filter processing unit l ib to calculate the pixel value p ′ (n, u, v) of the filter output image is as follows.

[0169] [Equation 29] pn, u, v) = p (n, u, v)-q _pred in, u, v)

Note that the above steps Slb to S3b are filter processing operations for calculating the pixel values of 4 pixels arranged in the V-th row among the 16 pixels arranged in 4 rows and 4 columns belonging to the block B. However, the pixel values of all the pixels belonging to the block B are calculated by executing the above steps Slb to S3b sequentially or in parallel for the first row and the fourth row of the block B.

[0171] In the filter processing in the vertical direction, the inverse filter processing executed by the inverse filter processing unit 12b includes the following steps Tlb to T3b.

[0172] (Step Tib) The inverse filter processing unit 12b performs processing on the Vth row of the block B in the predicted image. The average pixel value <q> of the arranged four pixels and the Vth row of block B in the predicted image

Calculate the average pixel value q> of the four pixels arranged in π and ν η + 1. Inverse filter processing unit 12b

n + l,

The calculation formula used to calculate these average pixel values is the same as [Equation 22].

[0173] (Step T2b) The inverse filter processing unit 12b uses the average pixel values <q> and <> obtained in Step Tib to calculate the predicted value q for each pixel in the Vth row of block B.

(n, u, v) and the predicted value q (n + 1, u, v) for each pixel in the v-th row of block B are calculated. The inverse filter processing unit 12b calculates the predicted values q (n, u, v) and q (n + 1, u, v).

pred pred

The calculation formula used to output is the same as [Equation 27] and [Equation 28].

[0174] (Step T3b)

The inverse filter processing unit 12b adds the predicted value q (n, u, v) obtained in step T2b to the pixel value P (n, u, v) of the image to be inverse filtered, thereby obtaining a filter. Output image

pred

Calculate pixel values (n, u, v). The calculation formula used by the inverse filter processing unit 12b to calculate the pixel value P ′ (n, u, v) of the filter output image is as follows.

[0175] [Equation 30] '(«, M, V) = P (n, u, v) + q _pred (n, u, v)

Note that the above steps Tlb to T3b are filter processing operations that calculate pixel values of 4 pixels arranged in the V-th row among 16 pixels arranged in 4 rows and 4 columns belonging to the block B. However, the pixel values of all the pixels belonging to the block B are calculated by executing the above steps Tlb to T3b sequentially or in parallel with respect to the first row and the fourth row of the block B.

[0177] As described above, the filter processing unit ib calculates a pixel value from which the image data of the target image to be filtered is also subtracted from the image data of the predicted image. In addition, the inverse filter processing unit 12b calculates the image data force of the predicted image using the same method as the filter processing means, with respect to the pixel value to be added to the image data of the processing target image to be subjected to the inverse filter processing. That is, the pixel value subtracted from the processing target image by the filter processing unit l ib and the pixel value added by the inverse filter processing unit 12b to the processing target image are calculated with the same predicted image power. The same image value. Therefore, according to the filter processing unit l ib and the inverse filter processing unit 12b, compared with the moving image encoding apparatus 300 that performs the filter processing 'inverse filter processing based on the processing target image having a difference in the quantization error level, The frequency component that generates block distortion removed from the processing target image by the filter processing unit 1 lb can be more completely restored by the inverse filter processing unit 12b.

[Additional Notes]

The present invention is not limited to the above-described embodiment, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention. For example, the present invention can be configured as follows.

[0178] The moving image encoding apparatus according to the present invention is a moving image encoding apparatus that divides and encodes an image into a plurality of blocks, and performs a predetermined filtering process on the image in units of blocks. It may be configured to include a filter processing means and an inverse filter processing means for performing an inverse filter process that is an inverse transformation of the filter processing means.

[0179] In addition, in the video encoding apparatus according to the present invention, the filter processing unit is configured to apply a predetermined filter to the encoding target image of the block based on the encoding target images of a plurality of adjacent blocks. It is configured to do the processing.

[0180] Further, the moving picture coding apparatus according to the present invention includes prediction means for performing intra-screen prediction or inter-screen prediction for each block, and the filter processing means outputs the prediction means. Based on the predicted images of a plurality of adjacent blocks, a predetermined filtering process may be performed on the encoding target image of the block.

[0181] Further, the moving picture coding apparatus according to the present invention includes a prediction unit that performs intra-frame prediction or inter-screen prediction for each block, and the filter processing unit outputs the prediction unit. Based on the predicted images of a plurality of adjacent blocks, a predetermined filter process may be performed on the difference image between the encoding target image and the predicted image of the block.

[0182] Also, the moving picture decoding apparatus according to the present invention is a moving picture decoding apparatus for decoding moving picture data encoded for each block, and is configured to include the inverse filter processing means. May be. [0183] Finally, the above-described blocks of the moving image encoding apparatus 300, 300a, 300b of the present invention and the moving image decoding apparatus 400, 400a, 400b of the present invention are particularly adapted to filter processing means 11. 11a • l ib , And the inverse filter processing means 12'12a'12b may be constituted by hardware logic, or may be realized by software using a CPU as follows. .

[0184] That is, the above-described video encoding device 'video decoding device includes a CPU (central processing unit) that executes instructions of a control program that realizes each function, and a ROM (read only memory) that stores the program. A random access memory (RAM) for expanding the program, and a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is to provide a computer with program codes (execution format program, intermediate code program, source program) of a control program for the above-described moving image encoding device “moving image decoding device” which is software for realizing the above-described functions. The recording medium recorded so as to be readable can be supplied to the moving picture encoding apparatus / moving picture decoding apparatus, and the computer (or CPU or MPU) can read and execute the program code recorded on the recording medium. Is achievable.

[0185] The recording medium includes, for example, a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy disk Z hard disk, and an optical disk such as CD-ROMZMOZ MD / DVD / CD-R. Disk systems, IC cards (including memory cards) Z optical cards and other card systems, or mask ROMZEPROMZEEPROMZ flash ROM and other semiconductor memory systems can be used.

[0186] Further, the moving image encoding device / moving image decoding device may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite A communication network or the like is available. In addition, the transmission medium constituting the communication network is not particularly limited. For example, IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc. can be used, such as IrDA or remote control. Infrared, Bluetooth (registered trademark), 802.11 wireless, HDR, mobile phone network, satellite It can also be used by radio such as lines and terrestrial digital networks. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

[0187] As described above, the moving image encoding apparatus according to the present invention divides an image into a plurality of blocks, quantizes the image data of each block, and uses the quantized representative value obtained by the quantization. The moving image encoding apparatus for encoding includes a filter processing means for performing a filter process for removing a frequency component that generates block distortion on the image data before quantization.

Accordingly, it is possible to generate code key data that can restore a decoded image with reduced block distortion without missing a specific frequency component.

In addition, as described above, the video decoding device according to the present invention decodes encoded data obtained by encoding image data from which frequency components that generate block distortion are removed. The apparatus includes an inverse filter processing unit that performs an inverse filter process for restoring the removed frequency component on the image data of the decoded image obtained by decoding.

[0190] Accordingly, a decoded image with reduced block distortion can be decoded without missing a specific frequency component.

The specific embodiments or examples made in the detailed description section of the invention are to clarify the technical contents of the present invention, and are limited to such specific examples. The present invention should not be construed as follows, and can be implemented with various modifications within the spirit of the present invention and the scope of the following claims.

Industrial applicability

[0191] The present invention relates to a moving image storage device that encodes and stores a moving image, a moving image transmission device that encodes and transmits a moving image, a moving image reproducing device that decodes and reproduces a moving image, and the like. Therefore, it can be suitably used. Specifically, for example, it can be suitably used for a hard disk recorder or a mobile phone terminal.

Claims

The scope of the claims

[1] In a video encoding device that divides an image into a plurality of blocks, quantizes the image data of each block, and encodes the quantized representative value obtained by the quantization,

A moving picture coding apparatus comprising: filter processing means for performing filter processing for removing frequency components that generate block distortion on the image data before quantization

[2] The filter processing performed by the filter processing means is to keep the average pixel value of each block unchanged before and after the processing.

The moving picture encoding device according to claim 1, wherein

[3] predicted image generation means for generating a predicted image;

An inverse filter processing means for performing an inverse filter process corresponding to the inverse transform of the filter process on the locally decoded image;

The predicted image generation means generates the predicted image using the locally decoded image subjected to the inverse filter processing as a reference image.

3. The moving picture coding apparatus according to claim 1 or 2, wherein

[4] The quantization target image to be quantized is a difference image between the predicted image and the original image subjected to the filtering process.

The locally decoded image is an image obtained by adding the image data obtained by dequantizing the quantized representative value to the moving image coding apparatus and the image data of the predicted image. is there,

4. The moving picture coding apparatus according to claim 3, wherein

[5] The quantization target image to be quantized is obtained by performing the filtering process on the difference image between the predicted image and the original image.

4. The moving picture coding apparatus according to claim 3, wherein

[6] The filter processing means includes an image data of a processing target image to be filtered. The pixel value to be subtracted is calculated as the image data force of the predicted image, and the inverse filter processing means adds the pixel value to be added to the image data of the processing target image to be subjected to the inverse filter processing. The image data force of the predicted image is also calculated by the same method as the filter processing means.

The moving picture coding apparatus according to any one of claims 5 to 5, wherein the force is any one of the fifth term and the fifth term.

[7] The filter processing means includes:

An average pixel value calculation process for calculating an average pixel value in each block of the processing target image to be filtered;

A prediction process for predicting a pixel value of each pixel of the processing target image based on the average pixel value calculated in the average pixel value calculation process;

For each pixel of the processing target image,

The difference between the predicted pixel value of the pixel predicted in the prediction process and the average pixel value of the block to which the pixel belongs calculated in the average pixel value calculation process should be removed from the processing target image. A subtraction process for subtracting the removal component from the pixel value of the pixel of the processing target image as a removal component;

Is to perform

The moving picture coding apparatus according to any one of claims 6 to 6, wherein the first term force in claim 1 is also characterized in that.

[8] A moving picture decoding apparatus for decoding encoded data obtained by encoding image data from which frequency components that generate block distortion are removed,

A moving picture decoding apparatus comprising: an inverse filter processing means for performing an inverse filter process for restoring the removed frequency component on the image data of a decoded image obtained by decoding.

[9] The filter inverse processing means includes:

An average pixel value calculation process for calculating an average pixel value in each block of the processing target image to be subjected to the inverse filtering process;

Based on the average pixel value calculated in the average pixel value calculation process, A prediction process for predicting the pixel value of each pixel of the elephant image;

For each pixel of the image to be processed, the predicted pixel value of the pixel predicted in the prediction process and the average pixel of the block to which the pixel calculated in the average pixel value calculation process belongs. An addition process for adding the difference from the value to the pixel value of the pixel of the processing target image as an additional component to be added to the processing target image.

9. The moving picture decoding apparatus according to claim 8, wherein the moving picture decoding apparatus is characterized in that:

[10] A moving image recording apparatus comprising the moving image encoding device according to any one of claims 1 to 7, wherein the moving image encoded by the moving image encoding device is recorded. A moving image recording device.

[11] A moving image reproduction device comprising the moving image decoding device according to claim 8 or 9, wherein the moving image reproduction device reproduces a moving image decoded by the moving image encoding device.