JP2007143178A - Method for removing coding distortion - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for removing coding distortions, wherein because mosaic-shaped block noise occurs when a compressed video signal is reproduced, this block noise is removed, but since removing block noise from each block by using a deblocking filter imposes a significant load on the deblocking filter, the method reduces the load. <P>SOLUTION: It is determined whether or not coding distortion removal process is necessary, and the deblocking filter is used only when the process is necessary. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a coding distortion removing method for removing coding distortion generated when an image signal is coded, and a coding method and a decoding method capable of further increasing the compression rate using the coding distortion removing method. And a recording medium on which a program for implementing it by software is recorded.

  In recent years, with the era of multimedia that handles sound, images, and other pixel values in an integrated manner, conventional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, telephones, etc., to people have become multimedia. It has come to be taken up as a target. In general, multimedia refers to not only characters but also figures, sounds, especially images, etc. that are associated with each other at the same time. It is an essential condition.

  However, when the information amount of each information medium is estimated as a digital information amount, the information amount per character is 1 to 2 bytes in the case of characters, whereas 64 kbits per second (phone quality) in the case of speech. In addition, for a moving image, an information amount of 100 Mbits (current television reception quality) or more per second is required, and it is not realistic to handle the enormous amount of information in the digital format as it is with the information medium. For example, a video phone has already been put into practical use by an integrated services digital network (ISDN) having a transmission rate of 64 kbps to 1.5 Mbps, but it is not possible to send a video of a TV camera as it is through an ISDN. Is possible.

  Therefore, what is required is information compression technology. For example, in the case of a videophone, H.264 has been internationally standardized by ITU-T (International Telecommunication Union Telecommunication Standardization Sector). 261 and H.264. H.263 standard video compression technology is used. In addition, according to the information compression technology of the MPEG-1 standard, it is possible to put image information together with audio information on a normal music CD (compact disc).

  Here, MPEG (Moving Picture Experts Group) is an international standard for digital compression of moving image signals, and MPEG-1 has moving image signals up to 1.5 Mbps, that is, information of television signals is about 1/100. It is a standard that compresses up to. In addition, since the transmission speed for the MPEG-1 standard is mainly limited to about 1.5 Mbps, MPEG-2 standardized to meet the demand for higher image quality has 2 to 2 video signals. Compressed to 15 Mbps.

  Furthermore, MPEG-4 having a higher compression rate has been standardized by a working group (ISO / IEC JTC1 / SC29 / WG11) that has been standardizing with MPEG-1 and MPEG-2. MPEG-4 initially introduced not only high-efficiency coding at a low bit rate, but also a powerful error resilience technology that can reduce subjective image quality degradation even if a transmission path error occurs. . In ITU-T, H.264 is used as a next-generation screen encoding method. 26L standardization activities are progressing.

  H. Unlike conventional video coding, 26L employs a coding distortion removal method that involves complicated processing in order to remove coding distortion. It is known that in a block unit encoding method using orthogonal transform such as DCT, which is often used in moving image encoding, a lattice-like distortion called block distortion is seen at a block boundary. Since the image quality degradation of the low frequency component is more conspicuous than the image quality degradation of the high frequency component, the block unit coding mainly encodes the low frequency component faithfully than the high frequency. In addition, since a natural image taken with a camera or the like includes more low-frequency components than high-frequency components, the frequency components in the block essentially have more low-frequency components. As a result, there is almost no high-frequency component inside the block, and adjacent pixels inside the block tend to have almost the same pixel value. On the other hand, since encoding is performed in units of blocks, even if the pixel values are almost the same in the block, the pixel values are not necessarily the same between adjacent blocks at the block boundary, that is, the pixel values are continuously changed. No. As a result, as shown in the explanatory diagram of the concept of the coding distortion removal method in FIG. 31, when block unit coding is performed on the original image in FIG. 31 (a), as shown in FIG. 31 (b), Within the block, the pixel value changes smoothly and continuously, and block distortion occurs in which the pixel value becomes discontinuous only at the block boundary indicated by a broken line. Therefore, by correcting the pixel values to be continuous at the block boundary as shown in FIG. 31C, block distortion that is a big problem in image quality due to encoding can be reduced. This process of reducing block distortion is called a coding distortion removal method (deblocking method).

  When the coding distortion removal method is used for moving picture decoding, a configuration (post filter) as shown in the block diagram of the image decoding apparatus using the conventional decoding method in FIG. 32 and the conventional decoding in FIG. There is a configuration (in-loop filter) as shown in a block diagram of an image decoding apparatus using the method. Hereinafter, the configuration of each block diagram will be described.

In the block diagram of the image decoding apparatus using the conventional decoding method of FIG. 32, a variable length decoding unit 52 performs variable length decoding on the encoded signal Str and outputs a frequency code component DCoef. The inverse zigzag scanning unit 54 rearranges the frequency components of the frequency code component DCoef into a two-dimensional block, and outputs a frequency component FCoef that is a frequency component in units of blocks. The inverse cosine transform unit 56 performs inverse quantization and inverse DCT on the frequency component FCoe and outputs it as a difference image DifCoef.
On the other hand, the motion compensation unit 60 outputs the pixel at the position indicated by the motion vector MV inputted from the outside from the reference image Ref stored in the memory 64 as the motion compensated image MCpel. The adder 58 adds the difference image DifCoef and the motion compensated image Mcpel and outputs the result as a reproduced image Coef. The deblocking filter 62 removes coding distortion from the reproduced image Coef and outputs a decoded image signal Vout. The reproduced image Coef is stored in the memory 64 and used as a reference image Ref in subsequent image decoding.

  The block diagram of the image decoding apparatus using the conventional decoding method shown in FIG. 33 is almost the same as the block diagram of the image decoding apparatus using the conventional decoding method shown in FIG. The position of is different. That is, in the block diagram of the image decoding apparatus using the conventional decoding method of FIG. 33, the decoded image signal Vout subjected to the deblocking filter 62 is stored in the memory 64.

  A block diagram of an image decoding apparatus using the conventional decoding method of FIG. 32 is MPEG-1, MPEG-2, MPEG-4, and H.264. A block diagram of an image decoding apparatus using the conventional decoding method of FIG. 261, and H.264. It is also used in 26L TM8.

In the block diagram of the image decoding apparatus using the conventional decoding method of FIG. 32, the reproduced image Coef stored in the memory 64 does not depend on the method of the deblocking filter 62. Therefore, the deblocking filter 62 can be independently developed and mounted according to the capability and application of the device from various methods such as a complicated but high-performance filter or a filter that is not very effective but easy to process. Moreover, there exists an advantage which can use an appropriate thing for every apparatus.
On the other hand, in the block diagram of the image decoding apparatus using the conventional decoding method of FIG. 33, the decoded image signal Vout stored in the memory 64 depends on the method of the deblocking filter 62. Therefore, although there is a problem that it cannot be changed to an appropriate one according to the implementation and application of the device, there is an advantage that it can be assured that the same encoding distortion removal result can be obtained with any device.

  FIG. 34 is a block diagram of a coding distortion removing unit using a conventional coding distortion removing method. This is a detailed description of the structure of the deblocking filter 62 shown in FIGS. In order to efficiently remove only the coding distortion from the image signal including the coding distortion, by specifying the magnitude and tendency of the coding distortion included in the image signal and performing an appropriate filter process corresponding thereto, It is important not to degrade the original image signal. Since coding distortion contains a lot of high-frequency components, the proportion of high-frequency components in the image signal is investigated, and high-frequency components are found in pixels that are considered not to contain high-frequency components in the image signal. If there is a coding distortion (the image signal has a high correlation between adjacent pixels, pixel positions including high-frequency components are concentrated on the edge or the like. The process of performing a high frequency component suppression filter is a general idea of the coding distortion removal method. This deblocking filter 62 is an H.264 filter. The inventor of the present application created it based on the contents described in TML 8 of 26L.

  The filter target pixel number determination unit 84 determines a pixel position including coding distortion using the reproduced image Coef, and outputs the filter target pixel number FtrPel. The filter coefficient determination unit 86 determines the filter coefficient (including the number of filter taps) appropriate for removing the coding distortion of the pixel using the filter target pixel number FtrPel and the reproduced image Coef, and outputs the filter coefficient FtrTap. . The filter processing unit 88 performs filter processing for removing the coding distortion on the reproduced image Coef using a filter coefficient indicated by the filter coefficient FtrTap, and outputs a decoded image signal Vout.

MPEG-4 P290-P292 F.M. 3.1 Deblocking filter

Now, such a conventional coding distortion removal method has a problem that the coding distortion removal effect is large, but the processing is very complicated and difficult to implement. In addition, there is a problem that the amount of processing per unit time is large.
Also, no matter how effective the coding distortion removal method is used, it is impossible to accurately distinguish the image signal and the coding distortion without any special additional information. The image quality of the signal may be impaired. As shown in the block diagram of the image decoding apparatus using the conventional decoding method in FIG. 33, this affects the encoding result of the subsequent screen when the result of encoding distortion removal is used as a reference image. Therefore, the problem is particularly great.
The present invention is a coding distortion removal method that is easy to process, and by performing high-performance coding distortion removal that reduces the possibility of image quality deterioration of the image signal by performing coding distortion removal. An object of the present invention is to provide an encoding distortion removal method, an encoding method, and a decoding method that can reduce the possibility of impairing the image quality.

In order to achieve this object, the present invention provides a coding distortion removal method for removing coding distortion from an image, where a motion compensation unit boundary and a code are encoded when the motion compensation unit is larger than the coding unit. Encoding distortion is removed by a method that is different between a boundary that coincides with the boundary of the encoding unit, a boundary between the motion compensation unit, and a boundary that differs from the boundary of the encoding unit.
As a result, the encoding distortion at the boundary of the motion compensation unit and the encoding distortion at the boundary of the encoding unit are different from each other. Therefore, the encoding distortion can be reduced by switching the filter for removing the encoding distortion for each unit. Only the coding distortion can be efficiently removed from the included image signal.
Further, when the residual error of the motion compensation unit is 0, encoding distortion is removed only at the boundary of the motion compensation unit.

The present invention is also a coding distortion removal method for removing coding distortion from an image, the step of extracting image parameters from an image including coding distortion, and a pixel for removing coding distortion, wherein the image parameter is A first step that uses and specifies a method for removing encoding distortion, a second step that uses the image parameter to specify, and a method that uses the second step to specify the pixel that is specified in the first step. And a third step of removing coding distortion.
As a result, image parameters that can be commonly used in the first step of specifying pixels for removing coding distortion and the second step of specifying a method for removing coding distortion are calculated in advance, and the image parameters are used. Thus, by simplifying the processing of the first step and the second step, it is possible to reduce the processing of the coding distortion removal method without degrading the image quality.

The present invention also relates to a coding distortion removal method for removing coding distortion from an image, wherein it is determined whether or not coding distortion is to be removed in units of blocks, and a block to be removed by the above determination is further determined for each pixel. It is determined whether or not the coding distortion should be removed, and a pixel to be subjected to the coding distortion removing process is determined.
This makes it possible to reduce the processing amount of the coding distortion removal method by first determining whether or not to remove the coding distortion in units of blocks and omitting the pixel-by-pixel judgment for blocks that do not require coding distortion removal. . If image coding information is used, it is often easy to determine that the block does not require coding distortion removal (for example, a block that is stationary and whose reference image and pixels completely match).

The present invention also relates to a coding distortion removing method for removing coding distortion in an area sandwiching a boundary line between a first block and a second block adjacent to each other in a picture having a plurality of blocks constituting a moving image. The difference value between the pixel value in the pixel of the first block and the pixel value in the pixel of the second block, the quantization parameter in the first block, and the quantization parameter in the second block And a parameter for determining a method for removing the coding distortion corresponding to the average value. And a removing step for removing the coding distortion based on the result of the comparison in the comparison step.
Thereby, in the encoding distortion removal process in the block boundary from which a quantization parameter differs, the average value of the quantization parameter of an adjacent block can be used for the filter process of the both sides of a block boundary.

  The present invention also relates to a coding distortion removing method for removing coding distortion in an area sandwiching a boundary line between a first block and a second block adjacent to each other in a picture having a plurality of blocks constituting a moving image. A decoding step of decoding a parameter for determining a threshold when removing coding distortion, and a pixel value in a pixel of the first block and a pixel value in a pixel of the second block A comparison step of comparing the difference value with a predetermined threshold value based on the decoded parameter, and a removal step of switching a coding distortion removal method based on a comparison result in the comparison step.

As a result, a threshold parameter used when performing coding distortion removal is preliminarily superimposed on each coded signal, and coding distortion using an appropriate threshold parameter for each coded signal when coding distortion is removed. By performing the removal, the coding distortion can be efficiently removed from the image signal including the coding distortion. The threshold parameter indicates an appropriate coding distortion removal method suitable for the image signal, and does not explicitly switch the coding distortion removal method for each pixel or individual block. The amount of data required to be superimposed on the digitized signal is small.
The moving image has a slice composed of a plurality of blocks, and the parameter is included in header information of a slice in a code string obtained by encoding image data corresponding to the moving image.

The present invention is also a moving image encoding apparatus that performs encoding by referring to at least one of a plurality of reference images, and refers to a plurality of encoded images obtained by removing encoding distortion by a plurality of methods. An image.
As a result, a plurality of screens from which coding distortion has been removed in at least two ways are used as reference screens, and when referring to one of them, an appropriate one is sequentially selected to efficiently encode coding distortion from an image signal including the coding distortion. The removed image can be used as a reference image, and the compression rate of moving image coding is improved.
Of the plurality of methods, the first method is a method that does not remove coding distortion from the coded image, and the second method is a method that removes coding distortion from the coded image. It is.

The present invention is also a video decoding device that performs decoding by referring to at least one of a plurality of reference images, and refers to a plurality of decoded images obtained by removing coding distortion by a plurality of methods. An image.
As a result, a plurality of screens from which coding distortion has been removed in at least two ways are used as reference screens, and when referring to one of them, an appropriate one is sequentially selected to efficiently encode coding distortion from an image signal including the coding distortion. What has been removed can be used as a reference image, and the encoded signal can be correctly decoded.
The first method of the plurality of methods is a method that does not remove coding distortion from the decoded image, and the second method is a method that removes coding distortion from the decoded image. is there.

  The present invention is also a coding distortion removal method for removing coding distortion of an interlaced image composed of pixels of odd lines and pixels of even lines, wherein a predetermined number of pixels of the odd lines and a predetermined number of pixels are used. A picture having a frame structure block having pixels of the even lines, a picture having a block of one field structure composed of a predetermined number of pixels of the odd lines, or a predetermined number of pixels of the even lines. A determination step for determining whether the picture has a block of the other field structure, and when the block whose coding distortion is to be removed is a block of a picture having all the blocks of the frame structure, The coding distortion is removed between the blocks, and all the blocks for which the coding distortion is removed are the field structure. When a block of a picture having a block, and a removal step of removing coding distortion between blocks of the field structure adjacent.

Accordingly, the processing of the target block from which the coding distortion is removed can be switched depending on whether the picture has a frame structure block or the field structure block.
Further, when the block whose coding distortion is to be removed is a picture block having the frame structure block and the field structure block, the field structure block is converted into the frame structure block. A conversion step, a comparison step for comparing a difference value between a pixel value in the pixel of the block having the field structure and a pixel value in the pixel of the converted block with a predetermined threshold, and a result of the comparison in the comparison step And a removing step for removing the coding distortion based on.

The present invention also relates to a coding distortion removing method for removing coding distortion in an area sandwiching a boundary line between a first block and a second block adjacent to each other in a picture having a plurality of blocks constituting a moving image. The first block includes a predetermined number of pixels of the odd lines and a predetermined number of pixels of the even lines in an interlaced image including pixels of odd lines and pixels of even lines. A block having a frame structure, wherein the second block includes one field composed of a predetermined number of pixels of the odd lines in an interlaced image composed of pixels of odd lines and pixels of even lines; The other field composed of a predetermined number of pixels of the even line in an interlaced image composed of pixels of the line and pixels of the even line A conversion step of converting the first block having the frame structure into the block having the field structure, and a pixel value in the pixel of the second block having the field structure. And a comparison step for comparing the difference value between the pixel value in the pixel of the converted block and a predetermined threshold value, and a removal step for removing the coding distortion based on the comparison result in the comparison step.
As a result, when a block having a frame structure and a block having a field structure are adjacent to each other, it is possible to perform processing on a target block for adaptively removing coding distortion.

Also, the conversion from the first block having the frame structure to the block having the field structure is switched in units of macro blocks or in units of two adjacent macro blocks in the vertical direction.
Also, the second block having the field structure is not converted into the block having the frame structure.

The present invention also relates to a coding distortion removing method for removing coding distortion in an area sandwiching a boundary line between a first block and a second block adjacent to each other in a picture having a plurality of blocks constituting a moving image. The first block includes a predetermined number of pixels of the odd lines and a predetermined number of pixels of the even lines in an interlaced image including pixels of odd lines and pixels of even lines. A block having a frame structure, wherein the second block includes one field composed of a predetermined number of pixels of the odd lines in an interlaced image composed of pixels of odd lines and pixels of even lines; The other field composed of a predetermined number of pixels of the even line in the interlaced image composed of pixels of the line and pixels of the even line. A determination step of determining whether a target block from which coding distortion is to be removed is the frame structure block or the field structure block, and the target block has a field structure When it is a second block, the first block having the frame structure is converted into a block having the field structure, and when the target block is the first block having a frame structure, the field structure is obtained. A conversion step for converting the second block into a block having the frame structure, a comparison step for comparing a pixel value in a pixel of the target block with a predetermined threshold, and a code based on a result of the comparison in the comparison step And a removing step for removing the distortion.
As a result, when a block having a frame structure and a block having a field structure are adjacent to each other, it is possible to perform processing on a target block that adaptively removes coding distortion.

  Further, in the conversion from the frame structure block to the field structure block in the conversion step, one field after conversion is generated from the odd line pixels in the frame structure block, and the even number in the frame structure block is generated. The other field after conversion is generated from the pixels of the line, and the comparison between the difference value and the threshold value in the comparison step is performed by comparing the pixel value of the pixel in one field of the second block with the value of the first block. The pixel value in the pixel of one field after the conversion is compared, or the pixel value in the pixel of the other field of the second block and the pixel in the pixel of the other field after the conversion of the first block Compare the value.

The present invention also relates to a coding distortion removing method for removing coding distortion in an area sandwiching a boundary line between a first block and a second block adjacent to each other in a picture having a plurality of blocks constituting a moving image. The first block includes a predetermined number of pixels of the odd lines and a predetermined number of pixels of the even lines in an interlaced image including pixels of odd lines and pixels of even lines. A block having a frame structure, wherein the second block includes one field composed of a predetermined number of pixels of the odd lines in an interlaced image composed of pixels of odd lines and pixels of even lines; The other field composed of a predetermined number of pixels of the even line in the interlaced image composed of pixels of the line and pixels of the even line. A conversion step of converting the second block having the field structure into a block having the frame structure, and a pixel value in the pixel of the first block having the frame structure. And a comparison step for comparing the difference value between the pixel value in the pixel of the converted block and a predetermined threshold value, and a removal step for removing the coding distortion based on the comparison result in the comparison step.
As a result, when a block having a frame structure and a block having a field structure are adjacent to each other, it is possible to perform processing on a target block that adaptively removes coding distortion.

In addition, the conversion from the second block having the field structure to the block having the frame structure is performed in units of macroblocks or in units of two macroblocks adjacent in the vertical direction.
Also, the second block having the field structure is not converted into the block having the frame structure.
Further, in the conversion from the field structure block to the frame structure block in the conversion step, a converted frame is generated from the pixel of the one field block and the pixel of the other field block, and Compare the pixels in the odd-numbered pixels of the first block with the pixel values in the converted odd-numbered pixels of the second block, or compare the pixels in the even-numbered pixels of the first block with the second Are compared with the pixel values in the pixels of the even lines after the conversion in the block.
Further, in the comparison step, the difference value and the threshold value are compared for each of a plurality of pixels arranged in a line in the same direction as the boundary line at a target position with respect to the boundary line.
Thereby, a plurality of pixels can be collectively removed to remove coding distortion.

  The present invention also includes a decoding unit that decodes an encoded difference image and outputs a difference image, a motion compensation unit that outputs a motion compensated image from a reference image, and adds and combines the difference image and the motion compensated image. An image coding apparatus comprising: an adding unit that outputs an image; an encoding distortion removing unit that removes coding distortion of the synthesized image and outputting a reproduced image; and a storage unit that stores the reproduced image as the reference image In the encoding distortion removing unit, the encoding distortion is removed by any one of the encoding distortion removing methods.

  The present invention is also a program for removing coding distortion from an image, and removes coding distortion by any one of the coding distortion removing methods.

  The present invention also includes a decoding unit that decodes an encoded difference image and outputs a difference image, a motion compensation unit that outputs a motion compensated image from a reference image, and adds and combines the difference image and the motion compensated image. Image coding using an adding unit that outputs an image, an encoding distortion removing unit that removes coding distortion of the composite image and outputs a reproduced image, and a storage unit that stores the reproduced image as the reference image The coding distortion removing unit causes the computer to remove coding distortion by any one of the coding distortion removing methods.

  As described above, the coding distortion removal method according to the present invention includes a coding distortion removal method that is easy to process, a coding distortion removal method that is less likely to impair the image quality of an image signal by performing coding distortion removal, It is possible to provide an encoding method and a decoding method capable of reducing the possibility of impairing the image quality of the image signal by removing the encoding distortion, and its practical value is high.

Hereinafter, embodiments of the present invention will be described.
(Embodiment 1)
In the block diagram of the image decoding apparatus using the decoding method of FIG. 1, a variable length decoding unit 52 performs variable length decoding on the encoded signal Str and outputs a frequency code component DCoef. The inverse zigzag scanning unit 54 rearranges the frequency components of the frequency code component DCoef into a two-dimensional block, and outputs a frequency component FCoef that is a frequency component in units of blocks. The inverse cosine transform unit 56 performs inverse quantization and inverse DCT on the frequency component FCoe and outputs it as a difference image DifCoef.

  On the other hand, the motion compensation unit 60 outputs the pixel at the position indicated by the motion vector MV input from the outside as the motion compensation image MCpel from the reference image Ref stored in the memory 64, and indicates the size of the motion compensation block. The motion compensation block size Mcsize is output. The adder 58 adds the difference image DifCoef and the motion compensated image MCpel and outputs the result as a reproduced image Coef. The deblocking filter 62 receives the reproduced image Coef, the motion compensation block size MCsize, and the difference image DifCoef, performs coding distortion removal, and outputs a decoded image signal Vout. The reproduced image Coef is stored in the memory 64 and used as a reference image Ref in subsequent image decoding.

FIG. 2 is a block diagram of a deblocking filter (also referred to as a coding distortion removing unit) 62 using the coding distortion removing method of the present invention. This deblocking filter 62 is ITU-T H.264. This inventor was invented by referring to the contents of the deblocking filter described in TML 8 of 26L.
The filter target pixel number determination unit 4 determines a pixel position including encoding distortion for each input reproduced image Coef, and outputs the determined pixel position as the filter target pixel number FtrPel. That is, the number of filter target pixels FtrPel indicates the pixel position at which it is determined that the filter process should be performed. The filter coefficient determination unit 6 uses the filter target pixel count FtrPel and the pixel value of the reproduced image Coef to determine a filter coefficient (including the number of filter taps) appropriate for removing the coding distortion of the pixel, and the filter coefficient FtrTap Output as. The filter processing unit 8 performs a filter process for removing the coding distortion on the reproduced image Coef using a filter coefficient indicated by a filter coefficient FtrTap, and outputs a decoded image signal Vout.

  The motion compensation block boundary determination unit 2 receives the difference image DifCoef and the motion compensation block size MCsize, determines whether the difference image DifCoef of the processing target block is equal to or smaller than a predetermined value, for example, 0, and motion compensation It is determined whether it is a block boundary, and a motion compensation block boundary flag IsEdge is output.

  FIG. 3 shows ITU-T H.264. The example of the motion compensation block size employ | adopted as 26L TML8 is shown. In this example, the maximum value of the motion compensation block size is 16 × 16 pixels, which matches a size called a macroblock. 3A to 3G correspond to motion compensation block sizes of 4 × 4, 4 × 8, 8 × 4, 8 × 8, 8 × 16, 16 × 8, and 16 × 16, respectively. ITU-TH. In 26L TML8, an appropriate one is selected for each macroblock from the seven motion compensation block sizes and used for encoding / decoding. It should be noted that an appropriate unit can be selected and encoded / decoded in a unit of two vertically adjacent macroblocks, and a unit of the macroblocks is called a macroblock pair.

  On the other hand, ITU-T H.I. In 26L TML8, the unit of frequency conversion and encoding is 4 × 4 pixels. This unit of 4 × 4 pixels is called an encoding unit. The encoding unit may be a size other than 4 × 4 pixels. As shown in FIG. 3A, each of the 16 blocks A to P is composed of 4 × 4 pixels. As described above, the 4 × 4 pixels as the encoding unit and the motion compensation block size do not match in cases other than those shown in FIG. Since block distortion, which is visually disturbing as coding distortion, occurs at a minimum of 4 × 4 pixels, which is a coding unit, the conventional coding distortion removal method always performs processing in units of 4 × 4 pixels.

  By performing motion compensation encoding, when the correlation between screens is particularly strong, the residual error of motion compensation between screens becomes zero. In this case, since the difference image DifCoef to be encoded / decoded in units of 4 × 4 pixels is also 0, the pixel value associated with the encoding / decoding encoding distortion other than the boundary portion of the motion compensation block. Discontinuous values are not expected to occur. Therefore, for example, in the case of a motion compensation block as shown in FIG. 3B, a 4 × 4 pixel boundary (boundary boundary indicated by a dotted line) of AC, BD, EG, FH, IK, JL, MO, and NP in FIG. Line) does not require encoding distortion removal processing. Similarly, in the case of the motion compensation block as shown in FIG. 3C, the coding distortion is removed at the 4 × 4 pixel boundary of AB, CD, EF, GH, IJ, KL, MN, and OP in FIG. No processing is necessary. If the differential image DifCoef to be encoded / decoded in units of 4 × 4 pixels is also 0, encoding distortion removal processing is performed only on the boundary of the motion compensation block, and the unit of 4 × 4 pixels in the motion compensation block is used. Encoding distortion removal processing is not performed at the boundary. Thereby, compared with the case where the encoding distortion removal process is performed on all the block boundaries, the number of operations of the encoding distortion removal process can be reduced.

  Therefore, the motion compensation block boundary determination unit 2 turns off the selection units 10a and 10b (indicated by solid lines) when the difference image DifCoef of the processing target block is 0 and is not a boundary of the motion compensation block, and the selection unit 10b The reproduced image Coef is output as a decoded image signal Vout. The selection unit is switched by the motion compensation block boundary flag IsEdge. Thereby, the process of the filter object pixel number determination part 4, the filter coefficient determination part 6, and the filter process part 8 can be abbreviate | omitted. In other cases, the selection units 10a and 10b are turned ON (indicated by dotted lines), and the output of the filter processing unit 8 is output from the selection unit 10b as a decoded image signal Vout. The selection unit is switched by the motion compensation block boundary flag IsEdge.

As described above, according to the present embodiment, by introducing a mechanism that omits the processing of the filter target pixel number determination unit 4, the filter coefficient determination unit 6, and the filter processing unit 8 with the motion compensation block boundary flag IsEdge. It is possible to increase the speed by reducing the processing and to reduce the power required for the processing.
In this embodiment, an example in which encoding distortion removal processing is simply not performed has been shown, but instead of not performing encoding distortion removal processing, simple processing encoding distortion removal processing is used, and complicated processing is performed. May be switched to encoding distortion removal processing requiring 4 × 4 pixels.

(Embodiment 2)
In the present embodiment, a specific processing procedure for easily realizing the coding distortion removal method will be described. First, FIG. 4 is a flowchart showing a coding distortion removal method.
In step S18, it is determined whether or not the target block is a coding distortion removal target block. If it is an encoding distortion removal target block, the process proceeds to step S19, and if it is not a target block, the process proceeds to step S24.
In step S19, an appropriate coding distortion removal filter is selected, and in step S20, coding distortion removal processing is performed with the selected filter. In step S21, the target pixel is changed to an unprocessed pixel of the block. If no unprocessed pixel exists in the block in step S22, the process proceeds to step S24. If there is an unprocessed pixel, step 19 and subsequent steps are repeated.

In step S24, it is determined whether or not there is an unprocessed block on the screen. If there is an unprocessed block, the process proceeds to step S23, and if there is no unprocessed block on the screen, the coding distortion removal process on the screen is terminated.
In step S23, the target block is changed to an unprocessed block, and the process from step S18 is repeated.

FIG. 6 is a flowchart of determining the number of filter target pixels in the coding distortion removal method of the present invention, and is an example of the operation of the filter target pixel number determination unit 4 shown in FIG. FIG. 6 shows a case where the motion compensation block is shown in FIG. In addition, as shown in FIG.
p3, p2, p1, p0, q0, q1, q2, q3
And the encoded distortion-removed pixel values are P3, P2, P1, P0, Q0, Q1, Q2, Q3
And Here, each pixel value is obtained by adding a symbol to the order of arrangement of pixel positions, p0 to p3 and P0 to P3 are corresponding pixels in the same block, and q0 to q3 and Q0 to Q3 are correspondences of the same block. The pixel to perform is shown.

  The larger the quantization parameter QP, the coarser the quantization step and the larger the coding distortion. Therefore, it is effective to switch the filter depending on the magnitude of the quantization parameter QP. FIG. 5 shows a parameter correspondence table for quantization parameter QP and coding distortion removal. Table 1 below shows a correspondence table of parameters π, Ω, and n for parameter coding distortion removal processing for determining a parameter n that determines the number of target pixels of the filter. Note that π is, for example, an edge when the difference between pixel values is large, so that it is not filtered. Therefore, π should be determined so that filtering is not performed between pixels having a difference of less than π. In addition, for example, when the difference in pixel values is small, Ω is more influenced by coding errors rather than edges as the difference is smaller depending on whether the difference is very small (less than Ω) or slightly smaller (less than 2 × Ω). Therefore, Ω should be determined so as to apply a powerful filter (n is large).

Table 1

here,
dif1 = p0 q0
dif2 = p1 q1
dif1a = | dif1 |
Let dif2a = | dif2 |. That is, the flowchart of determining the number of filter target pixels in the coding distortion removal method of the present invention in FIG. 6 is summarized based on Table 1.

  In step S27, a pixel difference value DifPel, which is a parameter repeatedly calculated in the coding distortion removal process, is calculated. The pixel difference value DifPel is dif1a and dif2a calculated in step S27.

  In step S28, dif1a is compared with π, and if dif1a exceeds π, in step S29, n = 0 (that is, no coding distortion removal processing is performed), and the process is terminated. If dif1a is less than or equal to π, the process proceeds to step S30.

In step S30, dif2a is compared with Ω, and if dif2a is less than Ω, processing is performed in step S31 as n = 2 (that is, encoding distortion removal processing is performed from the block boundary to the second pixel in each adjacent block). finish. If dif2a is greater than or equal to Ω, the process proceeds to step S32.
In step S32, dif2a is compared with 2 × Ω, and if dif2a is less than 2 × Ω, in step S33, n = 1 (that is, encoding distortion removal processing is performed from the block boundary to the first pixel in each adjacent block). ) To end the process. If dif2a is 2 × Ω or more, the process ends in step S34 with n = 0 (that is, no coding distortion removal process is performed). dif2 is the absolute value of the difference between the pixel values near the boundary, and it is considered that the smaller the difference value is, the less high frequency components are in the vicinity of the boundary. Therefore, the smaller the dif2, the greater the number of pixels to be subjected to the coding distortion removal process. Thus, the encoding distortion at the boundary can be efficiently removed.

FIG. 7 is a flowchart of filter coefficient determination in the coding distortion removal method of the present invention, and is an example of the operation of the filter coefficient determination unit 6 shown in FIG.
In step S37, three conditions are compared using n and dif1a, dif2a, and φ, and if all three conditions are satisfied, a setting is made to perform a 3-tap filter process in step S39. . That is, φ is a threshold value for determining the number of taps of the filter. When n = 2 with a small number of high frequency components and when the pixel value changes little at the boundary (| dif2a−dif1a | <φ), the 3-tap filter become. Usually, a filter that strongly suppresses high frequency components is used for the 3-tap filter than for the 1-tap filter. Note that since the filter process can be switched depending on the value of n, it is possible to switch the type of filter instead of switching the number of pixels to be filtered using n thus obtained. Further, both the number of pixels to be filtered and the type of filter may be switched according to n thus obtained.

If the three conditions are not satisfied in step S37, it is determined in step S38 whether n is 1 or more. If n is 1 or more, a setting is made to perform 1-tap filter processing in step S40. On the other hand, if n is 0, it sets so that a filter process may not be performed by step S42.
Note that the quantization parameter QP may be changed for each block. However, the coding distortion removal process becomes complicated at block boundaries with different quantization parameters QP. In order to prevent this, in the present invention, even when the quantization parameter QP changes in the boundary block, at the boundary,
* Average value of quantization parameter QP of adjacent blocks (Note that fractions are rounded down, for example)
* Maximum value of quantization parameter QP of adjacent block * Minimum value of quantization parameter QP of adjacent block * Quantization parameter QP of left or upper adjacent block
The QP selected by any one of the above methods is used for filtering on both sides of the block boundary. Note that the difference due to the selection of the four quantization parameters QP is slight, and a preselected one may be used.
The encoding distortion can be easily removed by the method as described above.

  Next, FIG. 8A is a block diagram showing another example of the coding distortion removing unit 62 shown in FIG. Moreover, it is another example of the part enclosed with the dotted line shown in FIG. In the figure, devices that operate in the same manner as the devices described in the block diagram of the coding distortion removing unit using the conventional coding distortion removing method of FIG. 34 are assigned the same numbers, and descriptions thereof are omitted.

  The pixel difference calculation unit 20 calculates a pixel difference value at the block boundary from the reproduced image Coef, and outputs a pixel difference value DifPel. The pixel difference value DifPel includes signals corresponding to dif1a and dif2a. For example, as illustrated in FIG. 8B, the pixel difference value DifPel is obtained by comparing pixels at a position that is symmetrical with respect to the boundary line of the encoding unit block or a position that is symmetrical with respect to the top and bottom, and that difference (color difference). Or d1, d2, d3, and d4. If the average value of these differences (for example, (d1 + d2 + d3 + d4) / 4) is equal to or less than a certain value, it is considered that there is no image boundary line within the range of the width for which d4 is obtained. To work. On the other hand, if the average value is equal to or greater than a certain value, it is considered that there is an image boundary line, so control is performed so as not to execute the deblocking filter. Note that the comparison may be any one of d1, d2, d3, and d4, any two, or any one of the three. Further, instead of the average value, the largest difference may be compared with a constant value.

As an example of the operation of the filter target pixel number determination unit 4, there is a flowchart for determining the number of filter target pixels shown in FIG. An example of the operation of the filter coefficient determination unit 6 is a flowchart for determining the number of filter coefficients shown in FIG. Then, by referring to the pixel difference value DifPel as shown in FIG. 8B, the number of times of calculating the pixel difference can be reduced by each of the filter target pixel number determination unit 4 and the filter coefficient determination unit 6. Therefore, the filter target pixel number determination unit 4 and the filter coefficient determination unit 6 can determine the filter target pixel number and the filter coefficient without referring to the reproduced image Coef.
As described above, the amount of calculation can be reduced by repeatedly using the value calculated as the pixel difference value DifPel.

(Embodiment 3)
In this embodiment, an encoding device and a decoding device that can realize the encoding distortion removal method described in the other embodiments will be described.
FIG. 9 is a block diagram showing an encoding apparatus.

  The motion detection unit 30 compares the reference image Ref1 and the reference image Ref2 output from the first memory 38 and the second memory 40 with the image signal Vin, and detects a motion vector MV that is the amount of motion of the image signal Vin with respect to the reference image. To do. At this time, information indicating which of the reference image Ref1 and the reference image Ref2 is referred to reduces the prediction error is included in the motion vector MV and notified to the motion compensation unit 32. The motion compensation unit 32 extracts the image at the position indicated by the motion vector MV from the reference image Ref1 or the reference image Ref2, and outputs it as a motion compensation image MCpel.

The subtracting unit 42 calculates a difference value between the image signal Vin and the motion compensated image MCpel and outputs the difference value to the cosine transform unit (DCT) 46. The cosine transform unit 46 DCT transforms and quantizes the input difference value and outputs a frequency component FCoef. The zigzag scanning unit 48 outputs the frequency code component DCoef obtained by rearranging the order of the frequency components FCoef, and the variable length encoding unit 50 performs variable length encoding on the frequency code component DCoef and outputs the encoded signal Str.
On the other hand, the output of the DCT unit (cosine transform unit) 46 is input to the inverse DCT unit (inverse cosine transform unit). Then, the frequency component Fcoef and the motion compensated image Mcpel output from the motion compensation unit 32 are synthesized by the synthesis unit 34, and a synthesized image coef is output. On the one hand, the output composite image is output to the memory 38 as it is, and on the other hand, after being processed by the deblocking filter 36, the decoded image signal Vout from which the coding distortion has been removed is stored in the memory 40.
FIG. 10 is a block diagram showing the decoding apparatus. For example, the encoded signal Str encoded by the block diagram of the encoding apparatus shown in FIG. 9 can be correctly decoded. Devices that perform the same operations as those in the block diagram of the image decoding apparatus illustrated in FIG. 32 or 33 are denoted by the same reference numerals, and description thereof is omitted. An inverse DCT unit (inverse cosine transform unit) 56 performs inverse quantization and inverse DCT on the frequency component FCoe to output a difference image DifCoef, and an adder 58 adds the difference image DifCoef and the motion compensated image MCpel to reproduce a reproduced image. Get Coef. The reproduced image Coef is stored in the first memory 64, and the decoded image signal Vout obtained by removing the coding distortion from the reproduced image Coef by the deblocking filter 62 is stored in the second memory 66.

Through the above operation, the first memory 38 and the first memory 64 store the image from which the coding distortion has not been removed, and the second memory 40 and the second memory 66 store the image from which the coding distortion has been removed. become. The coding distortion removal process does not necessarily remove only the coding distortion, and a part of the original image signal may be lost. Therefore, in the encoding apparatus shown in FIG. 9, the motion detection unit 30 is provided with a mechanism that can always select an optimal one from both the first memory 38 and the second memory 40.
With this configuration, an appropriate reference image can be selected by referring to the first memory 38 even when a part of the original image signal is lost by removing the coding distortion. Similarly, an appropriate reference image can be selected also in the decoding apparatus shown in FIG.
In this embodiment, DCT is used as orthogonal transform, but Hadamard transform or wavelet transform may be used.

(Embodiment 4)
FIG. 11 is a block diagram showing an encoding distortion removing unit in the present embodiment, and corresponds to, for example, the deblocking filter 62 shown in FIG. In addition, a threshold value for determining a filter is determined. In this figure, devices that perform the same operations as those described in the block diagram of the conventional coding distortion removing unit shown in FIG.

  The filter determination parameter decoding unit 22 decodes the filter determination parameter signal FtrStr and outputs a filter parameter FtrPrm. The filter determination parameter signal FtrStr is not a threshold value itself but a parameter for determining the threshold value. The filter parameter FtrPrm corresponds to π, Ω, and φ in FIG. By decoding and obtaining the data obtained by optimizing these parameters π, Ω, and φ for each image from the filter determination parameter signal FtrStr, it is possible to perform coding distortion removal suitable for the image.

  FIG. 12 is a configuration diagram of the encoded signal Str in the encoding distortion removal method of the present invention. FIG. 12A shows an encoded signal corresponding to one screen, which includes picture data PicData that is data of one screen and a picture header PicHdr that is data common to all data of one screen. The picture header PicHdr includes a filter determination parameter signal FtrStr.

FIG. 12B shows the structure of the picture data PicData. The picture data PicData includes a slice signal SliceStr, which is an encoded signal of a slice composed of a set of a plurality of blocks.
FIG. 12C shows the structure of the slice signal SliceStr, which is composed of slice data SliceData which is data of one slice and a slice header SliceHdr which is data common to all data of one slice. By including the filter determination parameter signal FtrStr in the slice header SliceHdr, it is possible to correctly decode the encoded signal received in units of slice data SliceData.

  When the picture data PicData includes a plurality of slice signals SliceStr, instead of including the filter determination parameter signal FtrStr in all the slice headers SliceHdr, only some slice headers SliceHdr include the filter determination parameter signal FtrStr. You may make it. If the content of the filter determination parameter signal FtrStr is common to the slices, as shown in FIG. 12C, if the slice header SliceHdr does not have the filter determination parameter signal FtrStr, the filter determination parameter signal FtrStr of another slice header SliceHdr. It is also possible to suppress an increase in the number of bits due to repetition of the filter determination parameter signal FtrStr.

In addition, when the encoded signal Str is transmitted not in a continuous bit stream but in a packet or the like which is a unit of finely divided data, the header portion and the data portion other than the header may be separated and transmitted separately. In that case, the header part and the data part do not become one bit stream as shown in FIG. However, in the case of a packet, even if the transmission order of the header part and the data part is not continuous, only the header part corresponding to the corresponding data part is transmitted in another packet, and it becomes one bit stream. Even if not, the concept is the same as in the case of the bit stream described in FIG.
FIG. 13 shows a block diagram of an encoding apparatus. In the figure, devices that operate in the same manner as the devices in the block diagram of the encoding device shown in FIG.

  The memory 217 stores an input image signal Vin that is an image signal to be encoded. The image quality comparison unit 216 compares the image signal to be encoded read from the memory 217 with the decoded image Vout. The magnitude of the error obtained as a result of the comparison by the image quality comparison unit 216 is stored in the comparison result memory 218 together with the threshold value of the deblocking filter corresponding to the decoded image. The selection unit 219 selects the deblocking filter threshold corresponding to the smallest error among the errors stored in the comparison result memory 218 as the optimum threshold. The selected optimum threshold value is multiplexed in the threshold data addition unit 220 as an additional bit stream associated with the bit stream of the corresponding picture. Further, based on the optimum threshold output from the selection unit 219, the threshold control unit 215 generates a deblocking filter threshold candidate for the next picture, and notifies the deblocking filter 36 to notify the coding distortion removal processing threshold. And the comparison result memory 218 is notified of the threshold value currently in use.

FIG. 14 shows an encoding apparatus that more conceptually shows the block diagram of FIG. 13, for example, the optimum threshold selector 226 includes three components, the zigzag scan 48, the variable length encoder 50, and the threshold data adding unit 220 of FIG. This corresponds to the operation of the part excluding, or the operation of the memory 217, the image quality comparison unit 216, the comparison result memory 218, the selection unit 219, and the threshold control 215. Further, the video encoder 227 corresponds to the operation of a part other than the memory 217, the image quality comparison unit 216, the comparison result memory 218, the selection unit 219, and the threshold control 215 in FIG. The threshold value 228 corresponds to the optimum threshold value.
The optimum threshold value selector 226 selects an optimum threshold value. Here, the optimum threshold corresponds to, for example, a set of π, Ω, and φ determined for each quantization parameter QP shown in FIG. The selected optimum threshold value is stored in the memory 228 and added to the video encoder 227 as a filter determination parameter FtrStr. The encoded FtrStr is processed in the decoder by, for example, the filter determination parameter decoding unit 22 shown in FIG.

The optimum threshold value may be stored in a memory in the threshold control 215 shown in FIG. 13, and the threshold data may be sent to the threshold data adding unit 220 by the threshold control 215.
Next, the operation of determining the filter determination parameter FtrStr when removing the coding distortion will be described. FIGS. 15, 16, and 17 are flowcharts showing the operation of the encoding apparatus described in FIGS.
FIG. 15 is a flowchart illustrating an example of an operation for measuring image quality.

First, the target frame target_frame is set, and the first picture is output (step 229). Here, the target frame target_frame is a picture used to derive a threshold value.
Next, in the threshold control unit 215, a threshold range is set (step 230), and a value at one end of the range set in step 230 is output from the threshold control 215 as an initial value of the threshold (step 231).

Next, encoding distortion is removed by the deblocking filter 36 using the initial value of the set threshold, encoding of the image of the target frame target_frame is started (step 232), and the encoded first picture The image quality of the input image signal Vin is measured by the image quality comparator 216 (step 233).
The measurement result is recorded in the memory 218 (step 234), and the current frame number current_frame is incremented (step 235). That is, the first picture is switched to the next picture, and the next picture is output to, for example, the optimum threshold selector 226 and video encoder 227 shown in FIG. 14 or the memory 217, motion compensation unit 30, and subtraction unit 42 shown in FIG. .

In step 236, it is determined whether the current frame number has reached the target frame target_frame. If not, steps 233 to 235 are repeated. The image quality of the input picture is measured by the image quality comparison unit 216, and the measurement result is stored in the memory 218. If the current frame number has reached the target frame target_frame, the process proceeds to step 237 to return to the first picture that is the first picture.
In step 238A, the threshold is incremented in the threshold control 215. That is, the threshold value is set to the following value. Here, the next value is a value obtained by adding a predetermined increase amount from the initial value.

In step 238B, it is determined whether all threshold values have been tested up to the value at the other end of the threshold range. If the test has been performed for all the threshold values, the process for selecting the optimum threshold value is terminated. If all the threshold values have not been tested, the process returns to step 232, and the picture of the target frame target_frame is encoded again.
As described above, after the image quality is measured for all target frames target_frame with respect to one threshold, the threshold is added by a predetermined amount, and the image quality is measured again for all target frames target_frame, thereby measuring the image quality. be able to.
Next, using the flowchart of FIG. 16, image quality is measured for all the threshold values within the set threshold range for one picture, and then all the values within the set threshold range for the next picture are measured. A method of measuring the image quality with the threshold value will be described.

  First, the target frame target_frame is set, and the first picture is output (step 239). The current frame number current_frame is set to zero (step 240), and the first picture is set. Next, a threshold range is set in the threshold control unit 215 (step 241), and a threshold for the deblocking filter 36 is set (step 242).

Encoding (encoding distortion removal processing) is performed on the first picture using the initial value of the threshold (step 243), and the image quality of the encoded picture is measured by the image quality comparator 216 (step 244).
The measurement result is recorded in the memory 218 (step 245), and the threshold value control 215 unit increments the threshold value to the next value (step 246A).
Then, in step 246B, it is determined whether all thresholds have been tested. If all the threshold values have not been tested, the process returns to step 242 to measure the image quality of the same picture with different threshold values. If the image quality has been measured for all the threshold values, the process proceeds to step 247.

In step 247, the current frame number current_frame is incremented. That is, the first picture that is the first frame is switched to the second picture that is the next frame, and the next picture is shown in, for example, the optimum threshold selector 226 and the video encoder 227 shown in FIG. 14 or FIG. The data is output to the memory 217, the motion compensation unit 30, and the subtraction unit 42.
In step 248, it is determined whether the current frame number has reached the target frame target_frame. If the current frame number has not reached the target frame target_frame, the process returns to step 241. If it has reached, the image measurement process is terminated.

FIG. 17 is a flowchart illustrating a method for selecting an optimum threshold based on the threshold described in FIG. 15 or FIG. 16 and the image quality measurement result for the threshold.
In step 249 of FIG. 17, the selection unit 219 acquires a pair of image quality measurement result data and corresponding threshold data.
In step 250, the measurement results are arranged in a predetermined order.
In step 251, a picture having an optimum image quality is selected based on a predetermined condition, and a threshold corresponding to the picture is selected as the optimum threshold. The predetermined condition is, for example, that the SN ratio is small, the difference between the reproduced picture (image that has been deblocked by the threshold value) and the original picture (input image signal Vin) is the lowest, and the average of the differences Any one or combination of the lowest squared, etc. can be used.

In step 252, the selected optimum threshold is output to the video encoder 227 shown in FIG. 14, for example, as the filter determination parameter FtrStr.
Thus, the optimum threshold can be selected using the method described in FIG.
In Embodiment 4 described above, the image quality is measured for all threshold values in the range, the image quality measurement results are collected, and the optimum threshold value is selected from the measurement results. Alternatively, the image quality may be measured, the measurement of the image quality may be terminated when the optimum image quality measurement result is obtained, and the threshold value corresponding to the image quality measurement result may be selected as the optimum threshold value. This makes it possible to reduce the number of image quality measurements.
Further, in the process of removing the coding distortion of a certain block, the pixel value of the block is compared with the pixel value of the adjacent block. In this case, for the adjacent block, the block for which the encoding distortion removal process is completed and the pixel value is corrected is used.

For example, in FIG. 18, when the encoding distortion removal processing of the block G is performed, the encoding distortion can be removed from each of the four blocks E, D, H, and M adjacent to the block G. In this case, it is possible to perform more accurate coding distortion removal by performing the processing with a block that has already been subjected to coding distortion removal processing among adjacent blocks.
Therefore, the coding distortion is removed in a line sequential manner and in a scanning sequential manner. That is, it is preferable to remove the coding distortion in the scanning direction of the horizontal scanning line of the image and in the scanning line order.

That is, in FIG. 18, encoding distortion removal processing is performed in the order of A, B, E, F, C, D,... As the first scanning line, and there are four boundary lines for each block. Of these, it is preferable to perform a coding distortion removal process on adjacent blocks in contact with the upper boundary line and the left boundary line.
In this case, for the block A, since there is no adjacent block in contact with the upper boundary line and the left boundary line, the coding distortion is not removed.
For block B, since there is no adjacent block in contact with the upper boundary line, coding distortion is removed from adjacent block A in contact with the left boundary line.

For block G, the adjacent blocks in contact with the upper boundary line and the left boundary line are blocks E and D. Therefore, for block G, the coding distortion is removed for blocks E and D. The coding distortion is not removed for the blocks H and M.
In this way, for a new block, the coding distortion is removed with respect to the adjacent block for which the coding distortion has been removed, and the coding is performed with respect to the adjacent block for which the coding distortion has not yet been removed. If the distorted distortion is not removed, a more accurate coding distortion removal process can be performed.

(Embodiment 5)
In the present embodiment, as shown in FIG. 19, first, an example will be described in which a plurality of pixels, for example, four pixels arranged in a row, are grouped as one group, calculation is performed in pairs of groups, and coding distortion removal processing is performed. . The term “encoding distortion removal processing” as used herein refers to determination of whether or not to perform deblocking filter processing in a region sandwiching a boundary line between blocks and / or deblocking filter processing. The block refers to either a block including 4 × 4 16 pixels, which is an encoding processing unit, or a block that performs motion compensation described with reference to FIG.

  As shown in FIG. 19, the four pixels constituting one group are the four pixels in the block arranged in the same direction as the block boundary line. FIG. 19 shows groups r1, r2, r3, r4 including four pixels. Data of groups r1, r2, r3, r4 can be held in four registers (for example, SIMD registers). The groups r1 and r2 are in positions symmetrical with respect to the boundary line. The groups r3 and r4 are also symmetric with respect to the boundary line. The pixel values in the group r1 and the pixel values in the group r2 are compared, and the coding distortion is removed based on the difference.

  The calculation for obtaining these differences is the difference 1 between the pixel at the top of the group r1 and the pixel at the top of the group r2, the pixel next to the top of the group r1, and the top of the group r2. Difference 2 from the pixel, difference 3 between the pixel above the lower end of the group r1 and pixel above the lower end of the group r2, difference between the pixel at the lower end of the group r1 and the pixel at the lower end of the group r2 4 is expressed as a representative difference such as the average value of these differences 1, 2, 2, 3 and 4 or the sum of the absolute values of these differences 1, 2, 2, 3 and 4 as representative differences. Is compared with a predetermined threshold. Other calculation methods are possible. Since the calculation is performed in units of four pixels in the same group, parallel calculation is possible compared to the case where calculations are performed in units of pixels, and high-speed mounting by parallelization is facilitated.

  In the above, the case where the group r1 and the group r2 are compared has been described. Further, when the accuracy is required, the brightness of the pixels in the group r3 is compared with the brightness of the pixels in the group r4. It is also possible to perform coding distortion removal processing by adding or averaging both the representative difference of the comparison results obtained in the groups r1 and r2 and the current representative difference. Although the case where the boundary line is in the vertical direction has been described above, the same applies to the horizontal direction. When the boundary line is in the horizontal direction, four pixels arranged in the horizontal direction are captured as a group.

  Incidentally, FIGS. 20A and 20B show a case where the scanning lines on the screen are composed of interlaces. Here, an interlaced image is an image in which one frame is composed of two fields having different times. In interlaced image encoding and decoding processing, one frame may be processed as a frame, processed as two fields, or processed as a frame structure or a field structure for each block in the frame. it can. In the drawing, a small square of a gray color indicates pixels of odd lines, and a small square of white indicates pixels of even lines. In other words, one field is composed of gray-colored pixels composed of odd lines, and the other field of the frame is composed of white pixels composed of even-numbered lines. In the interlaced image signal, one frame is composed of two fields (even field and odd field) having different times. In the case of a still image, since the pixel value does not change with time, the correlation between the upper and lower lines of the frame is stronger than the correlation between the upper and lower lines of the field. On the other hand, in the case of a moving image, the image changes greatly with time, so the pixel values of the two fields differ greatly, and the correlation between the upper and lower lines of the field is stronger than the correlation between the upper and lower lines of the frame. Therefore, it is efficient to process still images as frames and moving images as fields.

When the picture is an interlaced image, for example, <1> blocks are all pictures having a frame structure (the frame structure will be described later), <2> blocks are all pictures having a field structure (the field structure will be described later), <3 > Conceivable is a picture in which a block can have a frame structure and a field structure block.
For example, when the picture is <1> (in the case of a frame), the deblocking filter processing is performed in units of the frame structure, and when the picture is <2> (in the case of a field), all the field structure Deblocking / filtering processing is performed in units. When the picture is <3>, deblocking / filtering processing is performed by adaptively converting from the field structure to the frame structure or from the frame structure to the field structure. Each will be described in detail below.

  An interlaced image having a still or small motion is processed in units of one frame including an odd field and an even field as shown in FIG. 20A (referred to as a frame structure). In the frame structure, as shown on the right side of FIG. 20A, an odd line pixel and an even line pixel are included in one block of 16 pixels. The coding distortion removal process is performed between blocks having a frame structure. That is, as described with reference to FIG. 8B, coding distortion removal processing is performed at block boundaries.

  A large motion interlaced image is processed in units of one field divided into an odd field and an even field as shown in FIG. 20B (referred to as a field structure). As shown on the right side of FIG. 20B, the field structure is divided into an odd field composed of odd lines and an even field composed of even lines. In the odd field, blocks composed of odd lines are divided. The even field includes a block composed of even lines. Coding distortion removal processing is performed only between odd-line blocks having a field structure or only between even-line blocks.

  FIG. 21A shows an example in which a part of an interlaced image is formed with a frame structure and another part is formed with a field structure. Preferably, in the image, the moving image portion is formed with a field structure and the still image portion is formed with a frame structure. The minimum unit formed by the field structure or the frame structure is a macroblock (or a super macroblock obtained by integrating a plurality of them) that is the maximum unit for performing orthogonal transformation such as DCT or motion compensation. Here, it is assumed that the square part including the automobile in FIG. 21A is formed with a field structure and the remaining part is formed with a frame structure.

A description will now be given of how encoding distortion removal processing is performed in a portion formed by the field structure and a portion adjacent to the portion formed by the frame structure.
In FIG. 21 (b), the blocks included in columns C1, C2, C3, and C4 belong to a portion where an automobile is located and are formed in a field structure because of movement, while included in columns C5, C6, C7, and C8. The blocks that belong to the part without the automobile are formed with a frame structure that is efficient for still images. In this example, the macroblock has 16 pixels both horizontally and vertically, and the block has 4 pixels both horizontally and vertically. In FIG. 21 (b), columns C4 and C5 are shown spaced apart, but are adjacent on the image. For the boundary line between the columns C3 and C4 and the boundary line between the columns C5 and C6, the coding distortion removal process shown in FIG. 8B is performed. For the boundary line between the column C4 and the column C5, as shown in FIG. 21 (c), the frame structure block in the column C5 is first converted to the field structure block. This conversion is performed, for example, by converting the pixels on the odd-numbered lines in the column C5 shown in FIG. 21B into blocks composed of the gray-colored pixels in the column C5 shown in FIG. 21C, and the column C5 shown in FIG. The even-line pixels are converted into blocks of column C5 made up of white pixels as shown in FIG. Subsequently, the coding distortion removal processing shown in FIG. 8B is performed on the boundary line between the columns C4 and C5.

  As described above, the frame structure block is converted to the field structure block. When the field structure block is changed to the frame structure block in the presence of motion, the vertical pixel correlation is lost and the vertical coding is performed. This is because unnatural degradation occurs when the distortion removal processing is performed. On the other hand, if the frame structure block is a field structure when it is stationary, the effect of suppressing the coding error in the high frequency component in the vertical direction will be reduced, but the correlation between the pixels in the vertical direction will not be lost. Deterioration is difficult to occur.

  Note that the frame structure block is converted to the field structure block in order to reduce the processing amount (only the frame is converted into the field). However, the processing amount is not considered, that is, the frame is converted into the field, and the field is further converted. Compared with the previous case, the amount of processing for converting the field into a frame is newly increased, but the processing amount is increased. It is determined whether the pixel whose pixel value is to be changed is within a frame structure block or a field structure block, and the pixel that is the target of the coding distortion removal process is the field structure If the block has a frame structure, the block having the frame structure has the field structure (the block having the pixel to be subjected to coding distortion removal processing) If the pixel subject to coding distortion removal processing is within the frame structure block, the field structure block is converted to the frame structure (coding distortion removal processing). May be converted into a block of a type of a block structure having a pixel to be processed.

  Hereinafter, the operation in the case where the frame structure and the field structure coexist will be described based on the flowchart of FIG.

  The frame of the interlaced image signal sequence is composed of two fields that are scanned at different times (time instant). That is, it is possible to perform frame coding (frame structure coding) in which two fields are combined into one, or field coding (field structure coding) that handles fields individually. Furthermore, these encoding methods can be grouped into the following two categories. In other words, either fixed encoding that encodes the entire screen by either frame encoding or field encoding, and the screen is divided into a plurality of sections, and either frame encoding or field encoding is performed for each section. This is adaptive coding performed by switching between the two. The fixed coding includes frame fixed coding corresponding to the frame structure and field fixed coding corresponding to the field structure. In fixed coding, an interlaced video sequence is always coded with either frame coding or field coding regardless of content. On the other hand, in the adaptive coding, it is possible to select either one of frame coding or field coding depending on the content, the screen, or the division unit in the screen. Segmentation within the screen can be done at a granularity down to the macroblock level. Therefore, in adaptive encoding, a macroblock can be encoded by either frame encoding or field encoding. In the following description, a macro block is simply abbreviated as a block.

  In a frame-coded block, that is, a block having a frame structure, the coding distortion is removed in the same manner as in the case of removing the coding distortion of non-interlaced video. In a field-coded block, that is, a block having a field structure, a field is divided into an even field and an odd field, each field is treated as an individual screen, and coding distortion is removed for each field.

  Step 63 determines whether the target block is field-encoded or frame-encoded. If the block is field encoded, steps 64-69 are executed. If the block is frame encoded, steps 70-72 are executed.

Steps 64 to 66 process even field structure blocks, and steps 67 to 69 process odd field structure blocks. The processing in steps 64 to 66 corresponds to the processing for removing the coding distortion between white pixels at the boundary between columns C3 and C4 in FIG. The processing in steps 67 to 69 corresponds to the processing for removing the coding distortion between the gray color pixels at the boundary between the columns C3 and C4 in FIG.
In step 64, the brightness between the pixels is compared to determine whether the coding distortion needs to be removed. In step 65, the number of pixels to be filtered is determined. In step 66, coding distortion is removed in the field mode.

In steps 67, 68 and 69, the same processing as in steps 64, 65 and 66 is performed.
In steps 70 to 72, a block having a frame structure is processed. This processing corresponds to the coding distortion removal processing at the boundary between columns C5 and C6 in FIG. In step 70, the brightness between the pixels is compared to determine whether the coding distortion needs to be removed. In step 71, the number of pixels to be filtered is determined. In step 72, coding distortion is removed in the frame mode.
In step 73, it is determined whether or not all blocks have been processed, and the process ends.

FIG. 23 shows a case where steps 64 and 67 of FIG. 22 are unified. That is, it is determined whether the coding distortion needs to be removed from both the even field block and the odd field block. Based on the determination result, both the even field block and the odd field block remove the coding distortion. It is. Thereby, it is possible to simplify the process of removing the coding distortion.
FIG. 24 shows a case where steps 65 and 68 of FIG. 23 are unified. That is, the number of pixels from which coding distortion is to be removed is determined for both the even field block and the odd field block. Based on this determination, the coding distortion of the even field and odd field blocks is removed. Thereby, it is possible to further simplify the process of removing the coding distortion.

FIG. 25 shows a boundary line between a frame structure block and a field structure block when a frame coding block and a field coding block are mixed in one screen as shown in FIG. The flowchart of a process in case it exists in the both sides on both sides of is shown.
In step 95, whether the boundary line between the blocks for which coding distortion is removed is a specific boundary line, that is, a boundary line having a frame structure block on one side of the boundary line and a field structure block on the other side. Judgment is made. This determination corresponds to determination of whether or not the boundary line is between the columns C4 and C5 in FIG. If it is determined that the boundary line is a specific boundary line, the process proceeds to step 96.

In step 96, the frame structure block on one side of the boundary is converted to a field structure block. This conversion corresponds to the conversion from the block shown in column C5 in FIG. 21B to the block shown in column C5 in FIG. The converted block is called a conversion block.
In step 97, it is determined whether the coding distortion needs to be removed between the field structure block on the other side of the boundary line and the transform block. This determination corresponds to the determination at the boundary line between columns C4 and C5 in FIG. If it is determined that it is necessary, the routine proceeds to step 98.

In step 98, the number of pixels to be filtered is determined.
In step 99, encoding distortion is removed in the field mode.
FIG. 25 shows an example in which the frame structure is converted to the field structure when the frame structure and the field structure block are adjacent in adaptive coding, and the coding distortion is removed in the field. It may be converted into a frame structure and coding distortion may be removed by the frame. If the coding distortion is removed in the field as shown in FIG. 25, even if the image signal has a fast motion, the coding distortion is removed using only the pixels at the same time, so that there is an advantage that unnatural image quality degradation is unlikely to occur. However, if the coding distortion is removed in the frame, the pixel correlation in the vertical direction is stronger in the image signal with small motion than in the field, so that the high frequency component is less deteriorated than in the case of the deblocking filter in the field. There are advantages. Accordingly, each has advantages, and one may be selected when manufacturing the device, or the user may be able to select one.

  Furthermore, even in the case of adaptive coding, coding distortion removal processing may be determined not on a block basis but on a screen basis (by frame or field). Implementation can be simplified by setting the deblock filter to one of the field mode or the frame mode for each screen. Either the field mode or the frame mode may be fixed to one, or may be switched for each screen. When switching is possible for each screen, the encoding device determines which mode is appropriate, and an identification signal indicating in which mode the deblocking filter of the decoding device should perform encoding distortion processing. It may be transmitted by being included in the header part of the code string.

If field / frame switching is possible in block units, coding distortion is removed in field units. If switching is impossible (prohibited), for example, information relating to pictures (picture parameters) cannot be switched in the picture. In this case, encoding distortion removal processing may be performed in units of frames.
Note that the deblocking filters shown in the first to fifth embodiments can be used both as the out-of-loop filter shown in FIG. 32 and the in-loop filter shown in FIG.

  In the in-loop filter, data before the coding distortion removal processing is stored in the memory 64, so that an image from which block distortion has not been removed is referred to as a predicted image, and the block distortion has been removed. The encoded image quality is slightly deteriorated as compared with the case where an image is referred to as a predicted image. On the other hand, in the out-of-loop filter, the result of the coding distortion removal process is not a reference image, so that even if an arbitrary one is mounted as the deblocking filter 62, the decoded image does not deteriorate greatly. For example, a small-computation amount filter is used as a deblocking filter 62 in a portable device where power consumption is given priority, and a high-precision, high-accuracy, high-quality filter is used as a deblocking filter 62 in an entertainment stationary device where image quality is the highest priority Can be used as

(Embodiment 6)
Further, by recording a program for realizing the configuration of the encoding distortion removal method, the encoding method, and the decoding method shown in each of the above embodiments on a storage medium such as a floppy disk, The processing shown in the embodiment can be easily performed in an independent computer system.

  FIG. 26 is an explanatory diagram in the case of implementing by a computer system using a floppy disk storing the encoding distortion removing method, encoding method, and decoding method of the first to fifth embodiments.

  FIG. 26B shows an external appearance, a cross-sectional structure, and a floppy disk when viewed from the front of the floppy disk, and FIG. 26A shows an example of a physical format of the floppy disk which is a recording medium body. The floppy disk FD is built in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the floppy disk storing the program, an encoding distortion removing method, an encoding method, and a decoding method as the program are recorded in an area allocated on the floppy disk FD.

  FIG. 26C shows a configuration for recording and reproducing the program on the floppy disk FD. When recording the program on the floppy disk FD, the encoding distortion removing method, the encoding method, and the decoding method as the program are written from the computer system Cs via the floppy disk drive. When the above-described encoding distortion removal method, encoding method, and decoding method are constructed in a computer system by a program in a floppy disk, the program is read from the floppy disk by a floppy disk drive and transferred to the computer system.

In the above description, a floppy disk is used as the recording medium, but the same can be done using an optical disk. Further, the recording medium is not limited to this, and any recording medium such as a CD-ROM, a memory card, and a ROM cassette that can record a program can be similarly implemented.
Furthermore, application examples of the moving picture coding method and the moving picture decoding method shown in the above embodiment and a system using the same will be described.
FIG. 27 is a block diagram showing an overall configuration of a content supply system ex100 that implements a content distribution service. The communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed wireless stations, are installed in each cell.

The content supply system ex100 includes, for example, a computer ex111, a personal digital assistant (PDA) ex112, a camera ex113, a mobile phone ex114, a camera via the Internet ex101, the Internet service provider ex102, the telephone network ex104, and the base stations ex107 to ex110. Each device such as the attached mobile phone ex115 is connected.
However, the content supply system ex100 is not limited to the combination shown in FIG. 27, and any combination may be connected. Also, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.

  The camera ex113 is a device capable of shooting a moving image such as a digital video camera. In addition, the mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global Mobile System). Alternatively, PHS (Personal Handyphone System) or the like may be used.

  In addition, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution or the like based on the encoded data transmitted by the user using the camera ex113 becomes possible. The encoded processing of the captured data may be performed by the camera ex113, or may be performed by a server or the like that performs data transmission processing. The moving image data shot by the camera 116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device such as a digital camera that can shoot still images and moving images. In this case, the moving image data may be encoded by the camera ex116 or the computer ex111. The encoding process is performed in the LSI ex117 included in the computer ex111 and the camera ex116. Note that image encoding / decoding software may be incorporated into any storage medium (CD-ROM, flexible disk, hard disk, or the like) that is a recording medium readable by the computer ex111 or the like. Furthermore, you may transmit moving image data with the mobile phone ex115 with a camera. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.

  In this content supply system ex100, the content (for example, video shot of music live) taken by the user with the camera ex113, camera ex116, etc. is encoded and transmitted to the streaming server ex103 as in the above embodiment. On the other hand, the streaming server ex103 stream-distributes the content data to the requested client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like that can decode the encoded data. In this way, the content supply system ex100 can receive and play back the encoded data at the client, and can also receive a private broadcast by receiving, decoding, and playing back at the client in real time. It is a system that becomes possible.

For the encoding and decoding of each device constituting this system, the moving image encoding device or the moving image decoding device described in the above embodiments may be used.
A mobile phone will be described as an example.
FIG. 28 is a diagram illustrating the mobile phone ex115 that uses the moving picture coding method and the moving picture decoding method described in the above embodiment. The mobile phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a video from a CCD camera, a camera unit ex203 capable of taking a still image, a video shot by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display that displays data obtained by decoding received video and the like, a main body unit composed of a group of operation keys ex204, an audio output unit ex208 such as a speaker for audio output, and audio input To store encoded data or decoded data, such as a voice input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. Recording media ex207 and mobile phone ex115 with recording media ex207 And a slot portion ex206 to ability. The recording medium ex207 stores a flash memory element which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is a non-volatile memory that can be electrically rewritten and erased in a plastic case such as an SD card.

  Further, the cellular phone ex115 will be described with reference to FIG. The cellular phone ex115 controls the main control unit ex311 which is configured to control the respective units of the main body unit including the display unit ex202 and the operation key ex204. The power supply circuit unit ex310, the operation input control unit ex304, and the image encoding Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronization bus ex313 Are connected to each other.

When the end of call and the power key are turned on by the user's operation, the power supply circuit unit ex310 starts up the camera-equipped digital cellular phone ex115 in an operable state by supplying power from the battery pack to each unit. .
The mobile phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305 based on the control of the main control unit ex311 including a CPU, ROM, RAM, and the like. The modulation / demodulation circuit unit ex306 performs spectrum spread processing, and the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits the result via the antenna ex201. In addition, the cellular phone ex115 amplifies the received signal received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, performs spectrum despreading processing by the modulation / demodulation circuit unit ex306, and analog audio by the voice processing unit ex305 After conversion into a signal, this is output via the audio output unit ex208.

  Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.
The image encoding unit ex312 has a configuration including the image encoding device described in the present invention, and an encoding method using the image data supplied from the camera unit ex203 in the image encoding device described in the above embodiment. The image data is converted into encoded image data by compression encoding, and is sent to the demultiplexing unit ex308. At the same time, the cellular phone ex115 sends the voice collected by the voice input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 via the voice processing unit ex305 as digital voice data.

The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit Spread spectrum processing is performed in ex306, digital analog conversion processing and frequency conversion processing are performed in the transmission / reception circuit unit ex301, and then transmitted through the antenna ex201.
When receiving data of a moving image file linked to a homepage or the like in the data communication mode, the received signal received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is obtained. Data is sent to the demultiplexing unit ex308.

  In addition, in order to decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data to generate an encoded bitstream of image data and an encoded bitstream of audio data. The encoded image data is supplied to the image decoding unit ex309 via the synchronization bus ex313, and the audio data is supplied to the audio processing unit ex305.

  Next, the image decoding unit ex309 is configured to include the image decoding apparatus described in the present invention, and a decoding method corresponding to the encoding method described in the above embodiment for an encoded bit stream of image data. To generate playback moving image data, which is supplied to the display unit ex202 via the LCD control unit ex302, thereby displaying, for example, moving image data included in the moving image file linked to the homepage . At the same time, the audio processing unit ex305 converts the audio data into an analog audio signal, and then supplies the analog audio signal to the audio output unit ex208. Thus, for example, the audio data included in the moving image file linked to the home page is reproduced. The

  It should be noted that the present invention is not limited to the above system, and recently, digital broadcasting using satellites and terrestrial waves has become a hot topic. As shown in FIG. Any of the decoding devices can be incorporated. Specifically, in the broadcasting station ex409, a coded bit stream of video information is transmitted to a communication or broadcasting satellite ex410 via radio waves. Receiving this, the broadcasting satellite ex410 transmits a radio wave for broadcasting, and receives the radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, such as a television (receiver) ex401 or a set top box (STB) ex407. The device decodes the encoded bit stream and reproduces it. In addition, the image decoding apparatus described in the above embodiment can also be implemented in a playback apparatus ex403 that reads and decodes an encoded bitstream recorded on a storage medium ex402 such as a CD or DVD as a recording medium. is there. In this case, the reproduced video signal is displayed on the monitor ex404. Further, a configuration in which an image decoding apparatus is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on a monitor ex408 of the television is also conceivable. At this time, the image decoding apparatus may be incorporated in the television instead of the set top box. It is also possible to receive a signal from the satellite ex410 or the base station ex107 by the car ex412 having the antenna ex411 and reproduce the moving image on a display device such as the car navigation ex413 that the car ex412 has.

  Further, the image signal can be encoded by the image encoding device shown in the above embodiment and recorded on a recording medium. Specific examples include a recorder ex420 such as a DVD recorder that records image signals on a DVD disk ex421 and a disk recorder that records images on a hard disk. Further, it can be recorded on the SD card ex422. If the recorder ex420 includes the image decoding device described in the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

The configuration of the car navigation ex 413 is, for example, the configuration shown in FIG. 29 excluding the camera unit ex 203, the camera interface unit ex 303, and the image encoding unit ex 312, and the same applies to the computer ex 111 and the television (receiver). ) Ex401 can also be considered.
In addition to the transmission / reception type terminal having both the encoder and the decoder, the terminal such as the mobile phone ex114 has three mounting formats, that is, a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.
Note that the configurations of the encoding device, the decoding device, and the like described in Embodiments 1 to 6 are not limited to those described.

  As described above, the moving picture encoding method or the moving picture decoding method described in the above embodiment can be used in any of the above-described devices and systems, and as a result, the above-described embodiment has been described. An effect can be obtained.

  The present invention is applicable to a coding distortion removal method, a moving image coding method, a moving image decoding method, and an apparatus and a program for realizing them.

Block diagram of an image decoding apparatus using a decoding method according to the present invention. The block diagram of the encoding distortion removal part using the encoding distortion removal method which concerns on Embodiment 1 of this invention. The figure which shows the example of the motion compensation block size The flowchart of the encoding distortion removal method which concerns on Embodiment 2 of this invention. The figure which shows each parameter correspondence of quantization parameter QP and encoding distortion removal which concerns on Embodiment 2 of this invention. Flowchart for determining the number of filter target pixels in the coding distortion removal method according to Embodiment 2 of the present invention. Flowchart for determining the number of filter coefficients in the coding distortion removal method according to Embodiment 2 of the present invention The block diagram of the encoding distortion removal part using the encoding distortion removal method which concerns on Embodiment 2 of this invention, and the figure showing the arrangement | sequence of a pixel The block diagram of the encoding apparatus using the encoding method which concerns on Embodiment 3 of this invention. Block diagram of a decoding apparatus using a decoding method according to Embodiment 3 of the present invention The block diagram of the encoding distortion removal part using the encoding distortion removal method which concerns on Embodiment 4 of this invention. Configuration diagram of encoded signal Str in the encoding distortion removing method according to Embodiment 4 of the present invention. Block diagram showing video encoding process with loop filter Block diagram showing the placement of automated threshold selection in a video encoding loop A flowchart showing how to collect data to find the optimal threshold Flow chart showing another way to collect data to find the optimal threshold Flow chart illustrating a method for selecting an optimal threshold Diagram showing the vicinity of blocks that share the same boundary that can skip the deblocking process Explanatory drawing which shows the group containing multiple pixels (A) is a frame structure, (b) is an explanatory diagram of a field structure. (A) is an explanatory diagram when a frame structure and a field structure are mixed in one picture, and (b) and (c) are steps for removing coding distortion at the boundary between the frame structure and the field structure. Illustration showing Flowchart of coding distortion removal processing when frame structure and field structure are mixed 22 is a flowchart in which steps 64 and 67 are integrated. 23 is a flowchart in which steps 65 and 68 are integrated. Flowchart of processing when a frame structure block and a field structure block are on both sides of a boundary line Explanatory drawing about the storage medium for storing the program for implement | achieving the variable length encoding method and variable length decoding method of Embodiment 1-Embodiment 2 by computer system regarding Embodiment 6 of this invention. Block diagram showing the overall configuration of the content supply system Example of mobile phone using moving picture coding method and moving picture decoding method Mobile phone block diagram Example of digital broadcasting system The figure which shows the signal level of a pixel in order to demonstrate the concept of an encoding distortion removal method. Block diagram of an image decoding apparatus using a conventional decoding method Block diagram of an image decoding apparatus using a conventional decoding method Block diagram of coding distortion removing unit using conventional coding distortion removing method

Explanation of symbols

30 motion detection units 32, 60 motion compensation units 50 variable length coding unit 52 variable length decoding unit 46 cosine transform unit 56 inverse cosine transform unit 48 zigzag scan unit 54 inverse zigzag scan unit 38, 40, 64 memory 62 deblock filter 34, 58 Addition unit 42 Subtraction unit 20 Pixel difference calculation unit 4 Filter target pixel number determination unit 6 Filter coefficient determination unit 8 Filter processing unit 22 Filter determination parameter decoding unit 2 Motion compensation block boundary determination unit 10a Selection unit 10b Selection unit Cs Computer system FD floppy disk FDD floppy disk drive

Claims (3)

A coding distortion removing method for removing coding distortion from an image,
Extracting an image parameter from an image including coding distortion; and a first step of specifying a pixel from which coding distortion is removed using the image parameter;
A second step of identifying a method for removing coding distortion using the image parameter;
A coding distortion removing method comprising: a third step of removing coding distortion from the pixel identified in the first step by the method identified in the second step.   A coding distortion removal method for removing coding distortion from an image, which determines whether or not to remove coding distortion in units of blocks, and further removes coding distortion for each pixel in the block to be removed based on the above determination. An encoding distortion removal method for determining a pixel to be subjected to an encoding distortion removal process by determining whether or not to perform an encoding distortion. An encoding distortion removal method for generating an encoded image by encoding an image divided into a plurality of macroblocks, and removing an encoding distortion of a decoded image obtained by decoding the encoded image as a reference image,
The pixel size of the motion compensation block is smaller than the pixel size of the macro block, and the pixel size of the motion compensation block is larger than the pixel size of the coding unit, and is adjacent to the macro block. When the difference image between the two motion compensation blocks is less than a predetermined value or 0,
The coding distortion is not removed at the block boundary inside each of the two motion compensation blocks,
Remove coding distortion at the block boundary between the two motion compensation blocks;
And a coding distortion removing method.

Images