JP2001204029A - Noise detection method, noise detector and image recorder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a detection method by which a block border causing block noises or a block producing a mosquito noise can surely be detected. SOLUTION: Ths method includes a coding information extract step where coding information including an orthogonal transform coefficient and a motion vector as to each block from a code sequence coded by using motion compression prediction with respect to an image and orthogonal transform and quantization in the unit of blocks, a reference area extract step where a reference area of each block is obtained from a reference frame, and a coded noise detection step where a coded noise to be eliminated is detected on the basis of the distribution of each frequency component of the orthogonal transform coefficient as to each block and a block in the reference frame overlapped in the reference area.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for decoding a compressed and coded image, and more particularly to a technique for detecting and removing coding noise caused by coding.

[0002]

2. Description of the Related Art In recent years, MPEG (Moving) has been used as a highly efficient compression and encoding method for images in the fields of broadcasting, communication and storage.
Picture Experts Group) method is widely used. In the MPEG system, encoding is performed by removing redundancy in a spatial direction and a temporal direction from an image.

In order to remove the redundancy in the spatial direction, a discrete cosine transform (hereinafter referred to as DCT) and a quantization process are used. First,
After the image is divided into units called blocks of 8 × 8 pixels, coefficients in the frequency domain (hereinafter, DC
(Referred to as a T coefficient), and performs a quantization process on the DCT coefficient.

[0004] The quantization process is a process of dividing a DCT coefficient by using both a quantization matrix having a value corresponding to each frequency in the DCT area and a quantization scale. By this quantization processing, the value of the frequency component having a small DCT coefficient value becomes zero. Generally, since the energy of an image signal is concentrated in a low band, a high frequency component is deleted by this processing. However, human visual characteristics become worse as the frequency becomes higher. Therefore, if the quantization scale used in the quantization process is small (the quantization step is small), the image quality degradation is not noticeable.

[0005] In order to remove temporal redundancy, motion compensation is used. In motion compensation, 16 × 16
A region closest to the reference frame is selected in units of a macroblock of pixels. Then, the difference value from the reference frame is encoded. If the movement is not so fast, the difference value is almost 0, so that the temporal redundancy can be reduced.

Normally, in the MPEG system, when the transmission bit rate of a code is high (the compression ratio is low), the image quality is hardly noticeable. However, as the bit rate decreases (compression rate increases), coding noise starts to stand out,
Image quality deteriorates. Representative examples of the coding noise in the MPEG system include block noise (also called block distortion) and mosquito noise (also called ringing noise and corona noise).

[0007] Block noise is a phenomenon in which block boundaries look clear and tile-like. This occurs because an image signal in a block has only low-frequency components and frequency component values differ between adjacent blocks.

[0008] Mosquito noise is noise that occurs as if a mosquito were flying around the edge. This is caused by the fact that the high frequency components originally contained in the image signal are eliminated by the quantization processing.

Block noise and mosquito noise are:
Unlike analog noise, it is noticeable as image quality degradation, and several methods have been proposed to remove them.

An example of a method for removing block noise is disclosed in Japanese Patent Application Laid-Open No. 5-308623 (first conventional example). In this conventional example, the frequency characteristic of the filter is determined and filtered according to the highest frequency of the DCT coefficient and the amount of motion obtained from the motion vector.

In Japanese Patent Application Laid-Open No. Hei 7-177521, a filter characteristic is determined using a quantization scale and a magnitude of a motion vector, and a filter is applied to a block boundary of a decoded image using the determined filter. Accordingly, a method of removing block noise has been disclosed (second conventional example). In this conventional example, when the quantization scale is large and the motion vector is small, the filter characteristics are determined using only the quantization scale. When the quantization scale is small and the motion vector is large, the filter characteristics are determined using only the magnitude of the motion vector. If the quantization scale and the magnitude of the motion vector are both medium, the filter characteristics are determined using both.

On the other hand, an example of a method for removing mosquito noise is disclosed in Japanese Patent Laid-Open No. Hei 6-31499 (third conventional example). In this conventional example, the absolute sum of a specific DCT coefficient of each block is obtained, and the threshold value of the filter is changed according to the value. In addition, for a frame on which inter-frame reference encoding has been performed, a mosquito noise is removed from the difference image, and thereafter, a decoded image is obtained by adding reference frames that have already been subjected to mosquito noise removal processing. .

[0013]

Generally, the occurrence of block noise is not determined by the highest frequency of the DCT coefficient, but rather by the difference between the distribution of each frequency component of the DCT coefficient and the distribution of each frequency component of the DCT coefficient between adjacent blocks. Is determined by However, if the filter characteristic is determined using only the highest frequency component of the DCT coefficient as in the first conventional example, the ability to remove block noise is reduced.

The method of the first conventional example has the following problems. That is, when the filter characteristics are individually determined for each of the horizontal direction and the vertical direction, the amount of computation required for the filter determination increases. Further, when the filter characteristic is determined by the DCT coefficient, the block noise detection ability is reduced in a case where inter-frame coding (non-intra coding) is performed. Furthermore, if the amount of movement is large, apply a strong filter,
When such control is performed, the property of the reference frame is not considered. For example, when the motion compensation is accurately performed even if the motion amount is large, the difference image is filtered even though it is 0, and the image quality deteriorates such as blurring. Therefore, image quality deteriorates for a non-intra-coded image.

[0015] Even when the motion vector is large, the decoded image may contain high frequency components.
For example, in the case of a frame that has been subjected to inter-frame coding, even if the quantization scale is large,
The reference frame may include a high frequency component, and in such a case, the decoded image includes the high frequency component. As in the second conventional example, if a filter is applied to the decoded image in such a case, the image quality deteriorates.

The method of the third conventional example has the following problems. That is, when the absolute sum of a specific DCT coefficient of each block is obtained, DCT
A large amount of processing is required for calculating the absolute sum of the coefficients. Furthermore, when a decoded image is obtained by adding reference frames that have already been subjected to the mosquito noise removal processing, an error occurs in the decoded image because the reference frame at the time of encoding differs from the reference frame at the time of decoding. The accumulated image quality is noticeably deteriorated.

The present invention solves such a problem, and can reliably detect a block boundary where block noise occurs or a block where mosquito noise occurs. Provided is a noise detection method and a noise detection device capable of accurately detecting the encoding noise of the above, and an image decoding device capable of reliably removing the encoding noise while minimizing the deterioration of the image quality. That is the task.

[0018]

Means for Solving the Problems To solve the above-mentioned problems, the invention according to claim 1 employs, as a noise detection method, motion compensation prediction for an image, orthogonal transform in block units, and quantization processing. From the code sequence coded by using the coding information extraction step of extracting the coding information including the orthogonal transform coefficient and the motion vector for each block, based on the motion vector for each block, from the reference frame A reference region extraction step for obtaining a reference region of each block, and for each block and a block in a reference frame overlapping the reference region, coding noise to be removed based on a distribution of each frequency component of the orthogonal transform coefficient. And a coding noise detecting step of detecting.

According to the first aspect of the present invention, not only the distribution of each frequency component of the orthogonal transform coefficient obtained from the code string,
Using the motion vector, encoding noise is detected based on the distribution of each frequency component of the orthogonal transform coefficient of the block overlapping the reference area in the reference frame. Therefore, in the non-intra coded block, the coding noise can be reliably detected, and erroneous detection is less likely.

According to a second aspect of the present invention, in the noise detection method according to the first aspect, block noise is detected as the coding noise. Based on the distribution of each frequency component of the orthogonal transform coefficients, the data is classified into a plurality of classes, and a class of a processing target block and an adjacent block adjacent thereto, and a reference frame in a reference frame overlapping each reference region of these blocks. A new class of each of the processing target block and the adjacent block is obtained based on a class of the block, and a magnitude of block noise generated between the processing target block and the adjacent block based on the new class. Is detected.

According to the second aspect of the present invention, noise is detected not only based on the class of the block to be subjected to noise removal but also based on the class of the block overlapping the reference area in the reference frame using the motion vector. In an intra-coded block, it is possible to reliably detect a block boundary where block noise occurs. Further, since the classification of the class is performed without dividing the horizontal direction and the vertical direction, it is possible to detect the block noise with a small amount of calculation processing.

According to a third aspect of the present invention, there is provided the noise detection method according to the first aspect, wherein mosquito noise is detected as the coding noise. Based on the distribution of each frequency component of the orthogonal transform coefficients, classified into a plurality of classes, based on the class of the block to be processed and the class of the block in the reference frame overlapping the reference region of the block to be processed, A new class of the processing target block is obtained, and a magnitude of mosquito noise generated in the processing target block is detected based on the new class.

According to the third aspect of the present invention, the noise is detected not only based on the class of the block to be processed for noise removal but also based on the class of the block overlapping the reference area in the reference frame using the motion vector. In an intra-coded block, a block in which mosquito noise occurs can be reliably detected. Further, since the classification of the classes is performed without dividing the classes into the horizontal direction and the vertical direction, mosquito noise can be detected with a small amount of calculation processing.

According to a fourth aspect of the present invention, in the noise detection method according to the first aspect, a block noise is detected as the coding noise. The DC component of the orthogonal transform coefficient for the processing target block and the adjacent block adjacent thereto is extracted, and in addition to the distribution of each frequency component of the orthogonal transform coefficient, the DC component between the processing target block and the adjacent block is extracted. A magnitude of block noise generated between the processing target block and the adjacent block is detected based on an absolute value of a difference between DC components.

According to the fourth aspect of the present invention, the absolute value of the difference between the DC components between the processing target block and the adjacent block is checked, so that block noise can be detected with high accuracy.

According to a fifth aspect of the present invention, in the noise detection method according to the fourth aspect, in the encoding noise detecting step, a quantization scale for the processing target block is extracted from the encoded information, In addition to the distribution of each frequency component of the orthogonal transform coefficient, the magnitude of block noise generated between the processing target block and the adjacent block based on the absolute value of the difference between the DC components and the quantization scale. Is detected.

According to the fifth aspect of the present invention, the absolute value of the difference between the DC components between the processing target block and the adjacent block is checked according to the quantization scale of the processing target block.
Block noise can be detected with high accuracy.

According to a sixth aspect of the present invention, in the noise detection method according to the first aspect, in addition to the distribution of each frequency component of the orthogonal transform coefficient, the motion vector of the processing target block or an adjacent block adjacent thereto is processed. The encoding noise to be removed is detected on the basis of the magnitude of.

According to the sixth aspect of the present invention, generally, there is a tendency that the larger the magnitude of the motion vector is, the larger the coding noise is. Therefore, the noise is appropriately detected by using the magnitude of the motion vector. be able to.

According to a seventh aspect of the present invention, in the noise detection method according to the sixth aspect, a block noise is detected as the coding noise. The DC component of the orthogonal transform coefficient for the processing target block and an adjacent block adjacent thereto is extracted, and in addition to the distribution of each frequency component of the orthogonal transform coefficient and the size of the motion vector, the processing target block and A magnitude of block noise generated between the processing target block and the adjacent block is detected based on an absolute value of a difference of the DC component between the adjacent block.

According to the seventh aspect of the present invention, since the absolute value of the difference between the DC components between the processing target block and the adjacent block is checked, block noise can be detected with high accuracy.

In the noise detection method according to the present invention, the encoding noise detecting step may include extracting a quantization scale for the processing target block from the encoded information, In addition to the distribution of each frequency component of the orthogonal transform coefficient and the magnitude of the motion vector of each block, based on the absolute value of the difference between the DC components and the quantization scale, the processing target block and the adjacent block And the magnitude of block noise occurring between the two is detected.

According to the eighth aspect of the present invention, the absolute value of the difference between the DC components between the processing target block and the adjacent block is checked according to the quantization scale of the processing target block.
Block noise can be detected with high accuracy.

According to a ninth aspect of the present invention, in the noise detection method according to the first aspect, block noise and mosquito noise are detected as the coding noise, and the block noise and the mosquito noise of one block are detected. Is selected as coding noise to be removed based on the magnitude of these noises.

According to the ninth aspect of the present invention, one of the block noise and the mosquito noise is selected for one block, so that the amount of arithmetic processing and memory required for noise removal can be reduced. Can be
Both block noise and mosquito noise can be appropriately removed.

According to a tenth aspect of the present invention, in the noise detection method according to the first aspect, encoding noise detection processing is performed for each interlace image for each field.

According to the tenth aspect, encoding noise can be detected with high accuracy even in an interlaced image.

According to an eleventh aspect of the present invention, as a noise detection device, each block obtained from a code sequence coded by using motion compensation prediction for an image, orthogonal transform in units of blocks, and quantization is used. The coding information including the orthogonal transformation coefficient and the motion vector is input, and the reference region of each block is obtained from the reference frame based on the motion vector of each block. Means for detecting coding noise to be removed, based on the distribution of each frequency component of the orthogonal transform coefficients, and outputting the detection result.

According to the eleventh aspect of the present invention, not only the distribution of each frequency component of the orthogonal transform coefficient obtained from the code string, but also each frequency of the orthogonal transform coefficient of the block overlapping the reference area in the reference frame using the motion vector. The coding noise is detected based on the distribution of the components. Therefore, in the non-intra coded block, the coding noise can be reliably detected, and erroneous detection is less likely.

According to a twelfth aspect of the present invention, as the image decoding device, the noise detecting device according to the eleventh aspect, the code sequence is decoded, and the orthogonal transform coefficients for each block and the processing target block are processed. A decoding unit that outputs encoded information including a motion vector; and a coding noise removing unit that removes coding noise based on a detection result output by the noise detection device.

According to the twelfth aspect of the present invention, in the non-intra coded block, the coding noise can be reliably detected, and the image quality may be degraded by removing the erroneously detected coding noise. Few.

Further, according to the invention of claim 13, according to claim 12,
In the image decoding device according to the above, the noise detection device detects mosquito noise as the encoding noise, and extracts a quantization scale for the processing target block from the encoding information. The encoding noise elimination unit uses a value corresponding to the quantization scale as a threshold for detecting an edge pixel not used for noise elimination.

According to the thirteenth aspect of the present invention, the pixels used for removing the mosquito noise can be selected according to the quantization scale, so that the mosquito noise can be reliably removed while suppressing the deterioration of the image quality at the edge portion. it can.

[0044]

Embodiments of the present invention will be described below with reference to the drawings. In the following embodiment, it is assumed that a compression code string of an image is generated by the MPEG-2 method, for example. In the following, in order to represent the difference between a set of DCT coefficient values and a set of pixel values,
A block composed of coefficients is called a DCT coefficient block, and a block composed of pixels is called a pixel block.

(First Embodiment) In the first embodiment,
An image decoding device that detects and removes block noise as coding noise when decoding an intra-coded frame will be described.

FIG. 1 is a block diagram of an image decoding apparatus according to the first embodiment. The image decoding apparatus shown in FIG. 1 includes a variable length decoding unit 110, an inverse quantization unit 111, an inverse DC
It includes a T unit 112, switches 113 and 114, a frame memory 115, a block noise removing unit 116, a block noise detecting unit 130, a parameter memory unit 118, and an adding unit 119. The block noise detection means 130 operates as a noise detection device.

The compression code string (input bit stream) is
First, it is input to the variable length decoding means 110. The compression code string is a motion compensated prediction for an image, 8
This is a code string coded using DCT, quantization, and variable-length coding in units of × 8 pixels. The variable length decoding unit 110 decodes the variable length code of the input compressed code sequence, and quantizes the DCT coefficients and parameters (motion vector, quantization scale, etc.) as coding information used in coding. ) Is extracted and output. The quantized DCT coefficients are 64 coefficients of 8 horizontal frequencies × 8 vertical frequencies, and constitute a block of quantized DCT coefficients.

FIG. 2 is an explanatory diagram of the configuration of a macroblock when the image format is the 4: 2: 0 format. As shown in FIG. 2, a macroblock is a region of 16 × 16 pixels represented by a luminance signal, and is represented by four DCT coefficient blocks of a luminance signal and two DCT coefficient blocks of a color difference signal. The pixels of the color difference signal are thinned out, and the number of pixels in both the horizontal and vertical directions is half that of the luminance signal.

The quantized DCT coefficients obtained by the variable length decoding means 110 are input to the inverse quantization means 111 in the order of macro blocks. The inverse quantization means 111 performs inverse quantization of the quantized DCT coefficients for each block included in the macroblock, using the quantization scale and the quantization matrix obtained by the variable length decoding means 110. , DC
Obtain a T coefficient block. The inverse quantization means 111 uses the DCT
The coefficient block is output to the inverse DCT unit 112 and the block noise detection unit 130. The inverse DCT means 112
Inverse DCT is performed on the CT coefficient block to obtain a pixel block. The inverse DCT unit 112 performs inverse DCT on all blocks included in the macroblock, and then converts the luminance signal data into a frame structure using the DCT mode obtained by the variable length decoding unit 110.

Here, the DCT mode is a flag indicating whether the DCT is performed on the luminance signal data in a frame structure or a field structure. However, if the frame is a progressive (sequentially scanned) image, the DCT mode has only a frame structure, so that it is not necessary to convert the luminance signal data into a frame structure.

The pixel block obtained by the inverse DCT means 112 is input to the switch 113. Switch 113,
Reference numeral 114 switches according to the macroblock coding mode output from the variable length decoding unit 110. The macroblock coding mode is information indicating whether the macroblock has been subjected to intra-frame coding (intra coding) or inter-frame coding using a reference frame (non-intra coding). If the macroblock is intra-coded, switches 113 and 114 are connected to a and c, respectively. If the macroblock is non-intra-coded, the switch 11
3, 114 are connected to b and d, respectively.

Therefore, in the case of a macroblock that has been intra-coded, each pixel block of the macroblock output by the inverse DCT means 112 is stored in the frame memory 115 as it is. Also, each pixel block of the macroblock that has been non-intra coded is input to the adder 119. In the MPEG-2 system, since motion compensation is performed in units of macroblocks, an image in a reference frame corresponding to a macroblock, which is obtained using the motion vector output from the variable length decoding unit 110, is also added from the frame memory 115. Input to the device 119. Adder 1
19 adds the pixel block input from the switch 113 and the image in the reference frame obtained from the frame memory 115, and stores the result in the frame memory 115 via the switch 114.

The DCT coefficient is input to the block noise detection means 130 from the inverse quantization means 111, and the quantization scale and the coding mode of the macroblock are input from the variable length decoding means 110. The block noise detecting means 130 detects coding noise to be removed based on the distribution of each frequency component of the DCT coefficient, and determines the type of filter used for noise removal by the block noise removing means 116.
Notify. In addition, the block noise detection means 130
Parameters for each block are extracted from the input data, and the parameters are input / output to / from the parameter memory unit 118 as needed.

The block noise removing means 116 receives the type of filter to be applied to the boundary of each pixel block from the block noise detecting means 130, applies a filter to the block boundary of the image output from the frame memory 115, and removes the block noise. Output the removed image.

FIG. 3 is a block diagram showing the configuration of the block noise detecting means 130. The block noise detecting means 130 shown in FIG.
Coefficient extracting means 132 and quantization scale extracting means 133
And a filter determining means 134.

The DCT pattern determination means 131 receives the DCT coefficient block from the inverse quantization means 111,
Each block is classified from the distribution of each frequency component of the T coefficient block. Next, a method of classifying blocks will be described.

FIG. 4 is an explanatory diagram of a DCT pattern used for classifying blocks. The shaded portions in FIGS.
Each of 8 × 8 DCT coefficient block patterns (DC
T pattern), and one square corresponds to one coefficient. The upper left coefficient indicates a DC component, and the right one indicates a high horizontal frequency component, and the lower one indicates a high vertical frequency component. In the following, of the DCT coefficients,
The DC component is called a DC coefficient.

The DCT pattern determining means 131 determines whether or not the input DCT coefficient block satisfies the DCT pattern. That is, if the input DCT coefficient block has a coefficient of a frequency component included in the DCT pattern of FIG. 4 whose absolute value is larger than a predetermined value, the input DCT coefficient block Is determined to satisfy the DCT pattern, otherwise, it is determined not to be. For example, assuming that the predetermined value is 0, among the coefficients of the hatched frequency components,
If at least one has a non-zero coefficient, the input DCT
The coefficient block is determined to satisfy the DCT pattern.

The DCT pattern determination means 131 classifies the input DCT coefficient block into one of a plurality of DCT classes based on the result of the determination, and outputs the classification result to the parameter memory means 118.

Since the DCT pattern is set so as to include a high frequency component, generally, block noise is less likely to occur in a DCT coefficient block that satisfies a certain DCT pattern than in a DCT coefficient block that does not.

The DCT pattern determining means 131 determines whether the input DCT coefficient block is an intra-coded block as determined from the macroblock coding mode.
The DCT pattern PTN1 of (a) and the DC of FIG.
The T pattern PTN2 is used. Since block noise is likely to occur around a block having only low-frequency DCT coefficients, it can be said that a block satisfying only the DCT pattern PTN2 is more likely to generate block noise than a block satisfying the DCT pattern PTN1. .

FIG. 5 is a flowchart showing the flow of processing for classifying intra-coded blocks. Here, DCT pattern PTN1 and DCT pattern PTN2
The case where is used will be described. As shown in FIG. 5, the input DCT coefficient block first receives a D
This is compared with the CT pattern PTN1. If the input DCT coefficient block satisfies the DCT pattern PTN1, the block is classified into DCT class I1 (step S).
13). Input DCT coefficient block is DCT pattern PT
If N1 is not satisfied, next, in step S12, comparison with the DCT pattern PTN2 is performed. Input DC
If the T coefficient block satisfies the DCT pattern PTN2, the block is classified into DCT class I2 (step S14). If the input DCT coefficient block does not satisfy the DCT pattern PTN2, the block is
It is classified into CT class I3 (step S15). As described above, the DCT pattern determination unit 131 classifies each block into one of the DCT classes I1, I2, and I3, and outputs the classification result to the parameter memory unit 118.

The DC coefficient extracting means 132 outputs the input D
Only the DC coefficients are extracted from the CT coefficients and output to the parameter memory means 118. Quantization scale extracting means 133
Extracts the quantization scale from the encoded information output by the variable length decoding means 110,
Output to

In the following, such DCT class, DC
The coefficient and the quantization scale are collectively referred to as a block noise parameter. The parameter memory means 118
The input block noise parameter is accumulated.

By performing the above operation for each macro block, block noise parameters for each block are stored in the parameter memory means 118.

When the block noise parameters for the blocks for one frame are accumulated, the filter determining means 134 determines a filter to be applied to the boundary of each block while referring to the block noise parameters. The operation will be described below.

FIG. 6 is an explanatory diagram of the arrangement of the pixel blocks. In FIG. 6, one square indicates one pixel block. It is assumed that pixel blocks are arranged as shown in FIG. 6 and a case where a filter to be applied to a block boundary of the pixel block 501 is determined is considered. Here, the pixel block 501 is a processing target block that is currently subjected to the noise detection processing. First, the pixel block 50
Consider a case where a filter to be applied to a block boundary 511 between the pixel block 502 and the pixel block 502 is determined.

FIG. 7 is a flowchart showing the procedure for obtaining the type of filter. In step S31, the filter determining unit 134 acquires the block noise parameters of the pixel blocks 501 and 502 from the parameter memory unit 118. Then, the DCT class is compared among the block noise parameters, and the type of the filter is determined. Table 1 shows the type of filter.
Perform according to.

[0069]

[Table 1]

Table 1 shows an example of a determination method in the case where there are three types of filters, F1, F2, and F3. Here, filter strength (ability to remove block noise)
Is weakest in F1 and strongest in F3. Next, step S
At 32, the filter determination unit 134 changes the type of the filter using the DC coefficient and the quantization scale among the block noise parameters. Here, the DC coefficients of the pixel blocks 501 and 502 are respectively DC1 and DC
2. Assuming that the quantization scales are QS1 and QS2, respectively, the following equation (1) or (2): abs (DC1-DC2)> QS1 × k (QS1 <QS2) (1) abs (DC1- DC2)> QS2 × k (QS1 ≧ QS2) If any one of the following conditions is satisfied, the type of the filter is changed to F1. Here, k is a constant, and abs indicates an absolute value operation.

Similarly, the filter determining means 134
Pixel block 501 and pixel blocks 503, 504, 5
Block boundaries 512, 513, and
The type of filter to be applied to 514 is determined using the block noise parameter of each block. Then, the filter determining unit 134 outputs the determined filter type to the block noise removing unit 116.

The type of filter determined in this way corresponds to the magnitude of the detected block noise.
That is, if the determined filter type is F3, it can be said that the filter determination unit 134 has detected strong block noise, and if the determined filter type is F1, it has detected weak block noise. Alternatively, it can be said that no block noise was detected. An example of the operation of the block noise removing unit 116 will be described below.

FIG. 8 is an explanatory diagram of a horizontal filtering process for a boundary between blocks. FIG. 8 shows the pixels of the pixel block 501 and the pixel block 502 in FIG. 6, and one square represents one pixel. Now, consider a case where a filter is applied to the boundary 511 between the pixel blocks 501 and 502.

When the block noise detecting means 130 detects that the boundary 5
If the type of filter to be applied to 11 is determined to be F1, the block noise removing unit 116 does not apply a filter to the boundary 511. Block noise detection means 130 is at boundary 5
If the type of filter applied to 11 is determined to be F2, the block noise removing unit 116 applies a weak filter to pixels near the boundary 511. In this case, considering the pixels on the fourth line from the top in FIG. 8A, for example, a filter is applied to pixels b, c, d, and e. As the filter, a low-pass filter or the like can be used.

FIG. 9 is a graph showing an example of the frequency characteristic of the filter. When the type of the filter is F2, for example,
A low-pass filter having a frequency characteristic indicated by L2 in FIG. 9 is used. By applying the filter in this way, the pixel values shown in FIG. 8B become as shown in FIG. 8C, and the block noise is removed. Here, FIG.
In (b) and (c), the vertical axis indicates the pixel value, and the horizontal axis indicates the pixel position in the horizontal direction. The pixel positions here correspond to the pixel positions in FIG.

The block noise detecting means 130 detects the boundary 51
When the type of filter to be applied to 1 is determined to be F3, the block noise removing unit 116 applies a strong filter to pixels near the boundary 511. In this case, the filter is applied to pixels in the same or wider range as when the type of the filter is F2. Considering pixels on the fourth line from the top in FIG. 8A, for example, pixels a, b, c, d, e,
Filter f. As the filter, a low-pass filter or the like can be used as in the case of the filter type F2.

When the type of the filter is F3, for example,
A low-pass filter having a frequency characteristic indicated by L3 in FIG. 9 is used. When using a low-pass filter,
As shown in FIG. 9, when the type of filter is F3 (L
The cutoff frequency of 3) is set lower than that of F2 (L2). Thus, the ability to remove block noise is greater when the type of filter is F3 than when it is F2. By applying the filter in this way, FIG.
The pixel value shown in (d) becomes as shown in FIG. 8 (e), and the block noise is removed.

Here, the case where the filter is applied in the horizontal direction has been described.
The same applies to the case of applying a filter in the vertical direction in 13, 514 and the like.

As described above, the block noise removing means 11
The frame subjected to the filtering in 6 is output as an output image.

As described above, in the image decoding apparatus according to the first embodiment, each block is classified into a plurality of DCT classes using the DCT coefficients obtained from the compressed code string. Also,
DCT class in adjacent blocks, quantization scale,
A filter to be applied to the block boundary is determined using the DC coefficient. Here, when the filter is determined, the strength of the filter is set to be stronger as the DCT coefficient equal to or larger than the predetermined value is distributed only in the lower band. And
Based on the determined filter, a filter is applied to pixels around the block boundary of the decoded image.

As described above, according to the image decoding apparatus of the first embodiment, a block boundary where block noise occurs can be reliably detected from the distribution of DCT coefficients of adjacent blocks, and the DC coefficient Inspection of the absolute value of the difference eliminates erroneous detection of block noise. Then, prepare multiple filters with different intensities,
By selecting and using the block noise according to the magnitude of the block noise, it is possible to surely remove the block noise while minimizing blurring of the image. In addition, since the classification of the DCT class is performed without dividing the horizontal direction and the vertical direction,
Classification can be performed with a small amount of processing.

In this embodiment, the DCT pattern for the intra-coded block is shown in FIG.
(B) using two DCT coefficient blocks
Although the case of classifying into the T class has been described, the number of DCT patterns is not limited to two, and the number of DCT classes is three.
Not just one. The frequency distribution of the DCT pattern is
The distribution is not limited to those shown in FIGS.

Although the right side of the equations (1) and (2) is an equation using a quantization scale, the value on the right side of these equations may be a fixed value regardless of the quantization scale.

Although three types of filters are used in the block noise removing means 116, any number of types may be used.

Although the case where the filter used in the block noise removing means 116 is a low-pass filter has been described, other filters such as a median filter may be used as long as the filter removes block noise. .

Further, it has been described that no filter is applied to the block boundary when the type of filter is F1, but a weaker filter may be applied than when the type of filter is F2.

Also, a case has been described where the filter is applied to four pixels near the block boundary when the type of filter is F2, and the filter is applied to six pixels near the block boundary when the type of filter is F3. However, the range of pixels to be filtered may be a range different from that of the present embodiment.

Although the case where the range in which the filter F2 is applied is different from the range in which the filter F3 is applied has been described, the range may be the same.

The case where the filter determining means 134 determines the filter when the block noise parameter for one frame is stored in the parameter memory means 118 has been described. Good.

(Second Embodiment) In the second embodiment,
An image decoding apparatus that detects and removes block noise as coding noise when decoding a non-intra-coded frame will be described.

The image decoding apparatus according to the second embodiment
In the image decoding apparatus of FIG. 1, a block noise detecting means 140 is used instead of the block noise detecting means 130. Variable length decoding means 110, inverse quantization means 111, inverse DCT means 112, switches 113 and 11
4. Frame memory 115, block noise removing means 1
16, parameter memory means 118, and adding means 11
9 is the same as that of the first embodiment, and thus the same reference numeral is assigned and the description is omitted. Here, the block noise detection means 140 operates as a noise detection device.

FIG. 10 is a block diagram showing the structure of the block noise detecting means 140. The block noise detection unit 140 in FIG. 10 includes a DCT pattern determination unit 141,
DC coefficient extracting means 142 and quantization scale extracting means 1
43, filter determining means 144, switch 145
, A parameter correction unit 146, and a reference area parameter determination unit 147. The DCT coefficient is input to the block noise detection unit 140 from the inverse quantization unit 111, and the quantization scale,
The coding mode of the macroblock and the motion vector are input. It is also assumed that the parameter memory unit 118 holds the block noise parameter for the already decoded frame obtained by the method described in the first embodiment.

The DCT pattern determination means 141 receives the DCT coefficient block from the inverse quantization means 111,
Each block is classified from the distribution of each frequency component of the T coefficient block, and the classification result is output to the switch 145.

Next, a method of classifying blocks will be described. D
As described in the first embodiment, the CT pattern determination unit 141 determines a coefficient of a frequency component which is hatched in the DCT pattern of FIG. 4 and whose absolute value is larger than a predetermined value. If the input DCT coefficient block has the input DCT coefficient block, it is determined that the input DCT coefficient block satisfies the DCT pattern.

The DCT pattern determination means 141 determines from the macroblock coding mode,
When the coefficient block is an intra-coded block, the DCT pattern PTN1 of FIG.
The DCT pattern PTN2 of (b) is used. On the other hand, based on the macroblock coding mode,
When the T coefficient block is a non-intra coded block, the DCT pattern PTN3 of FIG.
(D) The DCT pattern PTN4 is used.

The processing method for the intra-coded block has been described in the first embodiment and will not be described.
Hereinafter, a method of processing a non-intra-coded block will be described.

FIG. 11 is a flowchart showing the flow of processing for classifying non-intra-coded blocks. As shown in FIG. 11, the input DCT coefficient block is first compared with the DCT pattern PTN3 in step S21. Input DCT coefficient block is DCT pattern PTN3
Is satisfied, the block is classified into the DCT class N1 (step S23). If the input DCT coefficient block does not satisfy the DCT pattern PTN3, a comparison with the DCT pattern PTN4 is performed in step S22. Input DCT coefficient block is DCT pattern P
If TN4 is satisfied, the block is DCT class N
2 (step S24). If the input DCT coefficient block does not satisfy the DCT pattern PTN4,
The block is classified into DCT class N3 (step S25). As described above, the DCT pattern determining unit 141 classifies each block into the DCT classes N1, N2, N3.
And outputs the classification result to the switch 145.

The DC coefficient extracting means 142 outputs the input D
Only the DC coefficient is extracted from the CT coefficient and output to the switch 145. The quantization scale extraction unit 143 extracts a quantization scale from the encoded information output from the variable length decoding unit 110, and outputs it to the switch 145.

The switch 145 is connected to the variable length decoding unit 11
The connection is switched by using the macroblock coding mode output by 0. When the macroblock coding mode is the intra coding, the switch 145 is connected to b. The operation in this case is the same as in the first embodiment.
When the macroblock coding mode is non-intra coding, the switch 145 is connected to a. Therefore, the block noise parameter is input to the parameter correction unit 146. Hereinafter, a case where the macroblock coding mode is non-intra coding will be described.

The reference area parameter determining means 147 refers to the block noise parameter of the reference block held by the parameter memory means 118 using the motion vector output from the variable length decoding means 110, and calculates the block noise parameter of the reference area. decide. Here, the reference area is a block in a reference frame that is referred to based on a motion vector of the block to be processed, which is a target of noise removal, when decoding the block. The reference block is a pixel block in the reference frame that overlaps the reference area. The detailed operation of the reference area parameter determination means 147 will be described below.

FIG. 12 is an explanatory diagram of a block to be processed and its reference area. FIG. 12A shows a processing target block 521 to be subjected to noise removal in a frame being decoded, and FIG. 12B shows a reference area 526 and a reference block 522 in a reference frame.
To 525. The reference area 526 is a block to be referred to using a motion vector when decoding the processing target block 521.

As shown in FIG. 12A, the address in the frame of the block 521 (the horizontal and vertical position of the upper left pixel of the block in terms of the number of pixels, starting from the upper left of the frame as a base point) is represented by (x , Y), and the motion vector is (MV)
x, MVy), the address of the reference area 526 is (x + MVx, y + MVy) as shown in FIG. The reference area parameter determining means 147 determines whether the reference area 5
From the address 26, a block in which the reference area 526 overlaps is found. In the case of FIG. 12B, the reference area 526
Overlaps with the blocks 522 to 525, the reference area parameter determination unit 147 determines
The block noise parameter No. 5 is obtained from the parameter memory unit 118, the block noise parameter of the reference area 526 is obtained using these block noise parameters, and output to the parameter correction unit 146.

First, a method of obtaining the DCT class from the block noise parameters of the reference area 526 will be described. Here, as an example, blocks 522 to 525
Are sequentially I2, I1, I2, and I3, and the number of pixels of the overlapping portion of the blocks 522 to 525 and the reference area 526 is 36, 12, 12,.
Consider the case of 4.

As a first method for obtaining the DCT class of the reference area 526, a value obtained by weighting and averaging the DCT classes of the blocks 522 to 525 with the number of pixels of a portion where the reference area 526 overlaps each block is used. , The DCT class of the reference area 526. This is because the DCT class I1 of the blocks 522 to 525 corresponds to the value of 0, the DCT class I2 corresponds to the value of 1, and the DCT class I3 corresponds to the value of 2.
This is a method in which an average value weighted by the number of pixels of a portion where a block belonging to a class overlaps is obtained, and a DCT class corresponding to a value closest to the average value is selected.

In this method, the weighted average value of the number of pixels of the value corresponding to the DCT class is (1 × 36 + 0 × 1).
2 + 1 × 12 + 2 × 4) /64=0.875. This average value is 1 when rounded to the first decimal place, and DC
Since the T class I2 corresponds to the value 1, the reference area 52
The DCT class of No. 6 is defined as I2.

As a second method for obtaining the DCT class of the reference area 526, there is a method of selecting a DCT class of a block having the largest number of pixels in a portion overlapping the reference area 526. In this method, the DCT class of the reference area 526 is set to the DCT class I2 of the block 522.

As a third method for obtaining the DCT class of the reference area 526, there is a method of obtaining the minimum value or the maximum value of the DCT class of each block. In this method, if the minimum value is selected, the DCT class of the reference area 526 becomes I1
When the maximum value is selected, the DCT class of the reference area 526 becomes I3.

As a fourth method of obtaining the DCT class of the reference area 526, the fourth method of blocks 522 to 525 is used.
As far as the blocks are concerned, there is a way to select the DCT class into which the most blocks have been classified. Here, since there are the most blocks classified into I2, the DCT class of the reference area 526 is I2 in this method.

Next, a method for obtaining DC coefficients among the block noise parameters of the reference area 526 will be described.

As a method of obtaining the DC coefficient of the reference area 526, a value obtained by weighting and averaging the DC coefficients of the blocks 522 to 525 with the number of pixels of the portion where the reference area 526 and each block overlap each other is calculated as follows. There is a method of using 526 DC coefficients.

As another method of calculating the DC coefficient of the reference area 526, there is a method of calculating the average of the DC coefficients of the blocks 522 to 525.

Next, a method of obtaining the quantization scale from the block noise parameters of the reference area 526 will be described.

As a method of obtaining the quantization scale of the reference area 526, a value obtained by weighting and averaging the quantization scales of the blocks 522 to 525 with the number of pixels of the portion where the reference area 526 overlaps each block is Reference area 5
There are 26 quantization scales.

As another method of obtaining the quantization scale of the reference area 526, there is a method of obtaining the minimum value or the maximum value of the quantization scale in blocks 522 to 525.

The parameter correction means 146 is a switch 1
45, the block noise parameter of the processing target block output by the DCT pattern determination unit 141, the DC coefficient extraction unit 142, and the quantization scale extraction unit 143, and the block noise parameter of the reference area output by the reference area parameter determination unit 147 And as input. Then, the block noise parameter of the block to be processed is corrected with the block noise parameter of the reference area and output to the parameter memory unit 118. The parameter memory means 118 stores the input block noise parameters.

First, the DCT class determined by the DCT pattern determining means 141 is stored in the reference area parameter determining means 1.
The correction is performed using the DCT class of the reference area determined by 47. Table 2 shows an example of this correction method.

[0117]

[Table 2]

In the correction method shown in Table 2, of the DCT class determined by the DCT pattern determination means 141 and the DCT class of the reference area determined by the reference area parameter determination means 147, the one which is less likely to generate block noise is selected. Will be.

Table 3 shows an example of another correction method.

[0120]

[Table 3]

In the correction method shown in Table 3, the DCT class obtained by averaging the DCT class determined by the DCT pattern determining means 141 and the DCT class determined by the reference area parameter determining means 147 is selected.

Next, the DC coefficient extracted by the DC coefficient extracting means 142 is corrected using the DC coefficient obtained by the reference area parameter determining means 147. As an example of this correction method, there is a method of adding the DC coefficient obtained by the reference area parameter determining means 147 to the DC coefficient extracted by the DC coefficient extracting means 142.

Next, the quantization scale extracted by the quantization scale extraction means 143 is corrected using the quantization scale obtained by the reference area parameter determination means 147. As an example of this correction method, the quantization scale extraction means 143
There is a method of using the smaller value of the quantization scale extracted by the reference value and the quantization scale obtained by the reference area parameter determining means 147. There is also a method of using the average value or the larger value of these quantization scales.

When the block noise parameters for one frame's worth of blocks are accumulated, the filter decision means 1
A filter 44 determines a filter to be applied to the boundary of each block based on, for example, Table 1 and Expressions (1) and (2) while referring to the block noise parameter. The filter determining unit 144 is a filter determining unit 1 according to the first embodiment.
34, and the description thereof is omitted. Here, it can be said that the determined type of filter corresponds to the magnitude of the detected block noise.

As described above, in the image decoding apparatus according to the second embodiment, each block is classified into a plurality of DCT classes using DCT coefficients obtained from a compressed code string. Also,
In the case of a non-intra coded block, a reference area in a reference frame is obtained using a motion vector. Thereafter, the DCT class, the DC coefficient, and the quantization scale of the reference area are obtained using the DCT class, the DC coefficient, and the quantization scale of the block that overlaps the reference area, and the DCT class of the processing target block that is obtained from the information of the compression code string; Correct DC coefficient and quantization scale. In addition, DC in adjacent blocks
A filter to be applied to the block boundary is determined using the T class, the quantization scale, and the DC coefficient. Here, when the filter is determined, the strength of the filter is increased as the DCT coefficient equal to or more than the predetermined value is distributed only in the lower band.

As described above, according to the image decoding apparatus of the second embodiment, it is possible to reliably detect a block boundary where block noise occurs from the distribution of DCT coefficients.
At this time, by using a DCT pattern different from the intra-coded block for the non-intra-coded block, classification suitable for the characteristics of the non-intra-coded can be performed. Further, by inspecting the absolute value of the difference between the DC coefficients, erroneous detection of block noise does not occur. In a non-intra coded block, the block noise parameter is determined using the block noise parameter of the reference frame, so that the block noise can be detected with higher accuracy. By preparing a plurality of filters having different intensities and selecting and using the filters according to the magnitude of the block noise, it is possible to surely remove the block noise while minimizing blurring of the image.

In this embodiment, non-intra coded blocks are converted to DCT patterns as shown in FIG.
(D), the DCT coefficient block is divided into three DCT coefficients.
Although the case of classifying into the T class has been described, the number of DCT patterns is not limited to two, and the number of DCT classes is three.
Not just one. The frequency distribution of the DCT pattern is
The distribution is not limited to the distributions shown in FIGS.

The reference area parameter determining means 147
Although several examples of the method of determining the block noise parameter of the reference area in the above are described, the method of determination is not limited to the method described in the present embodiment.

Further, the case where Table 2 or Table 3 is used as a method of correcting the block noise parameter in the parameter correcting means 146 has been described, but the correction method is not limited to the method of Table 2 or Table 3.

The reference area parameter determining means 147
DCT class using the parameter correction means 146 and
Although the method of correcting the DC coefficient and the quantization scale has been described, any of the parameters may not be corrected.

Also, a case has been described where the filter determining means 144 determines the filter when the block noise parameters for one frame are stored in the parameter memory means 118, but this does not have to be the time when the block noise parameters have been stored for one frame. Good.

Further, the case has been described where the DCT pattern determining means 141 uses different DCT patterns for the intra-coded block and the non-intra-coded block, but the same DCT pattern may be used.

(Third Embodiment) In the third embodiment,
An image decoding apparatus that determines the strength of a filter for removing block noise as coding noise based on the magnitude of a motion vector in addition to the DCT coefficient and removes noise will be described.

FIG. 13 is a block diagram of an image decoding apparatus according to the third embodiment. The image decoding device in FIG.
Variable length decoding means 110, inverse quantization means 111, inverse DCT means 112, switches 113 and 114, frame memory 115, block noise removal means 116
, Block noise detecting means 230 and adding means 119
And Variable length decoding means 110, inverse quantization means 111, inverse DCT means 112, switches 113 and 11
4. Frame memory 115, block noise removing means 1
16 and the adding means 119 are the same as those in the first embodiment, so the same numbers are assigned and the description is omitted. Here, the block noise detection means 230 operates as a noise detection device.

FIG. 14 is a block diagram showing the structure of the block noise detecting means 230. 14 includes a DCT pattern determination unit 231 and a block noise detection unit 230.
Motion vector extracting means 232 and filter determining means 23
3 is provided. The block noise detection unit 230 receives a DCT coefficient from the inverse quantization unit 111 and a motion vector from the variable length decoding unit 110.

The DCT pattern determination means 231 receives the DCT coefficient block from the inverse quantization means 111. DC
The T pattern determination means 231 determines whether or not the input DCT coefficient block includes a high frequency component from the distribution of each frequency component of the DCT coefficient block.

FIG. 15 is an explanatory diagram of a DCT pattern used for classifying blocks. DCT pattern determination means 231
Means that the input DCT coefficient block is D
It is determined whether or not the CT pattern is satisfied, and the result is output to the filter determining means 233. About this judgment,
This is the same as that described in the first embodiment.

For the sake of explanation, the DCT pattern determination means 231 outputs a determination result of “Yes” when the DCT coefficient block satisfies the DCT pattern and “No” when the DCT coefficient block does not satisfy the DCT pattern.
Output. Since the DCT pattern is used to determine whether or not the block has a high frequency component, for example, instead of the DCT pattern of FIG. A pattern may be used.

The motion vector extracting means 232 extracts a motion vector from the encoded information output from the variable length decoding means 110, finds the size of the motion vector, and outputs the magnitude to the filter determining means 233. Here, as a method for obtaining the magnitude of the motion vector, a method for obtaining the sum of squares of the horizontal and vertical components of the motion vector, a method for obtaining the sum of absolute values of the horizontal and vertical components of the motion vector, Among the horizontal and vertical components of the vector, there is a method of obtaining the larger absolute value.

The filter determining means 233 determines a filter to be applied to the boundary of each block, using the determination result output from the DCT pattern determining means 231 and the magnitude of the motion vector output from the motion vector extracting means 232. The determination method will be described below.

Assume that pixel blocks are arranged as shown in FIG. Also, the pixel blocks 501, 503, 50
5,506 constitute a pixel macroblock. A case will be considered where what kind of filter is to be applied to the boundary of the pixel macro block.

FIG. 16 is a flowchart showing the flow of processing for classifying blocks. Filter determination means 233
Determines the type of filter according to the procedure shown in FIG. Here, the pixel block 501 and the pixel block 50
The case where the filter to be applied to the block boundary 511 between the two is determined will be described as an example.

First, in step S41 of FIG.
The output of the DCT pattern determination unit 231 for two blocks sandwiching the block boundary, that is, the pixel blocks 501 and 502, is inspected. If the determination result of at least one block is “Yes”, no filter is applied to the block boundary (Step S).
44). Otherwise, the process of step S42 is performed.

In step S42, the magnitude of the motion vector output from the motion vector extracting means 232 is compared with a predetermined threshold value TH1, and if the magnitude of the motion vector is smaller than the threshold value TH1, a weak filter (Step S45). Otherwise, the process of step S43 is performed. In step S43,
The magnitude of the motion vector and a predetermined threshold T
Compare with H2. Here, TH1 <TH2. If the magnitude of the motion vector is smaller than the threshold value TH2, a medium filter is applied (step S4).
6). Otherwise, a strong filter is applied (step S47).

Here, in comparing the magnitudes of the motion vectors in steps S42 and S43, the magnitudes of the motion vectors of both blocks, that is, the pixel blocks 501 and 502, may be used. Only the size may be used. For example, using the magnitudes of the motion vectors of both blocks, if one of the values is larger than the threshold value, the conditions of steps S42 and S43 may be satisfied, or both values may be smaller than the threshold value. If it is larger, the conditions of steps S42 and S43 may be satisfied. Here, the strength of the filter depends on the cutoff frequency of the filter when the filter is a low-pass filter, for example. In this case, it can be said that the filter is stronger as the cutoff frequency is lower.

Similarly, the filter determination means 233
The type of filter to be applied to another macroblock boundary is determined, and the determined type of filter is output to the block noise removing unit 116.

FIG. 17 is a block diagram showing another example of the configuration of the block noise detecting means of FIG. 17 includes a DCT pattern determination unit 231, a motion vector extraction unit 232, a filter determination unit 243, and a DC coefficient extraction unit 244. The DCT pattern determination unit 231 and the DC coefficient extraction unit 244 receive DCT coefficients from the inverse quantization unit 111, and the motion vector extraction unit 232 receives a motion vector from the variable length decoding unit 110.

The DCT pattern determining means 231 and the motion vector extracting means 232 are the same as those described with reference to FIG.

The DC coefficient extraction means 244 extracts only DC coefficients from the DCT coefficient block output from the inverse quantization means 111, and outputs it to the filter determination means 243.

The filter determining means 243 determines the result of the determination output by the DCT pattern determining means 231, the magnitude of the motion vector output by the motion vector extracting means 232,
Using the DC coefficient output from the C coefficient extraction unit 244,
A filter to be applied to the boundary of each block is determined. The operation will be described below.

FIG. 18 is a flowchart showing the flow of processing when classifying blocks using DC coefficients. The filter determining means 243 determines the type of the filter according to the procedure shown in FIG. The difference between the processing method of FIG. 18 and the processing method of FIG.
2 is that the process of step S51 is added. Thus, the processing in step S51 will be described.

In step S51, the absolute value of the difference between the DC coefficients between two adjacent blocks is determined, and it is determined whether the absolute value is greater than a predetermined threshold TH3.
If the absolute value of the difference is larger than the threshold value TH3, it is determined that no filter is applied to the block boundary 511 (step S44). Otherwise, step S4
Step 2 is performed.

The filter determining means 243 determines the type of the filter determined in this manner by the block noise removing means 1.
16 is output.

FIG. 19 is a block diagram showing still another configuration example of the block noise detecting means of FIG. FIG.
The block noise detecting unit 250 includes a DCT pattern determining unit 231, a motion vector extracting unit 232, a filter determining unit 253, a DC coefficient extracting unit 244, and a quantization scale extracting unit 255.

The DCT pattern determining means 231, the motion vector extracting means 232, and the DC coefficient extracting means 244 are the same as those described with reference to FIGS. 14 and 17, and therefore will not be described here.

The quantization scale extracting means 255 extracts a quantization scale from the coded information input from the variable length decoding means 110 and outputs it to the filter determining means 253.

The filter determination means 253 determines the determination result output from the DCT pattern determination means 231, the magnitude of the motion vector output from the motion vector extraction means 232,
Using the DC coefficient output from the C coefficient extraction unit 244 and the quantization scale output from the quantization scale extraction unit 255, a filter to be applied to the boundary of each block is determined. The operation will be described below.

The filter determining means 253 performs almost the same processing as the processing procedure shown in FIG. However, in the filter determining means 253, the threshold value T in step S51 in FIG.
As H3, a value obtained by multiplying the quantization scale output by the quantization scale extraction unit 255 by a constant is used. The filter determining unit 253 outputs the type of the filter determined in this way to the block noise removing unit 116.

The block noise removing means 116 operates almost the same as in the first embodiment. However, the filter having the medium strength corresponds to the filter F2 in the first embodiment, and the filter having the strong strength corresponds to the filter F3.
Is equivalent to If the type of the determined filter is a weak filter, the block noise removing unit 116
Applies a filter to pixels in the same or narrower range as when the filter type is a medium filter. When a low-pass filter is used, the cutoff frequency of a weak filter is set higher than that of a medium filter, as indicated by a frequency characteristic L1 in FIG. Here, it can be said that the determined type of filter corresponds to the magnitude of the detected block noise.

The same applies to the filter in the vertical direction. In the case of an interlaced (interlaced scanning) image, the filter in the vertical direction may be applied in the same field.

As described above, in the image decoding apparatus according to the third embodiment, whether or not each block has a high frequency component is determined based on the frequency distribution of orthogonal transform coefficients such as DCT coefficients obtained from a compressed code string. judge. Then, a filter to be applied to the block boundary is determined using the determination result of the frequency distribution of the orthogonal transform coefficient in the adjacent block, the size of the motion vector, the DC coefficient, and the quantization scale.

Here, the method of determining the filter is as follows. That is, it is determined that no filter is applied to a block having a high frequency component. If the block does not have a high frequency component, the filter strength is set to increase as the size of the motion vector increases. When using the DC coefficient,
If the absolute value of the difference between the DC coefficients in the adjacent blocks is larger than a predetermined value, it is determined that no filtering is performed. In addition, a value corresponding to the quantization scale can be used as the predetermined value at this time. Then, based on the determined filter, block noise is removed by applying a filter to pixels around the block boundary of the decoded image.

As described above, according to the image decoding apparatus of the third embodiment, it is possible to reliably detect a block boundary where block noise occurs from the magnitude of a motion vector of an adjacent block, and to detect a high frequency component. By not applying a filter to the block having the above, it is possible to eliminate erroneous detection and prevent image quality deterioration. Further, by checking the absolute value of the difference between the DC coefficients, it is possible to prevent erroneous detection such as detecting block noise at a block boundary between blocks having a large luminance difference. Also, DC
By using a value corresponding to the quantization scale as the threshold in the test of the absolute value of the coefficient difference, an adaptive test corresponding to the image quality can be performed. Then, by using a plurality of filters having different intensities according to the magnitude of the motion vector, it is possible to perform filtering according to the magnitude of the block noise. With these effects,
Block noise can be removed reliably and without erroneous detection while minimizing image blurring.

In the present embodiment, two DCT patterns shown in FIGS. 15A and 15B have been described as examples of DCT patterns.
It is not limited to these distributions.

Although three types of filters are used in the block noise removing means 116, any number of types may be used.

In this embodiment, the case where two thresholds TH1 and TH2 are used for the magnitude of the motion vector has been described. However, the number of thresholds may be other than two.

The case where the filter used in the block noise removing means 116 is a low-pass filter has been described. However, this is a filter that removes block noise, and may be another filter such as a median filter or a nonlinear filter. There may be.

(Fourth Embodiment) In the fourth embodiment,
In addition to the DCT coefficient of the processing target block and the size of the motion vector, the strength of a filter for removing block noise as coding noise is determined based on parameters related to the reference region of this block, and noise is removed. The following describes an image decoding device that performs this operation.

[0169] The image decoding apparatus according to the fourth embodiment comprises:
In the image decoding apparatus of FIG. 1, block noise detecting means 260 is used instead of block noise detecting means 130. Variable length decoding means 110, inverse quantization means 111, inverse DCT means 112, switches 113 and 11
4. Frame memory 115, block noise removing means 1
16, parameter memory means 118, and adding means 11
9 is the same as that of the first embodiment, and thus the same reference numeral is assigned and the description is omitted. Here, the block noise detection means 260 operates as a noise detection device. The DCT coefficient is input to the block noise detection unit 260 from the inverse quantization unit 111, and the variable length decoding unit 1
10, a motion vector is input.

FIG. 20 is a block diagram showing the structure of the block noise detecting means 260. 20 includes a DCT pattern determination unit 261 and a block noise detection unit 260.
Motion vector extracting means 232 and filter determining means 23
3, a reference area parameter determination unit 266, and a parameter correction unit 267. The DCT coefficient is input to the DCT pattern determination unit 261 from the inverse quantization unit 111, and the variable length decoding unit 110 is input to the motion vector extraction unit 232 and the reference area parameter determination unit 266.
Is input. Further, the DCT pattern determination unit 261 receives encoded information from the variable length decoding unit 110.

Since the motion vector extracting means 232 is the same as that described in the third embodiment with reference to FIG. 14, the description is omitted.

The DCT pattern determination means 261 receives the DCT coefficient block from the inverse quantization means 111. DC
The T pattern determination unit 261 determines whether or not the input DCT coefficient block includes a high frequency component based on the distribution of each frequency component of the DCT coefficient block. This determination method is the same as that described in the third embodiment, and a description thereof will be omitted. The DCT pattern determination unit 261 outputs the determination result to the parameter correction unit 267, and if the frame currently being decoded is a frame that is referred to when decoding another frame (for example, MP
In the case of an I picture or a P picture in EG-2), the determination result is also output to the parameter memory unit 118.

The reference area parameter determination means 266 refers to the DCT pattern determination result of the block including the pixels of the reference area using the motion vector output from the variable length decoding means 110.

As described with reference to FIG. 12 in the second embodiment, the reference area parameter determining means 266 sets the reference area 526 and the reference area 526 in the reference frame to be referred to when decoding the processing target block 521. Are found in blocks 522 to 525 where. The reference area parameter determining means 266 determines the DCT of blocks 522 to 525.
The pattern determination result is obtained from the parameter memory unit 118, the DCT pattern determination result of the reference area 526 is obtained using these DCT pattern determination results, and output to the parameter correction unit 267 and the parameter memory unit 118.

As a method of obtaining the DCT pattern determination result of the reference area 526 using the DCT pattern determination results of the blocks 522 to 525, the DCT pattern determination result of the block having the largest number of pixels in the portion overlapping the reference area 526 is selected. If at least one of the blocks 522 to 525 has a DCT pattern determination result of “Yes”, the determination result of the reference area 526 is “Y”.
es "method, blocks 522-525
And the like by a majority decision of the DCT pattern determination result.

The parameter correction unit 267 receives as input the DCT pattern determination result output by the DCT pattern determination unit 261 and the DCT pattern determination result of the reference region output by the reference region parameter determination unit 266. The parameter correction unit 267 corrects the determination result determined by the DCT pattern determination unit 261 using the determination result determined by the reference area parameter determination unit 266.
In this case, if at least one of the determination results is “Yes”, “Yes” is output to the filter determining unit 233.

The filter determining means 233 receives as input the determination result output from the parameter correcting means 267 and the magnitude of the motion vector output from the motion vector extracting means 232, and determines a filter to be applied to the boundary of each block using these. I do. This determination method is the same as the method described for the filter determination means 233 in FIG. The type of filter determined by the filter determining unit 233 is output to the block noise removing unit 116.

The block noise removing means 116 filters the block boundaries of the image output from the frame memory 115 to remove the block noise, as in the third embodiment. The block noise removing means 116
An image from which noise has been removed is output as an output image.

FIG. 21 is a block diagram showing another example of the configuration of the block noise detecting means of FIG. The block noise detecting means 270 in FIG. 21 includes a DCT pattern determining means 261, a motion vector extracting means 232, a filter determining means 243, a DC coefficient extracting means 244, a reference area parameter determining means 276, and a parameter correcting means 27.
7 is provided. The DCT coefficient is input to the DCT pattern determination unit 261 and the DC coefficient extraction unit 244 from the inverse quantization unit 111, and the motion vector extraction unit 232 and the reference area parameter determination unit 276 are supplied with the motion vector from the variable length decoding unit 110. Is entered. Further, the DCT pattern determination unit 261 receives encoded information from the variable length decoding unit 110.

The DCT pattern determining means 261 and the motion vector extracting means 232 are the same as those described with reference to FIG. 20, and the DC coefficient extracting means 244 is the same as that described with reference to FIG. Is omitted.

The outputs of the DCT pattern determining means 261 and the DC coefficient extracting means 244 are output to the parameter correcting means 277.
Is input to If the frame currently being decoded is a frame referred to when decoding another frame, the outputs of the DCT pattern determination unit 261 and the DC coefficient extraction unit 244 are also output to the parameter memory unit 118. Is done. The output of the motion vector extracting means 232 is output to the filter determining means 243.

The reference area parameter determining means 276 uses the motion vector output from the variable length decoding means 110 to read the reference frame D from the parameter memory means 118.
Acquire the determination result of the CT pattern and the DC coefficient. The method of obtaining these and the method of obtaining the DCT pattern determination result of the reference region are the same as the method described for the reference region parameter determination unit 266 in FIG. Also,
As a method of obtaining the DC coefficient of the reference area, there is a method of obtaining an average value of DC coefficients of a block including pixels of the reference area, a method of obtaining a weighted average value by an area of an overlapping portion of a block overlapping the reference area, and the like. .

The reference area parameter determination means 276 outputs the DCT pattern determination result and DC coefficient of the reference area obtained as described above to the parameter correction means 277 and the parameter memory means 118.

The parameter correction means 277 calculates the DCT pattern determination result output from the DCT pattern determination means 261, the DC coefficient output from the DC coefficient extraction means 244,
The DCT pattern determination result and the DC coefficient of the reference region output from the reference region parameter determination unit 276 are received as inputs. First, the parameter correction means 277
Corrects the determination result determined by the DCT pattern determination unit 261 using the determination result determined by the reference area parameter determination unit 276. This method is the same as the method described for the parameter correction unit 267 in FIG.

Next, the parameter correction means 277
The DC coefficient output from the coefficient extracting unit 244 is corrected using the DC coefficient output from the reference area parameter determining unit 276. This correction is performed by adding the DC coefficient output from the reference area parameter determining means 276 to the DC coefficient output from the DC coefficient extracting means 244. The DCT pattern determination result and the DC coefficient corrected by the parameter correction unit 277 are output to the filter determination unit 243.

The filter determining means 243 determines a filter to be applied to the boundary of each block using the determination result and the DC coefficient output from the parameter correcting means 277 and the magnitude of the motion vector output from the motion vector extracting means 232. . This determination method is the same as the method described for the filter determination means 243 in FIG. The type of filter determined by the filter determining unit 243 is output to the block noise removing unit 116.

FIG. 22 is a block diagram showing still another example of the configuration of the block noise detecting means of FIG. FIG.
The block noise detecting means 280 includes a DCT pattern determining means 261, a motion vector extracting means 232, a filter determining means 253, a DC coefficient extracting means 244, a quantization scale extracting means 255, a reference area parameter determining means 286, , Parameter correction means 287. The DCT coefficient is input to the DCT pattern determination unit 261 and the DC coefficient extraction unit 244 from the inverse quantization unit 111, and the variable length decoding unit 110 is input to the motion vector extraction unit 232 and the reference area parameter determination unit 286.
Is input. Further, the quantization scale is input from the variable length decoding unit 110 to the quantization scale extraction unit 255, and the DCT pattern determination unit 2
The encoded information is input to 61.

The DCT pattern determination means 261 and the motion vector extraction means 232 are the same as those described in FIG. 20, and the DC coefficient extraction means 244 is the same as that described in FIG. 255 is
Since this is the same as that described with reference to FIG. 19, the description thereof is omitted here.

The outputs of the DCT pattern determining means 261, the DC coefficient extracting means 244 and the quantization scale extracting means 255 are input to the parameter correcting means 287. Also,
If the frame currently being decoded is a frame that is referred to when decoding another frame, the outputs of the DCT pattern determination unit 261, the DC coefficient extraction unit 244, and the quantization scale extraction unit 255 are stored in the parameter memory unit 11
8 is also output. The output of the motion vector extracting unit 232 is output to the filter determining unit 253.

The reference area parameter determining means 286 uses the motion vector output from the variable length decoding means 110 to read the reference frame D from the parameter memory means 118.
The determination result of the CT pattern, the DC coefficient, and the quantization scale are obtained. The method of obtaining these, the method of determining the DCT pattern determination result of the reference region, and the method of determining the DC coefficient of the reference region are described in reference region parameter determination unit 2 in FIG.
This is the same as the method described with reference to FIG. As a method of obtaining the quantization scale of the reference region, a method of obtaining an average value of the quantization scale of a block including pixels of the reference region, and a method of obtaining a weighted average value by an area of an overlapping portion of a block overlapping the reference region There are a method, a method of taking the minimum value, a method of taking the maximum value, and the like.

The reference area parameter determining means 286 outputs the DCT pattern determination result of the reference area, the DC coefficient and the quantization scale obtained as described above to the parameter correcting means 287 and the parameter memory means 118.

The parameter correction means 287 calculates the DCT pattern judgment result output from the DCT pattern judgment means 261 and the DC coefficient output from the DC coefficient extraction means 244,
It receives as input the quantization scale output from the quantization scale extraction unit 255 and the DCT pattern determination result, DC coefficient, and quantization scale of the reference region output from the reference region parameter determination unit 286. First, the parameter correction unit 287 includes the DCT pattern determination unit 261
The determination result determined by the reference area parameter determination means 2
Correction is performed using the determination result determined by 86. This method is the same as the method described for the parameter correction unit 267 in FIG.

Next, the parameter correction means 287
The DC coefficient output from the coefficient extracting unit 244 is corrected using the DC coefficient output from the reference area parameter determining unit 286. This correction method is the same as the method described for the parameter correction unit 277 in FIG.

Subsequently, the parameter correction means 287 corrects the quantization scale output from the quantization scale extraction means 255 using the quantization scale output from the reference area parameter determination means 286. This correction method includes:
There is a method of calculating an average value of the quantization scale output from the quantization scale extraction unit 255 and the quantization scale output from the reference area parameter determination unit 286. In addition, there is a method of obtaining the minimum value or the maximum value of these two quantization scales. Also, the quantization scale extracting means 2
The quantization scale output by the output 55 may be output without correction. The determination result, DC coefficient, and quantization scale of the DCT pattern corrected by the parameter correction unit 287 are:
It is output to the filter determining means 253.

The filter determining means 253 determines the DCT pattern output from the parameter correcting
Using the C coefficient, the quantization scale, and the magnitude of the motion vector output by the motion vector extraction unit 232,
A filter to be applied to the boundary of each block is determined. This determination method is the same as the method described for the filter determination means 253 in FIG. The type of the filter determined by the filter determining unit 253 is determined by the block noise removing unit 11.
6 is output. Here, it can be said that the determined type of filter corresponds to the magnitude of the detected block noise.

As described above, the image decoding apparatus according to the present embodiment determines whether each block has a high frequency component based on the frequency distribution of orthogonal transform coefficients (for example, DCT coefficients) obtained from a compressed code string. I do. Further, a reference region in the reference frame is obtained using the motion vector, and a frequency distribution, a DC coefficient, and a quantization scale of the reference region are obtained using the frequency distribution, the DC coefficient, and the quantization scale of the block overlapping the reference region. Then, using these values, the frequency distribution, DC coefficient, and quantization scale of the processing target block obtained from the information of the bit stream are corrected. Then, a filter to be applied to the block boundary is determined using the determination result of the frequency distribution of the orthogonal transform coefficient in the adjacent block, the magnitude of the motion vector, the DC coefficient, and the quantization scale.

Here, the method of determining the filter is as follows. That is, if the orthogonal transform coefficient distribution is a block having a high frequency component, it is determined that no filter is applied. If the orthogonal transform coefficient distribution is a block having no high-frequency component, the filter strength is set to increase as the size of the motion vector increases. When the DC coefficient is used, if the absolute value of the difference between the DC coefficients in the adjacent blocks is larger than a predetermined value, it is determined that no filtering is performed. In addition, a quantization scale can be used as the predetermined value at this time. Then, using the filter determined as described above, block noise is removed by applying a filter to pixels around the block boundary of the decoded image.

With such an operation, according to the image decoding apparatus of the present embodiment, it is possible to reliably detect a block boundary where block noise occurs from the magnitude of a motion vector of an adjacent block, and to detect a high frequency component. By not applying a filter to the block having the above, it is possible to eliminate erroneous detection and prevent image quality deterioration. Further, by inspecting the absolute value of the difference between the DC coefficients, erroneous detection can be further prevented. By using a quantization scale as a threshold at that time, an inspection corresponding to image quality can be performed. In addition, when motion compensation is performed, the filter is determined using the DCT pattern determination result of the reference frame, the DC coefficient, and the quantization scale, so that the characteristics of the decoded image after motion compensation are more accurately determined. Block noise can be detected. Then, by using a plurality of filters having different intensities according to the magnitude of the motion vector, it is possible to reliably remove the block noise while minimizing the blur of the image.

(Fifth Embodiment) In the fifth embodiment,
An image decoding device that detects and removes block noise as encoding noise when decoding an intra-coded frame from a compressed code string in which an interlaced image is encoded will be described.

[0200] The image decoding apparatus according to the fifth embodiment comprises:
In the image decoding apparatus of FIG. 1, a block noise detecting means 330 is used instead of the block noise detecting means 130. Variable length decoding means 110, inverse quantization means 111, inverse DCT means 112, switches 113 and 11
4. Frame memory 115, parameter memory means 11
8 and the addition means 119 are the same as those in the first embodiment, and therefore, are denoted by the same reference numerals and description thereof is omitted. Here, the block noise detection means 330 operates as a noise detection device.

FIG. 23 is a block diagram showing the configuration of the block noise detecting means according to the fifth embodiment. 23 includes a DCT pattern determination unit 331, a DC coefficient extraction unit 332, a quantization scale extraction unit 333, and a filter determination unit 334. The block noise detection means 330 includes a DCT
The coefficients are input from the inverse quantization means 111, and the quantization scale, macroblock coding mode, and DCT mode are input from the variable length decoding means 110.

The DCT pattern determination means 331 receives the DCT coefficient block from the inverse quantization means 111 and the DCT mode from the variable length decoding means 110 as an input. D
The CT pattern determination unit 331 classifies each DCT coefficient block from the distribution of each frequency component of the DCT coefficient block. The DCT coefficient blocks are classified in the same manner as in the first embodiment, and each DCT coefficient block is classified into DCT classes I1, I2, and I3.

The DCT class determined here is assigned to each block on a field basis. Here, for simplification of the description, the description will be made using only the luminance signal block.

FIG. 24 is an explanatory diagram showing the relationship between DCT coefficient macroblocks and pixel blocks represented by fields when the DCT mode is the field mode. FIG. 24A is an explanatory diagram showing DCT coefficient blocks, and it is assumed that DCT coefficient blocks 601 to 604 form a DCT coefficient macroblock. FIG.
Since the DCT mode is the field mode, the DCT coefficient macro block of FIG.
01 and 603 indicate DCT coefficients of the first field,
CT coefficient blocks 602 and 604 correspond to the second field D
The CT coefficient is shown. Therefore, D in FIG.
FIG. 24B shows the state of the pixel macroblock obtained by performing the inverse DCT on the CT coefficient macroblock.

FIG. 24B is an explanatory diagram showing a pixel macro block in a frame structure.
8 shows a pixel macro block composed of a pixel macro block 08.
For example, the pixel block 605 has a first field of DC
The second field is composed of pixels obtained by inverse DCT of the DCT block 602.

FIG. 24C shows pixel blocks 605 to 60.
FIG. 8 is an explanatory diagram showing 8 for each field. Hereinafter, among the pixels included in one pixel block, a group of pixels (8 × 4 pixels) in one field is defined as one unit and is referred to as a field block. For example, the field blocks 609 and 610 are obtained by collecting pixels of the first field and the second field of the pixel block 605, respectively. Similarly, the field blocks 611 and 612 are composed of the pixels of the first field and the second field of the pixel block 606, respectively. Therefore, the DCT class determined for the DCT block 601 is assigned to the field blocks 609 and 611. Similarly, the DCT class determined for DCT block 602 is the DCT class determined for DCT block 603 for field blocks 610 and 612.
The class is for the field blocks 613 and 615
The DCT class determined for the DCT block 604 is assigned to the field blocks 614, 616.

FIG. 25 is an explanatory diagram showing the relationship between DCT coefficient macroblocks and pixel blocks represented by fields when the DCT mode is the frame mode. FIG. 25A is an explanatory diagram showing DCT coefficient blocks.
It is assumed that a CT coefficient macro block is configured. FIG.
Since the DCT mode of the DCT coefficient macroblock in (a) is the frame mode, the state of the pixel macroblock obtained by applying the inverse DCT to the DCT coefficient macroblock in FIG. 25A is as shown in FIG. .

FIG. 25B is an explanatory diagram showing a pixel macro block in a frame structure.
The figure shows a pixel macroblock consisting of 58.
For example, the pixel block 655 includes a DCT block 651
Of the pixel block 656
Are composed of pixels obtained by inverse DCT of the DCT block 652.

FIG. 25C shows pixel blocks 655 to 65.
FIG. 8 is an explanatory diagram showing 8 for each field. For example, the field blocks 659 and 660 are obtained by collecting pixels of the first field and the second field of the pixel block 655, respectively. Similarly, field blocks 661,
Reference numeral 662 includes pixels of the first field and the second field of the pixel block 656, respectively. Therefore, the DCT class determined for DCT block 651 is assigned to field blocks 659 and 660. Similarly, the DCT class determined for DCT block 652 is the field block 661,
662, the DCT class determined for the DCT block 653 is the field block 663, 664.
, The D determined for the DCT block 654
CT classes are assigned to field blocks 665 and 666.

The DC coefficient extraction means 332 receives the DCT coefficient block and the DCT mode as inputs, and
Only the C coefficient is extracted and output to the parameter memory means 118. The DC coefficient is assigned to each field block, similarly to the DCT pattern determination unit 331.

[0211] The quantization scale extracting means 333 extracts the quantization scale from the encoded information output from the variable length decoding means 110, and outputs it to the parameter memory means 118. The quantization scale is determined by the DCT pattern determination unit 331.
In the same manner as in the above, it is assigned for each field block.

The block noise parameter, ie, D
DCT class output by the CT pattern determination means 331,
The DC coefficient output from the DC coefficient extraction unit 332 and the quantization scale output from the quantization scale extraction unit 333 are stored in the parameter memory unit 1 in field block units.
18 is output. Parameter memory means 118
Accumulates the input block noise parameter in field block units.

After accumulating the block noise parameters for the field blocks for one frame, the filter determining means 334 determines a filter to be applied to the boundary of each field block with reference to the block noise parameters. The operation will be described below.

FIG. 26 is an explanatory diagram of the arrangement of field blocks. Now, it is assumed that the blocks are arranged as shown in FIG. FIG. 26A shows an arrangement of blocks in a frame structure, and one square indicates one block. FIGS. 26B and 26C show the arrangement of the field blocks of the first field and the second field in FIG. 26A, respectively. For example, the block 551 is divided into a field block 551a composed of pixels of the first field and a field block 551b composed of pixels of the second field.

The filter to be applied to the boundary is determined on a field block basis. For example, the field block 551a
For the boundaries 561a, 562a, 563a, 56
4a is determined and the field block 551b is determined.
For the boundaries 561b, 562b, 563b, 56
4b is determined. The method for determining the filter is the same as the method described in the first embodiment. For example, when determining the filter of the boundary 561a, the block noise parameters of the field blocks 551a and 552a are used. The type of filter determined by the filter determining unit 334 is output to the block noise removing unit 116. Here, it can be said that the determined type of filter corresponds to the magnitude of the detected block noise.

[0216] The block noise removing means 116 receives the type of filter to be applied to the boundary of each block from the block noise detecting means 330. And the frame memory 115
Is applied to the block boundaries of the image output from.
The horizontal filtering process is the same as in the first embodiment, and a description thereof will not be repeated. An example of the vertical filtering performed by the block noise removing unit 116 will be described below.

FIG. 27 is an explanatory diagram of the filtering process in the vertical direction for the boundary between blocks. FIG.
FIG. 26 shows pixels of blocks 551 and 555 in FIG. 26, and one square corresponds to one pixel. Now, consider a case where a filter is applied to the boundary 564 between the blocks 551 and 555. Assuming that the type of filter to be applied to the boundary 564 is determined to be F1 by the block noise detecting unit 330, the block noise removing unit 116 does not apply a filter to the boundary 564. Block noise detecting means 330
If the type of filter to be applied to the boundary 564 is determined to be F2, the block noise removing unit 116 applies a filter to pixels near the boundary 564.

For example, in FIG.
Explaining using the pixels in the fifth and fifth vertical lines, a filter is applied to the pixels c and d and the pixels c ′ and d ′. Here, the pixels of the first field (pixels c and d) are filtered using only the pixels of the first field, and the pixels of the second field (pixels c ′ and d ′) are filtered. Filtering is performed using only the pixels in the field.
Also, a low-pass filter or the like can be used as the filter.

By applying a filter in this way,
The pixel values shown in FIG. 27B become as shown in FIG. 27C, and the block noise is removed. Here, FIG.
In (b) and (c), the horizontal axis represents the pixel value, and the vertical axis represents the pixel position in the vertical direction. The pixel positions here correspond to the pixel positions in FIG.

If the type of filter to be applied to the boundary 564 is determined to be F3 by the block noise detecting means 330, the block noise removing means 116
In the vicinity of 64, the filter is applied to a wider range of pixels than when the type of filter is F2. The description will be made using the pixels of the fourth and fifth vertical lines from the left.
For example, for pixels in the first field, pixels b, c,
The filter is applied to d and e, and the pixels b ', c', d 'and e' are applied to the pixels in the second field. As the filter, a low-pass filter or the like can be used as in the case of F2. Further, as shown in the frequency characteristic example of the filter in FIG. 9, when the type of the filter is F3 (L
3) has a lower cutoff frequency than the case of F2 (L2), and accordingly has a greater ability to remove block noise.
By applying the filter in this way, FIG.
The pixel value indicated by becomes as shown in FIG. 27E, and the block noise is removed.

Here, an example of a method of determining the number of taps of a filter when a one-dimensional filter is used as a filter will be described. When a one-dimensional filter is used, both pixels of two field blocks adjacent to each other with a boundary therebetween are used for the filtering process, and pixels other than these two field blocks are not used. This is shown in FIG.
This will be described with reference to FIG.

For example, assuming that the tap length of the filter is 5, to apply the filter to the pixel c, the pixels a, b,
The value of the pixel c is obtained using c, d, and e. These pixels straddle blocks 551 and 555, and pixels in blocks other than blocks 551 and 555 are not used. However, in the case where a filter is to be applied to the pixel a, since the pixels of the block 555 are not used, a filter having a tap length of 5 cannot be applied to the pixel a. That is, when a filter having a tap length of 5 is used, the filters can be applied to the pixels b, c, d, and e among the pixels a to f. According to the same concept, when a filter having a tap length of 7 is used, a filter can be applied to the pixels c and d among the pixels a to f.

[0223] The frame filtered by the block noise removing means 116 as described above is output as an output image.

As described above, in the image decoding apparatus according to the present embodiment, each block is classified into a plurality of DCT classes using the DCT coefficients obtained from the compressed code string. This DCT
Classification is performed in units of field blocks obtained by dividing each block into fields. Then, a filter to be applied to the field block boundary is determined using the DCT class, quantization scale, and DC coefficient between adjacent field blocks. Here, when the filter is determined, the strength of the filter is increased as the DCT coefficient equal to or more than the predetermined value is distributed only in the lower band.

As described above, according to the image decoding apparatus of this embodiment, it is possible to reliably detect a block boundary where block noise occurs from the distribution of DCT coefficients, and
Inspection of the absolute value of the difference between the DC coefficients prevents erroneous detection of block noise. Then, by using a plurality of filters having different intensities according to the magnitude of the block noise, it is possible to reliably remove the block noise while minimizing blurring of the image. Also,
Since block noise is detected in units of field blocks, block noise can be detected more accurately for interlaced images. In addition, since block noise is removed in units of field blocks, block noise can be removed without affecting the image. When a filter is used to remove block noise, both pixels of two field blocks adjacent to each other across a boundary are used for filtering, and pixels other than these two field blocks are not used. Thus, the ability to remove block noise can be improved.

In this embodiment, when the type of filter is F2, the filter is applied to two pixels at the block boundary, and when the type of filter is F3, the filter is applied to four pixels at the block boundary. Was explained,
The range of pixels to be filtered may be a range different from that of the present embodiment.

In this embodiment, the case where the range in which the filter F2 is applied is different from the range in which the filter F3 is applied is described. However, the range may be the same.

Also, in the present embodiment, a case has been described where the filter determining means 334 determines a filter when the block noise parameters for one frame are stored in the parameter memory means 118. It does not have to be at the time.

(Sixth Embodiment) In the sixth embodiment,
An image decoding device that detects and removes block noise as coding noise when decoding a non-intra-coded frame from a compressed code string in which an interlaced image is coded will be described.

The image decoding apparatus according to the sixth embodiment comprises:
In the image decoding apparatus of FIG. 1, block noise detecting means 340 is used instead of block noise detecting means 130. Variable length decoding means 110, inverse quantization means 111, inverse DCT means 112, switches 113 and 11
4. Frame memory 115, block noise removing means 1
16, parameter memory means 118, and adding means 11
9 is the same as that of the first embodiment, and thus the same reference numeral is assigned and the description is omitted. Here, the block noise detection means 340 operates as a noise detection device.

FIG. 28 is a block diagram showing the configuration of the block noise detecting means according to the sixth embodiment. 28 includes a DCT pattern determination unit 341, a DC coefficient extraction unit 342, a quantization scale extraction unit 343, a filter determination unit 344, a switch 345, a parameter correction unit 346, and a reference area. And a parameter determining unit 347. The DCT coefficient is input to the block noise detection unit 340 from the inverse quantization unit 111, and the quantization scale, the macroblock coding mode, the motion vector, and the DCT mode are input from the variable length decoding unit 110. It is also assumed that the parameter memory unit 118 holds the block noise parameter for the already decoded frame obtained by the method described in the fifth embodiment.

The DCT pattern determination means 341
It receives the coefficient block, the DCT mode, and the macroblock coding mode, classifies each block into one of the DCT classes from the distribution of each frequency component of the DCT coefficient block, and outputs the classification result to the switch 345.

As described in the first and second embodiments, the blocks are classified into DCs having a frequency distribution as shown in FIG.
This is performed using a T pattern. DCT coefficient blocks are classified in the same manner as in the first embodiment in the case of intra-coded blocks, and the blocks are classified into DCT classes I1, I2, and I3. In the case of a non-intra coded block, the same method as in the second embodiment is used, and each block is classified into DCT classes N1, N2, and N3. The DCT class determined here is assigned to each field block by the same method as that described in the fifth embodiment.

A DCT coefficient block and a DCT mode are input to the DC coefficient extraction means 342. DC coefficient extracting means 342 extracts only DC coefficients from DCT coefficients,
Output to the switch 345. DC coefficient extracting means 342
Assigns the extracted DC coefficient to each field block, similarly to the DCT pattern determination unit 341.

The quantization scale extraction means 343 extracts a quantization scale from the encoded information output from the variable length decoding means 110, and outputs it to the switch 345. The quantization scale extracting unit 343 is a DCT pattern determining unit 341.
Similarly to the above, the extracted quantization scale is assigned to each field block.

The switch 345 is connected to the variable length decoding unit 11
The switch is switched using the macroblock coding mode output by 0. When the macroblock coding mode is the intra coding, the switch 345 is connected to b. The operation in this case is the same as in the fifth embodiment. When the macroblock coding mode is non-intra coding, the switch 345 is connected to a. Therefore, the block noise parameters (DCT class, D
C coefficient and quantization scale)
6 is input.

The reference area parameter determining means 347 refers to the block noise parameter of the reference block using the motion vector output from the variable length decoding means 110, and obtains the block noise parameter of the reference area. The detailed operation of the reference area parameter determining means 347 will be described below.

FIG. 29 is an explanatory diagram of a method of obtaining a block noise parameter of a reference area. The reference area parameter determining unit 347 uses the motion vector output from the variable length
Then, a block noise parameter of the reference frame is obtained from 18. The block noise parameter of the reference frame is obtained on a field block basis.

FIG. 29A shows a frame currently being decoded. Here, attention is paid to block 701. A field block composed of the pixels of the first field of the block 701 is denoted by 701a. FIG.
(B) to (f) are frames referred to when decoding FIG. 29A, respectively, and are blocks 702 to 705.
Indicates a block in the reference frame. Below,
A method of obtaining a block noise parameter of a reference frame for the field block 701a will be described.

FIG. 29B is a diagram for explaining a first example when the motion vector is in the frame mode. Now, the address in the frame of the block 701 is set to (x,
y), assuming that the motion vector is (MVx, MVy), the block 701 refers to the reference area 706. Here, assuming that the area of the first field of the reference area 706 is the area 706a, the area referenced by the field block 701a is the area 706a. The address of the reference area 706 is (x + MVx, y + MVy). The reference area parameter determination unit 347 finds a block where the area 706a overlaps from the address of the reference area 706. FIG.
In (b), the area 706a includes blocks 702 to 705.
Overlaps the first field. So block 7
Among the block noise parameters of the field block composed of the pixels of the first field from 02 to 705,
The DCT class is obtained from the parameter memory unit 118.

FIG. 29C is a view for explaining a second example in the case where the motion vector is in the frame mode. The area referred to by the block 701 is referred to as a reference area 707. Here, the area of the first field of the reference area 707 is set to the area 7
If 07a is set, the area referred to by the field block 701a is the area 707a. The reference area parameter determining means 347 calculates the area 70 from the address of the reference area 707.
Find the block where 7a overlaps. FIG. 29 (c)
Then, the second fields of the blocks 702 to 705 overlap. Therefore, the DCT class is obtained from the parameter memory unit 118 among the block noise parameters of the field blocks composed of the pixels of the second field of the blocks 702 to 705.

FIG. 29D is a view for explaining a third example in the case where the motion vector is in the frame mode. The area referred to by the block 701 is referred to as a reference area 708. Here, the area of the first field of the reference area 708 is set to the area 7
If 08a is set, the area referenced by the field block 701a is the area 708a. The reference area parameter determining means 347 calculates the area 70 from the address of the reference area 708.
Find the block where 8a overlaps. FIG. 29 (d)
In the example, the vertical motion vector has a size of a half pixel unit, and the average value of the pixels in the first field and the second field of the reference area 708 is the reference pixel value.
Therefore, the reference area parameter determining unit 347 stores the DCT class in the parameter memory of the block noise parameters of the field blocks composed of the pixels of the first field and the field blocks composed of the pixels of the second field in the blocks 702 to 705. Means 118
To get from.

FIG. 29E is a diagram for explaining a first example in the case where the motion vector is in the field mode. Here, the case where the field of the reference destination is the first field will be described. The area referred to by the block 701 is referred to as a reference area 709. Here, assuming that the area of the first field of the reference area 709 is the area 709a, the area referenced by the field block 701a is the area 709a. The reference area parameter determination unit 347 determines whether the reference area 7
From the address 09, a block where the area 709a overlaps is found. Since the reference destination field is the first field, the reference area parameter determining means 347
Stores the DCT class among the block noise parameters of the field blocks composed of the pixels of the first field of the blocks 702 to 705 in the parameter memory means 1.
Obtained from 18.

FIG. 29F is a diagram for explaining a second example in the case where the motion vector is in the field mode. Here, a case where the reference destination field is the second field will be described. The area referred to by the block 701 is referred to as a reference area 710. Here, assuming that the area of the first field of the reference area 710 is the area 710a, the area referenced by the field block 701a is the area 710a. The reference area parameter determination unit 347 determines whether the reference area 7
From the ten addresses, a block where the area 710a overlaps is found. Since the reference destination field is the second field, the reference area parameter determining means 347
Stores the DCT class among the block noise parameters of the field blocks composed of the pixels of the second field of the blocks 702 to 705 in the parameter memory means 1.
Obtained from 18.

As described above, the parameter memory means 11
8 using the DCT class obtained from
The method for obtaining the class is the same as the method described in the second embodiment, and a description thereof will not be repeated.

The reference area parameter determining means 347 outputs the DCT class of the reference area obtained for each field block as described above to the parameter correcting means 346.

The parameter correction means 346 is
45, a DCT pattern determination unit 341, a DC coefficient extraction unit 342, and a quantization scale extraction unit 343
And the DCT of the reference area output by the reference area parameter determination means 347.
Take a class as input. Then, first, the DCT class for each field block determined by the DCT pattern determination unit 341 is corrected using the DCT class determined by the reference area parameter determination unit 347. This correction method is the same as that described in the second embodiment, and a description thereof will not be repeated. However, the DCT class is corrected on a field block basis.

The parameter correcting means 346 determines the DCT class determined as described above and the DC coefficient extracting means 34.
2 and the quantized scale extracting means 34
3 is output to the parameter memory means 118 as a block noise parameter. The parameter memory means 118 stores the input block noise parameters in field block units.

When the block noise parameters for the block for one frame are accumulated, the filter decision means 3
Reference numeral 44 determines a filter to be applied to the boundary of each field block with reference to the block noise parameter. The operation at this time is the same as the operation of the filter determining unit 334 in the fifth embodiment. Here, it can be said that the determined type of filter corresponds to the magnitude of the detected block noise.

The block noise removing means 116 receives the type of filter applied to the boundary of each field block from the block noise detecting means 340. Then, a filter is applied to the field block boundary of the image output from the frame memory 115. The operation of the block noise removing means 116 is the same as that described in the fifth embodiment, and a description thereof will be omitted. Also in the present embodiment, when a filter is used in the block noise removing unit 116, the method for determining the tap length of the filter described in the fifth embodiment can be applied.

As described above, the image decoding apparatus according to the present embodiment classifies each block into a plurality of DCT classes using the DCT coefficients obtained from the compressed code string. The classification into the DCT class is performed in units of field blocks obtained by dividing each block into fields. In the case of a non-intra coded block, a reference area in a reference frame is obtained using a motion vector. Thereafter, the DCT class, DC coefficient, and quantization scale of the reference area are obtained using the DCT class, DC coefficient, and quantization scale of the block overlapping the reference area, and the processing target block obtained using information obtained from the bit stream is obtained. Correct DCT class, DC coefficient and quantization scale. Then, a filter to be applied to the block boundary is determined using the DCT class, quantization scale, and DC coefficient of the adjacent block. Here, when the filter is determined, the strength of the filter is increased as the DCT coefficient equal to or more than the predetermined value is distributed only in the lower band.

According to such an operation, the image decoding apparatus of the present embodiment can reliably detect a block boundary where block noise occurs from the distribution of DCT coefficients. In this case, by using a DCT pattern different from the intra-coded block for the non-intra-coded block, classification suitable for the characteristics of the non-intra-coded can be performed. Further, by inspecting the absolute value of the difference between the DC coefficients, erroneous detection of block noise does not occur. Further, in a non-intra coded block, the block noise parameter is determined using the block noise parameter of the reference frame, so that the block noise can be detected with higher accuracy. Then, by using a plurality of filters having different intensities according to the magnitude of the block noise, it is possible to reliably remove the block noise while minimizing blurring of the image. In addition, since block noise is detected in units of field blocks, block noise can be detected more accurately for interlaced images. In addition, since block noise is removed in units of field blocks, block noise can be removed without affecting the image.

In the present embodiment, a case has been described where the filter determining means 344 determines a filter when the block noise parameters for one frame are stored in the parameter memory means 118. It does not have to be at the time.

(Seventh Embodiment) In the seventh embodiment,
An image decoding device that detects and removes mosquito noise as encoding noise will be described.

FIG. 30 is a block diagram of an image decoding apparatus according to the seventh embodiment. The image decoding device in FIG.
Variable length decoding means 110, inverse quantization means 111, inverse DCT means 112, switches 113 and 114, frame memory 115, mosquito noise removal means 126
Mosquito noise detecting means 430, parameter memory means 118, and adding means 119. Variable length decoding means 110, inverse quantization means 111, inverse DCT
Means 112, switches 113 and 114, frame memory 115, parameter memory means 118, and addition means 1
19 is the same as that of the first embodiment, and thus the same reference numeral is assigned and the description is omitted. Here, the mosquito noise detection means 430 operates as a noise detection device.

FIG. 31 shows the mosquito noise detecting means 430.
FIG. 3 is a block diagram showing the configuration of FIG. The mosquito noise detection means 430 of FIG.
Quantization scale extraction means 433, filter determination means 434, switch 435, parameter correction means 4
36 and a reference area parameter determining means 437. The DCT coefficient is input to the block noise detection unit 430 from the inverse quantization unit 111, and the quantization scale,
The macroblock coding mode and the motion vector are input from the variable length decoding unit 110.

First, the operation when the input block is an intra-coded block will be described.

The DCT pattern determination means 431 receives the DCT coefficient block from the inverse quantization means 111. DC
The T pattern determination unit 431 classifies each block into one of m (m is a natural number) classes {M1, M2,..., Mm} from the distribution of each frequency component of the DCT coefficient block. Here, it is assumed that class M1 is a class in which mosquito noise is least likely to occur, and class Mm is a class in which mosquito noise is most likely to occur.

FIG. 32 is an explanatory diagram of a DCT pattern used for classifying blocks. FIGS. 32A to 32D respectively show patterns of 8 × 8 DCT coefficient blocks, and one square corresponds to one coefficient. The upper left coefficient is D
The C coefficient is shown, and the right one shows a high horizontal frequency component, and the lower one shows a high vertical frequency component.

The DCT pattern determining means 431 determines whether or not the input DCT coefficient block satisfies the DCT pattern, similarly to the DCT pattern determining means 131 described in the first embodiment.

If the input block is an intra-coded block determined from the macroblock coding mode, the DCT pattern determination means 431 determines whether the input block is a DCT pattern PTN11 in FIG. 32 (a) or a DCT pattern in FIG. The pattern PTN12 is used. Here, since mosquito noise is likely to occur when the DCT coefficient has a higher frequency band, it can be said that a block satisfying the DCT pattern PTN12 is more likely to generate mosquito noise than a block satisfying the DCT pattern PTN11. .

FIG. 33 is a flowchart showing the flow of processing for classifying blocks. FIG. 33 shows a case where each block is classified into one of three classes {M1, M2, M3} using the DCT patterns PTN11 and PTN12. As shown in FIG. 33, the input DCT coefficient block first receives the DC
This is compared with the T pattern PTN11. If the input DCT coefficient block does not satisfy the DCT pattern PTN11, that is, if all the absolute values of the coefficients in the hatched portions in FIG. 32A are equal to or smaller than a predetermined value, the block is classified into the DCT class M1 (step S73). ). If the input DCT coefficient block satisfies the DCT pattern PTN11, then in step S72, the DCT coefficient block is compared with the DCT pattern PTN12. Input DC
If the T coefficient block does not satisfy the DCT pattern PTN12, the block is classified into the DCT class M2 (step S74). Input DCT coefficient block is DCT
If the pattern PTN12 is satisfied, the block is classified into the DCT class M3 (step S75). As described above, the DCT pattern determination unit 431 classifies each block into one of the classes M1, M2, and M3, and outputs the classification result.

The quantization scale extracting means 433 extracts a quantization scale from the encoded information output from the variable length decoding means 110 and outputs it.

Here, the DCT pattern determining means 431
And the quantization scale output by the quantization scale extraction unit 433 are collectively referred to as noise parameters. When the macroblock coding mode is the intra coding, the switch 435 is connected to b. Therefore, the noise parameter is output to the parameter memory means 118. The parameter memory means 118 stores the input noise parameters.

By performing the above operation for each macro block, noise parameters for each block are stored in the parameter memory means 118.

Next, the operation when the input block is a non-intra coded block will be described. Here, it is assumed that the parameter memory unit 118 holds the noise parameter for the already decoded reference frame obtained by the method described in the present embodiment.

The DCT pattern determination means 431 receives the DCT coefficient block from the inverse quantization means 111. DC
The T pattern determination unit 431 classifies each block from the distribution of each frequency component of the DCT coefficient block.

The DCT pattern determination means 431 determines from the macroblock coding mode that the input block is a non-intra coded block, and
DCT pattern PTN13 and the DCT pattern in FIG.
The pattern PTN14 is used.

FIG. 34 is a flowchart showing the flow of processing for classifying non-intra-coded blocks. FIG. 34 shows DCT pattern PTN13 and DCT pattern P
Each block is divided into three classes {M1, M
2, M3}. As shown in FIG. 34, the input DCT block is first compared with the DCT pattern PTN13 in step S81. If the input DCT block does not satisfy the DCT pattern PTN13, the block is classified into the DCT class M1 (step S83). Input DCT block is D
If the CT pattern PTN13 is satisfied, next, in step S82, the comparison with the DCT pattern PTN14 is performed. Input DCT block is DCT pattern PTN1
If the block does not satisfy 4, the block is classified as DCT class M
2 (step S84). If the input DCT block satisfies the DCT pattern PTN14, the block is classified into the DCT class M3 (step S8).
5). As described above, the DCT pattern determining means 43
1 classifies each block into one of the DCT classes M1, M2, M3, and outputs the classification result.

[0270] The quantization scale extracting means 433 extracts the quantization scale from the encoded information output from the variable length decoding means 110, and outputs it.

The DCT class output from the DCT pattern determination means 431 and the quantization scale output from the quantization scale extraction means 433 are output to the switch 435.

The switch 435 is connected to the variable length decoding means 11
If the macroblock coding mode output as 0 is non-intra coding, the switch is connected to a. Therefore, the noise parameter is determined by the parameter correction means 436.
Is input to

The reference area parameter determination means 437 obtains a reference area of a processing target block from which noise is to be removed using the motion vector output from the variable length decoding means 110, and obtains a noise parameter of a reference block overlapping the reference area. To determine the noise parameter of the reference area.

The reference area parameter determining means 437 obtains the noise parameter of the reference block. This is done by referring to the reference blocks 522 to 52 by the reference area parameter determining means 147 as described with reference to FIG.
5, the description is omitted.

A method for obtaining the noise parameter of the reference area from the noise parameter obtained by the reference area parameter determining means 437 will be described.

First, as for the method of obtaining the DCT classes (M1 to M3) from the noise parameters of the reference area, as described in the second embodiment, the DCT classes (I1 to I3) of the reference area 526 are obtained. Since this is the same as the method, its description is omitted.

Next, a method of obtaining the quantization scale from the noise parameters of the reference area will be described with reference to FIG.

As a method of obtaining the quantization scale of the reference area 526, a value obtained by weighting and averaging the quantization scales of the reference blocks 522 to 525 with the number of pixels of the portion where the reference area 526 and each reference block overlap is obtained. Is the quantization scale of the reference area 526.

As another method for obtaining the quantization scale of the reference area 526, reference blocks 522 to 525 are used.
There is a method of calculating the minimum value or the maximum value of the quantization scale of

As still another method for obtaining the quantization scale of the reference area 526, there is a method for obtaining the minimum value, the maximum value, or the average value of the quantization scale in the reference block whose DCT class is a predetermined class. is there. For example, the quantization scale of the reference area 526 can be obtained based on the quantization scale of the reference block whose DCT class is M2 or M3.

The reference area parameter determination means 437 outputs the noise parameter of the reference area obtained as described above to the parameter correction means 436.

The parameter correction means 436 compares the noise parameter output by the DCT pattern determination means 431 and the quantization scale extraction means 433 via the switch 435 with the noise parameter of the reference area output by the reference area parameter determination means 437. Take as input. First, the DT determined by the DCT pattern determination means 431
The CT class is corrected using the DCT class determined by the reference area parameter determining unit 437. Table 4 shows an example of this correction method.

[0283]

[Table 4]

The correction method shown in Table 4 indicates that, of the DCT class determined by the DCT pattern determination means 431 and the DCT class determined by the reference area parameter determination means 437, the one which is more likely to generate mosquito noise is selected. ing.

Table 5 shows an example of another correction method.

[0286]

[Table 5]

In the correction method shown in Table 5, the DCT class determined by the DCT pattern determining means 431 and the DCT class determined by the reference area parameter determining means 437 are
This shows that the averaged DCT class is selected.

Next, the quantization scale extracted by the quantization scale extraction means 433 is corrected using the quantization scale determined by the reference area parameter determination means 437. As an example of this correction method, the quantization scale extraction means 43
There is a method of selecting the smaller value of the quantization scale extracted by the third area 3 and the quantization scale determined by the reference area parameter determining means 437. Further, an average value or a maximum value of the two may be obtained.

As another method of correcting the quantization scale, there is a method of determining whether or not to correct the quantization scale according to the DCT class of the reference area. For example, when the DCT class of the reference area is M1, the quantization scale extracted by the quantization scale extraction unit 433 is output without correction and the DCT class of the reference area is M2, M
If it is 3, the quantization scale is corrected by the above method, and the corrected value is output.

The parameter correction means 436 outputs the noise parameter corrected as described above to the parameter memory means 118. Parameter memory means 118
Accumulates the input noise parameters.

Next, the operation of the filter determining means 434 will be described. The operation of the filter determination unit 434 is the same for an intra-coded block and a non-intra-coded block.

The filter determining means 434 determines the type of filter to be applied to each block with reference to the DCT class among the noise parameters. Table 6 shows the type of filter.
Performed by

[0293]

[Table 6]

Table 6 shows an example of a determination method in the case where there are three types of filters, F1, F2, and F3. here,
Filter strength (ability to remove mosquito noise)
Is assumed that F1 is the weakest and F3 is the strongest. And
The type of the filter determined by the filter determining unit 434 is
Output to mosquito noise removing means 126.
Here, it can be said that the determined filter type corresponds to the magnitude of the detected mosquito noise.

The mosquito noise removing unit 126 receives the type of filter applied to each pixel block from the mosquito noise detecting unit 430. And the frame memory 11
5 is filtered. An operation example of the mosquito noise removing unit 126 will be described below.

FIG. 35 is an explanatory diagram of an example of pixels used for the filter processing. It is assumed that one square corresponds to one pixel. The filter determining means 434 determines whether the pixel block 5
Assume that it is determined that the filter 44 is to be subjected to the filter F2. In this case, a filter is applied to all the pixels in the block 544.

Now, consider the case where a filter is applied to the pixel d. In this case, first, a pixel that becomes an edge pixel from the pixel d toward the top, bottom, left, and right is detected. The edge pixel is detected by calculating the absolute value of the difference between the pixel value of the pixel d and another pixel, and detecting the absolute value of the difference value as an edge pixel if the absolute value is larger than a predetermined value. As the predetermined value, a value obtained by multiplying a quantization coefficient, which is a noise parameter of each block, by a predetermined coefficient can be used. For example, when detecting an edge pixel from the pixel d toward the left, the absolute value of the difference value between the pixel d and the pixels c, b, and a is obtained and compared with a predetermined value. Similarly, edge detection is performed on pixels j, i, h upward from pixel d, pixels k, l, m downward, and pixels e, f, g downward.

It is assumed that the detected edge pixels are a, b, f, and i. In this case, the pixel d is filtered using the pixels c, d, e, g, h, j, k, l, and m other than the edge pixels. The types of filters are
For example, a low-pass filter can be used.

FIG. 36 is an explanatory diagram of an example of filter processing for removing mosquito noise. Here, FIG.
The case where the filtering process is performed on the pixel d of FIG. 6A using the pixels a to g will be described.

FIG. 36B shows the pixel values of the pixels a to g. Here, of the pixels a to g, the pixel whose absolute value of the difference from the pixel value of the pixel d is equal to or smaller than a predetermined value, that is, FIG.
In 6 (b), the filtering process is performed using only the pixels between the two lines indicating the threshold value for the pixel d. Therefore, in FIG. 36 (c), the pixels a, b, and f represented by black circles are not used, and the pixels c,
Filter processing is performed on the pixel d using only d, e, and g. Here, the pixels a, b, and f represented by black circles are edge pixels.

FIG. 36D shows a pixel d after the filtering process. FIG. 36E shows the value of each pixel after the filter processing is performed on other pixels in the same manner. The case where the filtering process is performed using the pixels on the same line has been described, but the same applies to the case where the pixels not on the same line are used as shown in FIG.

[0302] Alternatively, as the pixels used for the filter, only pixels inside the edge pixels (pixels closer to the edge pixels) as viewed from the pixel d may be used. In this case, the pixel d is filtered using the pixels c, d, e, j, k, l, and m.

The pixels a, b, f, and
For i, the pixel value may be replaced with the pixel value of the pixel d, and the pixel d may be filtered using the pixels am.

If the filter decision means 434 decides to apply the filter F3 to the pixel block 544,
A filter having a higher strength than the filter F2 is applied. For example, when a filter is applied to the pixel d in FIG. 35, a pixel further to the left of the pixel a, a pixel further above the pixel h, a pixel further right to the pixel g, and a pixel m
This can be realized by using a lower pixel and a wider range of pixels. Alternatively, the pixel used is the filter F2.
In the same manner as in the above case, a low-pass filter in which a filter coefficient is set so as to lower the cutoff frequency may be applied.

If the filter decision means 434 decides to apply the filter F1, which is the weakest filter, to the pixel block 544, no filter is applied to the pixel block 544, or a filter having a lower intensity than the filter F2 is used. Apply.

FIG. 37 is an explanatory diagram of another example of the pixel used for the filter processing. The pixels used for the filter processing may be those shown in FIG. 37 in addition to those described in FIG. That is, in FIG. 37, when a filter is applied to the pixel m, the pixels a to y are used in FIG.
The filter is applied in the same way as in the case of.

The predetermined value to be compared with the absolute value of the difference between pixel values in edge detection may be a fixed value without using a quantization parameter. When the quantization scale is not used at the time of edge detection, the quantization scale extraction unit 433, the parameter correction unit 436, the reference area parameter determination unit 437, and the parameter memory unit 1
In 18, it is not necessary to deal with the quantization scale. That is,
In this case, the noise parameter is only the DCT class.

As described above, in the image decoding apparatus according to the present embodiment, for an intra-coded block, each block is classified into a plurality of DCT classes using DCT coefficients obtained from a compressed code sequence. In the case of a non-intra coded block, a reference area in a reference frame is obtained using a motion vector. Thereafter, the DCT class and the quantization scale of the reference area are obtained using the DCT class and the quantization scale of the block overlapping the reference area, and the DCT class and the quantization scale of the processing target block obtained from the information of the compressed code string are corrected. . Then, a filter to be applied to the block is determined using the DCT class of each block.
Here, when the filter is determined, a DCT equal to or more than a predetermined value is used.
The higher the distribution of the coefficients in the higher band, the stronger the filter strength. Then, a filter is applied to each block of the decoded image based on the determined filter. When a filter is applied, edge detection is performed using the quantization scale of each block or a predetermined fixed value as a threshold, and a filter is applied except for at least edge pixels to remove mosquito noise.

As described above, according to the image decoding apparatus of this embodiment, it is possible to reliably detect the block in which mosquito noise occurs and the magnitude of mosquito noise from the distribution of DCT coefficients of each block. In this case, by using a DCT pattern different from the intra-coded block for the non-intra-coded block, classification suitable for the characteristics of the non-intra-coded can be performed. For non-intra coded blocks, DC
By correcting the noise parameter for the difference image obtained from the T coefficient using the noise parameter of the reference area, the noise parameter of the decoded image is determined with high accuracy even for a non-intra coded block, and the mosquito noise The size can be determined. Further, since the noise parameter of the reference area is obtained using the noise parameter of the block overlapping the reference area, a highly accurate noise parameter can be obtained.

By using a plurality of filters having different intensities according to the magnitude of the mosquito noise, and performing the edge detection and performing the filtering without using the edge pixels, the blur of the image, particularly at the edge portion, is obtained. Mosquito noise can be reliably removed while minimizing the noise. In addition, by using the quantization scale of each block as a threshold value for detecting an edge pixel, the threshold value can be adaptively changed for each block, and more accurate edge detection can be performed. This makes it possible to more reliably remove mosquito noise while minimizing blurring particularly at edge portions.

In the present embodiment, a DCT pattern is used for an intra-coded block as shown in FIG.
(B) are classified into three DCT classes, and non-intra-coded blocks are classified into three DCT classes using two DCT patterns in FIGS. 32 (c) and (d). However, the number of DCT patterns is not limited to two, and the number of DCT classes is not limited to three. Further, the frequency distribution of the DCT pattern is not limited to the distributions shown in FIGS. Further, the same DCT pattern may be used for an intra-coded block and a non-intra-coded block.

Further, in the present embodiment, three types of filters are used in the mosquito noise removing means 126, but any number of types may be used.

Further, in the present embodiment, the case where the filter used in the mosquito noise removing means 126 is a low-pass filter has been described. However, this is another filter, such as a median filter or the like, as long as it is a filter for removing mosquito noise. It may be. Further, the position of the pixel used for the filter is not limited to the example of FIG. 35 or FIG.

In this embodiment, several examples of the method of determining the noise parameter of the reference area by the reference area parameter determining means 437 have been described. However, this determination method is not limited to the method described in this embodiment. Absent.

Further, in the present embodiment, the case where Table 4 and Table 5 are used as the noise parameter correction method in the parameter correction means 436 has been described, but the correction method is not limited to the method of Table 4 or Table 5.

Further, in the present embodiment, the method of correcting the DCT class and the quantization scale by using the reference area parameter determining means 437 and the parameter correcting means 436 has been described. Good.

In the present embodiment, as a method of detecting an edge by the mosquito noise removing means 126, for example, when a filter is applied to the pixel d of FIG. 35, a method of using a difference value between the pixel d and another pixel is used. Was explained. However, the method of edge detection is not limited to this method.
A method of performing edge detection using a difference value between adjacent pixels may be used.

(Eighth Embodiment) In the eighth embodiment,
An image decoding device that detects and removes mosquito noise as encoding noise when decoding a compressed code string in which an interlaced image is encoded will be described.

[0319] The image decoding apparatus according to the eighth embodiment comprises:
In the image decoding apparatus of FIG. 30, the mosquito noise detecting means 44 is used instead of the mosquito noise detecting means 430.
0 is used. Variable length decoding means 110, inverse quantization means 111, inverse DCT means 112, switch 11
3, 114, the frame memory 115, the parameter memory means 118, and the adding means 119 are the same as those in the first embodiment, so the same numbers are assigned and the description is omitted. Here, mosquito noise detection means 440
Operate as a noise detection device.

FIG. 38 is a block diagram showing the structure of the mosquito noise detecting means according to the eighth embodiment. FIG.
The mosquito noise detection means 440 includes a DCT pattern determination means 441, a quantization scale extraction means 443,
Filter determining means 444, switch 445, parameter correcting means 446, reference area parameter determining means 4
47. Mosquito noise detection means 440
, The DCT coefficient is input from the inverse quantization means 111,
The quantization scale, the macroblock coding mode, and the motion vector are input from the variable length decoding unit 110.

The DCT pattern determination means 441 receives a DCT coefficient block from the inverse quantization means 111 and a DCT mode from the variable length decoding means 110 as an input. D
The CT pattern determination means 441 classifies each DCT coefficient block from the distribution of each frequency component of the DCT coefficient block. The DCT coefficient blocks are classified in the same manner as in the seventh embodiment, and the DCT coefficient blocks are classified into DCT classes M1, M2, and M3.

The DCT class of each block determined here is assigned to each field in the block. This is the same as that described with reference to FIGS. DC
When the T mode is the field mode, for example, the DCT class for the DCT block 601 is assigned to the field blocks 609 and 611 as described with reference to FIG. When the DCT mode is the frame mode, for example, as described in FIG.
The DCT class for the T block 651 is assigned to the field blocks 659 and 660.

The quantization scale extracting means 443 extracts a quantization scale from the coded information output from the variable length decoding means 110 and outputs it. The quantization scale is assigned to each field block, similarly to the DCT pattern determination unit 441.

The switch 445 is connected to the variable length decoding unit 11
The switch is switched using the macroblock coding mode output by 0. When the macroblock coding mode is the intra coding, the switch 445 is connected to b. In this case, the DCT class output from the DCT pattern determination unit 441 and the quantization scale extraction unit 44
3 is output to the parameter memory means 118 in field block units. When the macroblock coding mode is non-intra coding, the switch 445 is connected to a. In this case, the noise parameter is input to the parameter correction unit 446.

When the macroblock coding mode is non-intra coding, the reference area parameter determining means 447 refers to the noise parameter of the reference block to determine the noise parameter of the reference area.

The reference area parameter determining means 447 obtains the noise parameter of the reference frame from the parameter memory means 118 using the motion vector output from the variable length decoding means 110 on a field block basis. The acquisition of the noise parameter of the reference frame is the same as that described with reference to FIG. 29 in the sixth embodiment, and a description thereof will not be repeated.

The method for obtaining the DCT class and the quantization scale of the reference area using the DCT class and the quantization scale obtained from the parameter memory means 118 is the seventh method.
Since the method is the same as that described in the embodiment, the description thereof will be omitted.

The reference area parameter determination means 447 outputs the noise parameter of the reference area obtained for each field block as described above to the parameter correction means 446.

The parameter correcting means 446 is provided by the switch 4
The noise parameter output from the DCT pattern determination unit 441 and the quantization scale extraction unit 443 via the reference region parameter correction unit 45 is corrected by the noise parameter of the reference region output from the reference region parameter determination unit 447. The method of correcting the noise parameter is the same as in the seventh embodiment, and a description thereof will not be repeated. Parameter correction means 446
Outputs the corrected noise parameter to the parameter memory unit 118. Parameter memory means 11
In step 8, the input noise parameters are stored in field block units.

Next, the operation of the filter determining means 444 will be described. The operation of the filter determination means 444 is the same for an intra-coded block and a non-intra-coded block. The filter determining means 444 determines the type of filter to be applied to each field block as D
Determined with reference to the CT class. The determination of the type of the filter is performed according to Table 6 as in the seventh embodiment. The type of filter for each field block determined by the filter determining unit 444 is determined by the mosquito noise removing unit 12.
6 is output. Here, it can be said that the determined filter type corresponds to the magnitude of the detected mosquito noise.

[0331] The mosquito noise removing means 126 receives from the mosquito noise detecting means 440 the type of filter applied to the boundary of each block. And the frame memory 1
A filter is applied to the block boundary of the image output from the block 15. An operation example of the filter of the mosquito noise removing unit 126 will be described below.

FIG. 39 is an explanatory diagram of an example of pixels used for the filter processing. It is assumed that one square corresponds to one pixel. Now, it is assumed that the filter determination unit 444 determines to apply the filter F2 to the pixel block 594. In this case, a filter is applied to all the pixels in the block 594.

Here, consider the case where a filter is applied to the pixel d. In this case, first, edge pixels are detected from pixel d in the vertical, horizontal, and horizontal directions with respect to the pixels in the same field.
Since the pixel d is a pixel in the first field, edge detection is performed using the pixel in the first field. The detection of the edge pixel is performed by detecting the edge as an edge if the absolute value of the difference between the pixel value of the pixel d and another pixel is larger than a predetermined value. As the predetermined value, a value obtained by multiplying a coefficient by a quantization scale, which is a noise parameter of each block, can be used.

For example, when detecting an edge pixel from the pixel d to the left, the absolute value of the difference value between the pixel d and the pixels c, b, a is obtained and compared with a predetermined value. Similarly, pixel d
Edge detection is performed for pixels j, i, h upward, pixels k, l, m downward, and pixels e, f, g upward.

It is assumed that the edge pixels detected in this way are a, b, f, and i. In this case, the pixel d is filtered using the pixels c, d, e, g, h, j, k, l, and m other than the edge pixels. That is, the filter is applied to the pixel d using only the pixels in the first field. As a type of the filter, for example, a low-pass filter can be used.

Consider a case where a filter is applied to pixel d '. In this case, since the pixel d 'is a pixel in the second field, edge detection is performed using the pixel in the second field. That is, for the pixel d ', an edge pixel is detected using the pixels a' to m '. The method of detecting an edge pixel is the same as the above method.

Assume that the edge pixels detected in this way are a ', b', f ', and i'. In this case, the pixels c ′, d ′, e ′, g ′, h ′,
The pixel d 'is filtered using j', k ', l', and m '. That is, the filter is applied to the pixel d 'using only the pixels in the second field. As the type of the filter, for example, a low-pass filter can be used as in the case of the first field.

Further, as the pixels used for the filter, only the pixels (pixels closer to the edge pixels) inside the edge pixels as viewed from the pixels d and d 'may be used. In this case, for example, for pixel d, pixels c, d, e, j, k,
The pixel d is filtered using l and m.

If the filter decision means 444 decides to apply the filter F3 to the pixel block 594,
A filter having a higher strength than the filter F2 is applied. For example, when a filter is applied to the pixel d in FIG. 39, a pixel on the left side of the pixel a, a pixel on the upper side of the pixel h, a pixel on the right side of the pixel g, and a pixel m
This can be realized by using pixels in the same field over a wider range, such as pixels on the lower side of the field. Alternatively, a low-pass filter in which a filter coefficient is set so that the cutoff frequency is lower may be applied by using the same pixel as that of the filter F2.

If the filter decision means 444 decides to apply the filter F1, which is the weakest filter, to the pixel block 594, the filter is not applied to the pixel block 594, or a filter having a lower intensity than the filter F2 is used. Apply.

FIG. 40 is an explanatory diagram of another example of the pixel used for the filter processing. The pixels used for the filter processing may be those shown in FIG. 40 in addition to those described in FIG. That is, in FIG. 40, the pixels a to y are used when the filter is applied to the pixel m, and the pixels a ′ to y ′ are used when the filter is applied to the pixel m ′. Apply a filter in the same way as.

The predetermined value to be compared with the absolute value of the difference between the pixel values in edge detection may be a fixed value without using a quantization parameter. When the quantization scale is not used at the time of edge detection, the quantization scale extraction unit 443, the parameter correction unit 446, the reference area parameter determination unit 447, and the parameter memory unit 1
In 18, it is not necessary to deal with the quantization scale. That is,
In this case, the noise parameter is only the DCT class.

[0343] The frame filtered by the mosquito noise removing means 126 as described above is output as an output image.

As described above, in the image decoding apparatus according to the present embodiment, for an intra-coded block, each block is classified into a plurality of DCT classes using DCT coefficients obtained from a compressed code sequence. The classification into the DCT class is performed in units of field blocks obtained by dividing each block into fields. In the case of a non-intra coded block, a reference area in a reference frame is obtained using a motion vector. Then, using the DCT class and quantization scale of the block overlapping the reference area,
The CT class and quantization scale are obtained, and the D of the processing target block obtained using information obtained from the bit stream is obtained.
Correct the CT class and quantization scale. Then, a filter to be applied to pixels in the field block is determined using the DCT class and quantization scale in each field block. Here, when the filter is determined, the strength of the filter is set to be stronger as the DCT coefficient of a predetermined value or more is distributed in a higher frequency band. Then, a filter is applied to each field block of the decoded image based on the determined filter. When a filter is applied, edge detection is performed using a quantization scale or a predetermined fixed value of each field block as a threshold, and a filter is applied without using at least edge pixels to remove mosquito noise.

As described above, according to the image decoding apparatus of the present embodiment, the block in which mosquito noise occurs and the magnitude of mosquito noise can be reliably detected from the distribution of the DCT coefficients of each block in field units.
In this case, by using a DCT pattern different from the intra-coded block for the non-intra-coded block, classification suitable for the characteristics of the non-intra-coded can be performed. For non-intra coded blocks, the noise parameters for the difference image obtained from the DCT coefficients are corrected using the noise parameters in the reference area, so that even non-intra coded blocks can be decoded with high accuracy. Determine the noise parameters of the image,
The magnitude of the mosquito noise can be determined. Also, since the noise parameter of the reference area is obtained using the noise parameter of the block overlapping the reference area,
A highly accurate noise parameter can be obtained.

Then, by using a plurality of filters having different intensities according to the magnitude of the mosquito noise, and performing the edge detection and performing the filtering without using the edge pixels, the blur of the image, particularly at the edge portion, is obtained. Mosquito noise can be reliably removed while minimizing the noise. In addition, since mosquito noise is detected and removed in units of field blocks, mosquito noise can be detected more accurately for interlaced images. In addition, by using the quantization scale of each block as a threshold value for detecting an edge pixel, the threshold value can be adaptively changed for each block, and more accurate edge detection can be performed. This makes it possible to more reliably remove mosquito noise while minimizing blurring particularly at edge portions. In addition, since mosquito noise is detected and removed in units of field blocks, mosquito noise can be reliably detected in finer units.
Mosquito noise can be removed without adversely affecting a portion where no mosquito noise is generated.

In the present embodiment, the DCT coefficient blocks are classified into three DCT classes, and three types of filters are used in the mosquito noise removing means 126. However, the types of DCT classes and filters are not limited to these. Absent.

In this embodiment, the case where the filter used in the mosquito noise removing means 126 is a low-pass filter has been described. However, this is another filter that removes mosquito noise, such as a median filter. , May be. The positions of the pixels used for the filter are not limited to the examples shown in FIGS.

In the present embodiment, as a method of performing edge detection by the mosquito noise removing means 126, for example, when a filter is applied to the pixel d of FIG. 39, a method of using a difference value between the pixel d and another pixel is used. explained. However, the method of edge detection is not limited to this method. For example, a method of performing edge detection using a difference value between adjacent pixels may be used.

(Ninth Embodiment) In the ninth embodiment,
Regarding block noise and mosquito noise related to one block, an image decoding apparatus that selects one of them as coding noise to be removed will be described.

FIG. 41 is a block diagram of an image decoding apparatus according to the ninth embodiment. The image decoding device in FIG.
Image decoding means 101 and block noise detection means 10
2, a mosquito noise detection unit 103, a noise removal region determination unit 104, and a noise removal unit 105. Block noise detecting means 102, mosquito noise detecting means 103, noise removing area determining means 10
4 operates as a noise detection device.

The image decoding means 101 receives a code string as input and decodes it by a method suitable for the code string to obtain a decoded image. For example, if the code string is JPEG (joint photog
If it is encoded by the raphic image coding experts group), it is decoded by the JPEG method and MPE
If it is encoded by the G method, it is decoded by the MPEG method. The image decoding unit 101 outputs the decoded image to the block noise detection unit 102, the mosquito noise detection unit 103, and the noise removal unit 105.

On the basis of the image data of the decoded image, the block noise detecting means 102 detects a block boundary where block noise occurs, and the mosquito noise detecting means 10
3 detects a block in which mosquito noise occurs.
The block noise detecting means 102 and the mosquito noise detecting means 103 replace the image data of the decoded image with
Noise may be detected based on coding information obtained from a code string input to the image decoding unit 101.

As a method of detecting block noise in the block noise detecting means 102 and a method of detecting mosquito noise in the mosquito noise detecting means 103, a conventional method can be used. In addition, the block boundary where block noise occurs is detected by the method described in the first to sixth embodiments, and the block where mosquito noise occurs is detected by the method described in the seventh and eighth embodiments. Is also good.

FIG. 42 is an explanatory diagram showing an example of a place where block noise and mosquito noise have occurred. In FIG. 42, one square indicates one block. FIG.
2A shows a block boundary at which the block noise detection unit 102 has detected the occurrence of block noise. In FIG. 42A, a block boundary B indicated by a thick solid line
B is a block boundary at which occurrence of block noise is detected. FIG. 42B shows a block in which the mosquito noise detection unit 103 has detected the occurrence of mosquito noise. In FIG. 42B, a hatched block MM is a block in which the occurrence of mosquito noise has been detected. The detection result of the block noise detection means 102 and the detection result of the mosquito noise detection means 103 are output to the noise removal area determination means 104.

The noise removal area determination means 104 selects the noise to be removed based on the detection results of the block noise and mosquito noise output from the block noise detection means 102 and the mosquito noise detection means 103, and selects an area in the screen. Determines whether to remove noise.
The area from which noise is to be removed refers to pixels around a detected block boundary for block noise, and a detected block for mosquito noise. The method of this determination differs depending on which of the block noise and the mosquito noise is given priority for removing the noise.

FIG. 43 is an explanatory diagram showing an area where it is determined that noise removal should be performed when noise is generated as shown in FIG. 43 is shown in FIG.
Is the same as

First, the case where priority is given to the removal of block noise will be described. In this case, the noise removal area determination means 104 determines that the block noise should be removed as it is from the block boundary BB where the block noise detection means 102 has detected that block noise has occurred. At this time, the block MM which is not in contact with the block boundary from which the block noise has been removed and which is detected by the mosquito noise detection means 103 as generating mosquito noise is a mosquito noise. Should be removed.

Then, the mosquito noise detecting means 103
Is detected by the block noise detecting means 1
When 02 touches the detected block boundary BB, the mosquito noise of this block MM is not removed, and only the block noise of this block boundary BB is selected as noise to be removed. FIG. 43A shows an area where it is determined that noise removal should be performed when priority is given to block noise.

Next, the case where priority is given to the removal of mosquito noise will be described. In this case, the noise removal area determination means 104 determines whether the mosquito noise detection means 103 has detected that the mosquito noise has occurred.
It is determined that mosquito noise should be removed as it is. A block boundary that is not in contact with the block from which mosquito noise has been removed at this time and that is detected by the block noise detecting means 102 as generating block noise is a block boundary BB. Determine that noise should be removed.

Then, the mosquito noise detecting means 103
Is detected by the block noise detecting means 1
When 02 touches the detected block boundary BB, the block noise of this block boundary BB is not removed, and only the mosquito noise of this block MM is selected as noise to be removed. FIG. 43B shows an area where it is determined that noise removal should be performed when priority is given to mosquito noise.

As a method of determining a region from which block noise and mosquito noise are to be removed by the noise removing region determining means 104, there is a method using the magnitude of noise. In this case, the block noise detecting means 102 and the mosquito noise detecting means 103 detect the locations where the block noise and the mosquito noise occur, respectively, and also determine the magnitude thereof. The larger the magnitude of the noise, the more noticeable the noise.

As a method of obtaining the magnitude of the noise, for example, as described in the first to eighth embodiments, the DCT
The block is divided into several classes (DCT classes) based on the distribution of the frequency components of the coefficients, and the type of filter for noise removal is determined based on these classes, and the type of the filter is made to correspond to the magnitude of the noise. There is a way. The magnitude of the noise may be determined using another method. The location and magnitude of the noise detected by the block noise detection means 102 and the mosquito noise detection means 103 are input to the noise removal area determination means 104.

FIG. 44 is an explanatory diagram showing an area where noise removal is to be performed in consideration of the magnitude of noise. The noise removal area determining means 104 selects noise to be removed so as to give priority to the removal of the noise having the larger magnitude from the block noise and the mosquito noise, and determines the area from which the noise is to be removed.

FIG. 44A shows a block boundary at which the occurrence of block noise has been detected by the block noise detecting means 102. In FIG. 44A, a block boundary indicated by a thick line is a portion where it is determined that block noise occurs, and a block boundary BB indicated by a thick solid line
1 indicates that the magnitude of the block noise is larger than the block boundary BB2 indicated by the thick broken line.

FIG. 44B shows a block in which the occurrence of mosquito noise has been detected by the mosquito noise detection means 103. In FIG. 44 (b), the shaded blocks are blocks for which mosquito noise has been determined to occur, and the blocks MM1 shaded with high density are less mosquito noise than the blocks MM2 shaded with low density. This indicates that the size is large.

When the block detected by the mosquito noise detecting means 103 is in contact with the block boundary detected by the block noise detecting means 102, priority is given to removal of the larger noise. When the magnitudes of the mosquito noises are the same, for example, priority is given to removing the block noise.

FIG. 44 (c) shows the area from which the noise removal is determined by the noise removal area determination means 104 and the type of the noise. The noise removal area determination unit 104 outputs information on the area from which noise is to be removed and the type of the noise, that is, information on the noise to be removed to the noise removal unit 105.

The noise removing unit 105 receives the decoded image from the image decoding unit 101 as input, and receives information on noise to be removed from the noise removing area determining unit 104. Then, the noise removing unit 105 removes noise to be removed from the decoded image.

[0370] For example, for a block boundary from which block noise is to be removed, the noise removing means 105 applies a low-pass filter to pixels around the boundary to remove the block noise and remove the mosquito noise from the block. For, mosquito noise is removed by applying a low-pass filter to pixels other than the edge pixels. Noise removal means 1
In step 05, a low-pass filter is applied to pixels around the block boundaries BB, BB1 and BB2 according to, for example, FIG. 43 and FIG. , A filter is applied to pixels other than the edge pixels. With such an operation, block noise and mosquito noise can be removed.
The noise removing unit 105 outputs a decoded image from which block noise and mosquito noise have been removed as an output image.

As described above, the image decoding apparatus according to the present embodiment specifies an area where block noise occurs and an area where mosquito noise occurs in a decoded image obtained by decoding a code string. Then, by giving priority to either the block noise or the mosquito noise, the noise removal area is determined so that there is no area for removing both the block noise and the mosquito noise. Then, block noise and mosquito noise are removed from the decoded image by performing noise removal according to the determined noise removal area and the type of noise.

As described above, according to the image decoding apparatus of this embodiment, since there is no area for removing both the block noise and the mosquito noise, the amount of arithmetic processing required for noise removal and the amount of memory to be used are reduced. Can be reduced. Therefore, for example, when the processing is realized by software, the processing amount and the memory amount can be reduced. In addition, when realized by hardware such as an LSI (large-scale integration), the chip area, power consumption, memory amount, and the like can be reduced.

In this embodiment, a low-pass filter is applied to a block boundary determined to generate block noise, and a low-pass filter is applied to pixels other than edge pixels to a block determined to generate mosquito noise. Has been described to remove the block noise and the mosquito noise, but this may be another method.

In the above embodiment, the case where the MPEG-2 system employing DCT as the orthogonal transform is used as the encoding system has been described. However, in the first, fifth and ninth embodiments, the orthogonal Any other encoding method may be used as long as it is an encoding method using conversion, and the second to fourth and sixth encoding methods may be used.
In the eighth to eighth embodiments, other encoding schemes may be used as long as the encoding scheme uses orthogonal transform and motion compensation.

Also, in the first to eighth embodiments, the DCT coefficient block is supplied from the inverse quantization means 111 to the block noise detection means 130 or the like or the mosquito noise detection means 430.
Has been described, but the block noise detection means 130 or the like or the mosquito noise detection means 430
Etc. are used to examine the distribution of each frequency component of the DCT coefficient.
Since the CT coefficient is used, the block of the quantized DCT coefficient before performing the inverse quantization from the variable length decoding unit 110 may be input to the block noise detection unit 130 or the mosquito noise detection unit 430 or the like.

[0376]

As described above, according to the present invention, encoding noise can be reliably detected, and erroneous detection is reduced. Therefore, it is possible to prevent deterioration in image quality caused by noise removal processing. it can. For non-intra coded blocks, the coding information of the reference area is used, so that coding noise can be reliably detected. Since the encoding noise detection processing is performed for each field on the interlaced image, the encoding noise can be accurately detected. Further, since the noise removal processing is performed according to the magnitude of the noise, it is possible to reliably remove the coding noise while minimizing the deterioration of the image quality.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image decoding device according to a first embodiment.

FIG. 2 is an explanatory diagram of a configuration of a macro block when an image format is a 4: 2: 0 format.

FIG. 3 is a block diagram illustrating a configuration of a block noise detection unit.

FIG. 4 is an explanatory diagram of a DCT pattern used for block classification.

FIG. 5 is a flowchart showing a flow of processing for classifying intra-coded blocks.

FIG. 6 is an explanatory diagram of an arrangement of pixel blocks.

FIG. 7 is a flowchart showing a procedure for obtaining a type of a filter.

FIG. 8 is an explanatory diagram of a horizontal filtering process on a boundary between blocks.

FIG. 9 is a graph showing an example of a frequency characteristic of a filter.

FIG. 10 is a block diagram illustrating a configuration of a block noise detection unit according to a second embodiment.

FIG. 11 is a flowchart illustrating a flow of processing for classifying non-intra-coded blocks.

FIG. 12 is an explanatory diagram of a processing target block and a reference area thereof.

FIG. 13 is a block diagram of an image decoding device according to a third embodiment.

FIG. 14 is a block diagram illustrating a configuration of a block noise detection unit in the image decoding device in FIG. 13;

FIG. 15 is an explanatory diagram of a DCT pattern used for block classification.

FIG. 16 is a flowchart illustrating a flow of processing for classifying blocks.

FIG. 17 is a block diagram showing another example of the configuration of the block noise detection means of FIG. 14;

FIG. 18 is a flowchart showing the flow of processing when classifying blocks using DC coefficients.

FIG. 19 is a block diagram showing still another configuration example of the block noise detection means of FIG.

FIG. 20 is a block diagram illustrating a configuration of a block noise detection unit according to a fourth embodiment.

FIG. 21 is a block diagram showing another configuration example of the block noise detecting means of FIG. 20;

FIG. 22 is a block diagram showing still another configuration example of the block noise detecting means of FIG. 20;

FIG. 23 is a block diagram illustrating a configuration of a block noise detection unit according to a fifth embodiment.

FIG. 24 is an explanatory diagram of the relationship between DCT coefficient macroblocks and pixel blocks represented by fields when the DCT mode is the field mode.

FIG. 25 is an explanatory diagram of the relationship between DCT coefficient macroblocks and pixel blocks represented by fields when the DCT mode is the frame mode.

FIG. 26 is an explanatory diagram of an arrangement of field blocks.

FIG. 27 is an explanatory diagram of filter processing in the vertical direction for a boundary between blocks.

FIG. 28 is a block diagram illustrating a configuration of a block noise detection unit according to a sixth embodiment.

FIG. 29 is an explanatory diagram of a method of acquiring a block noise parameter of a reference area.

FIG. 30 is a block diagram of an image decoding device according to a seventh embodiment.

FIG. 31 is a block diagram illustrating a configuration of a mosquito noise detection unit in the image decoding device in FIG. 30;

FIG. 32 is an explanatory diagram of a DCT pattern used for classifying blocks.

FIG. 33 is a flowchart showing the flow of processing for classifying blocks.

FIG. 34 is a flowchart showing the flow of a process for classifying non-intra-coded blocks.

FIG. 35 is an explanatory diagram of an example of pixels used for filter processing.

FIG. 36 is an explanatory diagram of an example of filter processing for removing mosquito noise.

FIG. 37 is an explanatory diagram of another example of the pixel used for the filter processing.

FIG. 38 is a block diagram illustrating a configuration of a mosquito noise detection unit according to an eighth embodiment.

FIG. 39 is an explanatory diagram of an example of pixels used for filter processing.

FIG. 40 is an explanatory diagram of another example of the pixel used for the filter processing.

FIG. 41 is a block diagram of an image decoding device according to a ninth embodiment.

FIG. 42 is an explanatory diagram showing an example of a place where block noise and mosquito noise have occurred.

FIG. 43 is an explanatory diagram showing a region where it is determined that noise removal should be performed when noise is occurring as in FIG. 42;

FIG. 44 is an explanatory diagram showing an area where it is determined that noise removal should be performed in consideration of the magnitude of noise.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 101 Image decoding means 102 Block noise detection means 103 Mosquito noise detection means 104 Noise removal area determination means 105 Noise removal means 110 Variable length decoding means 111 Inverse quantization means 112 Inverse DCT means 113,114 Switch 115 Frame memory 116 Block noise Removal means 118 Parameter memory means 119 Addition means 126 Mosquito noise removal means 130, 140, 230, 240, 250, 260, 2
70,280,330,340 Block noise detecting means 430,440 Mosquito noise detecting means 131,141,231,261,331,341,4
31, 441 DCT pattern determining means 132, 142, 244, 332, 342 DC coefficient extracting means 133, 143, 255, 333, 343, 433, 4
43 Quantization scale extraction means 134, 144, 233, 243, 253, 334, 3
44, 434, 444 filter determining means 145, 345, 435, 445 switch 146, 267, 277, 287, 346, 436, 4
46 Parameter correction means 147, 266, 276, 286, 347, 437, 4
47 Reference area parameter determination means 232 Motion vector extraction means PTN1 to PTN4, PTN11 to PTN14 DCT
Pattern BB, BB1, BB2 Block boundary in which block noise occurs MM, MM1, MM2 Block in which mosquito noise occurs

 ──────────────────────────────────────────────────続 き Continued on the front page (31) Priority claim number Japanese Patent Application No. 11-322569 (32) Priority date November 12, 1999 (November 12, 1999) (33) Priority claim country Japan (JP) (72) Inventor Hiroshi Taniuchi 1006 Kadoma, Kadoma, Osaka Pref. Matsushita Electric Industrial Co., Ltd. KK04 MA00 MA04 MA05 MA23 MC11 MC32 MC34 MC38 ME01 NN01 NN23 NN28 NN38 PP04 PP16 PP25 SS01 SS06 SS11 TA69 TB08 TC04 TC10 TC12 TC42 TD08 TD12 UA05 UA11 UA33 5J064 AA02 BA09 BA13 BA16 BB03 BB07 BC01 BC01 BC01 BC02

Claims (13)

[Claims] An encoding information including an orthogonal transformation coefficient and a motion vector for each block from a code sequence encoded by using motion compensation prediction, a block-based orthogonal transformation, and a quantization process for an image. Encoding information extraction step of extracting a reference area of each block from a reference frame based on a motion vector of each block; and a block in a reference frame overlapping each block and the reference area. A coding noise detecting step of detecting coding noise to be removed based on a distribution of each frequency component of the orthogonal transform coefficients. 2. The noise detection method according to claim 1, wherein the block noise is detected as the coding noise. In the coding noise detection step, each block is set to each frequency of the orthogonal transform coefficient. Based on the component distribution, the data is classified into a plurality of classes. Based on the class of the block to be processed and the adjacent block adjacent thereto, and the class of the block in the reference frame overlapping each of the reference regions of these blocks. Calculating a new class of each of the processing target block and the adjacent block, and detecting a magnitude of block noise generated between the processing target block and the adjacent block based on the new class. Noise detection method. 3. The noise detection method according to claim 1, wherein mosquito noise is detected as the coding noise. In the coding noise detection step, each block is converted into each frequency of the orthogonal transform coefficient. Classifying into a plurality of classes based on the distribution of the components, a new class of the processing target block based on the class of the processing target block and the class of the block in the reference frame overlapping the reference region of the processing target block. A noise detection method comprising: obtaining a class; and detecting a magnitude of mosquito noise generated in the processing target block based on the new class. 4. The noise detection method according to claim 1, wherein a block noise is detected as the coding noise, and in the coding noise detection step, a processing target block and an adjacent block are detected from the coding information. Extract the DC component of the orthogonal transform coefficient for the adjacent block to be added, in addition to the distribution of each frequency component of the orthogonal transform coefficient, the absolute value of the difference of the DC component between the processing target block and the adjacent block A noise detection method for detecting a magnitude of block noise generated between the processing target block and the adjacent block based on the block noise. 5. The noise detection method according to claim 4, wherein, in the coding noise detection step, a quantization scale for the processing target block is extracted from the coding information, and each frequency component of the orthogonal transform coefficient is extracted. In addition to the distribution, the absolute value of the difference between the DC components, based on the quantization scale, to detect the magnitude of block noise generated between the processing target block and the adjacent block, Noise detection method. 6. The noise detection method according to claim 1, wherein, in addition to the distribution of each frequency component of the orthogonal transform coefficient, a magnitude of a motion vector of the processing target block or an adjacent block adjacent thereto is calculated based on: A noise detection method comprising detecting coding noise to be removed. 7. The noise detection method according to claim 6, wherein a block noise is detected as the coding noise. In the coding noise detection step, the processing target block and the processing target block are determined from the coding information. Extract the DC component of the orthogonal transform coefficient for the adjacent block adjacent to, in addition to the distribution of each frequency component of the orthogonal transform coefficient and the magnitude of the motion vector, between the block to be processed and the adjacent block A noise detection method, comprising: detecting a magnitude of block noise generated between the processing target block and the adjacent block based on an absolute value of the difference between the DC components. 8. The noise detection method according to claim 7, wherein in the coding noise detection step, a quantization scale for the processing target block is extracted from the coding information, and each frequency component of the orthogonal transform coefficient is extracted. Block noise generated between the block to be processed and the adjacent block based on the absolute value of the difference between the DC components and the quantization scale in addition to the distribution of the motion vector and the magnitude of the motion vector of each block. A noise detection method characterized by detecting the size of a noise. 9. The noise detection method according to claim 1, wherein a block noise and a mosquito noise are detected as the coding noise, and one of the block noise and the mosquito noise related to one block is detected by the noise. A noise detection method for selecting as coding noise to be removed based on the magnitude of 10. The noise detection method according to claim 1, wherein a coding noise detection process is performed for each interlace image for each field. 11. A code including an orthogonal transform coefficient and a motion vector for each block, obtained from a code sequence coded by using motion compensation prediction, a block unit orthogonal transform, and a quantization process for an image. Input, and obtains a reference region of each block from a reference frame based on a motion vector for each block, and calculates each frequency of the orthogonal transform coefficient for each block and a block in a reference frame overlapping the reference region. A noise detection device comprising: means for detecting coding noise to be removed based on a distribution of components and outputting a result of the detection. 12. The noise detection device according to claim 11, wherein the decoding unit decodes the code string and outputs encoding information including the orthogonal transform coefficient for each block and a motion vector for a processing target block. An image decoding device comprising: a coding noise removal unit configured to remove coding noise based on a detection result output by the noise detection device. 13. The image decoding device according to claim 12, wherein the noise detection device is configured to detect mosquito noise as the encoding noise, and to calculate the mosquito noise from the encoding information. Extracting a quantization scale, wherein the coding noise removing unit uses a value corresponding to the quantization scale as a threshold value for detecting an edge pixel not used for noise removal. Image decoding device.