JPH05308623A - Image signal decoder - Google Patents

Abstract

PURPOSE:To eliminate distortion at high speed without generating blurring in an image by finding the band area of a signal at every block from a conversion coefficient in distortion elimination processing, and changing the characteristic of distortion elimination at every block based on a result of such processing, a motion vector at every block, and block type information, etc. CONSTITUTION:The data at every block of the image is decoded by a variable length encoder/decoder circuit 11, and is returned to an inter-frame differential signal by an inverse quantization circuit 12 and a reverse DCT circuit 13. The inter-frame differential signal is added on a prediction signal from a motion compensation circuit 15 at an adder circuit 14, and after the optimum distortion elimination processing being applied to it at a distortion elimination processing circuit 16, it is outputted to an image output part 17. A decision circuit 18 finds a frequency band area at every block in a crisscross direction from the output of the inverse quantization circuit 12, and decides the optimum distortion elimination processing based on the result of the band area, the motion vector, and a block type, and controls the distortion elimination processing circuit 16.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal decoding apparatus, and more particularly to an image signal decoding apparatus which decodes an image transmitted or recorded after being highly compression coded.

[0002]

2. Description of the Related Art When a moving image signal picked up by a solid-state image pickup device such as a CCD is recorded as digital data in a storage device such as a magnetic disk or a magnetic tape, the amount of data becomes enormous. .. Therefore, in order to record in a limited storage capacity range, it is necessary to compress the obtained image signal data with some high efficiency.

As a moving image compression method, a method such as the method proposed by ISO (see Journal of Image Electronics Engineers, Vol. 20, No. 4) is generally used to achieve high compression by utilizing inter-frame correlation. is there. This method will be briefly described with reference to FIG.

FIG. 4 is a block diagram showing a conventional moving image compression method utilizing inter-frame correlation. In the figure, the prediction error signal from which the motion compensation interframe prediction image is subtracted by the difference circuit 1 is DCT (discrete cosine transform).
After the DCT is performed for each block in the circuit 2, the quantization is performed in the quantization circuit 3. Further, the quantization result is the coding circuit 4
A variable length code is assigned and recorded at. Further, the quantization result is decoded by the inverse quantization circuit 5 and the inverse DCT circuit 6, and is added by the addition circuit 7 to the motion-compensated interframe prediction image. Then, a motion compensation prediction circuit 8 incorporating an image memory having a variable delay function for motion compensation
Then, the motion vector is calculated in order to create the motion-compensated inter-frame predicted image of the next frame.

All the frames are compressed by repeating this series of processing, but the difference is not always coded.
The input image itself is sometimes coded, and this is called an I picture. In addition, the following 2
There are different types.

[0006] One of them is called a P picture, which is temporally located before the image to be encoded now.
This is the difference from the already decoded image of the I picture or P picture. In practice, a so-called intra-coding method, which has a higher efficiency in encoding the difference from the motion-compensated predicted image or encoding without taking the difference, is selected.

The other is called a B picture, and is the most significant of the three types of difference coding and intra coding of temporally forward, backward, or interpolated images created from forward and backward. Select the most efficient one. This prediction method can be switched in block units, and selection information is added to the code as a block type.

In addition, DCT and variable length codes are used for the signal after motion compensation, the difference between images is obtained, and the redundancy in the time axis direction is reduced to reduce the redundancy in the spatial direction. There is. An encoding method using orthogonal transform such as DCT is also widely used for still image compression, and this method will be described below with reference to FIG.

FIG. 5 illustrates the operation of compressing a still image by a coding method using DCT. First, when a signal f whose redundancy in the time axis direction has already been reduced is input (101). , The input image data f is divided into blocks of a predetermined size to obtain f _b (102), and two-dimensional DCT is performed as orthogonal transformation for each divided block to obtain F _b.
(103). Next, linear quantization is performed according to each frequency component (104), and the quantized value F
Huffman coding is performed as variable length coding on Q (105). The result is transmitted or recorded as the compressed data C. At this time, the quantization width of the linear quantization is provided with a quantization matrix representing the relative quantization characteristics in consideration of the visual characteristics for each frequency component,
The quantization width is determined by multiplying this quantization matrix by a constant.

On the other hand, when the image data is reproduced from the compressed data, the quantized value FQ of the transform coefficient is obtained by decoding the variable length code (C) (106).
However, it is impossible to obtain the true value F before quantization from this value, and the result obtained by the inverse quantization becomes F'which includes an error (107). Therefore, IDCT (Inverse Discrete Cosine Transform) is performed on this value (F ') (108), the resulting value fb' is inversely blocked (109), and the resulting image data f'also has an error. It will be included.

Therefore, the image quality of the (110) reproduced image f'reproduced and output by the image reproducing apparatus is deteriorated. That is, the error in the value (F ') obtained as a result of the inverse quantization causes deterioration of the image quality of the reproduced image (f') as a so-called quantization error.

More specifically, first, the input image data is a block having a predetermined size (for example, 8 × 8).
Of pixels), and the two-dimensional DCT is performed as an orthogonal transform for each of the divided blocks.
Sequentially stored in an 8 × 8 matrix.

When viewed in a two-dimensional plane, the image data has a spatial frequency which is frequency information based on the distribution of grayscale information. Therefore, by performing the above DCT, the image data has a direct current component (DC) and an alternating current component (A).
C), and data indicating the value of the direct current component (DC) is stored at the origin position (0, 0 position) on the 8 × 8 matrix. Then, data indicating the maximum frequency value of the AC component (AC) in the horizontal axis is at the 0,7 position, and data indicating the maximum frequency value of the AC component (AC) in the vertical axis is at the 7,0 position. , And the AC component (A
Data indicating the maximum frequency value of C) is stored respectively. Further, at the intermediate position, the frequency data in the direction associated with each coordinate position is stored in such a manner that frequencies having higher frequencies appear from the origin side.

Next, the stored data at each coordinate position in this matrix is divided by the quantization width for each frequency component to perform linear quantization according to each frequency component, and this quantization is performed. Huffman coding is performed as variable-length coding on the value. At this time, with respect to the DC component (DC), the difference value from the DC component of the neighboring block is Huffman-encoded. Regarding the AC component (AC), a scan from a low frequency component called a zigzag scan to a high frequency component is performed, and a continuous number (zero run number) of invalid (value “0”) components, 2 of the effective component value that follows
Dimensional Huffman coding is performed to obtain data.

[0015]

In this method, the compression rate is generally controlled by changing the quantization width of the above quantization, and the higher the compression rate, the more
The quantization width becomes large. Therefore, the quantization error becomes large, and the image quality deterioration of the reproduced image becomes noticeable.

The quantization error of the transform coefficient tends to appear as so-called block distortion, in which discontinuity occurs at a block boundary portion in a reproduced image, or mosquito noise in which a haze-like object is seen in a flat portion near an edge. is there. Since these distortions are visually conspicuous, even if the S / N is good, the subjective impression is deteriorated.

Therefore, a method of applying a low-pass filter (low-pass filter) to the image reproduced by the decoder as distortion removal processing has been devised. This filter can remove distortion relatively well, but if an image contains edges, etc., they will be blurred, and conversely, the degree of low frequencies that pass will be reduced to reduce blur. If it is made loose, there is a problem that the block distortion cannot be completely removed.

In order to solve this problem, the presence or absence of edges in the image and distortion are detected, and whether to apply the filter or not is switched according to the result, and only the portion where the distortion exists is filtered. There is also.

However, the distortion removal method as shown in the conventional example still has a drawback that an image is blurred, and since it is necessary to calculate the block distortion amount and the like, a long processing time is required and therefore the circuit is The size and power consumption of the device had become very large. Therefore, it has been difficult to apply the above-described method to products that place importance on downsizing and high speed, particularly those that handle moving images.

The present invention has been made in view of the above problems, and it is an object of the present invention to provide an image signal decoding apparatus capable of removing distortion at high speed without causing blurring in an image with a simple circuit. To aim.

[0021]

That is, the present invention provides an image signal decoding apparatus for decoding moving image data compressed using orthogonal transformation, quantization and variable length coding for each block. Distortion removal processing means for removing block distortion from the image data obtained by the conversion, and the band of the signal for each block is obtained from the transform coefficient, and the result and the motion vector and block type information for each block are obtained. And a determination means for controlling the distortion removal processing means by determining the distortion removal processing for changing the distortion removal characteristics for each block based on the distortion removal processing.

[0022]

According to the present invention, it is obtained by decoding in an image signal decoding apparatus for decoding compressed moving image data by performing orthogonal transformation, quantization and variable length coding for each block. A distortion removal processing unit that removes block distortion for the result is provided. The distortion removal processing is performed by obtaining the signal band for each block from the transform coefficient, and changing the distortion removal characteristics for each block based on the result and the motion vector and block type information for each block. ..

[0023]

First, the concept of the present invention will be described.

The degree of conspicuousness of image distortion varies depending on the spatial frequency of nearby images. For example, when block distortion occurs in a portion having a fine structure and having a component up to a high spatial frequency, the block distortion is not so noticeable. On the contrary, when the block distortion is generated in the portion having only the low spatial frequency component whose change is relatively gradual, the block distortion becomes conspicuous.

On the other hand, since the block distortion is due to the discontinuity at the block boundary, it has a component up to a very high spatial frequency. Therefore, the block distortion can be made inconspicuous by removing the spatial frequency component higher than the spatial frequency of the image near the distortion.

Next, paying attention to the movement, in the moving portion, the degree of conspicuousness with respect to the blur of the image becomes less noticeable as the movement increases. Therefore, the frequency band, which may be excluded in terms of image quality, changes depending on the magnitude of the movement, and even for a block having a fine structure, for example, even for a block that is moving violently, , I try to do strong low-pass filtering.

Further, the mosquito noise has the characteristic that it is most noticeable when the movement is relatively slow. Therefore, when it is determined that the mosquito noise is conspicuous from the amount of movement, the characteristics of the distortion removal process are set to match those of the mosquito noise removal.

On the other hand, in a block having a large amount of motion, the motion vector cannot be detected and the motion cannot be compensated.
Intra coding may be selected. Therefore,
Although a motion vector is used as a means for knowing the amount of motion, the type of each block can also be used as information for determining the distortion removal characteristics. In the present invention, the distortion removal characteristic is adaptively changed as described above by using the transform coefficient, the motion vector for each block, the block type information, and the like.

Now, if the orthogonal transform coefficient of the block of interest is a DCT coefficient of 8 pixels × 8 pixels and the absolute value of each coefficient is compared with a threshold value, in a general block, a coefficient corresponding to a frequency lower than a certain frequency is significant. Judged as a coefficient. In this case, it can be seen that this block includes a structure that can be almost completely represented by the information up to the frequency of the significant coefficient in the horizontal direction and the vertical direction. Therefore, the filter characteristic for this block should be such that frequencies higher than the significant coefficient are cut in both the horizontal and vertical directions.

However, in the case of performing filtering in the spatial frequency plane, it is impossible to perform processing while changing the characteristics of the filter. Therefore, it is necessary to block the image, filter in the spatial frequency plane, and synthesize after inverse transformation. .. Then, there is a problem that the influence of blocking must be taken into consideration at that time.
Therefore, this filtering is realized by convolution in the real space, and the convolved coefficient is changed for each block.

Since the kernel size of this filtering is finite, it is impossible to obtain an ideal sharp cutoff characteristic, but there is no problem in practical use, and the filter coefficient and the kernel size can be arbitrarily determined, and the calculation It is determined in consideration of time and cutoff characteristics.

Therefore, a convolutional low-pass filter that preserves the band of the significant data of the transform coefficient obtained by decoding the compressed data in this way is adaptively applied to each block. The distortion can be removed without blurring the structure. In addition, since the characteristics of the filter are determined by using the transform coefficient that is the intermediate result of the decoding process, it is not necessary to detect the presence or absence of an edge or block distortion in the image, and it can be realized with a very simple circuit configuration. Moreover, since the processing content is simply compared with the threshold value, the time required for the processing can be shortened. Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the structure of an image data decoding apparatus to which the image signal decoding apparatus according to the present invention is applied. In the figure, 11 is a variable length code decoding circuit, which decodes code data for each block of an image. Then, it is output to the addition circuit 14 via the inverse quantization circuit 12 and the inverse DCT circuit 13. The output of the adder circuit 14 is added to itself via the motion compensation prediction circuit 15 and also the image output unit 17 through the distortion removal processing circuit 16.
Is supplied to. Further, the determination circuit 18 is the inverse quantization circuit 1
It receives the output of 2 and determines the optimum distortion removal processing, and controls the distortion removal processing circuit 16.

In the thus constructed decoding device, the code data for each block of the image is decoded by the variable length code decoding circuit 11, and the inverse quantization circuit 12 and the inverse DCT are used.
The circuit 13 restores the interframe difference signal. Then, this inter-frame difference signal is added to the prediction signal from the motion compensation prediction circuit 15 in the addition circuit 14. The result is output to the motion compensation prediction circuit 15 to obtain the prediction signal of the next frame, and the distortion removal processing circuit 1
After performing the optimum distortion removal processing in 6, the image output unit 17
Is output to.

The motion compensation prediction circuit 15 outputs a motion compensation prediction signal for the next frame based on the reproduced image signal and the motion vector and block type information sent together with the image code data. In addition, the determination circuit 18 obtains the frequency band of each block in both the vertical and horizontal directions from the output of the inverse quantization circuit 12, determines the optimum distortion removal processing based on the result, the motion vector and the block type, and performs the distortion removal processing. It controls the circuit 16.

In this way, the band of information recorded in block units can be filtered with almost no loss. In other words, strong low-pass filtering that averages over a wide range is performed on blocks with only low spatial frequency components, and conversely, weak low-frequency filtering that blocks relatively high spatial frequency components does not blur too much. By performing the low-pass filtering, it is possible to realize the low-pass filtering to the extent that the structure in the block is not blurred.

That is, the optimum filter is selected for each pixel of interest from the convolution filters having different cutoff frequency characteristics. The characteristics of the filter are such that the coefficient and size of the filter are changed, and in the case of strong low-pass filtering, the coefficient is an average of pixel values in a relatively wide range around the pixel of interest, and the filter size is large. I am trying. On the other hand, in the case of weak low-pass filtering, the coefficient is an average of pixel values in a relatively narrow range around the target pixel, and the filter size is also small.

Further, such filtering can be performed on a block for which stronger low-pass filtering or mosquito noise removal filtering is determined to be appropriate according to the motion, so that distortion that is more noticeable depending on the amount of motion Can be effectively removed, and deterioration such as blurring does not occur in the image.

FIG. 2 is a block diagram showing an example of the judgment circuit 18 shown in FIG. In the figure, first, the output from the inverse quantization circuit 12 enters the absolute value circuit 20 from the input terminal 19 and the absolute value is calculated. The calculation result is compared with a predetermined threshold value in the threshold value comparison circuit 21, and is divided into a significant coefficient and a non-significant coefficient. The result is
The maximum frequency in the horizontal direction is supplied to the maximum frequency determination circuit 22 and the maximum frequency in the vertical direction determination circuit 23, the maximum frequency in each direction is obtained, and the maximum frequencies are supplied to the horizontal filter determination circuit 24 and the vertical filter determination circuit 25. Thus, the kernel size and coefficient of the filter in each direction are determined. At this time, the motion vector and block type information are also taken into consideration. Then, these pieces of information are output from the output terminals 26 and 27 to the distortion removal processing circuit 16.

Then, the respective maximum frequencies in the horizontal and vertical directions are supplied to the horizontal direction filter determination circuit 24 and the vertical direction filter determination circuit 25, and the kernel size and coefficient of the filter in each direction are determined. At this time, the motion vector and block type information are also taken into consideration. The determination of the highest frequency and the determination of the filter will be described below by taking the case of the horizontal direction as an example.

The data of the block is zigzag scanned and given by the threshold comparison circuit 21 in the order shown in FIG. Therefore, in the horizontal direction maximum frequency determination circuit 22, the number of the coefficient of the incoming significant coefficient is checked depending on the coefficient of the column, and the maximum column number of all the significant coefficients is used. The highest horizontal frequency is required.

For example, when the first coefficient is a significant coefficient,
If “1” is output to the temporary memory and the 14th coefficient is the significant coefficient next, it can be seen from FIG. 3 that it is in the second row, fourth column. Therefore, the value in the temporary memory is compared with the column number 4, and the larger value is stored in the temporary memory. In this way, the values in the temporary memory are output when the determination is completed for all the significant coefficients in the block. Therefore, for example, even if the 20th coefficient after the 14th is a significant coefficient, since it is the coefficient in the second column, the output of the horizontal maximum frequency determination circuit 22 does not change.

After obtaining the highest frequency in this way, the horizontal filter determination circuit 24 selects the filter corresponding to the highest frequency. Here, if it is determined that the motion is large based on the motion vector and the block type, the selected filter is changed to a stronger low-pass filter. When it is determined that the movement is slow, the distortion removal processing circuit 16 is instructed to perform the mosquito noise removal filtering after the low-pass filtering.

As for the type of low-pass filter, since this output can take values from "0" to "8", the horizontal direction filter determination circuit 24 only needs to prepare 9 types of filters. However, 4 to 5 or less are usually sufficient.

The filters are determined in the same manner in the vertical direction, and the order of the coefficients, the correspondence between the row number and the column number, and the filter characteristic are held in advance as a table. A sigma filter is used as the mosquito noise removal filter. Here, an example of the sigma filter used for mosquito noise removal will be described.

The difference between the target pixel and ± 2 pixels in the vicinity of the target pixel is calculated, and when the absolute value is smaller than the threshold value, the difference is weighted and added to the target pixel. With this filter, haze-like noise that appears in the flat portion is reduced.

It is also possible to obtain all the data for determining the filter characteristics and the reproduction data of each block before filtering and store it in the memory for filtering. Next, another embodiment of the present invention will be described.

When the compression rate is increased in the coding of the apparatus to which the present invention is applied, the quantization width increases and the probability that the coefficient is quantized to 0 increases. In particular, high frequency components generally have little power, so most of them are zero.
Tends to be quantized into. Therefore, in order to determine whether the conversion coefficient is a significant coefficient, instead of comparing the absolute value of each coefficient with a threshold value, the method of determining whether the value of each coefficient is zero is also effective. is there. In this case, the input to the decision circuit 18 of FIG.
You will be able to use the one from.

As still another embodiment, when there are consecutive blocks in which only significant frequency components or very few significant coefficients are continuous, they are collectively regarded as a macro block, and this macro block is selected. There is one that applies a strong low-pass filter in a wide range.

This is because, for example, a portion where the gradation is changing very slowly, such as the sky or a white wall in the image, is highly compressed and all the AC components are lost, resulting in a staircase. In addition, even if distortion removal processing is performed by filtering in block units, the distortion is not completely removed from a macro perspective, and when this is observed as an image, the edges still remain due to the human visual characteristics. It seems to exist. Therefore, if a block of several blocks or tens of blocks in which almost no AC component is lost is regarded as a macro block and smoothing is performed in a large range within this macro block, a smooth gradation can be obtained.

The present invention is not limited to the block size, the type of orthogonal transform, the type of variable length coding, etc. used in the above-mentioned embodiment. Further, the filters are separately applied in the horizontal direction and the vertical direction, but a two-dimensional filter may be applied at once. Furthermore, instead of applying to the entire block, the filtering may be applied only to the vicinity of the block boundary.

[0052]

As described above, according to the present invention, it is possible to provide an image signal decoding apparatus capable of removing distortion at high speed without causing blurring in an image with a simple circuit. The cost and size of the applied device can be reduced, and the device can be used for a digital electronic camera with a moving image reproducing function. Further, even with respect to the standard compression method, the effect can be improved only by devising the decoding device, of course, the conventional reproduction can be performed, and the degree of distortion removal can be freely set.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image data decoding device to which an image signal decoding device according to the present invention is applied.

FIG. 2 is a block configuration diagram showing an example of a determination circuit 18 of FIG.

FIG. 3 is a diagram showing data of a block given by a threshold comparison circuit 21 of FIG.

FIG. 4 is a block diagram showing a conventional moving image compression method using conventional inter-frame correlation.

FIG. 5 is a diagram for explaining the operation of compressing a still image by a conventional coding method using DCT.

[Explanation of symbols]

1 ... Difference circuit, 2 ... DCT circuit, 3 ... Quantization circuit, 4 ...
Encoding circuit, 5, 12 ... Inverse quantization circuit, 6, 13 ... Inverse D
CT circuit, 7, 14 ... Addition circuit, 8, 15 ... Motion compensation prediction circuit, 11 ... Variable length code decoding circuit, 16 ... Distortion removal processing circuit, 17 ... Image output section, 18 ... Judgment circuit.

Claims (1)

[Claims] 1. An image signal decoding device for decoding moving image data compressed using orthogonal transformation, quantization and variable length coding for each block, with respect to image data obtained by the above decoding. Distortion removal processing means for removing block distortion, and the signal band for each block is obtained from the transform coefficient, and the distortion removal characteristics are determined for each block based on the result and the motion vector and block type information for each block. The image signal decoding apparatus according to claim 1, further comprising: a determination unit that determines the distortion removal processing to be changed to and controls the distortion removal processing unit.