JPH0522709A - Image signal decoder - Google Patents

Abstract

PURPOSE:To remove block distortion without the occurrence of an out-of-focus image, etc. CONSTITUTION:Output data of a reverse quantization circuit 12 is divided into two and one of them obtains the image signal of actual space in a reverse orthogonal transformation circuit 13. An absolute value in the space frequency component of the other one is compared with a threshold by a coefficient decision circuit 14 at every block so that information of a coefficient which is larger than the threshold is transmitted to a distortion removing characteristic deciding circuit 17 as the singnificant coefficient. The output image signal of the reverse orthogonal transformation circuit 13 is decomposed so as to be a sub-block in a sub-blocking circuit 15 and, then, executed an optimum distortion removing processing at every sub-block. That is, distortion removal is processed by low pass characteristic which is decided by a distortion removing characteristic deciding circuit 17 and which preserve a significant coefficient band in the block of a compression unit, which is obtained by the coefficient decision circuit 14 when the distortion removing processing is execute by the sub-block unit.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal decoding apparatus for decoding an image transmitted or recorded after being highly compression coded, and more particularly to improvement of block distortion removal processing.

[0002]

2. Description of the Related Art When an image signal picked up by a solid-state image pickup device typified by a CCD (charge coupled device) is recorded as digital data in a storage device such as a memory card, a magnetic disk, or a magnetic tape, The amount of data will be enormous. Therefore, in order to record in a limited storage capacity range, it is necessary to perform some high-efficiency compression on the obtained image signal data.

As such a highly efficient image data compression method, an encoding method utilizing orthogonal transform encoding for image data divided into blocks of a predetermined size is widely known.

However, according to this method, a coefficient obtained by performing two-dimensional DCT (discrete cosine transform) or the like as orthogonal transformation for each block is linearly quantized according to each frequency component, and variable length coding is performed. Since the result is transmitted or recorded, so-called block distortion occurs in which the quantization error, that is, the degree of image quality deterioration differs for each block, and the block boundary becomes conspicuous.

This situation will be described in more detail below. By performing the DCT, the image data is converted into a DC component DC and an AC component AC. At this time, for example, assuming that each pixel is divided into 8 × 8 pixels, data indicating the value of the DC component DC is stored at the origin position ((0,0) position) on the matrix, and the (0,7) position is stored. Is data showing the maximum frequency value of the AC component AC in the horizontal axis direction.
Data indicating the maximum frequency value of the AC component AC in the vertical axis direction is stored in the (7,0) position, and further data indicating the maximum frequency value of the AC component AC in the oblique direction is stored in the (7,7) position. To be done. Then, at the intermediate position, the frequency data in the direction associated with each coordinate position is stored in such a manner that the frequency data having a higher frequency sequentially from the origin side appears.

Next, the stored data at each coordinate position in this matrix is divided by the quantization width for each frequency component to perform linear quantization according to each frequency component,
Huffman coding is performed on this quantized value as variable-length coding. At this time, for the DC component DC, the difference value with the DC component of the neighboring block is Huffman-encoded. Regarding the AC component AC, a scan from a low frequency component called a zigzag scan to a high frequency component is performed, and the number of consecutive invalid components (value “0”) (zero run number) is followed by the effective number. Two-dimensional Huffman coding is performed on the values of various components to obtain data.

In this method, the compression rate is generally controlled by changing the quantization width of the quantization. The higher the compression rate, the greater the quantization width, and thus the quantization. The error becomes large, and the deterioration of the image quality of the reproduced image becomes noticeable.

The quantization error of the transform coefficient appears as a so-called block distortion in which a discontinuity occurs at a block boundary portion in a reproduced image, and this block distortion is visually noticeable, so that S / N is good. Even so, the subjective impression gets worse.

Therefore, a method of applying a low-pass filter to the image reproduced by the decoder as a distortion removal process has been devised. This post-filter can remove the distortion relatively well, but if the image contains edges etc., they will be blurred, and conversely if the degree of low pass is loosened to reduce the blur. There was a problem that the block distortion could not be completely removed.

In view of such a point, for example, Japanese Unexamined Patent Publication No.
As disclosed in Japanese Patent No. 46482, there has been devised a method of determining whether the block distortion as described above has occurred and changing the degree of the distortion removal processing according to the degree of the distortion.

Further, in Japanese Laid-Open Patent Publication No. 1-311782, it is determined how much transform coefficients are to be transmitted, and the coefficients within the range are quantized and encoded and transmitted, and at the same time, the range transmitted is also information. There is also disclosed a method in which the degree of the post-filter is locally changed by utilizing the information on the decoder side by multiplexing and transmitting.

[0012]

However, in the method disclosed in Japanese Patent Laid-Open No. 3-46482 mentioned above,
When obtaining the degree of block distortion, it is difficult to distinguish between image information and distortion, and not only distortion but also image edge information may be lost.

Further, in the latter method disclosed in Japanese Patent Laid-Open No. 1-311782, a signal indicating the range to which the transform coefficient is sent is sent on the encoder side, and the degree of the filter is changed according to the signal. Therefore, there is a problem that the information is wasted because the signal indicating the range in which the conversion coefficient is sent must be transmitted, and therefore the compression rate cannot be increased so much.

The present invention has been made in view of the above points, and provides an image signal decoding apparatus capable of adaptively performing block distortion removal processing on a highly compressed image with a simple circuit. The purpose is to

[0015]

In order to achieve the above object, an image signal decoding apparatus according to the present invention divides image data into blocks, performs orthogonal transformation on each of the divided blocks, and then outputs the transformed output. Variable length code decoding means for receiving the compressed image data obtained by quantizing the compressed output and then compressing the quantized output by variable length coding, and the decoding output from the variable length code decoding means. Dequantizing means for inverse quantizing, inverse orthogonal transforming means for inverse orthogonal transforming the inverse quantized output from the inverse quantizing means, and the decoded image data of the transformed output from the inverse orthogonal transforming means Subblocking means for dividing into blocks, distortion removing means for performing distortion removal processing on the output of the subblocking means, and an evaluation value for each subblocking It is characterized by comprising a strain relief characterization means for changing the in distortion removal properties to said strain relief means the evaluation value.

[0016]

Function: That is, in general, the degree of conspicuousness of block distortion varies depending on the spatial frequency of an image in the vicinity. That is, when the block distortion occurs in a portion having a fine structure and having a component up to a high spatial frequency, the block distortion is not so noticeable. On the contrary, when the block distortion is generated in the portion having only the low spatial frequency component that changes relatively gently, the block distortion becomes conspicuous.

On the other hand, block distortion has a component up to a very high spatial frequency because it is caused by discontinuity at the block boundary. Therefore, the block distortion can be made inconspicuous by removing the spatial frequency component higher than the spatial frequency of the image near the distortion.

Therefore, in the image signal decoding apparatus according to the present invention, the decoded image data is divided into sub-blocks,
The evaluation value is obtained for each sub-block. However, the evaluation value for each sub-block is obtained based on information such as to what extent the sub-block includes frequency components, and how much the sub-block affects the block distortion. Then, the characteristic of distortion removal is adaptively changed based on the evaluation value. In other words, strong low-pass filtering that averages over a wide range is performed for sub-blocks that have only low spatial frequency components, so that filtering is performed so that the band of information recorded in block units is hardly lost. By performing a weak low-pass filtering that does not blur the sub-block that includes a relatively high spatial frequency component, the block distortion can be removed without causing blurring in the image.

[0019]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of an image signal decoding apparatus according to the first embodiment of the present invention. Image data is divided into blocks, orthogonal transformation is performed for each divided block, the transformed output is quantized, and then the quantized output is variable length coded and compressed, and then transmitted or recorded. The compressed image data is given to the variable length code decoding circuit 11. The variable length code decoding circuit 11 decodes the given variable length code of the compressed image data and sends it to the inverse quantization circuit 12 for inverse quantization. This dequantization result is a transform coefficient for each block, which is a compression unit, and thus corresponds to a spatial frequency component in the block.

The output data of the inverse quantization circuit 12 is divided into two, and one of them is sent to the inverse orthogonal transformation circuit 13 to obtain an image signal in the real space. The other one is sent to the coefficient determination circuit 14. The coefficient determination circuit 14 compares the absolute value of the spatial frequency component with a threshold value for each block, sets a coefficient larger than the threshold value as a significant coefficient, and sends the information to the distortion removal characteristic determination circuit 17.

On the other hand, the image signal obtained by the inverse orthogonal transform circuit 13 is sent to the sub-blocking circuit 15 and decomposed into sub-blocks, and then sent to the distortion removing circuit 16 to optimally remove distortion for each sub-block. It is processed. The distortion removal processing in the distortion removal circuit 16 is a low-pass characteristic that preserves the band of significant coefficients of the block of the compression unit obtained by the coefficient determination circuit 14 when the distortion removal processing is performed in sub-block units. Is determined by the distortion removal characteristic determination circuit 17, and this characteristic is used.

More specifically, the orthogonal transform coefficients of the three blocks given to the inverse transform circuit 13 are shown in FIG.
8 × 8 pixel DCT coefficients, and the first, second, and third blocks are arranged in order from the left, and the 4 × 4 portion 21 and the 3 × 3 portion 2 are shown by hatching.
It is assumed that the coefficient of the 2, 1 × 1 portion 23 is not zero. In the coefficient determination circuit 14, since the hatched portions 21, 22, and 23 shown in the figure are non-zero, they are determined to be significant coefficients. (By the way, regarding the determination method of this significant coefficient, When the compression rate is increased in the encoding where is applied, the quantization width increases,
In particular, since high frequency components are generally low in power, most of them are quantized to 0, and few remain as significant coefficients, so the method of determining non-zero as a significant coefficient is effective).

On the other hand, in the sub-blocking circuit 15, the distortion removing circuit 16 is applied to the image data divided into sub-blocks.
To perform convolution filtering. Here, for simplicity, considering only one-dimensional horizontal filtering as the distortion removal processing, the distortion removal circuit 16
As shown by the broken line in FIG. 2B, distortion removal processing is performed on the image signal divided into sub-blocks so as to preserve the band of the significant coefficient of each block. That is, since the A sub-block spans the first and second blocks, and the B sub-block spans the second and third blocks, respectively,
The distortion removal characteristic determination circuit 17 selects a filter that preserves the wider band of the blocks straddling the sub-blocks. Therefore, the filter is such that the band of the first block is stored in the case of the A sub-block and the band of the second block is stored in the case of the B sub-block. The reason why the block boundary in which block distortion exists and the sub-block boundary in which the distortion removal processing is performed are different is as follows.

FIG. 3A shows an example in which the block boundary where block distortion exists and the sub-block boundary are made equal. The block A on the left side has nothing, and is a block having only a DC component in terms of frequency. The shaded portion 24 of the block B on the right side is a shadow portion having a large contrast difference with other portions, and is a block having a strong edge.
It also includes high frequency components.

Therefore, if filtering is performed on all the pixels of the block A so as to preserve the frequency band of the block A, the pixels in the vicinity of the block B are also subjected to a strong low pass, so that the tap of the filter is performed. When the block B includes a shaded portion, a shadow portion is also formed in the block A. However, since the low pass is hardly applied to the pixels of the block B, the edge remains as it is. Therefore, after filtering,
As shown in (B), the distortion may occur such that the edges are doubled.

Such distortion can be prevented by making the block boundary where block distortion exists and the sub-block boundary for which distortion removal processing is performed different, as in the first embodiment.

FIG. 4 is a block configuration diagram of an image signal decoding apparatus according to the second embodiment of the present invention. The second embodiment enables filtering of transform coefficients in the frequency domain.

That is, in the image signal decoding apparatus of the second embodiment, first, the variable length code decoding circuit 41 receives the transmitted or recorded compressed image data and decodes the variable length code of the compressed image data. To do. The decoded output is given to the dequantization circuit 42 and dequantized. The inversely quantized result is divided into two and one is sent to the inverse orthogonal transform circuit 43. The inverse orthogonal transform circuit 13 performs an inverse orthogonal transform on the supplied inverse quantized output to obtain an image signal in the real space, sends it to the subblocking circuit 45, and decomposes it into subblocks. At this time, the block boundary is different from the sub-block boundary on which the distortion removal processing is performed.

The output of the sub-blocking circuit 45 is sent to the orthogonal transform circuit 46 and converted into orthogonal transform coefficients. The signal converted into the orthogonal transform coefficient is divided into two and one is sent to the distortion removing circuit 49. The other is converted to an absolute value by the absolute value circuit 47 and then sent to the distortion removal characteristic determination circuit 48.

The other output of the inverse quantization circuit 42 is similarly supplied to the absolute value circuit 4 in the distortion removal characteristic determining circuit 48.
It is converted into an absolute value by 4 and given. Therefore, the distortion removal characteristic determination circuit 48 compares the absolute values of both orthogonal transform coefficients, determines the distortion removal characteristic, and sends the information to the distortion removal circuit 49.

The distortion removing circuit 49 performs optimum distortion removing processing for each sub-block based on the above information. The distortion removal processing at this time is frequency filtering in the orthogonal transform coefficient plane. The method of determining the characteristics of the filter in the distortion removal characteristic determining circuit 48 is filtering so that the absolute value of the orthogonal transform coefficient for each sub-block has an energy that is substantially equal to that of the blocks of the sub-blocks.

This filter may be determined only by the output from the absolute value circuit 44 or by the output of the coefficient judging circuit as in the first embodiment. In these cases,
The output of the orthogonal transformation circuit 46 need only be sent to the distortion removal circuit 49, and the absolute value circuit 47 is therefore unnecessary. Then, after filtering, it is subjected to inverse orthogonal transform again by the inverse orthogonal transform circuit 50 and output by the output circuit 51.

Next, a third embodiment of the present invention will be described. In the third embodiment, the sub-block is made smaller than the block size at the time of compression, and the filter characteristic is changed depending on each position.

In FIG. 5, the solid line indicates a block boundary at the time of compression, and the broken line indicates a 4 × 4 sub-block boundary for distortion removal processing. Since the sub-blocks shown by hatching in the figure do not include block boundaries, they are portions that hardly affect the removal of block distortion due to high frequencies. Therefore, it may be wasteful to make the distortion removal characteristic of such a sub-block the same as that of other block boundaries.

Therefore, in the third embodiment, the degree of low-pass is weak except when it is determined in the distortion removal characteristic determination circuit that a very strong low-pass is applied to a sub-block that does not include a block boundary. I am trying to change it. Therefore, a strong low-pass is applied to a region having only low frequency components such as the sky, and different filters are applied to the region including the high frequency component in the vicinity of the block boundary and the other region.

Specifically, for example, in the case of the configuration of the first embodiment, the sub-blocking circuit 15 informs the distortion removal characteristic determining circuit 17 of information indicating which block the sub-block belongs to. It suffices to send information indicating whether the block boundary is included.

As described above with reference to the first to third embodiments, according to the present invention, a low-pass filter that preserves the band of significant data of transform coefficients obtained by decoding compressed data. By adaptively applying for each sub-block, it becomes possible to remove the distortion without blurring the structure in each block. Also, since the distortion removal characteristics are determined using only the data that is the intermediate result of the decoding process, it is not necessary to add information for distortion removal on the encoder side, and the presence or absence of edges in the image and block distortion can be determined. Naturally, no processing for detection is required, so that the circuit can be realized with a very simple configuration.

When the present invention is applied to a color image, the luminance signal and the color difference signal may be separately processed, or the luminance signal may be processed first and the color difference signal may have the distortion removal characteristic of the luminance signal. The processing may be determined using the result. Further, a preferable one of the distortion removal characteristics determined by both signals may be selected and the same processing may be performed on both signals. In addition, as an evaluation value that determines the degree of distortion removal processing, the code amount of each block, the number of significant coefficients of transform coefficients of each block, or the sequence of each significant coefficient of transform coefficients of each block is preset. You may make it use the sum of the defined weight.

Further, as described in detail in the above embodiment, the kernel size of the convolution filter is arbitrary, and all the data for determining the filter characteristics and the reproduction data of each block are obtained and stored in the memory prior to the filtering. It may be stored and filtered.

The present invention is not limited to the block size, the type of orthogonal transform, the type of variable length coding, etc. used in the above-mentioned embodiment. Further, the filter may be any one other than the convolution filter as long as it has the effect of removing distortion, such as filtering in the frequency plane.

[0042]

As described in detail above, according to the present invention,
It is possible to provide an image signal decoding device capable of adaptively performing block distortion removal processing on a highly compressed image with a simple circuit.

That is, the present invention relates to a compressed image decoding apparatus, which can adaptively remove block distortion by using only image information transmitted or recorded.
Moreover, since the circuit has a simple structure, the cost and size of the applied device can be reduced, and the device can be used not only for still images but also for digital electronic cameras with a function of reproducing moving images.

Further, according to the present invention, the encoding device may have the conventional configuration. That is, even with respect to the standard compression method, the effect can be improved only by devising the decoding device, the reproduction can be performed as usual, and the degree of distortion removal can be freely set.

[Brief description of drawings]

FIG. 1 is a block configuration diagram of an image signal decoding apparatus according to a first embodiment of the present invention.

FIG. 2A is a diagram for explaining the operation of a coefficient determination circuit, and FIG. 2B is a diagram for explaining the operation of a distortion removal characteristic determination circuit.

FIGS. 3A and 3B are diagrams showing states of blocks before and after filtering in a distortion removing circuit when a block boundary in which block distortion exists and a sub-block boundary are equal to each other.

FIG. 4 is a block configuration diagram of an image signal decoding apparatus according to a second embodiment of the present invention.

[Fig. 5] Fig. 5 is a diagram illustrating a relationship between block boundaries at the time of compression and sub-blocks for distortion removal processing for explaining the operation of the image signal decoding apparatus according to the third embodiment of the present invention.

[Explanation of symbols]

11, 41 ... Variable length code decoding circuit, 12, 42 ... Inverse quantization circuit, 13, 43, 50 ... Inverse orthogonal transform circuit, 14
... coefficient determination circuit, 15, 45 ... sub-blocking circuit,
16, 49 ... Distortion removal circuit, 17, 48 ... Distortion removal characteristic determination circuit, 18, 51 ... Output circuit, 44, 47 ... Absolute value circuit, 46 ... Orthogonal transformation circuit.

Claims (1)

Claim: What is claimed is: 1. Image data is divided into blocks, orthogonal transformation is performed for each divided block, the transformed output is quantized, and then the quantized output is variable-length coded. Variable-length code decoding means for receiving the compressed compressed image data and decoding the compressed image data; dequantizing means for dequantizing the decoded output from the variable-length code decoding means; and dequantizing Inverse orthogonal transform means for inverse orthogonal transforming the inverse quantized output from the means, subblocking means for dividing the decoded image data of the transformed output from the inverse orthogonal transform means into subblocks, output of the subblocking means Distortion removing means for performing the distortion removing process, and an evaluation value for each of the sub-blocking, and the distortion removing characteristic of the distortion removing means is determined by the evaluation value. An image signal decoding apparatus comprising: a distortion removal characteristic determining unit that changes.