JPH10336489A - Image expansion device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the image expansion device that is configured such that a capacity of a block data memory is reduced to eliminate noise specific to digital compression and filter processing for noise elimination is conducted simultaneously by having only to conduct conventional expansion processing from the standpoint of a decoder. SOLUTION: The memory capacity is reduced by using a memory storing noise elimination block data in common for a memory used by a decoder circuit and the hardware is configured such that a filter circuit for noise elimination is regarded as a mere memory circuit from the standpoint of the decoder circuit. Specifically, the device is made up of the decoder circuit 31, an expansion arithmetic memory circuit 32 and a filter circuit 33 that is inserted between the circuits 31, 32 and intervenes input and output of a data signal.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image decompression device provided with a noise removal device for removing noise (block noise and mosquito noise) peculiar to digital compression when decompressing digitally compressed image data.

[0002]

BACKGROUND OF THE INVENTION Over the past few years, MPEG-1, MP
EG-2, JPEG, H.E. Many image compression methods have been proposed and standardized for transmitting digital images, such as H.261. Most of these compression standards have effectively lost a lot of data to achieve high compression ratios. When the compressed digital image data is reconstructed (expanded) into the original digital image due to the lack of information due to the data loss, specific noise has occurred. Generally, this noise includes block noise that makes the screen look like a tile on a screen with a high compression ratio or a screen with a lot of movement, and mosquito (ringing) noise that is generated in a wavy shape around characters and tree branches. is there.
Post-processing on the decompressed digital image is then required to reduce or eliminate these noises.

FIG. 4 shows a basic configuration example of a conventional digital image compression expansion circuit, that is, a decoder circuit. As shown in the figure, a general decoder circuit includes a decoder LSI1
1 and its working memory (DRAM) 12. In order to remove block noise and mosquito noise peculiar to the digitally compressed image included in the image data decoded (expanded) by the decoder circuit, a noise removing device 13 as shown in FIG. 2 is provided. The noise removing device 13 is supplied with the decoded image data output from the decoder LSI 11, and for example,
It comprises a smoothing filter circuit 14 and a working memory 15 acting as a buffer. Then, the smoothing filter circuit 14 obtains a difference between adjacent pixels of the decoded image data, determines block noise based on the magnitude of the difference, and performs smoothing filtering to remove the block noise.

The image decompression apparatus having such a configuration has a complicated circuit configuration such as providing a working memory 15 unique to the smoothing filter circuit 14 or requiring a complex circuit based on FIR or IIR as the smoothing filter circuit 14. Had become. In addition, even when there is a difference in the image from the beginning, the image quality may be degraded due to the smoothing process, and there is a problem in the filter performance. Therefore, in order to avoid this, a "linking noise elimination device for digital images" disclosed in JP-A-8-204999 receives block data of a decoded image directly from a decoder circuit, and The state of the pixel is accurately grasped, and the filter processing is adaptively performed.

A "digital image linking noise elimination apparatus" disclosed in Japanese Patent Application Laid-Open No. 8-204999.
Is shown in FIG. 6 and will be briefly described below. The apparatus for removing ringing noise of a digital image shown in FIG. 1 includes an image decoder 1 for restoring compressed image data, and a ringing noise detector 2 for detecting ringing in the restored image.
A state memory 3 for storing a detection state of the ringing noise detector 2, a reorder memory 4 for converting pixel data in a block format into a raster scan, and acting as a storage circuit for another decoder system. A ringing noise filter 5 for classifying pixels to be filtered and pixels not to be filtered and removing ringing noise only for pixels to be filtered, and interpolation / post-processing for post-processing the reconstructed digital image And a processing device 6.

When the compressed image data 40 is input to the image decoder 1 in order to reconstruct the original image, it is expanded and output as reconstructed image pixel data 41. The reconstructed image pixel data 41 output from the image decoder 1 is in a block format (for example, of 8 × 8 pixels). The output of the reconstructed image pixel data 41 is output to the reorder memory 4 and the ringing noise detector 2. The reorder memory 4 rearranges the frames of the image decoder 1 and converts the data in the block format into a raster scan format. Then, when the pixel data included in the block format of the reorder memory 4 is converted into the raster scanning format, the pixel data is sent to the ringing noise filter 5.

From each block of the reconstructed image pixel data 41 supplied to the ringing noise detector 2,
High frequency regions, which are potential sources of ringing noise, are detected. This detected frequency level depends on the detection threshold that can be adjusted. Next, the detected block detection state is stored in the state memory 3. This detection state is used for smoothing control. Then, the signal supplied from the reorder memory 4 and the state memory 3 is filtered by the ringing noise filter 5, and reproduced image data is output via the interpolation / post-processing device 6.

[0008]

However, Japanese Patent Application Laid-open No.
The apparatus disclosed in JP-A-204999 has a problem that a raster scan conversion process and a memory for performing the raster scan conversion process are required to receive a decoder output in a block format, thereby increasing a circuit scale. Also, for image data output from the image decoder,
Since the noise removal processing is performed in the portion indicated by reference numeral 7 in FIG. 6, the image output is delayed by the time required for the noise processing, and thus the display of the decoded image is delayed.

Therefore, in the present invention, in the conventional configuration shown in FIG. 4, a block data memory for removing noise peculiar to digital compression is reduced, and ordinary decompression processing is performed when viewed from a decoder. It is an object of the present invention to provide an image decompression device configured so that filter processing for removing noise is performed at the same time.

[0010]

As means for achieving the above object, a decoder circuit for expanding a digitally compressed image and a memory for temporarily storing data for performing the expanding process by the decoder circuit are provided. An image decompression device, which is provided between the input and output circuits, performs a noise process on the image data from the supplied image data and coding information, and includes a filter circuit that outputs the image data to the decoder circuit.
Alternatively, there is provided an image decompression device comprising a decoder circuit for decompressing a digitally compressed image, a memory circuit for temporarily storing data for decompression processing by the decoder circuit, and a filter circuit for noise processing. The filter circuit includes: a control unit that inputs and outputs data to and from the memory circuit and the decoder circuit; and a Difficulty operation & that calculates a state coefficient representing an image state of each macroblock.
Block activity, Max for calculating a memory unit, and a state coefficient representing a state of each block of luminance data and color difference data.
An arithmetic and memory unit, a state of each macro block supplied from the difficult arithmetic and memory unit, and the block activation
an Alpha operation and memory unit for calculating a block filter coefficient calculated from a block state of luminance data and color difference data supplied from the ty, Max operation and memory unit;
ha NR operation unit for performing noise removal and enhancement processing of pixels in the horizontal direction by block filter coefficients supplied from the operation and memory unit, and lines of pixels subjected to noise removal processing in the horizontal direction for each luminance data and color difference data A line memory unit that accumulates data in units, and a vertical NR operation unit that performs noise removal and enhancement processing by predicting vertical pixel noise based on the block filter coefficient supplied from the Alpha operation and memory unit. It is intended to provide an image decompression device characterized by the following.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention aims to reduce the number of memories by sharing a memory for storing block data for noise removal with a memory used by a decoder circuit. This is an image decompression device that is configured by hardware so that a filter circuit for noise removal can be regarded as a mere memory circuit, thereby enabling noise removal without using a special interface.

An embodiment of an image decompression device according to the present invention will be described with reference to the drawings. As shown in FIG. 1, the image decompression device of the present invention includes a decompression circuit (decoder circuit) 31 and a decompression operation memory circuit (DRAM) 32, which are interposed between the decompression circuit 31 and the data signal input / output. And a filter circuit 33 that performs the operation. When actually configured by hardware, as shown in FIG. 2, the expansion operation memory circuit 32 and the filter circuit 33 may be used in an integrated manner. Is equivalent to that of FIG.

Since the decoder circuit 31 is constituted by hardware, data necessary for the filter circuit 33 such as decode information from the supplied bit stream and decoded image data to be output is determined according to a predetermined rule. The data is written to a fixed address of the memory circuit 32 based on this. Therefore, the filter circuit 33 can determine the type of data currently passing through the filter circuit 33 by detecting the fixed address indicated by the address control signal output from the decoder circuit 31 to the memory circuit 32. it can.

That is, of the data stored in the memory circuit 32 at the time of decoding the image data, a fixed address where necessary data is stored is searched, and only the necessary data is supplied to the filter circuit 33. Can be stored in the internal memory of the filter circuit 33. At this time, since there is no need to store data other than data directly necessary for the filter processing in the internal memory of the filter circuit 33, it is not necessary to incorporate a large-capacity memory in the filter circuit 33.

Further, by inserting the filter circuit 33 between the decoder circuit 31 and the memory circuit 32 as described above, the filter processing can be performed during the decoding of the image data, and the delay of the image display due to the noise removal processing can be achieved. Is almost gone.

Here, a configuration example of the filter circuit 33 is shown in FIG.
Shown in The filter circuit 33 shown in FIG.
2 and an interface unit (control unit) 51 for inputting and outputting data to and from the decoder circuit 31; a Difficulty operation & memory unit 52 comprising an operation circuit for calculating a state coefficient representing an image state for each macroblock and its storage memory , An arithmetic circuit for calculating a state coefficient representing the state of each block of the luminance data and the color difference data, and its storage memory
Block operation, Max operation & memory unit 53, Alpha operation & memory unit comprising an operation circuit for calculating a block filter coefficient calculated from the state of each macroblock and the state of the block of luminance data and color difference data, and its storage memory 54, a horizontal NR calculation unit 55 that performs noise removal and enhancement processing of horizontal pixels, and a line memory unit 56 that accumulates pixels subjected to horizontal noise removal (NR) processing in units of lines for each of luminance data and color difference data And a vertical direction NR calculation unit 57 that predicts noise of pixels in the vertical direction and performs noise removal and enhancement processing.

The filter circuit 3 having such a configuration
In 3, the noise is predicted for each block from the image data and the coding information supplied from the memory circuit 32 and the decoder circuit 31, and then the noise is predicted for each pixel. Further, the noise is adaptively removed from the two prediction noises and the image data using the least squares method for the luminance data and the color difference data.

The image data and encoded information supplied from the memory circuit 32 and the decoder circuit 31 are first sent to the control unit 51.
Supplied to The control unit 51 outputs the quantized value (Q scale) data included in the supplied encoding information to the Difficulty operation & memory unit 52, and outputs the block data included in the encoding information to the Block Activity, Max operation & memory unit. Output to 53.

In the Difficulty operation & memory unit 52, a state coefficient (Diff) representing the state of an image for each macroblock (16 pixels × 16 pixels) is obtained from the supplied Q scale data.
iculty). That is, although DCT (discrete cosine transform) data is quantized in macroblock units, the Q scale is not always given as an appropriate value in relation to the total code amount, so that the Q scale of the surrounding macroblocks is used. By increasing the Q scale in accordance with the surrounding when the surrounding Q scale is large,
Seeking Difficulty. When the surrounding Q scale is small, the Q scale is used as it is.

In the Block Activity, Max calculation & memory unit 53, each block of luminance data and color difference data (8
A state coefficient representing a state for each pixel (8 pixels) is obtained. Here, the variation of the pixels representing the difficulty (dirty) of the image of each block is defined as Block Activity, and the maximum value of the difference between the pixel and the pixel average is defined as Block Max, which is obtained for each block. Difficulty, Bloc obtained in this way
k Activity and Block Max are stored in the Alpha calculation & memory unit 54
To calculate the block filter coefficient (Alpha). The block filter coefficient output from the Alpha calculation and memory unit 54 is calculated by the horizontal NR calculation unit 5.
5 and the vertical direction NR calculation unit 57.

The horizontal direction NR calculation unit 55 extracts a low-pass component of a pixel from the luminance data and color difference data of the pixel supplied from the control unit 51 by a 5-tap low-pass filter while shifting the pixel by one pixel in the horizontal direction. A high frequency component containing a lot of noise is removed by performing the processing described above, and the data of the high frequency component of the pixel is also extracted, and the noise is removed from the high frequency component. This noise removal processing predicts noise for a high-frequency component of a pixel from a block filter coefficient (Alpha) indicating a state of a block in which the pixel exists and a state of pixels surrounding the pixel. Perform noise removal and enhancement processing adaptively,
This is to obtain a high-frequency component from which noise has been removed. And
By combining the low-frequency component extracted by the low-pass filter with the high-frequency component, it is possible to re-synthesize a pixel from which horizontal noise has been removed.

The pixels processed by the horizontal NR calculation unit 55 are subjected to a low-pass filter process of 5 taps in the vertical direction by the vertical NR calculation unit 57. Stored in line units.

The vertical NR calculator 57 performs the same processing as the horizontal NR calculator 55 on the luminance data and color difference data of the pixels input from the line memory 56. That is, a process of extracting low-frequency components of pixels using a 5-tap low-pass filter while shifting each pixel in the vertical direction with respect to the luminance data and the color difference data of the pixels supplied from the line memory unit 56 is performed to increase noise. In addition to removing the high-frequency component included, the data of the high-frequency component of the pixel is also extracted, and noise is removed from the high-frequency component. This noise removal processing predicts noise for a high-frequency component of a pixel from a block filter coefficient (Alpha) indicating a state of a block in which the pixel exists and a state of pixels surrounding the pixel. This is to obtain a high-frequency component from which noise has been removed by adaptively performing noise removal and enhancement processing. Then, by synthesizing the low-frequency component extracted by the low-pass filter and the high-frequency component, it is possible to re-synthesize the pixel from which noise in the vertical direction has been removed.

The pixel data output from the vertical NR calculation unit 57 is output to the decoder circuit 31 via the control unit 51.

[0025]

In the image decompression apparatus according to the first aspect of the present invention, since the decoding process is performed by the decoding circuit and the noise removal process is also performed, the display delay of the image is almost eliminated. Then, the filter circuit obtains image coding information in a bit stream other than the decoded pixel data, such as the Q scale obtained during decoding, and does not need to buffer it. Since there is no need for buffering, it is only necessary to store only the image information obtained in block units, and the capacity of the internal memory can be greatly reduced.

In addition, the decoder circuit needs to read data from the memory circuit line by line when performing image data output processing. In the present invention, however, the decoder circuit performs noise removal processing in the image data output processing stage. Since they are performed at the same time, there is no need to perform block-raster conversion as in the related art, and there is an effect that horizontal and vertical noise removal processing can be performed in line units.

Further, the image decompression device according to the second aspect of the present invention can detect the image state of a macroblock or the image state of a block, and performs processing one pixel at a time in the horizontal and vertical directions. Thus, it is possible to perform detailed noise prediction and noise removal processing. As a result, there is an effect that only block noise and mosquito noise peculiar to digital compression, which could not be removed without a more complicated circuit in the past, can be beautifully removed.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing an embodiment of an image decompression device of the present invention.

FIG. 2 is a configuration diagram showing another embodiment of the image decompression device of the present invention.

FIG. 3 is a configuration diagram showing an example of a filter circuit of the image decompression device of the present invention.

FIG. 4 is a configuration diagram illustrating a basic configuration example of an image decompression device.

FIG. 5 is a configuration diagram illustrating an example of a conventional image decompression device.

FIG. 6 is a configuration diagram showing an example of a conventional image decompression device.

[Explanation of symbols]

 Reference Signs List 31 decoder circuit (decompression circuit) 32 memory circuit (DRAM) 33 filter circuit 51 control section (interface section) 52 Difficulty calculation & memory section 53 Block Activity, Max calculation & memory section 54 Alpha calculation & memory section 55 horizontal NR calculation section 56 line memory unit 57 vertical NR operation unit

Claims (2)

[Claims] An image data inserted and supplied between a decoder circuit for expanding a digitally compressed image and a memory circuit for temporarily storing data for performing the expanding process by the decoder circuit. An image decompressing apparatus, comprising: a filter circuit that performs noise processing on the image data from encoded information and outputs the result to the decoder circuit. 2. An image decompression device comprising a decoder circuit for decompressing a digitally compressed image, a memory circuit for temporarily storing data for decompression processing by the decoder circuit, and a filter circuit for noise processing. A control unit that inputs and outputs data to and from the memory circuit and the decoder circuit; a Difficulty operation & memory unit that calculates a state coefficient representing a state of an image for each macroblock; A Block Activity, Max operation & memory unit for calculating a state coefficient representing a state of each block of data and color difference data, a state of each macro block supplied from the Difficulty operation & memory unit, and the Block Activity, Max operation & Alpha operation & calculates the block filter coefficient calculated from the state of the block of the luminance data and the color difference data supplied from the memory unit. A memory section; a horizontal NR operation section for performing noise removal and enhancement processing of pixels in the horizontal direction by using a block filter coefficient supplied from the Alpha operation and memory section; A line memory unit that accumulates line-by-line data for each color difference data, and a vertical NR operation unit that predicts noise of pixels in the vertical direction based on a block filter coefficient supplied from the Alpha operation and memory unit and performs noise removal and enhancement processing An image decompression device characterized by comprising: