JPH06327002A - Moving image encoding device - Google Patents

Abstract

PURPOSE:To prepare a differential picture between a preceding frame and the present frame after allowing data to pass through a filter at the time of decoding the compressed code of the preceding frame in order to encode a moving image with a high quality. CONSTITUTION:A picture signal inputted to a raster/block conversion circuit 21 is supplied through a difference circuit 22, data selector 23, DCT circuit 24, and encoder 25 to a code memory 26. The data of the code memory 26 are supplied through a decoder 27 and an IDCT circuit 28, converted by a block/raster conversion circuit 29, and the output of the block/raster conversion circuit 29 and the output of the decoder 27 are supplied to an AFIR 30. The output of the AFIR 30 and the output of a data selector 31 are inputted to an adding circuit 32, and the output of the adding circuit 32 is inputted to a frame memory 33 and a raster/block conversion circuit 34. A block scan signal converted by the raster/block conversion circuit 34 is inputted through a data selector 35 to the difference circuit 22.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing device using digital moving images.

[0002]

2. Description of the Related Art A conventional moving image compression encoding method is shown in FIG.
As shown in (1), a mainstream is a mixture of a frame for intra-frame coding and a frame for inter-frame differential coding (inter-frame coding frame). Interframe coding is a very effective compression method when the correlation between frames is high. That is, by inserting the intra-frame coded frame between the plurality of inter-frame coded frames, it is possible to improve the compression efficiency and to prevent the image disturbance from being minimized even if the image is lost.

A conventional moving image processing apparatus will be described with reference to FIG. In the figure, an image signal of a raster signal (not shown) output from an image input device is converted into a block scan signal by a raster / block conversion circuit 1. Since the process from the raster block conversion circuit 1 is different between the intra-frame coded frame and the inter-frame coded frame, the intra-frame coded frame will be described first.

The image signal converted into a block scan signal is input to a DCT (discrete cosine transform) circuit 5 via a data selector 4. DCT with this DCT circuit 5
The image data converted into the coefficient is output by the encoder 6.
Quantization and Huffman coding are performed by a preset quantization table and recorded in the code memory 7. The data of one screen recorded in the code memory 7 is read to an external storage device (hard disk, tape streamer, magneto-optical storage device, etc.) not shown, and at the same time, input to the decoder 8.

In this decoder 8, Huffman decoding and dequantization are performed to generate DCT coefficients. The DCT coefficient is
It is input to the IDCT (inverse discrete cosine transform) circuit 9.
The data converted from the DCT coefficient into the image data by the IDCT circuit is input to the adding circuit 11. The adder circuit 11 is a circuit for reconstructing the original image from the difference frame in the inter-frame coded frame, and in the case of the intra-frame coded frame, "0" is selected by the data selector 10. Therefore, the output result does not change.

The output of the adder circuit 11 is the frame memory 1
At the same time as being written to 2, it is input to the data selector 2 and then input to the difference circuit 3 via this data selector 2. Next, the inter-frame coded frame will be described. The image signal converted into the block scan signal by the raster / block conversion circuit 1 is input to the difference circuit 3,
The difference from the compressed and expanded image of the intra-coded frame is taken and input to the DCT circuit 5 via the data selector 4. The image data converted into the DCT coefficient by the DCT circuit 5 is quantized and Huffman coded by a quantization table set in advance by the encoder 6 and recorded in the code memory 7.

The data of one screen recorded in the code memory 7 is simultaneously read out to the external storage device (not shown) and at the same time input to the decoder 8 where it is Huffman-decoded and dequantized to generate DCT coefficients. This DCT coefficient is the ID
It is input to the CT circuit 9. Then, in the IDCT circuit 9, D
The difference image data converted from the CT coefficient is added by the addition circuit 1.
Input to 1. The adder circuit 11 is a circuit for reconstructing an image by adding the image of the previous frame, and the data selector 10 selects data from the frame memory 12. The output of the adder circuit 11 is the frame memory 12
Is written to the data selector 2 at the same time.
The subsequent frame will be this repetition.

Next, the decoding procedure will be described. The coded data is read from an external storage device (not shown), buffered in the code memory 7 and then input to the decoder 8 for Huffman decoding. And inversely quantized to generate DCT coefficients. The DCT coefficient is input to the IDCT circuit 9 and converted into image data, and then input to the addition circuit 11.

In the case of an intra-frame coded frame, "0" is selected by the data selector 10, so that the output result does not change. In the case of the inter-frame coded frame, the image of the previous frame recorded in the frame memory 12 is selected by the data selector 10 and added by the adder circuit 11.
Is added and output with the difference image. The output is converted into a raster signal by the block / raster conversion circuit 13 and output as an image signal.

[0010]

In such a conventional technique, the decompression process is performed at the same time in the compression process, and the process performed at the time of decompression to use the differential frame by using the image also needs to be performed at the time of compression. There is. However, since the compression / expansion processing is all performed by the block scan signal, when applying the distortion removal filter at the block boundary as disclosed in, for example, Japanese Patent Application No. 3-1499, there is a problem that the FIR filter cannot be applied. is there.

Further, when the compression / expansion of the moving image and the compression / expansion of the still image are combined, the filtering algorithm of the moving image and the algorithm of the still image are different, so that it is necessary to prepare different circuits.

Further, it is difficult to see the compression and filtering coefficients of moving images and still images, and the state of image deterioration due to compression. Therefore, the present invention has been made in view of the above problems. When the compression / expansion of a moving image is combined with the compression / expansion of a still image, it is not necessary to prepare a different circuit. It is an object of the present invention to provide a moving picture coding device in which it is not difficult to see a filtering coefficient and a state of image deterioration due to compression.

[0013]

That is, according to the present invention, an input moving image that is temporally continuous is subtracted pixel by pixel between the input image of the current frame and the decoded image of the previous frame to obtain an interframe moving image code. In a video encoding device including encoding,
When decoding the previous frame image, it is provided with filter means for filtering the image signal of the previous frame image to remove the distortion of the block boundary, and the previous frame from which the distortion of the block boundary has been removed by this filter means. It is characterized in that a difference image between the image signal of the image and the input image of the current frame is created.

[0014]

In the moving picture coding apparatus according to the present invention, the time-continuous input moving picture is subtracted pixel by pixel between the current frame input picture and the preceding frame decoded picture to obtain an inter-frame moving picture. In a moving image coding apparatus including image coding, when decoding a previous frame image, the image signal of the previous frame image is filtered by a filter means to remove distortion at the block boundary. Then, a difference image between the image signal of the previous frame image from which the distortion of the block boundary has been removed by this filter means and the current frame input image is created.

[0015]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a first embodiment in which the present invention is applied to a moving image compression by a distortion removing filter at a block boundary of a compressed image. Here, the configuration of the processing circuit is shown, and the algorithm of the distortion removal filter is described in Japanese Patent Application No.
Since it is disclosed in Japanese Patent No. 14499, the description is omitted here as a reference.

In FIG. 1, an image signal from an image input device (not shown) input to the raster / block conversion circuit 21 is converted into a block scan signal and the difference circuit 2 is supplied.
It is output to the data selector 23 via 2 and directly. After that, the image data converted into the DCT coefficient by the DCT circuit 24 is quantized and Huffman coded by the encoder 25 and supplied to the code memory 26. A compressed file is input to and output from the code memory 26 with an external storage device (a hard disk, a tape streamer, a magneto-optical storage device, etc.) not shown.

The data of one screen of the code memory 26 is Huffman-decoded and dequantized by the decoder 27 to obtain a DC signal.
The T coefficient is generated, input to the IDCT circuit 28, and converted into image data. The converted image data is converted into a raster signal by the block / raster conversion circuit 29 and then supplied to an adaptive convolution filter (AFIR) 30 together with the DCT coefficient output from the decoder 27. The output of the adaptive convolution filter 30 is input to the adder circuit 32 together with the output of the data selector 31. The output of the adder circuit 32 is supplied to the frame memory 33 and also to the raster / block conversion circuit 34. The block scan signal converted by the raster / block conversion circuit 34 is input to the difference circuit 22 via the data selector 35.

In such a structure, an image signal of a raster signal output from an image input device (not shown) is converted into a block scan signal by the raster / block conversion circuit 21. As shown in FIG. 5 described above, the image configuration is such that a plurality of inter-frame encoded frames are sandwiched between the intra-frame encoded frames and the intra-frame encoded frames.

Since the processing is different between the intra-frame coded frame and the inter-frame coded frame, the intra-frame coded frame will be described first. The image signal converted into the block scan signal by the raster / block conversion circuit 21 is input to the DCT circuit 24 via the data selector 23. The image data converted into the DCT coefficient by the DCT circuit 24 is quantized and Huffman-encoded by the quantization table set in advance by the encoder 25 and recorded in the code memory 26. Code memory 26
The data of one screen recorded in (1) is read out to an external storage device (hard disk, tape streamer, magneto-optical storage device, etc.) not shown, and at the same time, it is input to the decoder 27 and Huffman-decoded and dequantized to obtain the DCT. Generate a coefficient.

The DCT coefficient is input to the IDCT circuit 28. The data converted from the DCT coefficient to the image data by the IDCT circuit 28 is the block / raster conversion circuit 2
9 is input and converted into a raster signal. The image signal converted into the raster signal is filtered by the adaptive convolution filter 30. The coefficient of the adaptive convolution filter 30, that is, the frequency characteristic is determined by the DCT coefficient of the output of the decoder 27. The differential image signal from which the block boundary distortion has been removed by the adaptive convolution filter 30 is input to the addition circuit 32.

The adder circuit 32 is a circuit for reconstructing the original image from the difference frame in the inter-frame coded frame. In the case of the intra-frame coded frame, "0" is set in the data selector 31. Since it is selected, the output result does not change. The output of the adder circuit 32 is written into the frame memory 33 and, at the same time, is input into the raster / block conversion circuit 34. Then, after being converted into a block scan signal here, it is input to the difference circuit 22 via the data selector 35.

Next, the inter-frame coded frame will be described. The signal converted into the block scan signal by the raster / block conversion circuit 21 is input to the difference circuit 22, and the difference from the compressed / expanded image of the intra-frame coded frame which has been processed previously is calculated to obtain the data selector. Two
3 is input to the DCT circuit 24. DCT circuit 2
The image data converted into the DCT coefficient in 4 is converted by the quantization table preset by the encoder 25 into
Quantization and Huffman coding are performed, and the code memory 26
Recorded in. The data of one screen recorded in the code memory 26 is read to the external storage device (not shown), and at the same time, is input to the decoder 27 and subjected to Huffman decoding and dequantization to generate DCT coefficients.

The DCT coefficient is input to the IDCT circuit 28. The data converted from the DCT coefficient to the image data by the IDCT circuit 28 is the block / raster conversion circuit 2
9 is input and converted into a raster signal. The image signal converted into the raster signal here is filtered by the adaptive convolution filter 30. The coefficient of the adaptive convolution filter 30, that is, the frequency characteristic is determined by the DCT coefficient of the output of the decoder 27.

The adaptive convolution filter 3
The difference image signal from which the distortion of the block boundary is removed by 0 is input to the addition circuit 32. The image of the previous frame, which is recorded in the frame memory 33, is selected by the data selector 31, added by the adder circuit 32, and output. This output is converted into a raster signal by the raster / block conversion circuit 34 and input to the data selector 35.

The subsequent frame is the above repetition. Next, the decoding procedure will be described. The encoded data is read from the external storage device, buffered in the code memory 26, input to the decoder 27, Huffman-decoded and dequantized to generate DCT coefficients. The DCT coefficient is input to the IDCT circuit 28. The data converted from the DCT coefficient to the image data by the IDCT circuit 28 is a block / raster conversion circuit 29.
Is input to and converted into a raster signal. The image signal converted into the raster signal is filtered by the adaptive convolution filter 30.

This adaptive convolution filter 3
The coefficient of 0, that is, the frequency characteristic is determined by the DCT coefficient of the output of the decoder 27. The image signal from which the distortion of the block boundary has been removed by the adaptive convolution filter 30 is input to the addition circuit 32. In the case of an intra-coded frame, the data selector 31 selects "0", so that the output result does not change. In the case of the inter-coded frame, the image of the previous frame recorded in the frame memory 33 is selected by the data selector 31, added by the adder circuit 32, and output as an image signal, and the frame is re-framed. Memory 3
Written in 3.

As described above, according to the first embodiment, it is possible to remove the distortion and the like that occur during image compression. FIG. 2 shows a second embodiment in which the present invention is applied to a block boundary distortion removal filter of a compressed image in moving image and still image compression. Since the processing of the moving image is the same as that of the first embodiment described above, the description thereof will be omitted, and the processing of the still image will be described.

The still image coding will be described below. In FIG. 2, the signal converted into the block scan signal by the raster / block conversion circuit 21 is input to the DCT circuit 24 via the data selector 23. This DC
The image data converted into the DCT coefficient by the T circuit 24 is quantized and Huffman coded by the quantization table set in advance by the encoder 25, and recorded in the code memory 26. The data of one screen recorded in the code memory is read to the external storage device (hard disk, tape streamer, magneto-optical storage device, etc.) not shown.

Next, decoding of a still image will be described. The encoded data is read from the external storage device (not shown) and buffered in the code memory 26. Then, it is input to the decoder 27 and Huffman-decoded and dequantized to generate DCT coefficients. This DCT
The coefficient is input to the IDCT circuit 28. The data converted from the DCT coefficient to the image data by the IDCT circuit 28 is input to the block / raster conversion circuit 29 and converted into a raster signal. The image signal converted into the raster signal is filtered by the adaptive convolution filter 30.

This adaptive convolution filter 3
The coefficient of 0, that is, the frequency characteristic is determined by the DCT coefficient of the output of the decoder 27. The image signal from which the distortion of the block boundary has been removed by the adaptive convolution filter 30 is input to the addition circuit 32. Since “0” is selected by the data selector 31, the result is recorded in the frame memory 33 without change.

When one frame worth of data is recorded in the frame memory 33, it is read out in the vertical direction this time. And switch 1
5 through the adaptive convolution filter 30 again.
Enter and is filtered by the previously stored vertical filter coefficients. The image signal is input to the adding circuit 32. Here, since “0” is selected by the data selector 31, the result remains unchanged and is recorded again in the frame memory 33. Frame memory for one screen 3
After being recorded in 3, the data is read out in the horizontal direction and output as an image signal.

With the configuration of the second embodiment, it is possible to perform processing suitable for moving images and still images in the same system. FIG. 3 shows a third embodiment in which the present invention is applied to the compression of a block boundary of a compressed image in a moving image compression.

In FIG. 3, an analog signal of an image output from an image input device such as a camera 36 is digitally converted by an A / D converter 37. Then, the raster image signal converted into a digital signal is converted into a block scan signal by the raster / block conversion circuit 21.

First, the intra-coded frame will be described. The image signal converted into the block scan signal is input to the DCT circuit 24 via the data selector 23. Then, the image data converted into the DCT coefficient by the DCT circuit 24 is quantized and Huffman-encoded by the quantization table set in advance by the encoder 25 and recorded in the code memory 26. The data of one screen recorded in the code memory 26 is read to an external storage device (a hard disk, a tape streamer, a magneto-optical storage device, etc.) not shown, and at the same time, is input to the decoder 27 to be subjected to Huffman decoding and dequantization. Is generated and DCT coefficients are generated. The DCT coefficient is IDC
It is input to the T circuit 28.

The data converted from DCT coefficients into image data by the IDCT circuit 28 is input to the block / raster conversion circuit 29, where it is converted into a raster signal. The image signal converted into the raster signal is filtered by the adaptive convolution filter 30. The coefficient of the adaptive convolution filter 30, that is, the frequency characteristic is determined by the DCT coefficient of the output of the decoder 27.

The image signal from which the distortion of the block boundary has been removed by the adaptive convolution filter 30 is input to the adding circuit 32. The adder circuit 32 is a circuit for reconstructing an original image from a difference frame in an inter-coded frame. In the case of an intra-frame coded frame, "0" is selected by the data selector 31, so that the output result does not change.

The output of the adder circuit 32 is the frame memory 3
At the same time as being written in 3, the data is input to the D / A converter 39, converted into an analog signal, and further output to the monitor 38. The output of the adder circuit 32 is also input to the raster / block conversion circuit 34, converted into a block scan signal, and then input to the difference circuit 22 via the data selector 35.

The inter-frame coded frame is as follows. The signal converted into the block scan signal by the raster / block conversion circuit 21 is input to the difference circuit 22, and the difference from the compressed and expanded image of the previously processed intra-coded frame is calculated, and the data selector 23
Is input to the DCT circuit 24 via. The image data converted into DCT coefficients by the DCT circuit 24 is quantized and Huffman-encoded by a quantization table set in advance by the encoder 25, and recorded in the code memory 26.

The data of one screen recorded in the code memory 26 is read out to the external storage device (not shown), and at the same time, is input to the decoder 27 to be Huffman-decoded and dequantized to obtain the DCT coefficient. To occur. In the IDCT circuit 28 to which this DCT coefficient is input, the DCT coefficient is converted into difference image data. Then, the data converted into the difference image data is input to the block / raster conversion circuit 29 and converted into a raster signal.

The difference image signal converted into the raster signal is
It is filtered by the adaptive convolution filter 30. This adaptive convolution filter 30
, The frequency characteristic is determined by the DCT coefficient of the output of the decoder 27. The differential image signal from which the distortion of the block boundary has been removed by the adaptive convolution filter 30 is input to the addition circuit 32. Here, the image of the previous frame recorded in the frame memory 33 is selected by the data selector 31, added by the adder circuit 32, and output.

The output of the adder circuit 32 is converted into a raster signal by the raster / block conversion circuit 34 and input to the data selector 35. The output of the adder circuit 32 is written into the frame memory 33 and, at the same time, is input into the D / A converter 39. Then, after being converted into an analog signal by the D / A converter 39, it is output to the monitor 38.

The subsequent frame is a repetition of this.
With such a configuration, it is possible to monitor a compressed image in real time. For example, a block boundary distortion removal filter or the like often depends on an image. That is, an appropriate filter coefficient can be selected by changing the filter strength while viewing the image.

FIG. 4 shows a fourth embodiment in which the present invention is applied to a block boundary distortion removal filter of a compressed image in moving image and still image compression. Since the processing of the moving image is the same as that of the first embodiment described above, the description thereof will be omitted, and the processing of the still image will be described.

The encoding of a still image will be described below. In FIG. 4, the signal converted into the block scan signal by the raster / block conversion circuit 21 is input to the DCT circuit 24 via the data selector 23. This DC
The image data converted into the DCT coefficient by the T circuit 24 is quantized and Huffman coded by the quantization table set in advance by the encoder 25, and recorded in the code memory 26.

The data of one screen recorded in the code memory 26 is stored in an external storage device (hard disk, not shown).
At the same time as being read out to a tape streamer, a magneto-optical storage device, etc., it is inputted to the decoder 27, Huffman-decoded and dequantized to generate DCT coefficients. The DCT coefficient is input to the IDCT circuit 28. And IDCT
The data converted from the DCT coefficient to the image data by the circuit 28 is input to the block / raster conversion circuit 29 and converted into a raster signal.

The image signal converted into the raster signal is filtered by the adaptive convolution filter 30. The coefficient of the adaptive convolution filter 30, that is, the frequency characteristic is the DC of the output of the decoder 27.
It is determined by the T coefficient. The image signal from which the distortion of the block boundary has been removed by the adaptive convolution filter 30 is input to the addition circuit 32. Here, since "0" is selected by the data selector 31, the result is unchanged and converted into a raster signal by the raster / block conversion circuit 34, and then input to the data selector 35. The data that has passed through the data selector 35 is input to the difference circuit 22 and the difference from the original image is obtained. This difference signal is S /
The data is input to the N determination circuit 40 and the S / N data is obtained.

This processing is performed by changing the coefficient of the adaptive convolution filter 30, that is, the frequency characteristic.
The image having the highest S / N coefficient is selected and the coefficient is written in the header of the image or the like. As a result, the best image can be obtained by removing the distortion of the block boundary with the coefficient of the filter during reproduction.

Further, in the fourth embodiment, since only the additional information is set in the header or the like and the actual data is not affected, the expansion is performed even in the reproducing apparatus which does not have the block boundary distortion removing filter. be able to.

Further, since most of the circuit configuration is shared with the moving image processing, there is an advantage that the circuit configuration is not increased. Incidentally, each configuration of the above-mentioned embodiment, various modifications,
Of course, it can be changed. For example, as the block boundary distortion removal filter, various image processing filters such as a block distortion removal filter and a mosquito noise removal filter can be used.

[0050]

As described above, according to the present invention, when the compression / expansion of a moving image and the compression / expansion of a still image are combined, it is not necessary to prepare a different circuit, and the compression of a moving image or a still image can be performed. It is possible to provide a moving picture coding apparatus in which it is not difficult to see the state of image deterioration due to compression, the coefficient of filtering, and it is possible to obtain the filter coefficient of a still image using the moving picture processing part. .

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment in which a moving image coding apparatus of the present invention is applied to a moving image compression by a distortion removing filter for block boundaries of a compressed image.

FIG. 2 is a second application of the present invention to a block boundary distortion removal filter of a compressed image in moving image and still image compression.
It is a block diagram showing an example of.

FIG. 3 is a block diagram showing a third embodiment in which the present invention is applied to a moving image compression by a distortion removing filter at a block boundary of a compressed image.

FIG. 4 is a fourth application example of the present invention in which a block boundary distortion removal filter of a compressed image is applied to moving image and still image compression.
It is a block diagram showing an example of.

FIG. 5 is a frame configuration diagram of a conventional moving image compression encoding method.

FIG. 6 is a block diagram showing a conventional moving image processing apparatus.

[Explanation of symbols]

21, 34 ... Raster / block conversion circuit, 22 ... Difference circuit, 23, 31, 35 ... Data selector, 24 ... DCT
(Discrete Cosine Transform) Circuit, 25 ... Encoder, 26 ...
Code memory, 27 ... Decoder, 28 ... IDCT (Inverse Discrete Cosine Transform) circuit, 29 ... Block / Raster transform circuit, 30 ... Adaptive convolution filter (AFI)
R), 32 ... Adder circuit, 33 ... Frame memory.

Claims (1)

[Claims] 1. An input moving image continuous in time,
An image of the previous frame image when the previous frame image is decoded in a moving image encoding apparatus that includes inter-frame moving image encoding by subtracting each pixel between the current frame input image and the previous frame decoded image A differential image between the image signal of the previous frame image and the current frame input image, the filtering unit removing the distortion of the block boundary by filtering the signal. A moving picture coding device, characterized in that: