JP2000036961A - Image coder, image coding method and image decoder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain the image coding method for keeping high image quality by minimizing deteriorated coding even in the case that a data rate of image data is high and a data amount is large. SOLUTION: The number of data equivalent to one screen of input image data of a non-interlace scanning system is thinned into a half and the image data are converted into image data of the interlace scanning system. A coding section 1 encodes image data after the conversion by the unit of one screen. As to pixel data at positions thinned by an image data number reduction means, a code representing an interpolation candidate pixel data with largest correlation with the corresponding pixel data in original image data is encoded among plural interpolation candidate pixel data. The image data encoded by the coding section 1 and the encoded interpolation candidate pixel code are multiplexed and the multiplexed data are transmitted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention encodes progressive scan image data in which the number of image data per unit time is twice that of interlaced scan image data, and interlaced scan image data. The present invention relates to an image encoding device, an image encoding method, and an image decoding device that can perform encoding with high efficiency without causing deterioration of encoding as much as possible.

[0002]

2. Description of the Related Art When image data is recorded on a recording medium such as a disk and reproduced, or transmitted by terrestrial television broadcasting, satellite television broadcasting, cable television broadcasting, and the like.
When image data is transmitted through a transmission medium having a limited transmission capacity, the image data is compressed and transmitted in order to effectively use the limited transmission capacity.

FIG. 8 is a block diagram of an image coding circuit of the MPEG system which has been widely used as this type of compressed image coding system.

[0004] Input digital image data Vi is image data of an effective screen portion from which a synchronization signal portion has been removed. The input digital image data Vi is supplied to the pre-processing circuit 11 of the encoding unit 1, where the image order for performing the motion compensation processing is changed, and the raster scan is converted into a block scan. That is, in this example, raster scan format image data in units of horizontal scanning lines is converted into block scan format image data in units of blocks of, for example, horizontal × vertical = 8 pixels × 8 pixels.

[0005] The image data in the block scan format from the pre-processing circuit 11 is passed through a subtraction circuit 12 for calculating the difference in the inter-frame or inter-field predictive encoding processing, and is subjected to DCT (Discrete Cosine Trans).
form; discrete cosine transform) processing circuit 13. The DCT processing circuit 16 performs a DCT operation for transforming a signal in the time domain into a DCT coefficient in the frequency domain with respect to data in block units (prediction error signals in P pictures and B pictures to be subjected to inter-frame prediction coding).

[0006] The DCT coefficient as a result of the arithmetic processing by the DCT processing circuit 13 is supplied to a quantization circuit 14 and quantized. The output of the quantization circuit 14 is supplied to the variable length encoding circuit 2 as an output of the encoding unit 1 and is also supplied to a circuit unit for motion compensation processing as described below.

That is, the output of the quantization circuit 14 is returned to the original difference information by the inverse quantization circuit 15 and the inverse DCT processing circuit 16 for the motion compensation processing,
7 is supplied. The prediction data of the corresponding block of the previous frame (or previous field) from the motion compensation circuit 19 through the switch circuit 21 is supplied to the addition circuit 17 to obtain the prediction data of the current frame (or previous field). Can be That is, the locally decoded data is obtained from the adding circuit 17, and the locally decoded data is stored in the frame memory 18.

A motion vector detecting circuit 20 calculates a motion vector for generating a predicted image signal by performing motion compensation in a motion compensating circuit 19 from the image data from the preprocessing circuit 11 and the image data in the frame memory 18.

The motion compensation circuit 19 includes a frame memory 18
, Using the motion vector from the motion vector detection circuit 20 for the locally decoded image data stored in
The motion compensation is performed, and the subtraction circuit 12 generates a predicted image signal of the next frame supplied from the preprocessing circuit. And
The predicted image signal is supplied to the subtraction circuit 12 through the switch circuit 21.

The switch circuit 21 is connected to the terminal a and supplies the output of the motion compensation circuit 19 to the subtraction circuit 12 when performing inter-frame predictive coding according to a switching control signal from a control circuit (not shown).

In the block for performing the intra-frame encoding process, the switch circuit 21 is switched to the terminal b side.
The data supplied to the subtraction circuit 12 is set to zero. That is, in this case, the subtraction circuit 12 does not perform the difference calculation.

The image data compressed and encoded by the encoding unit 1 as described above is subjected to variable-length encoding using, for example, Huffman code in a variable-length encoding circuit 2 to control the transmission amount. Is output as output data Vo through the buffer memory 3.

The rate control circuit 4 monitors the data occupation state of the buffer memory 3 and controls the quantization value of the quantization circuit 14 according to the detected data occupation state. As a result, the output transmission rate of the output image data Vo is stabilized, and the output image data Vo is controlled so as to effectively utilize the transmission capacity.

Although not shown, information on the motion vector detected by the motion vector detection circuit 20 is transmitted together with the encoded output data Vo.

[0015]

Recently, high-definition, high-quality television broadcasting such as satellite television broadcasting and terrestrial digital television broadcasting has become a reality. The number of horizontal lines per frame of this type of television broadcast is 1125, of which the number of effective horizontal lines is 10
The number is 80. As a scanning method (scanning method), there are two methods: an interlaced scanning method and a sequential scanning method.

In the interlaced scanning method, the field rate is 60 Hz and the frame rate is 30 Hz.
The image data of the interlaced scanning method can be compression-encoded by using the above-described image encoding apparatus of FIG. 8 while securing a certain level of image quality.

On the other hand, in the progressive scanning system, the frame rate is 60 Hz, and the transmission data rate is doubled. For this reason, when the image data of the progressive scanning method is compression-encoded by using the image encoding device of FIG. 8, the amount of generated codes becomes large as it is, and the same transmission as in the case of the interlaced scanning method is performed. Transmission may not be possible on the road. Therefore, in the case of the image data of the progressive scanning method,
It must be compressed at a higher compression ratio than the interlaced scanning image data. As a result, there is a possibility that more coding degradation will occur and image quality will be degraded than in the case of encoding interlaced image data.

In view of the above points, the present invention provides a transmission medium (transmission path) even for image data whose generated code amount becomes excessively large as a transmission data amount as it is.
It is an object of the present invention to provide an image coding apparatus and method capable of minimizing deterioration of image quality without increasing transmission capacity of the image.

[0019]

According to a first aspect of the present invention, there is provided an image encoding apparatus for encoding input image data in units of one screen and transmitting the encoded image data. An image data number reducing means for thinning out the number of data per one screen of the input image data; and image data from the image data number reducing means,
An image encoding unit for encoding in units of a screen, a decoding unit for generating decoded image data obtained by decoding the encoded image data, and the number of image data from the decoded image data from the decoding unit. For each of the pixel data thinned out by the reducing means, an interpolation candidate pixel preparing means for preparing a plurality of predetermined interpolation candidate pixel data; restored image data from the image data number restoring means; To correct the delay error with the data,
A delay unit that delays the input image data, pixel data at a position in the input image data from the delay unit that has been thinned out by the image data number reduction unit, and the interpolation candidate pixel preparation unit for the pixel data position. A correlation with each of the plurality of prepared interpolation candidate pixel data is detected, and the interpolation candidate determined to have the largest correlation with the corresponding pixel data in the input image data based on the correlation detection result. Correlation interpolation pixel detection means for outputting an interpolation pixel code for specifying a pixel, interpolation pixel code encoding means for encoding the interpolation pixel code from the correlation interpolation pixel detection means for each screen unit, One screen unit of encoded image data from the image encoding unit, and the encoded image data from the interpolation pixel code encoding unit. Multiplexing means for multiplexing the encoded output data of the interpolation pixel code with respect to image data of the same screen, and output means for outputting data from the multiplexing means for transmission. And

In the first aspect of the present invention, when the input image data is progressive scan image data, the image data number reducing means outputs data of every other horizontal line of the input image data. In addition to the extraction, in adjacent frames, it is preferable to output image data corresponding to the image data of the interlaced scanning method by changing the position of the horizontal line to be extracted.

According to the first aspect of the present invention, the image data number reducing means thins out, for example, data of horizontal lines every other horizontal line and converts image data corresponding to image data of the interlaced scanning system. The generated image data of the interlaced scanning method is supplied to the image encoding means, and the image data of half of the image data is encoded in units of one screen. Therefore, the image encoding means can handle image data of the same transmission rate as image data of the interlaced scanning method even in the case of image data of the progressive scanning method.

According to the first aspect of the present invention, the information of the remaining half of the image data of one screen unit which is not encoded by the image encoding means is included in a predetermined plurality of interpolation candidate pixels. , An interpolated pixel code that specifies an optimal interpolation candidate pixel that has the largest correlation with the original pixel data and is therefore closest to the original pixel data is encoded and transmitted.

In this case, the amount of the interpolated pixel code can be reduced as compared with the remaining interlaced scan image data which is thinned out from the original image data. The amount of code generation of the multiplexed data is smaller than that in the case where the encoding is performed by the encoding means. Therefore, encoding deterioration is significantly reduced, and deterioration of image quality can be minimized.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image coding apparatus according to the present invention will be described below with reference to the drawings. In the embodiment described below, the number of horizontal lines per frame is 1125, and the number of effective horizontal lines is 108
This is a case where image data HVi of 0 lines and of a progressive scanning method with a frame rate of 60 Hz is compression-coded by the MPEG method.

As shown in FIG. 1, the input image data HVi of the progressive scanning method is supplied to an image data number reduction circuit 31. The image data number reduction circuit 31 reduces the number of image data of the input image data HVi and reduces the transmission rate. In the case of this example, the image data of every other horizontal line is extracted, and the remaining image data of every other horizontal line is thinned out. In addition, in adjacent frames, the horizontal line position to be extracted is made different, and processing is performed so that the data rate in pixel data units is half that of the original input image data HVi of the progressive scanning method.

That is, in the case of this example, the image data number reduction circuit 31 applies a filter of the vertical low-pass characteristic in the image frame of the progressive scanning method, and then, for example, inputs the image data HVi of the input progressive scanning method. Among them, odd lines are thinned out in odd frames, and even lines are thinned out in even frames. Thus, the progressive scan image data at the frame rate of 60 Hz is converted to the interlaced scan image data at the frame rate of 30 Hz (field rate of 60 Hz). Therefore, the image data number reduction circuit 31 in this example performs the sequential-> interlaced scan conversion to reduce the image data number and the data rate by one.
/ 2.

In the image data number reduction circuit 31 in order
The interlaced scan conversion processing will be described with reference to FIGS.

In FIG. 2A, circles arranged in the vertical direction indicate horizontal lines of one frame. That is, for example, A0, A1, A2,... Indicate each horizontal line of the frame A, and B0, B1, B2,.

In the image data number reduction circuit 31, FIG.
As shown in (B), in the frame A, only the image data of every other horizontal line A0, A2, A4,... Is extracted. Every other horizontal line B1, B3, B5, ...
Is extracted. In frame C, every other horizontal line C at the same position as frame A
Only the image data of 0, C2, C4,... Are extracted. Hereinafter, the image data is extracted while changing the horizontal line position to be extracted.

As a result, the frame rate 6 shown in FIG.
As shown in FIG. 2B, image data IVi equivalent to the image data of the interlaced scanning method having a field rate of 60 Hz and a frame rate of 30 Hz can be obtained from the 0 Hz progressive scanning image data.

As described above, the number of image data obtained from the image data number reduction circuit 31 is reduced to half, and the image data IVi converted into the interlaced scanning format is supplied to the encoding unit 1. The encoding unit 1 has exactly the same configuration as that of the encoding unit 1 shown in FIG. 8, and compresses and encodes image data input thereto according to the MPEG method, as described above.

In this case, the processing speed of the encoding unit 1 is:
This is half the speed in the case of encoding the image data HVi of the progressive scanning method directly, and this is because the frame rate is 3
The processing speed is the same as that when encoding image data of 1080 effective scanning lines of the interlaced scanning method at 0 Hz and a field rate of 60 Hz.

As described above, the image data CVd compressed and coded by the coding unit 1 is supplied to the variable length coding circuit 2 in the same manner as in FIG. After the variable length coding is performed, the multiplexing circuit 3
8 and multiplexed with data from a buffer memory 37 to be described later.

As described above, information on half the image data of the progressive scanning input image data HVi is compression-encoded and transmitted as a part of multiplexed information. On the other hand, information on the remaining half of the input image data HVi that is not included as information in the compression coded data CVd is coded as a small information amount as described below, and is compressed and coded. The data is multiplexed and transmitted.

The image information has a feature that the correlation in the horizontal direction and the correlation in the vertical direction, and further, the correlation in a frame unit is strong. Therefore, at the time of decoding the coded and transmitted image data HVo, the information thinned out by the image data number reduction circuit 31 can be generated by an interpolation process using their correlation. For this reason, the input image data HV that is not included as information in the compressed image data CVd
Even if the remaining half of the thinned image data of i is not transmitted, decoded image data having a predetermined or higher image quality can be obtained.

In this embodiment, as the interpolation processing, motion adaptive interpolation as described later is mainly used.
In this motion adaptive interpolation, three pixels of two pixel data in the vertical direction of a pixel position to be interpolated and pixel data one frame before the pixel position to be interpolated,
This is to generate motion adaptive interpolation pixel data by performing adaptive synthesis in accordance with the size of the motion vector.

If this motion adaptive interpolation processing is used, the interpolation according to the image content is performed such that the pixel data of the previous frame is used in the still image portion, and the average value of the upper and lower two pixels is used in the moving image portion. Processing can be performed, and decoded image data having a predetermined or higher image quality can be obtained.

If the decoding side performs only this motion adaptive interpolation processing to restore the original progressive scan image data, the encoded data HV of the above-described compressed encoded image data is restored.
Only o needs to be transmitted. However, it is difficult to perform interpolation with little deterioration for all of various image contents by using one type of interpolation method.

For example, if the image content includes a diagonal line or a diagonal edge, the motion adaptive interpolation using the upper and lower pixel data and the pixel data at the same position in the previous frame will not be performed. It may not be possible to reproduce correctly. That is, it is necessary to use the pixel data in the oblique direction of the pixel position to be interpolated.

Therefore, in this embodiment, a plurality of candidates (interpolation candidate pixels) including the motion adaptive interpolation pixels are prepared as interpolation pixels at pixel positions to be interpolated in consideration of various image contents. Each interpolation candidate pixel is assigned a code for identifying it. Then, from the plurality of interpolation candidate pixels, a pixel having the largest correlation with the corresponding pixel of the original image data (input image data HVi), that is, the closest interpolation candidate pixel is detected, and the interpolation candidate pixel is determined. Data obtained by encoding the indicated code is transmitted to the decoding side together with the encoded data HVo of the compressed encoded image data.

In this way, on the decoding side, it is possible to recognize the optimal interpolation candidate pixel using the interpolated pixel code having the largest correlation, and to perform the interpolation process using the interpolated pixel. , Image data close to the original input image data HVi can be restored. In view of the above, in this embodiment, first, image data obtained by decoding compression-encoded image data is generated, and interpolation candidate pixels are generated from the decoded image data.

That is, first, the addition circuit 1 of the encoding unit 1
7 is stored in the frame memory 18
And to the post-processing circuit 32. The post-processing circuit 32 performs a process reverse to the pre-processing performed by the pre-processing circuit 11 of the encoding circuit 1, converts the data in the block scan format into the data in the raster scan format, and based on the image order. Perform return processing. The output data of the post-processing circuit 32 is supplied to the image data number restoration circuit 33.

The image data number restoring circuit 33 restores the image data thinned out by the image data number reducing circuit 31 by an interpolation process, and sequentially obtains the same number of image data as the input image data HVi and the same data rate. This is a circuit for restoring image data of the scanning method. In this case,
This is to perform a skip-to-sequential scan conversion process.

In the case of this example, the image data number restoring circuit 33 executes the interpolation processing by the motion adaptive interpolation circuit using the motion adaptive interpolation pixel Pe.

FIG. 3 is a block diagram of a configuration example of the image data number restoring circuit 33 in this embodiment.

The image data from the post-processing circuit 32 is supplied to the synthesizing circuit 331 and also to the average value interpolation data generating circuit 332, the previous frame data transmitting circuit 333 and the motion vector detecting circuit 334.

The average value interpolation data generation circuit 332 calculates the average value of the data of the pixels located directly above and below the pixel to be interpolated, and outputs it as the interpolation pixel data.

The previous frame data transmission circuit 333
Has a frame memory FL, and the frame memory F
From L, the pixel data of the previous frame corresponding to the pixel data to be interpolated is sequentially read.

The average value interpolation data generation circuit 332
Is output through the coefficient multiplication circuit 335 to the addition circuit 3.
37. Also, the previous frame data transmission circuit 3
The output data of 33 is supplied to an addition circuit 337 through a coefficient multiplication circuit 336.

The motion vector detection circuit 334 detects a motion vector from the image data from the post-processing circuit 32 and the image data of the previous frame in the frame memory FL. The motion vector detection output is output to the motion coefficient generation circuit 3.
38.

The motion coefficient generating circuit 338 determines that the motion coefficient is k = 1 in a portion where the motion is large and the image is completely determined to be a moving image according to the magnitude of the motion vector detected by the motion vector detecting circuit 334. Is output. Further, in an image portion where there is no motion and the image is completely determined to be a still image, k = 0 is output as the motion coefficient k. Further, in an image portion determined to be intermediate between a complete moving image and a still image, the motion coefficient k is a value corresponding to the motion vector detected by the motion vector detection circuit 334, and is in a range of 0 <k <1. The value of is output. Then, the motion coefficient generation circuit 338
Supplies the motion coefficient k to the coefficient multiplying circuit 335, and
The coefficient (1-k) is supplied to a coefficient multiplying circuit 336.

Therefore, the output data Pe of the addition circuit 337 is obtained by performing the following operation. That is, the i-th of the j-th horizontal line to be generated by interpolation
The pixel Pi, j (i, j is a positive integer greater than or equal to 0, and Pi, j is a pixel only at a pixel position thinned out by the image data number reduction circuit 31 among the pixels of the sequential scanning method) ) Represents the pixel at the same position one frame earlier than P
When i, j ′, Pi, j = k × {Pi, (j−1) + Pi, (j + 1)} / 2+ (1−k) × Pi, j ′ (Equation 1) It will be.

In this (Equation 1), in the image portion determined to be a still image, the motion coefficient k = 0, and the value of the interpolation pixel Pi, j in that portion is determined by the previous frame data transmission circuit 33.
3, the previous value is held by the pixels Pi, j 'of the previous frame at the same position as the pixels Pi, j.

In an image portion in which the motion vector is large and the moving image is determined to be a moving image, the motion coefficient k = 1, and the value of the interpolated pixel Pi, j is the pixel Pi, (j-1) on one line in the frame. ) And the pixel Pi, (j +
Average value interpolation with 1) is performed.

If the motion coefficient k is a value in the range of 0 <k <1, the interpolation pixel Pi, j is calculated from the pixel of the previous frame and the upper and lower pixels of the frame by the magnitude of the motion vector. It is required adaptively according to.

The thus obtained interpolation pixel from the adding circuit 337, that is, the data Pe of the motion adaptive interpolation pixel
Is supplied to the synthesizing circuit 331 and is synthesized with the image data from the post-processing circuit 32, whereby the interpolation processing is executed, and the number of image data is returned to the original number of image data of the progressive scanning method. That is, in the case of this embodiment, the image data number restoring circuit 33 performs the sequential-> interlaced scan conversion processing.

In this way, in the image data number restoring circuit 33, adaptive interpolation processing using the motion vector is performed, and restored progressive scanning image data RVi corresponding to the input image data HVi is obtained. The restored progressively scanned image data RVi can be represented as shown in FIG.

In FIG. 2C, horizontal lines A0, A2, A4,..., B1, B3, B5,.
.., C0, C2, C4,.
(B) is obtained by decoding the interlaced image data IVi, and if there is no coding deterioration in the coding unit 1, these original horizontal lines A0 and A
2, A4, B1, B3, B5, C0, C
The image data of 2, C4,... Correctly returns to the original without coding deterioration.

In FIG. 2C, horizontal lines a1, a3, a5,..., B0, b2, b4,
.., C1, c3, c5,... Are data composed of motion adaptive interpolation pixel data generated by interpolation processing.

As described above, the image data number restoration circuit 3
3 is supplied to the correlation interpolation pixel detection circuit 34.

On the other hand, the input progressively scanned image data HVi is
After being delayed by the delay circuit 35, the signal is supplied to the correlation interpolation pixel detection circuit 34. The delay circuit 35 performs a delay corresponding to the processing delay time in the encoding unit 1, the post-processing circuit 32, and the image data number restoring circuit 33, and outputs the image data RVi from the image data number restoring circuit 33 and the input image data HVi.
Adjust the timing with

In the correlation interpolation pixel detection circuit 34, the interpolation candidate pixel data and the delay circuit 3
5 and the correlation between the original progressively scanned image data HVi and the corresponding pixel data is detected.

The correlation interpolation pixel detection circuit 34 also has a function of an interpolation candidate pixel preparation means. As the interpolation candidate pixel data, in addition to the data of the motion adaptive interpolation pixel from the image data number restoration circuit 33, Nine interpolation candidate pixels are prepared.

That is, the input progressively scanned image data HVi
For example, as shown in FIG. 4A, assume that nine pixels A to I span three lines. Of the nine pixels A to I, pixels D, E, and F are pixels that are thinned out by the image data number reduction circuit 31.

FIG. 4B shows the image data number restoring circuit 33.
Are pixels corresponding to the above-mentioned pixels A to I in the image data restored in step (1). That is, the pixel A ′ in FIG.
B ′, C ′, G ′, H ′, and I ′ are decoded pixels when the compression-encoded image data is decoded, and d,
e and f are pixels interpolated by the image data number restoring circuit 33, that is, motion adaptive interpolation pixels.

Here, it is assumed that the pixel to be interpolated is the pixel at the position of the motion application interpolation pixel e. in this case,
Interpolation candidate pixels prepared by the correlation interpolation pixel detection circuit 34 are:
There are ten pixels (including the pixels required for the operation) as shown in the table of FIG. As shown on the right side of the table in FIG. 5, a code for identification (this code is referred to as an interpolated pixel code) is given to these ten pixels.

The correlation interpolation pixel detection circuit 34 detects the correlation between each of these ten interpolation candidate pixels and the corresponding pixel E of the input image data HVi from the delay circuit 35. In this example, in the correlation interpolation pixel detection circuit 34,
A difference between each data value of the interpolation candidate pixel and the data value of the pixel E is obtained, an interpolation candidate pixel having the smallest difference is detected as a pixel having the largest correlation (correlation interpolation pixel), and the detected correlation interpolation is performed. Pixel interpolation pixel code CD
Is output from the correlation interpolation pixel detection circuit 34.

In this way, the interpolated pixel code CD of the interpolated pixel closest to the original pixel data is obtained from the correlated interpolation pixel detection circuit 34. In this example, since there are ten interpolation candidate pixels, the interpolation pixel code CD has 4 bits.

The interpolation pixel code CD from the correlation interpolation pixel detection circuit 34 is supplied to, for example, a variable length coding circuit 36 using a Huffman code, and is subjected to variable length coding. In the case of this example, the variable length encoding circuit 36 encodes the code so that the code length becomes shorter as the appearance probability from the correlation interpolation pixel detection circuit 34 is larger.

For example, since the appearance probability of the motion adaptive interpolation pixel is considered to be the highest, the variable length coding circuit 36
When the interpolation pixel code CD of the motion adaptive interpolation pixel is encoded, it is encoded so as to have the shortest code length. Subsequently, the output code length is assigned to the variable length coding circuit 36 so that the code length becomes shorter as the probability of appearance from the correlation interpolation pixel detection circuit 34 becomes larger.

The assignment of the code length to the code of each interpolation candidate pixel can be determined by obtaining the probability of detecting a standard image as a correlation interpolation pixel.

The encoded data of the interpolated pixel code CD variable-length encoded by the variable-length encoding circuit 36 is supplied to the multiplexing circuit 38 as the correlation interpolated pixel data CCo via the buffer memory 37, as described above. , The compressed coded data CVd variable-length coded by the variable-length coding circuit 2
Are multiplexed.

FIG. 6 shows an example of the structure of the multiplexed data at this time. Multiplexing is performed for each frame of data. That is, the video stream CV including the variable-length encoded data of the compressed image data CVd of one frame is transmitted with the picture header PH added thereto. And the correlation interpolation pixel data stream CC
When o is multiplexed, a correlation interpolation pixel data stream header CCH is inserted after the video stream CV, and subsequently, the correlation interpolation pixel data stream CCo is multiplexed. Note that invalid data Nd is inserted after the correlation interpolation pixel data stream CCo.

Although not shown, the motion vector information detected by the motion vector detection circuit 20 of the encoder 1 is multiplexed with the output data HVo and transmitted, for example.

The data multiplexed by the multiplexing circuit 38 as described above is output as output data HVo through the buffer memory 3 for controlling the transmission amount.
Then, the rate control circuit 5 monitors the data occupation state of the buffer memory 3 and controls the quantization value of the quantization circuit 14 according to the detected data occupation state of the buffer. As a result, the output transmission rate of the output data HVo is stabilized, and control is performed to effectively utilize the transmission capacity.

As described above, in this embodiment, the progressive scan image data is half the data amount, and the interlaced scan image data IVi having a data rate of 1/2 is encoded. The image data compression-encoded by the unit 1 is multiplexed with the correlation interpolation pixel data CCo obtained by decoding the compression-encoded data and encoding the code of the optimal interpolation pixel to generate the original image data by interpolation. And transmit it.

For this reason, the encoding unit 1 has a field rate of 60 Hz and a frame rate of 30 in the interlaced scanning method.
Encoding is performed in exactly the same way as in the case of image data of Hz. Therefore, the coding degradation is the same as that of the image data of the interlaced scanning method with the field rate of 60 Hz.

The information of the image data which has been thinned out because it has been converted into the image data of the interlaced scanning method is restored by interpolation processing on the decoding side. Since the interpolated pixel data CCo obtained by encoding the interpolated pixel code is transmitted, an optimal interpolation process is performed on the decoding side. Therefore, a high-quality reproduced image can be obtained.

The encoded data CC of the interpolation pixel code
For o, the amount of transmission data is very small. Therefore, even if the encoded data CCo of the interpolated pixel code is multiplexed and transmitted together with the output image data HVo, the transmission load is only slightly increased. For this reason, it is possible to transmit the image data at a data rate twice as high as that of the image data of the interlaced scanning system through a transmission line that is substantially the same as the transmission line.

In particular, in this embodiment, since the motion adaptive interpolation pixel is used as one of the interpolation candidate pixels, the appearance probability of the motion adaptive interpolation pixel is considered to be the highest.
By minimizing the code length when the code of the motion adaptive interpolation pixel is subjected to variable length coding, it can be expected that the generated code amount will be extremely small.

As described above, according to this embodiment, it is possible to realize image coding that maintains high image quality while minimizing coding deterioration.

In the above description, in the image data number restoring circuit 33, the motion vector detecting circuit for adaptively switching between the first interpolation means and the second interpolation means is the image data number restoring circuit 33. However, since the delay circuit 35 also includes a frame memory, the delay circuit 35
, A motion vector detection circuit may be provided, and its detection output may be supplied to the motion coefficient generation circuit 338 of the image data number restoration circuit 33.

[Example of Decoding Apparatus] Image data encoded and transmitted by the above-described image encoding apparatus is decoded as follows. That is, FIG. 7 shows a block diagram of an embodiment of the image decoding apparatus.

As shown in FIG. 7, the transmitted multiplexed image data HVo is supplied to a demultiplexer (separation circuit) 42 through a buffer memory 41,
Are separated into a stream of compression-encoded image data and a difference data stream based on the header shown in FIG.

Then, the stream of compressed image data is
The data is supplied to the variable-length decoding circuit 51 and subjected to variable-length decoding.
The image data variable-length decoded by the variable-length decoding circuit 51 is supplied to an inverse quantization circuit 52, and is inversely quantized. The inversely quantized image data is supplied to an inverse DCT circuit 53.
Then, the DCT coefficients in the frequency domain are returned to the image data in block units in the time domain (difference information in the case of an inter-frame predictive coding part such as a P picture or a B picture). The output of the inverse DCT circuit 53 is supplied to an adding circuit 54.

The addition circuit 54 includes a switch circuit 57
The prediction data of the corresponding block of the previous frame from the motion compensation circuit 56 is supplied to obtain the prediction data of the current frame. That is, predictive decoded data is obtained from the adding circuit 54, and the predicted decoded data is stored in the frame memory 55. In the I picture and the non-predictive coding part, the switch circuit 57 is connected to the opposite side from the illustrated side, and no prediction data is supplied to the adder circuit.

The motion compensation circuit 56 has a frame memory 55
The motion compensation is performed using the prediction decoded image data and the motion vector information included in the transmission data and supplied to the motion compensation circuit 56, and the output is used as the prediction data as an addition circuit. 54.

The decoded data obtained from the adder circuit 54 is not progressive scanning image data, but is
This is the image data of the interlaced scanning method thinned out to 1/2 line number. The decoded data from the adding circuit 54 is supplied to a post-processing circuit 58.

In the post-processing circuit 58, the data in the block scan format from the adder circuit 54 is converted into the data in the raster scan format, and the image order is restored. The output data of the post-processing circuit 58 is supplied to the image data number restoring circuit 59.

On the other hand, the stream of the correlation interpolation pixel data CCo separated by the demultiplexer 42 is supplied to the variable length decoding circuit 61 and subjected to variable length decoding. Is obtained.
That is, the variable length decoding circuit 61 constitutes an interpolation pixel code decoding unit.

The interpolated pixel code CD obtained from the variable length decoding circuit 61 is interpolated when the decoded image data is interpolated by the image data number restoring circuit 59 to generate the restored sequential scanning image data DVi. This is the code of the interpolation candidate pixel that is optimal for each of the power pixel positions. The interpolation pixel code CD from the variable length decoding circuit 61 is supplied to the image data number restoring circuit 59.

The image data number restoring circuit 59 has the same configuration as that of the image data number restoring circuit 33 of the image encoding apparatus shown in FIG.
It has a function of performing optimal interpolation based on the interpolation pixel code CD obtained by decoding Co. That is, the image data number restoring circuit 59 can generate all of the ten interpolation candidate pixels described above and, out of the ten interpolation candidate pixels, the optimal interpolation pixel code indicated by the decoded interpolation pixel code. An object is selected and interpolation is performed.

The image data number restoring circuit 59 performs the interpolation processing so that the input image data H to the image encoding apparatus shown in FIG.
This is a circuit for restoring progressive scan image data having the same number of image data and the same data rate as Vi. In this example, it performs interlaced to progressive scan conversion processing.

More specifically, in the case of this example, the image data number restoring circuit 59 basically includes a frame memory, similarly to the image data number restoring circuit 33 shown in FIG.
A motion vector between the previous frame and the frame is detected, and a motion adaptive interpolation process using the motion vector as shown in the above (Equation 1) is performed.

When the interpolation pixel code CD obtained by decoding the correlation interpolation pixel data CCo is "0", the pixel subjected to the motion adaptive interpolation processing is used as it is.
When the interpolated pixel code CD obtained by decoding is another code, the corresponding interpolated pixel is replaced with an interpolation candidate pixel indicated by the code.

In this way, from the image data number restoring circuit 59, the decoded progressively scanned image data DVi corresponding to the original input image data HVi, which has been encoded and transmitted, is obtained.

In this case, instead of one type of interpolation, a pixel having the highest correlation with the corresponding pixel in the original image data selected from a plurality of interpolation candidate pixels is used for interpolation. Can be decoded with less coding degradation. For example, in the case of the present embodiment in which the pixels of the diagonal arithmetic mean and the pixels at the diagonal positions are used as interpolation candidate pixels, interpolation processing with good reproducibility can be performed even on diagonal lines and diagonal edge portions.

[Modification] In the above description of the embodiment, the correlation interpolation pixel detection circuit 34 outputs an optimum interpolation pixel code for each pixel position to be interpolated, and the variable length coding circuit 36 outputs The variable length coding is performed for each interpolation pixel code unit, and the coded data CC
However, the variable length encoding circuit 36 considers that the same interpolation pixel code for consecutive interpolation pixels is often the same as the optimal interpolation pixel code. The encoded data CCo may be generated as a pair with the number (length) of the interpolation pixels followed by the interpolation data.

In the correlation interpolation pixel detection circuit 34, the optimum interpolation candidate pixel is detected for each pixel position to be interpolated, but the interpolation pixel in the block is composed of a plurality of pixels. The optimal interpolation candidate pixel may be detected, and the optimal interpolation pixel code may be output for each block. If the size of the unit block is a DCT block or a macro block in the case of the MPEG compression method as in the above example, the processing is facilitated.

As described above, when the optimum interpolation candidate pixel is detected in units of blocks, the interpolation pixel code C generated from the correlation interpolation pixel detection circuit 34 per frame can be obtained.
The number of D is reduced, and the data amount of the encoded data CCo of the interpolation pixel code can be reduced accordingly.

It is needless to say that the example of the interpolation candidate pixel is not limited to the example described above. That is, the number of interpolation candidate pixels may be smaller or larger. Further, for example, pixels around the interpolation pixel (may include the previous frame as well as the same frame) are used, and the interpolation pixel and
Those that generate interpolation pixels by weighting them according to the spatial distance between them and the surrounding pixels may be used as interpolation candidate pixels, or may be arithmetically averaged between adjacent horizontal pixels (see FIG. 4). Then, (A ′ + B ′) / 2) can be used as the correlation candidate pixel.

In the above embodiment, half of the image data is thinned out in units of horizontal line image data. However, the method of thinning the image data to reduce the number of pixels is not limited to this. For example, the present invention is also applicable to a case where image data is thinned out for each pixel or a plurality of pixels.

Although the multiplexing circuit is provided before the buffer memory 3, the multiplexing circuit may be provided after the buffer memory 3. In this case, the rate control circuit 5 monitors not only the buffer memory 3 but also the buffer occupancy of data in the buffer memory 36,
The quantization value of the quantization circuit 14 is controlled.

Although the description has been given of the case where the input image data is of the progressive scanning system and the frame rate is 60 Hz, it is needless to say that the present invention can be applied to the case where the frame rate is 59.94 Hz or 50 Hz. Nor. In this case, the frame rate of the image data is 30 Hz, 29.75 Hz, or 25H in terms of the image data of the interlace scanning frame rate.
Even in the case of z, the image encoding method according to the present invention is applicable.

The encoding section 1 is not limited to the MPEG compression encoding system, and it goes without saying that various other image encoding systems can be used. Also, the variable length coding circuits 2 and 36 are not limited to those using Huffman codes, and various variable length coding methods can be used.

[0106]

As described above, according to the present invention, even when the data rate of image data is high and the data amount is large, the amount of code generation is suppressed to minimize coding deterioration. In addition, it is possible to realize image encoding that maintains high image quality.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of an image encoding device according to the present invention.

FIG. 2 is a diagram illustrating a processing operation of a main part of the embodiment.

FIG. 3 is a block diagram of a specific configuration example of a partial circuit according to the embodiment;

FIG. 4 is a diagram for explaining a processing operation of a main part of the embodiment.

FIG. 5 is a diagram for describing an example of an interpolation pixel candidate used in the embodiment.

FIG. 6 is a diagram for describing a multiplexing operation in the embodiment.

FIG. 7 is a block diagram of an embodiment of an image decoding device corresponding to the image encoding device of the first embodiment.

FIG. 8 is a block diagram illustrating an example of a compressed image encoding method.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Encoding part, 2 ... Variable length encoding circuit, 3 ... Buffer circuit, 4 ... Rate control circuit, 5 ... Rate control circuit, 11 ...
Preprocessing circuit, 12... Subtraction circuit (prediction error calculation circuit), 1
3 DCT processing circuit, 14 quantization circuit, 15 inverse quantization circuit, 16 inverse DCT processing circuit, 17 local addition circuit for decoding, 18 frame memory, 19 motion compensation circuit,
20: motion vector detection circuit, 31: image data number reduction circuit, 32: post-processing circuit, 33: image restoration circuit,
34 ... correlation interpolation pixel detection circuit, 35 ... delay circuit, 36 ...
Variable length encoding circuit, 37 buffer, 42 demultiplexer (separation circuit), 51, 61 variable length decoding circuit, 59 image data number restoration circuit

Continued on front page F term (reference) 5C059 KK22 KK30 LA07 LB04 LB12 LB15 LB16 LB18 MA00 MA02 MA03 MA05 MA23 MC11 MC38 ME02 NN01 RC00 RC40 SS02 SS05 SS13 SS24 TA09 TB04 TB07 TB08 TC12 UA02 UA05 UA33 UA34

Claims (16)

[Claims] An image encoding apparatus for encoding and transmitting input image data in units of one screen, wherein the number of image data is reduced by thinning out the number of pieces of input image data for each screen. Means, image encoding means for encoding the image data from the image data number reducing means in units of one screen, and decoding means for generating decoded image data obtained by decoding the encoded image data. From the decoded image data from the decoding means, for each of the pixel data thinned out by the image data number reduction means, interpolation candidate pixel preparation means for preparing a plurality of predetermined interpolation candidate pixel data, A delay unit that delays the input image data in order to correct a delay error between the restored image data from the image data number restoration unit and the input image data; The pixel data at the position of the image data number reduction unit in the input image data input from the delay unit and the plurality of interpolation candidate pixels prepared by the interpolation candidate pixel preparation unit for this pixel data position A correlation with each of the data is detected, and an interpolation pixel code for identifying the interpolation candidate pixel determined to have the largest correlation with the corresponding pixel data in the input image data based on the correlation detection result is output. Correlation interpolation pixel detection means, interpolation pixel code encoding means for encoding the interpolation pixel code from the correlation interpolation pixel detection means for each screen unit, and one screen unit from the image encoding means And the interpolated image from the interpolated pixel code encoding means for image data on the same screen as the encoded image data. An image coding apparatus comprising: a multiplexing unit that multiplexes encoded output data of a raw code; and an output unit that outputs data from the multiplexing unit for transmission. 2. The image encoding apparatus according to claim 1, wherein said input image data is image data of a progressive scanning method, and said image data number reducing means includes a unit for every other horizontal line of said input image data. An image encoding device that extracts image data corresponding to interlaced scanning image data by extracting the horizontal line positions in adjacent frames while extracting the data. 3. The image coding apparatus according to claim 2, wherein the frame rate of the input image data of the progressive scanning method is 60 Hz, 59.94 Hz or 50 Hz, An image coding device, wherein the frame rate is 30 Hz, 29.97 Hz or 25 Hz. 4. The image encoding apparatus according to claim 1, wherein said image encoding means performs an MPEG compression encoding process, and said decoding means performs said MPEG compression encoding processing. An image coding apparatus for generating image data that has been decompressed and decoded using a local decoded signal used in a motion compensation process in the process. 5. The image encoding apparatus according to claim 4, wherein said interpolation candidate pixel preparing means uses a memory for delaying the image data by one screen as one interpolation candidate pixel data. An image coding apparatus, wherein a motion adaptive interpolation pixel is prepared by adaptively using an interpolation process and an interpolation process using only image data in the same screen according to a motion detection signal. 6. The image encoding apparatus according to claim 1, wherein the interpolation candidate pixel preparing means generates the interpolation candidate pixel data by using pixel data at a pixel position in an oblique direction of the pixel data at the interpolation position. An image coding apparatus comprising: 7. An image encoding apparatus according to claim 1, wherein said correlation interpolation pixel detecting means is replaced with an optimal interpolation commonly used for a plurality of pixel data of said block in a block unit of a plurality of pixels. An image coding apparatus characterized in that a block unit correlation interpolation pixel detection means for selecting any one of a plurality of interpolation candidate pixels prepared by said interpolation candidate pixel preparation means is used as a candidate pixel. 8. An image encoding method for encoding and transmitting input image data in units of one screen, wherein the number of data of one screen of the input image data is reduced by thinning processing. A number reduction step, an image encoding step of encoding the image data of which data number has been reduced in the image data number reduction step in units of one screen, and decoding the image data encoded in the image encoding step. A decoding step; and preparing a plurality of predetermined interpolation candidate pixel data for each of the pixel data thinned out by the image data number reducing means from the decoded image data obtained in the decoding step. An interpolation candidate pixel preparing step, and delaying the input image data to correct a delay error between the restored image data from the image data number restoring unit and the input image data. A delay step; pixel data at a position in the input image data delayed in the delay step, which has been thinned out by the image data number reducing means; and the pixel data position is prepared in the interpolation candidate pixel preparing step To detect a correlation with each of the plurality of interpolation candidate pixel data, and to specify an interpolation candidate pixel determined to have the largest correlation with the corresponding pixel data in the input image data based on the correlation detection result. A correlation interpolation pixel detection step of generating an interpolation pixel code of; an interpolation pixel code encoding step of encoding the interpolation pixel code generated in the correlation interpolation pixel detection step for each screen unit; The same as the encoded image data generated in the interpolated pixel code encoding step, the encoded image data in units of one screen generated in the process. A multiplexing step of multiplexing the encoded output data of the interpolated pixel code with respect to image data of a screen; and an output step of outputting the data multiplexed in the multiplexing step for transmission. Encoding method. 9. The image encoding method according to claim 8, wherein the input image data is image data of a progressive scanning method, and in the image data number reduction step, every other horizontal line of the input image data is provided. An image encoding method which generates image data corresponding to interlaced scanning image data by extracting the horizontal line position in adjacent frames while extracting the data of adjacent frames. 10. The image encoding method according to claim 9, wherein the frame rate of the input image data of the progressive scanning method is 60 Hz, 59.94 Hz or 50 Hz, and the image generated in the image data number reduction step. Data frame rate is 30Hz, 29.97Hz or 25H
z is an image encoding method. 11. The image encoding method according to claim 8, wherein the image encoding step performs an MPEG compression encoding process, and the decoding step includes performing the image encoding step. An image encoding method characterized by generating image data that has been decompressed and decoded using a locally decoded signal used in the motion compensation processing in (1). 12. The image encoding method according to claim 11, wherein said interpolation candidate pixel preparation means uses a memory for delaying the image data by one screen as the one interpolation candidate pixel data. What is claimed is: 1. An image encoding method, comprising preparing a motion adaptive interpolation pixel obtained by adaptively using an interpolation process and an interpolation process using only image data in the same screen according to a motion detection signal. 13. The image encoding method according to claim 8, wherein in the interpolation candidate pixel preparation step, the interpolation candidate pixel data is generated using pixel data at a pixel position in an oblique direction of the pixel data at the interpolation position. An image encoding method comprising: 14. The image encoding method according to claim 8, wherein, in place of said correlated pixel detection code generating step, an optimum unit commonly used for a plurality of pixel data of said block in units of blocks composed of a plurality of pixels. An image encoding method using a block unit optimal interpolation candidate detection step of selecting any one of a plurality of interpolation candidate pixels prepared in the interpolation candidate pixel preparation step as the interpolation candidate pixel. 15. A method according to claim 15, wherein a plurality of predetermined interpolations are added to the coded pixel data encoded in a state where the number of image data is reduced by thinning out the image data as the data of the thinned image data position. Data obtained by multiplexing and transmitting data obtained by encoding an interpolation pixel code specifying one of the candidate pixel data is separated into the encoded image data and the encoded data of the interpolation pixel code. Separating means; image data decoding means for decoding the encoded image data separated by the separating means; and interpolated pixel code decoding for decoding encoded data of the interpolated pixel code separated by the separating means. The image data decoding means by interpolating using an interpolation candidate pixel indicated by the interpolation pixel code decoded by the interpolation pixel code decoding means. The number of image data of Goka image data, the image decoding device characterized by and an image data number restoring means for restoring the original number of image data. 16. The image data decoding means obtains interlaced scanning image data, and the image data number restoring means obtains sequential scanning image data. Claim 1.
6. The image decoding device according to 5.