JP2002010268A - Device and method for coding images - Google Patents

Abstract

PROBLEM TO BE SOLVED: To overcome the problem such that the entire quantization is stepped up when a bit rate has been decreased. SOLUTION: This device and method for coding images includes a preprocessing means 10 to preprocess original image data S1, a decoding means 20 for local decoding after coding of original image data S4 put out fro the preprocessing means 10, and a preprocessing control means 9 which computes the difference between local decoding data S5 put out from the decoding means 20 and original image data S4 and controls the preprocessing means 10 on its result.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding apparatus and an image encoding method for encoding original image data with high efficiency.

[0002]

2. Description of the Related Art Conventionally, techniques in such a field include:
MPEG-2 Tes specified in ISO / IEC JTC1 / SC29 / WG11 / N0400
An image coding device shown in t Model 5 is known.

FIG. 5 is a block diagram showing the configuration of the image encoding device 50. In the figure, reference numeral 51 denotes a subtractor that calculates a difference between original image data and image data that has been encoded and decoded in the past, and 52 denotes a DCT (orthogonal transform) that converts the difference data calculated by the subtractor 51 into frequency domain information. vessel),
53, a quantization unit for quantizing the data orthogonally transformed by the DCT 52; 54, a VLC (variable length encoder) for removing the redundancy of the quantized data; 55, a variable length code generated by the VLC 54 at a certain rate; A buffer for smoothing the data and sending it out to the transmission path, 56 is an inverse quantizer for inversely quantizing the data quantized by the quantizer 53, and 57 is an inverse transform for the data inversely quantized by the inverse quantizer 56. The inverse DCT 58 is an adder for adding the data inversely transformed by the inverse DCT 57 and the decoded data n frames before (hereinafter, the adder 58).
Is referred to as local decoded data).

Further, reference numeral 59 denotes a frame memory in a loop for storing the local decoded data added by the adder 58, and 60 denotes a motion vector information based on the change in the image based on the original image data and the local decoded data. Motion compensator for controlling reading of frame memory 59 by
Reference numeral 1 denotes a quantization control unit which controls a quantization step and determines a bit rate and an encoded image quality. Reference numeral 62 denotes an activity (8 × 8 pixels of a luminance signal in a frame or a field; It is an activity calculation unit that calculates the value obtained by subtracting the average value of the 64 values from the value (integrated value).

[0005] In the case of Main-Profile, which is a general encoding method in the MPEG-2 standard, an image signal is rearranged from a display order to an encoding order before encoding (omitted in the figure), and an I picture ( Encoding is performed according to the picture types of the intra-frame prediction), the P-picture (forward prediction), and the B-picture (pre / post / complementary prediction). These operations are described in a large number of documents, including the Journal of the Institute of Television Engineers of Japan, Vol. 49, No. 4, pp. 435-466 (1995). As a method for controlling the rate, (1) the amount of target information generated in the screen, (2) the feedback control based on the buffer storage amount, and (3) the quantization step is controlled by the activity of the original image data.

FIG. 6 is a block diagram showing a configuration of a conventional image coding apparatus 70 disclosed in Japanese Patent Application Laid-Open No. 05-103317. In the figure, reference numeral 71 denotes H. described above. Encoding unit represented by 26X or MPEG, 7
2 is a delay circuit for delaying the original image data, 73 is a difference calculator for obtaining a difference (distortion) between the original image data delayed by the delay circuit and the local decoded data, and 74 is a control parameter for the difference data obtained by the difference calculator 73. Is a quantization parameter control circuit that controls the quantization process.

[0007] As another conventional technique, Japanese Patent Application Laid-Open
Japanese Patent Application Laid-Open Nos. 0-23162, 10-108167, 5-103317 and the like are known.

[0008]

By the way, MPEG-
The motion-compensated inter-frame coding method represented by SD2 is SD
This is a system conceived mainly for digital broadcasting, transmission, and storage media from TV to HDTV. In particular, in BS / terrestrial digital broadcasting, HDTV is mainly used and a considerably low bit rate (20 Mbps or less) is assumed (for example, The Institute of Image Information and Television Engineers Vol.53, No11, pp1456-1459 (1
999)). The conventional MPEG-2 and the above-described conventional image encoding device 50 are basic control models, and the image quality cannot be said to be sufficient by themselves. Therefore, various quantization control methods have been devised in addition to the above. At present, when an HDTV signal is compression-coded based on the MPEG-2 standard, a bit rate that satisfies broadcast quality is set to 22 Mbps or more according to an evaluation method defined by ITU-R (see the above-mentioned document, Video Information Media). Journal Vol.5
3, No11, pp 1456-1459 (1999)).

Further, according to this document, it is understood that in order to realize SFN (Single Frequency Network) in digital terrestrial broadcasting, it is necessary to further reduce the bit rate (increase the video compression ratio). It is a well-known fact that if the bit rate is reduced under the conventional control, the quantization step is inevitably coarsened, block distortion is increased, and a sense of visual discomfort occurs.

Further, the above-described conventional image encoding apparatus 70
Is a method in which the distribution of quantization steps is changed based on the magnitude of the difference (distortion). When the bit rate is relatively high, the effect can be expected, but when the bit rate decreases, the overall quantization The problem was that the steps went up.

The present invention solves such a problem, and can reduce the quantization step even when the bit rate is reduced, and provides a visually superior quality coded image in which block distortion is inconspicuous. It is an object of the present invention to provide an image encoding device and an image encoding method that can obtain the following.

[0012]

According to the present invention, there is provided an image coding apparatus comprising: a preprocessing section for preprocessing original image data; and an encoding apparatus for local decoding after coding the original image data output from the preprocessing section. And a pre-processing control unit that obtains a difference between the local decoded data output from the encoding unit and the original image data, and controls the pre-processing unit based on the difference data.

Here, the pre-processing unit has a band limiter, and the pre-processing control unit controls the band limiter so as to narrow the frequency band as the difference between the local decoded data and the original image data increases. Is preferred.

The preprocessing unit has a pixel number converter, and the preprocessing control unit controls the pixel number converter so as to increase the thinning rate as the difference between the local decoded data and the original image data increases. Is preferred.

The image processing apparatus further includes a scene change detecting unit that inputs the original image data and determines that the scene has changed when the correlation between the previous and next frames is low. It is preferable to perform control so that the processing of the preprocessing unit is not performed on the frame.

According to the image encoding method of the present invention, a difference between locally decoded data obtained by encoding the original image data and then locally decoded and the original image data is obtained, and preprocessing for the original image data is controlled based on the difference data. It is characterized by doing.

Here, in the pre-processing control for the original image data, it is preferable to narrow the frequency band as the difference between the local decoded data and the original image data increases.

In the pre-processing control for the original image data, it is preferable to increase the thinning rate as the difference between the local decoded data and the original image data increases.

[0019]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an image encoding apparatus and an image encoding method according to the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a block diagram showing a configuration of an image coding apparatus 1 according to the present embodiment. In the figure, 2
Is a band-limiting filter (band limiter) capable of changing the characteristic for inputting the original image data S1 and limiting the frequency band, and 3 is for inputting the original image data S2 output from the band-limiting filter 2, A pixel number conversion filter (pixel number converter) 4 for changing the number of lines, a scan conversion unit 4 for inputting the original image data S3 output from the pixel number conversion filter 3 and converting from raster scan to block scan;
An encoding unit 0 inputs and encodes the original image data S4 output from the scan conversion unit 4. Note that the band-limiting filter 2 and the pixel number conversion filter 3 form the original image data S1.
Is configured.

A delay circuit 5 for matching the pixel position of each frame to obtain a difference between the local decoded data S5 output from the encoding unit 20 and the original image data S4;
Reference numeral 6 denotes a difference calculator (subtractor) for obtaining distortion between the original image data S4 and the local decoded data S5, 7 an absolute value part for converting the difference data S7 to an absolute value, and 8 a difference absolute value data S8 for an arbitrary pixel (block). An accumulation unit 9 for accumulating data in units of 9) is a preprocessing control unit that controls the filter characteristics of the band limiting filter 2 and the thinning rate of the pixel number conversion filter 3 based on the accumulation result data S9 to S11.

Further, reference numeral 21 denotes a subtractor for obtaining a difference between the original image data S4 and the previously encoded and decoded image data.
Reference numeral 22 denotes a DCT (orthogonal transformer) that converts the difference data calculated by the subtractor 21 into frequency domain information, and 23 denotes a DCT.
A quantization unit for quantizing the data orthogonally transformed at 22;
Reference numeral 4 denotes a VLC (Variable Length Encoder) for removing the redundancy of the quantized data; 25, a buffer for smoothing the variable length code generated by the VLC 24 at a certain rate and sending it out to a transmission path; Is an inverse quantization unit that inversely quantizes the data quantized by, 27 is an inverse DCT that inversely transforms the data that is inversely quantized by the inverse quantization unit 26, and 28 is an inverse DCT 27
Is an adder that adds the inversely transformed data and the decoded data n frames before and outputs local decoded data S5.

Reference numeral 29 denotes an in-loop frame memory for storing the local decoded data S5 added by the adder 28;
0 is a motion compensator that controls the reading of the in-loop frame memory 29 by using the change of the image as motion vector information based on the original image data S4 and the local decoded data S5, and 31 controls the quantization step. , A quantization control unit that determines the bit rate and the encoded image quality, and a scene change detection unit 32 that receives the original image data S4 and detects a scene change based on the correlation between the previous and next frames.

Next, the operation will be described. First, an overall schematic operation will be described with reference to FIG. The original image data S1 includes a luminance signal and a color difference signal (Pb, Pr or C
b, Cr). The original image data S1 is input to the band limiting filter 2 and the pixel number converting filter 3, and subjected to pre-processing such as band limiting and pixel number converting. For example, in the case of an HDTV signal, the frequency band is 30 MHz.
Hz or the horizontal effective pixel 1920 is set to 144
For example, it is thinned to 0 or 1280. Of course 1
It is also possible to limit only the frequency band with 920 pixels.

The pre-processed original image data S3 is input to the scan conversion unit 4, and from a scan in screen order (hereinafter, raster scan) to a scan in a block in which the screen is divided into a plurality of small blocks (hereinafter, block scan). ). M
Speaking of PEG-2, DCT performs processing in units of 8 × 8 blocks, and quantization and motion compensation perform processing in units of 16 × 16 macroblocks. In this example, the scan conversion unit 4 is located after the preprocessing unit 10. However, since the band limitation and the pixel number conversion can be performed after the scan conversion is performed, the scan conversion unit 4 is May be arranged at the preceding stage.

The original image data S4 output from the scan converter 4 is input to a delay circuit 5, and the original image data S4 is delayed in order to obtain the difference between the same pixels in each frame. The original image data S4 output from the scan converter 4 is also input to the encoder 20. In the encoding unit 20, first, the original image data S4 is input to the subtracter 21, and the difference between the original image data S4 and the image data stored in the in-loop frame memory 29 is obtained. The difference data is input to the DCT 22 and is converted into frequency domain information. Further, the data converted into the information in the frequency domain is input to the quantization unit 23, where quantization processing is performed. The quantized data has its redundancy removed by the VLC 24, smoothed at a predetermined bit rate by the buffer 25, and sent to the transmission path.

The data quantized by the quantization unit 23 is also input to the inverse quantization unit 26 and is inversely quantized. The inversely quantized data is inversely transformed by the inverse DCT 27, and is added by the adder 28 to the encoded data of n frames before. Adder 28
Is output to the in-loop frame memory 29, the motion compensator 30, and the difference calculator 6. The motion compensator 30 controls reading of the in-loop frame memory 29 based on the local decoded data S5 input from the adder 28 and the original image data S4 input from the scan converter 4.

The difference between the locally decoded data S5 input to the difference calculator 6 and the delayed data S6 whose delay is compensated by the delay circuit 5 is obtained. This difference is quantization distortion that occurs when quantization is performed by the encoding unit 20. The quantization distortion becomes smaller as the quantization step becomes smaller, and becomes larger as the quantization step becomes larger. This depends on the motion of the subject and the fineness of the picture. For example, if a certain subject is a moving image, the result obtained by taking the
When converted into frequency domain information at 22, a signal of a high frequency component is generated.

When the signal of the high frequency component is roughly quantized, the original difference value is not restored, so that the original image data S
Appears as a distortion to 4 and becomes a visual impairment. On the other hand, in the case of a flat picture, the difference value is small even if the inter-frame difference is calculated by the subtractor 21, and the signal of the high-frequency component is not generated so much.

The difference data S output from the difference calculator 6
7 is converted into an absolute value by an absolute value section 7 and the difference absolute value data S
Get 8. As shown in FIG. 2, the absolute difference value data S8 is input to the accumulator 8, and the accumulated result data S9 for each of an arbitrary number of macroblocks in the frame, the accumulated result data S10 for each frame, and a plurality of The accumulation result data S11 for each frame is obtained. The pre-processing control unit 9 controls the band limiting filter 2 and the pixel number conversion filter 3 based on the accumulation result data S9 to S11.

Next, the operation of the pre-processing control section 9 will be described with reference to FIG.
This will be described with reference to FIG. The accumulation result data S9 to S11 generated by the accumulation unit 8 are obtained from a plurality of signals obtained as a result of accumulation for each of an arbitrary number of macroblocks, for each frame, and for each of a plurality of frames in a frame as described above. Be composed. These accumulation result data S9 to S11 are input to the pre-processing control unit 9. The pre-processing control unit 9 uses the accumulation result data S9 in units of a plurality of macroblocks or pixels to determine that encoding is strict when the accumulation result data S9 is larger than a predetermined threshold K1, and to determine the frequency band. Is changed so that the frequency band width becomes narrower.

On the contrary, when the accumulation result data S9 is smaller than the predetermined threshold value K1, the characteristic of the band limiting filter 2 is changed so as to widen the frequency band (including the through band). As a result, even if the bit rate is reduced, it is possible to prevent the quantization step from becoming unnecessarily rough, and it is possible to obtain a coded image of excellent visual quality with no noticeable block distortion.

Although the system of the band limiting filter 2 is not limited, generally, an FIR type having little aliasing is often used. The unit for switching the characteristics of the band limiting filter 2 can be performed on a frame basis or locally within a frame. When a filter is locally applied to the screen, the accumulation result data S9 in units of macroblocks is sequentially stored in the memory 9a of the preprocessing control unit 9, and the quantization distortion distribution in the screen is obtained. The filter characteristics may be switched in an arbitrary unit depending on the size of. In addition, the distribution of quantization distortion is distributed over the entire screen,
In addition, when the accumulation result data S10 for each frame is higher than the predetermined threshold K2, the filter acts on the entire screen.

On the other hand, the pixel number conversion filter 3 is, for example, H
In the case of a DTV signal, the horizontal effective pixels 192 of the original signal are in accordance with the US and domestic digital broadcasting standards (for example, ARIB STD-B20).
With respect to 0 (120 macroblocks), it is recognized that transmission is performed by thinning out to 1440 pixels (90 macroblocks). The minimum unit of switching is a frame (or a field), and switching is possible only in a frame on which intra-frame coding (I picture) is performed. In the case of 1440 pixels, not only the bandwidth is narrowed, but also the number of macroblocks is reduced, which is an effective means at the time of a low bit rate or when encoding an image that is difficult to encode.

In order to effectively switch the number of pixels (format) for such a standard, accumulated result data S10 accumulated for each frame and accumulated result data S11 for a plurality of frames are used. Accumulation result data S10,
If S11 is larger than the predetermined threshold K3, it is determined that encoding is severe, and the next I picture is thinned out to 1440 pixels and output. Conversely, when the accumulation result data S10 and S11 are smaller than the predetermined threshold value K3, the data is output without skipping. As a result, even if the bit rate is reduced, it is possible to prevent the quantization step from becoming unnecessarily rough, and it is possible to obtain a coded image of excellent visual quality with no noticeable block distortion.

In this example, an example in which 1920 pixels are thinned out to 1440 pixels as an HDTV has been described. However, the present invention is not limited to these two types. In order to further increase the compression ratio, pixels are reduced to 1280 pixels. What is necessary is just to change the characteristic of the number conversion filter 3.

Accumulated result data S accumulated per frame
The use of 10 and the accumulation result data S11 of a plurality of frames can be selectively performed, for example, when all frames are performed by I picture (intra-frame prediction), the number of pixels can be switched in units of one frame. It is also possible to use only the accumulation result data S10. If the period of the I picture is long to some extent in the inter-frame prediction, the accumulation result data S1 of a plurality of past frames until immediately before encoding the I picture is obtained.
1 can be controlled.

The accumulated result data S10 is stored as a history for each frame in the memory 9a, and among a plurality of frames from the past I picture to immediately before the next I picture, the accumulated result data S10 is set to the threshold K3. Can be determined depending on how many frames have been exceeded. In this case, there is an effect of providing a hysteresis when switching is performed in units of one frame, and also an effect of preventing the screen from flashing due to frequent switching of the number of pixels.

As shown in FIG. 4, in the image encoding apparatus 1, the local decoding data S5 output from the encoding section 20 is output.
, The control of the preprocessing unit 10 using the accumulation result data S9 to S110 is reflected not from the current frame but from the next frame. Therefore, if there is a scene change in the frame to be reflected, if the preprocessing unit 10 is controlled based on the information on the differential distortion of the previous frame, meaningless processing will be performed.

Therefore, the original image data S4 is input to the scene change detecting section 32, and it is detected whether or not there is a scene change between frames of the original image data S4. In particular,
The degree of correlation between the frames before and after the original image data S4 is detected, and when the degree of correlation is lower than a predetermined threshold, it is determined that the scene has changed. The detection signal S14 obtained by detecting a scene change by the scene change detection unit 32 is input to the preprocessing control unit 9, and the preprocessing control unit 9 stops controlling the preprocessing unit 10 for one frame. As a result, the original image data S1
The processing that does not conform to the above characteristics is prevented from being performed in the pre-processing unit 9, and a coded image with excellent quality can be obtained.

The unit of the accumulation processing performed by the accumulation unit 8 is:
Since both the original image data S4 and the local composite data S5 are block scans, the macroblock is the minimum unit. However, the macroblock is not necessarily the block unit but may be an arbitrary pixel unit. However, in this case, it is necessary to perform scan conversion on the original image data S4 or the difference data S7.

The pre-processing unit 10 includes a band limiting filter 2 and a pixel number conversion filter 3.
Another configuration such as a selector for switching the output of the band limiting filter 2 and the output of the pixel number conversion filter 3 may be provided. In this case, the pre-processing control unit 9 also controls the selector based on the accumulation result data S9 to S110.

[0043]

Since the image encoding apparatus and the image encoding method according to the present invention are configured as described above, the following effects can be obtained. That is, the degree of difficulty of encoding is indirectly obtained by using the distortion between the locally decoded data actually decoded and the locally decoded data and the original image data, and the processing of the preprocessing unit is controlled by the preprocessing control unit. Therefore, even if the bit rate is reduced, it is possible to prevent the quantization step from becoming unnecessarily rough, and it is possible to obtain a coded image of excellent visual quality with no noticeable block distortion.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image encoding device according to the present embodiment.

FIG. 2 is a block diagram illustrating a configuration of an accumulation unit.

FIG. 3 is a block diagram illustrating a configuration of a preprocessing control unit.

FIG. 4 is a diagram illustrating an example of timing of calculation and control.

FIG. 5 is a block diagram illustrating a configuration of a conventional image encoding device.

FIG. 6 is a block diagram illustrating a configuration of a conventional image encoding device.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Image coding apparatus, 2 ... Band limiting filter (band limiting device), 3 ... Pixel number conversion filter (pixel number converter), 4 ...
Scanning conversion unit, 5 delay circuit, 6 difference operation unit, 7 absolute value unit, 8 accumulation unit, 9 preprocessing control unit, 20 encoding unit, 21 subtractor, 22 DCT, 23 ... Quantization unit, 2
4 VLC, 25 buffer, 26 inverse quantization unit, 27
... Inverse DCT, 28 ... Adder, 29 ... Frame memory in loop, 30 ... Motion compensation unit, 31 ... Quantization control unit, 32 ...
Scene change detection unit, A: pre-processing unit.

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C059 KK34 LA06 LB05 LB07 MA00 MA05 MA23 MC11 ME01 NN01 NN43 PP05 SS02 SS03 SS05 TA05 TA06 TA46 TA69 TC03 TC14 TD11 UA02 UA12 UA33 UA38 5J064 AA02 BA01 BA16 BC01 BC16 BC25 BC26

Claims (7)

[Claims] 1. A pre-processing unit that pre-processes original image data, an encoding unit that performs local decoding after encoding the original image data output from the pre-processing unit, and a local unit that is output from the encoding unit An image encoding apparatus comprising: a preprocessing control unit that obtains a difference between decoded data and the original image data, and controls the preprocessing unit based on the difference data. 2. The apparatus according to claim 1, wherein the preprocessing unit includes a band limiter, and the preprocessing control unit performs the band limiting so as to narrow a frequency band as a difference between the local decoded data and the original image data increases. 2. The image encoding apparatus according to claim 1, wherein the image encoding apparatus controls the encoder. 3. The pre-processing unit has a pixel number converter, and the pre-processing control unit is configured to increase the thinning rate as the difference between the local decoded data and the original image data increases. The image coding apparatus according to claim 1, wherein the image coding apparatus controls a number converter. 4. A scene change detecting unit which receives the original image data and determines that a scene has changed when the degree of correlation between preceding and succeeding frames is low, and wherein the pre-processing control unit determines that the scene has changed. The image encoding apparatus according to any one of claims 1 to 3, wherein control is performed such that the processing of the pre-processing unit is not performed on a frame immediately after. 5. The method according to claim 1, wherein a difference between locally decoded data obtained by encoding the original image data and then locally decoded and the original image data is obtained, and preprocessing for the original image data is controlled based on the difference data. Image encoding method. 6. The image coding apparatus according to claim 5, wherein the control of the pre-processing for the original image data narrows a frequency band as a difference between the local decoded data and the original image data increases. Method. 7. The pre-processing control for the original image data, wherein the thinning rate increases as the difference between the local decoded data and the original image data increases. Image encoding method.