JPH11196423A - Device and method for picture processing and presentation medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve picture quality with regard to a parameter before scene change when the parameter after the scene change becomes small in encoding processing after the scene change. SOLUTION: A parameter detector 10 detects a parameter for indicating picture complexity and supplies a rate controller 11 with it. The rate controller 11 compares the first parameter of the first picture that time-wise precedes the second and the second parameter of the second picture that time-wise follows the first and controls a quantization step so that the second picture data are quantized on the basis of the first parameter when the second parameter is smaller than the first parameter. A quantizer 2 quantizes the picture data with the quantization step that is controlled by the rate controller 11.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method, and a providing medium, and more particularly, to a case where a parameter after a scene change is lower than a parameter before a scene change in an encoding process after a scene change. The present invention relates to an image processing apparatus and method capable of preventing noise from being noticeable without reducing a quantization step, and a providing medium.

[0002]

2. Description of the Related Art MPEG2 (Moving Picture Experrts Group)
Phase 2) is, for example, HDTV (High Definion Television)
Has been adopted as an encoding method for digital television broadcasting represented by Digital television broadcasting can have more channels than analog television broadcasting. As a result, not only can the cost per channel be reduced, but also not only video and audio but also various types of data broadcasting, etc. Enable service.

For example, regarding satellite broadcasting, in North America Di
Digital broadcasts using MPEG2 are becoming more and more popular, with some plans being implemented, including recTv and PerfecTv and JSkyB in the country.

In these satellite broadcasts, the input source is
An encoder that encodes by the MPEG method is used. The encoder divides a television (hereinafter referred to as a television) image into m × n individual macroblocks, and, for each macroblock, converts pixel data into an orthogonal function (for example, DC
T (Discrete Cosine Transform), etc.) to concentrate the signal power and compress the total information volume.

[0005] For example, in the case of compressing and encoding an image in accordance with the MPEG2 method or the like, a TM (Test Model) is used as one of the algorithms for obtaining a decoded image of good image quality. Five are known. TM5 is
It is composed of three steps, and the following processing is performed in each step.

[Step 1] The rate controller sets the target code amount of the frame to be encoded this time, based on the complexity of the frame of the same picture type encoded last time.

[Step 2] The rate controller:
The data storage amount of the virtual buffer for managing the difference between the target code amount and the generated code amount for each picture type of I, P, and B is fed back, and the actual generated code amount is determined based on the data storage amount. Then, the rate control is performed for the next macroblock to be encoded so as to match (substantially match) the target code amount set in step 1.

[Step 3] The quantizer performs adaptive quantization based on the complexity of the macroblock to be encoded so as to improve the image quality of the decoded image in consideration of visual characteristics.

Therefore, in TM5, for example, a parameter that changes according to the complexity of the image is C, and a parameter corresponding to the activity of the image is a parameter Q for matching the generated code amount to the target code amount. Assuming that those corresponding to the provisional quantization steps determined from the amount of data stored in the virtual buffer are used, the final quantization step MQUANT for quantizing the macroblock is used.
Is determined, for example, according to the following equation. MQUANT = f (C, Q) Here, f (C, Q) is a function that outputs a value obtained by normalizing values corresponding to the arguments C and Q to values within a predetermined range.

[0010]

SUMMARY OF THE INVENTION As described above, TM5
In, the quantization step is determined from a parameter C corresponding to the complexity of the image and a parameter Q for causing the generated code amount to correspond to the target code amount.

However, when the scene change of an image is performed, the relationship between the amount of information generated in the previous frame and the next frame is suddenly interrupted. In this case, noise of a simple image immediately after a scene change is conspicuous, and there is a problem that image quality is deteriorated.

The present invention has been made in view of such a situation, and has been made in the case where a scene change of an image is performed.
This is to prevent the image quality from deteriorating.

[0013]

According to the first aspect of the present invention, there is provided an image processing apparatus comprising: a quantizing means for quantizing image data; a detecting means for detecting a parameter representing the complexity of an image; Comparing means for comparing the first parameter of the first image with the second parameter of the second image that is later in time; and when the second parameter is smaller than the first parameter, Control means for controlling the quantization means so as to quantize the image data based on the first parameter.

According to a second aspect of the present invention, there is provided an image processing method comprising: a quantization step of quantizing image data; a detection step of detecting a parameter representing complexity of the image; Comparing the first parameter with a second parameter of a second image that is temporally later; and when the second parameter is smaller than the first parameter, converting the image data of the second image into the first parameter. And a control step of controlling the quantization in the quantization step so as to perform the quantization based on the parameter (1).

According to a third aspect of the present invention, there is provided the providing medium, comprising: a quantization step of quantizing the image data; a detection step of detecting a parameter representing the complexity of the image;
Comparing the first parameter of the second image with the second parameter of the second image in time, and when the second parameter is smaller than the first parameter, the image data of the second image And a control step of controlling the quantization in the quantization step so that is quantized based on the first parameter.

An image processing apparatus according to claim 1 and claim 2.
In the image processing method according to the first aspect and the providing medium according to the third aspect, the image data is quantized, a parameter representing the complexity of the image is detected, and the first data of the first image that is temporally previous is detected. Parameters and the second of the second image in time
Are compared, and when the second parameter is smaller than the first parameter, the image data of the second image is
The quantization is controlled so as to be quantized based on the first parameter.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below. In order to clarify the correspondence between each means of the invention described in the claims and the following embodiments, each means is described. When the features of the present invention are described by adding the corresponding embodiment (however, an example) in parentheses after the parentheses, the result is as follows. However, of course, this description does not mean that each means is limited to those described.

An image processing apparatus according to a first aspect of the present invention includes a quantizing means for quantizing image data (for example, the quantizer 2 in FIG. 1) and a detecting means for detecting a parameter representing the complexity of an image. , The first parameter of the first image before the time, the second parameter after the time
Comparing means (for example, step S23 in FIG. 3) for comparing the second parameter of the image with the image data of the second image when the second parameter is smaller than the first parameter. To quantize based on
And a control unit (for example, steps S26 and S28 in FIG. 3) for controlling the quantization unit.

An embodiment of an image processing apparatus to which the present invention is applied will be described below with reference to FIG.

The arithmetic unit 1 calculates, from the input image data,
The prediction image data motion-compensated by the motion compensator 8 is subtracted, and the difference data is output to the quantizer 2. The quantizer 2 quantizes the input image data and outputs it to the variable length encoder 3 and the inverse quantizer 5. The variable-length encoder 3 includes the quantized data input from the quantizer 2 and
The motion vector supplied from the motion vector detector 9 is subjected to variable-length encoding and output to the buffer 4. Buffer 4
The variable length data supplied from the input variable length encoder 3 is output to a predetermined transmission path.

The inverse quantizer 5 inversely quantizes the quantized data input from the quantizer 2 and outputs the result to an arithmetic unit 6. The arithmetic unit 6 adds the predicted image data motion-compensated by the motion compensator 8 to the difference data input from the inverse quantizer 5 and converts the data into the original image data.
And store it. The motion vector detector 9 detects a motion vector of the input image data, and outputs the motion vector to the motion compensator 8 and the variable length encoder 3. The motion compensator 8 performs motion compensation on the image data read from the frame memory 7 in accordance with the motion vector, and generates predicted image data.

The parameter detector 10 detects a parameter corresponding to the complexity of the input image data using the output of the motion vector detector 9 and outputs it to the rate controller 11. The rate controller 11 is also supplied with data on the occupation amount of the data in the buffer 4. The rate controller 11 controls the quantization step of the quantizer 2 according to the data from the parameter detector 10 and the buffer 4.

The configuration on the transmitting side has been described above. Next, the configuration on the receiving side will be described. The buffer 31 temporarily stores the encoded data supplied via the transmission path, and outputs this to the variable length decoder 32. The variable length decoder 32
The input coded data is demodulated, the image data component is supplied to the inverse quantizer 33, and the motion vector is converted to the motion compensator 36.
Output to The arithmetic unit 34 adds the predicted image data supplied from the motion compensator 36 to the output of the inverse quantizer 33 and outputs the result as decoded image data. This decoded image data is supplied to the frame memory 35 and stored. The motion compensator 36 performs motion compensation on the image data stored in the frame memory 35 according to the motion vector, and generates prediction data.

Normally, the quantized data is transformed into a variable length code after the DCT transformation. Therefore, the inverse DCT transformation is performed before the inverse quantization. However, for the sake of simplicity, it is omitted here. ing.

Next, the basic operation will be described. On the transmitting side, the motion vector detector 9 detects a motion vector of the input image data and outputs the motion vector to the motion compensator 8. The motion compensator 8 performs motion compensation on the image data read from the frame memory 7 in accordance with the motion vector, generates predicted image data,
Output to arithmetic unit 1. The arithmetic unit 1 subtracts the predicted image data from the input image data, and outputs the result to the quantizer 2. The quantizer 2 quantizes the input image data,
Output to the variable length encoder 3. The variable-length encoder 3 converts the quantized data input from the quantizer 2 and the motion vector supplied from the motion vector detector 9 into a variable-length code, and transmits the variable-length code to the transmission path via the buffer 4. .

On the other hand, the quantized data output from the quantizer 2 is inversely quantized by the inverse quantizer 5 and then input to the arithmetic unit 6. The arithmetic unit 6 adds the predicted image data output by the motion compensator 8 to the output of the inverse quantizer 5, restores the original image data, supplies the original image data to the frame memory 7, and stores it.

On the receiving side, on the other hand, the encoded data transmitted from the transmission path is temporarily stored in the buffer 31 and then input to the variable length decoder 32. Variable length decoder 3
2 decodes the input coded data, outputs the image data component to the inverse quantizer 33, and supplies the motion vector to the motion compensator. The inverse quantizer 33 inversely quantizes the input image data and outputs the result to the arithmetic unit. The motion compensator 36 performs motion compensation on the image data stored in the frame memory 35 in accordance with the motion vector, generates predicted image data, and outputs it to the calculator 34. Arithmetic unit 34
Adds the output of the inverse quantizer 33 and the predicted image data output from the motion compensator 36, decodes the original image data, and outputs the decoded image data. This image data is also supplied to and stored in the frame memory 35 for generating predicted image data.

The above is the basic operation. Next, control of the quantization step in the quantizer 2 will be described. First, the parameter detection operation will be described.

FIG. 2 is a flowchart for explaining the operation of calculating the parameters of the image processing apparatus of FIG. First, in step S1, the parameter detector 10
It is determined whether or not the target macroblock is an intra macroblock. If the macroblock of interest is determined to intra-macro block, the process proceeds to step S2, the parameter detector 10, for each data x of the target macroblock, the difference between the average values x _AVE data (x-
sum of absolute values of x _AVE ) | xx _AVE | (MAE (MeanAbsolu
te Error)). Further, the parameter detector 10
Calculates the average value of the MAE (P-MAE) by dividing the total sum of each macroblock in the picture by the number n of macroblocks in the picture in step S3.

If it is determined in step S1 that the target and the macroblock are not intra macroblocks (inter macroblocks), the process proceeds to step S4.
The motion vector detector 9 calculates a value obtained by subtracting the search macroblock data x ′ from the target macroblock data x (x
−x ′) is calculated, and the sum (DIST) of the absolute value | xx ′ | of the sum of the sum is supplied to the parameter detector 10. Upon receiving the supply of the DIST from the motion vector detector 9, the parameter detector 10 divides the sum of the DIST in the picture by the number n of macroblocks in step S5, and
Calculate the average ST value (P-DIST).

After the processing in step S3 or step S5, the process proceeds to step S6, where the parameter detector 10 and the motion vector detector 9 determine whether or not the processing data has been completed. If it is determined that the processing data has ended, the processing ends. If it is determined that the processing data is not completed, the process returns to step S1, and the parameter detector 10 and the motion vector detector 9 repeat the same processing.

Next, the operation of the image processing apparatus of FIG. 1 for controlling the quantization step of the quantizer 2 after a scene change will be described with reference to the flowchart of FIG.

First, in step S21, the parameter detector 10 and the motion vector detector 9 perform parameter calculation processing as described with reference to the flowchart of FIG. The parameter detector 10 supplies the calculated new parameters P-MAE and P-DIST to the rate controller 11. The rate controller 11 determines in step S22
Then, the value of the new parameter supplied from the parameter detector 10 is set as the current parameter. In step S23, the rate controller 11 determines whether or not the value of the old parameter stored so far is twice or more the new parameter detected in step S21.

If YES is determined in the step S23, the process proceeds to a step S24, where the rate controller 11
Sets flag = 1, and if NO is determined, the process proceeds to step S27 to set flag = 0. When flag = 1 is set (when it is determined that the new parameter has greatly decreased from the reference value), the process further proceeds to step S25, and the rate controller 11 holds the new parameter detected in step S21 in an internal memory. . Rate controller 11
Is the old parameter (step S31)
Is set to the current parameter.

In step S27, when flag = 0 is set (when it is determined that the new parameter is not greatly reduced or increased), or in step S26.
After the end of the process, the process proceeds to step S28, and the rate controller 11 performs a process of controlling the quantization step.
Details of the control processing of this quantization step are shown in the flowchart of FIG.

In the control operation of the quantization step shown in FIG. 4, first in step S51, the rate controller 11
Calculates the information amount allocation from the following equation. {P-MAE / (P-MAE x I + P-DIST _p x P + P-DIST _b x B)}
× bit rate Here, the values of I, P, and B are I picture and P, respectively.
Indicates the ratio of the basic bit rate for allocating the data amount of a picture or a B picture, for example, I: P: B
= 1: 4: 10.

The above equation is an equation for an I picture, and for a P picture or a B picture, the numerator of the above equation is P-DIST _p or P-DIST _b .

The rate controller 11 determines in step S5
In step 2, it is determined whether the data amount of the buffer 4 overflows by allocating the bit amount. If it is determined that the buffer 4 overflows, step S
Proceeding to 53, the rate controller 11 adjusts the calculation result so that the buffer 4 does not overflow.

In step S52, when it is determined that the buffer 4 does not overflow, or in step S53
Is completed, the rate controller 11
Is based on the calculation result, and in step S54,
Is controlled. In step S55, the quantizer 2 performs quantization in the specified quantization step.
Further, the process proceeds to step S56, where the rate controller 1
1 judges whether or not the processing data is completed. If it is determined that the processing data is not completed, step S51
And the rate controller 11 repeats the same processing. When it is determined that the processing data has ended, the rate controller 11 ends the processing.

Returning to FIG. 3, the rate controller 11
Determines whether flag = 1 in step S29. When it is determined that flag = 1, the process proceeds to step S30, and the rate controller 11 reads the new parameter stored in step S25 and sets the new parameter as the current parameter. That is, the rate controller 11 determines in step S2
The processing reverse to that performed in step 6 is performed.

After the processing in step S30, and when it is determined in step S29 that flag = 0 (not flag = 1) (skipping the processing in step S30), the flow proceeds to step S31. In step S31, the rate controller 11 stores the current parameter as an old parameter. The rate controller 11 further performs step S3
At 2, it is determined whether the processing data is completed. If it is determined that the processing data has not been completed, the rate controller 11 returns to step S21 and repeats the same processing. When it is determined that the processing data has ended, the rate controller 11 ends the processing.

FIG. 5 is a diagram showing the operation of the image processing apparatus of FIG. 1 for controlling the parameters of the picture after the scene change. In this example, when the parameter after the scene change is A, it is larger than the parameter before the scene change, and when B, the parameter is smaller than the parameter before the scene change by a reference value or more (less than twice). ing).

As in the case of FIG. 5A, the fact that the parameter increases after a scene change is performed means that
This means that the amount of generated bits increases. In that case, the rate controller 11 attempts to allocate a larger number of bits (bits held in the buffer 4) to the subsequent frame in the quantization step, but already allocates a lower allocation to the preceding frame. As a result, no bits can be allocated in the buffer 4. Also, since the amount of generated bits is in the direction of increasing, noise is not so noticeable visually. Therefore, the rate controller 11 controls the quantization step as in the related art.

On the other hand, as in the case of FIG.
If the parameter becomes smaller, the rate controller 1
The number of bits that 1 allocates after a scene change decreases.
However, since a scene change has occurred, it is difficult for the rate controller 11 to predict the amount of generated bits based on the parameters of the immediately preceding picture, as in the past, in allocating bits (control of the quantization step). become. In this case, it is considered that the buffer 4 contains many bits. Therefore, the rate controller 11
Does not use the reduced parameter immediately after the scene change, but uses the large parameter immediately before the scene change as it is, so that the generated bit amount is not suppressed.

That is, as shown in FIG. 5, when pictures occur in the order of P, B, B after the scene change,
If the rate controller 11 calculates the control amount of the quantization step using the new parameter (small value) of the P picture immediately after the scene change, the generated bit amount is reduced. The calculation is performed using the (old parameter) (large value) as it is. Thereafter, the rate controller 11 restores the new parameters immediately after the scene change, and continuously performs the allocation processing.

The above processing is schematically described as shown in FIG. That is, as shown in FIG.
When a simple image F1 changes to a complex image F2 due to a scene change, the parameters increase, but in this case, the noise is not so noticeable.

On the other hand, as shown in FIG.
A complex image F11 changes to a simple image F after a scene change.
When the parameter changes to 12, the parameter changes to a value smaller than the reference value. A simple image is a flat image with little change, and when noise occurs therein, the noise becomes extremely noticeable. Therefore, as described above, in the present invention, this simple image F12 is replaced with a complicated image F12.
Assuming that the image is a complicated image as in the case of 11, the quantization step is controlled to prevent the noise in the simple image F12 from being noticeable.

Note that such a phenomenon is particularly noticeable at the time of a scene change, but it goes without saying that the present invention may be applied to a case other than a scene change. Further, in the above embodiment, the reference value of the parameter change is doubled, but may be another value.

Further, as a providing medium for providing a user with a computer program for performing the above-described processing, a recording medium such as a magnetic disk, a CD-ROM, and a solid-state memory, as well as a communication medium such as a network and a satellite are used. be able to.

[0050]

As described above, according to the image processing apparatus according to the first aspect, the image processing method according to the second aspect, and the providing medium according to the third aspect, the second parameter is set to the first parameter.
When the second image data is smaller than the parameter, the second image data is quantized based on the first parameter.
In particular, in the case of a scene change, it is possible to suppress the occurrence of noise assumed when a complex image is changed to a flat image, and to improve the image quality at the time of a scene change.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus to which the present invention has been applied.

FIG. 2 is a flowchart illustrating a parameter calculation operation of the image processing apparatus in FIG. 1;

FIG. 3 is a flowchart illustrating a quantizer control operation of the image processing apparatus in FIG. 1;

FIG. 4 is a flowchart illustrating details of quantization step control in step S28 in FIG. 3;

FIG. 5 is a diagram illustrating changes in parameters before and after a scene change of the image processing apparatus in FIG. 1;

FIG. 6 is a diagram illustrating the relationship between the complexity of images before and after a scene change.

[Explanation of symbols]

1 arithmetic unit, 2 quantizer, 3 variable length encoder,
4 buffer, 5 inverse quantizer, 6 arithmetic unit, 7
Frame memory, 8 motion compensator, 9 motion vector detector, 10 parameter detector, 11 rate controller, 31 buffer, 32 variable length decoder, 33 inverse quantizer, 34 arithmetic unit, 35 frame memory, 36 motion compensator

Claims (3)

[Claims] A quantizing means for quantizing the image data; a detecting means for detecting a parameter representing the complexity of the image; a first parameter of a first image which is temporally earlier; Comparing means for comparing a second parameter of the second image with the image data of the second image based on the first parameter when the second parameter is smaller than the first parameter. An image processing apparatus comprising: a control unit that controls the quantization unit so as to perform quantization. 2. A quantization step of quantizing the image data, a detection step of detecting a parameter representing the complexity of the image, a first parameter of the first image before the time, and a A comparing step of comparing a second parameter of the second image, and when the second parameter is smaller than the first parameter, converting the image data of the second image based on the first parameter. A control step of controlling quantization in the quantization step so as to perform quantization. 3. A quantization step of quantizing the image data, a detection step of detecting a parameter representing the complexity of the image, a first parameter of the first image before the time, and a A comparing step of comparing a second parameter of the second image, and when the second parameter is smaller than the first parameter, converting the image data of the second image based on the first parameter. Providing a computer program comprising: a control step of controlling quantization in the quantization step so as to perform quantization.