JP2003163933A - Image-coding equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To select superfineness in an image within a limited amount of code. <P>SOLUTION: The image coding equipment for coding an image has a conversion section 6 that carries out discrete cosine conversion to the image for calculating a discrete cosine conversion coefficient, a quantization section 7 that quantizes the discrete cosine conversion coefficient based on a quantization step for calculating a quantization coefficient, and a control section 19 that calculates an image characteristic parameter according to the discrete cosine conversion coefficient and decides the quantization step based on the image characteristic parameter. The image characteristic parameter is calculated from the discrete cosine coefficient, thus obtaining the image characteristic parameter by a small amount of operation. Additionally, the image characteristic parameter is used, thus accurately predicting the amount of generation code. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to the H.263 standard, MPE.
The present invention relates to an image coding apparatus suitable for image coding based on the G standard and the like, and particularly to an improvement for reducing the code amount.

[0002]

2. Description of the Related Art As a technique for compressing an image, that is, reducing the amount of an image signal by encoding a moving image, an encoding technique represented by the well-known MPEG standard is known. In the example of the encoding technology of the MPEG standard, interframe encoding is performed by incorporating motion compensation using a motion vector expressing the amount of local image motion between frames. As a result, the compression efficiency of moving images is dramatically improved.

[0003]

However, in the conventional image coding technique, in order to transmit a realistic moving image using a transmission path with a limited transmission capacity such as a telephone line, it is still necessary to transmit the encoded signal. The problem is to keep the signal amount, that is, the code amount low.

The present invention has been made to solve the above-mentioned problems in the conventional image coding technique, and an object thereof is to provide an image coding apparatus capable of reducing the code amount.

[0005]

According to a first aspect of the present invention, there is provided an image coding apparatus for coding an image, wherein the image coding apparatus performs a discrete cosine transform on the image to calculate a discrete cosine transform coefficient, and For the discrete cosine transform coefficient, a quantization unit that performs quantization based on a quantization step to calculate a quantized coefficient, and an image characteristic parameter is calculated from the discrete cosine transform coefficient, based on the image characteristic parameter, And a control unit that determines the quantization step.

The apparatus of the second invention is the image coding apparatus of the first invention, further comprising a variable length coding section for performing variable length coding on the quantized coefficient while referring to a variable length coding table. Further, the control unit calculates the image characteristic parameter based on an image type, a macroblock type, and a method adapted to the variable length coding table.

A third aspect of the invention is the image coding apparatus according to the first or second aspect of the invention, wherein the control section further comprises:
A feature is that the target code amount in the next macroblock to be encoded is determined based on the distribution history of the image characteristic parameters of the past image, the visual characteristic, and the generated code amount in the encoding. To do.

An apparatus according to a fourth aspect of the present invention is an image coding apparatus for coding an image, wherein a transform unit for performing discrete cosine transform on the image to calculate a discrete cosine transform coefficient and a discrete cosine transform coefficient for the discrete cosine transform coefficient. The quantization unit that performs quantization based on the quantization step to calculate the quantization coefficient, and for each macroblock, the position in the screen, and from the discrete cosine transform coefficient in the image to be encoded, An image characteristic parameter is calculated, based on the image characteristic parameter, a weighting parameter for performing weighting according to the degree of visual importance is calculated, and further, based on the weighting parameter, a target for each macroblock is calculated. A control unit that allocates a code amount and weights the quantization step is provided.

[0009]

BEST MODE FOR CARRYING OUT THE INVENTION <1. First Embodiment> First, an apparatus according to the first embodiment will be described.

<1-1. Overall Configuration of Apparatus> FIG. 1 is a block diagram showing the configuration of the image encoding apparatus according to the first embodiment.
As will be described later, in the device of the first embodiment, an image that has been inter-coded, which is coding between frames, has been partially subjected to intra-coding, which is intra-frame coding. The device portion (the mode switching unit 20 in FIG. 1) that incorporates an image has a main feature. Therefore, FIG. 1 exemplifies a device that performs image signal encoding based on the widely known MPEG standard. However, the invention embodied in the first embodiment is an intra-coded image and an inter-coded image. It is applicable to general image coding apparatuses that obtain and transmit both of these.

First, an outline of the overall configuration of the apparatus shown in FIG. 1 will be described, including the peripheral portion which is a prerequisite for the configuration of the mode switching unit 20. As shown in FIG. 1, an image input unit 1 such as a color camera outputs an image signal representing a captured image in a digital signal format. In this specification, in order to avoid unnecessary complication of description, the image signal representing an image and the image itself are equally expressed as “image” within a range where there is no possibility of confusion.

The image output from the image input unit 1 is temporarily stored in the frame memory 2 in frame units. After that, the frame memory reading unit 3 reads the images from the frame memory 2 in a fixed order for each frame, and the difference circuit 4 and the switching unit 5 as the current image 22.
Is input to.

The difference circuit 4 calculates the difference in pixel value between the current image 22 sent from the frame memory reading section 3 and the predicted image 12 sent from the prediction memory 11. The switching unit 5 uses the difference image 2 output from the difference circuit 4.
3 and the current image 22 from the frame memory reading unit 3
And output any one of them. The signal output by the switching unit 5 is input to the conversion unit 6.

The transform unit 6 executes so-called discrete cosine transform (DCT). In this discrete cosine transform,
For example, for each block including 8 × 8 pixels, the input image has the same number of spatial frequency components (DCT coefficients) as the number of pixels configuring the block (for example, 8 × 8 = 64).
Is converted into a set of. The DCT coefficient obtained by the transformation is input to the quantizer 7.

The quantizer 7 refers to the set of coefficient values defined in the quantization table provided therein and the quantization step instruction signal 25 sent from the encoding parameter controller 19 to refer to each DCT. Quantization is performed for each coefficient. That is, the DCT coefficient is converted into a quantized coefficient. The quantization step instruction signal 25 uniformly assigns a common scaling factor to a set of coefficient values defined in the quantization table, and thereby the quantization width based on the quantization table, that is, the quantization step, All DCT coefficients are corrected by a common magnification. The smaller the quantization step is, the more the deterioration in image quality is suppressed, but the larger the code amount.

The quantized coefficient obtained by the quantizing unit 7 is inputted to the variable length coding unit (VLC) 15 and at the same time inputted to the dequantizing unit 8. The inverse quantizer 8 performs an operation reverse to that of the quantizer 7. Therefore, a signal having the same format as the DCT coefficient before being quantized is obtained.

However, since the processing in the quantizer 7 is generally irreversible, the DC reconstructed in the inverse quantizer 8 is used.
The T coefficient is generally not the same as the DCT coefficient output from the conversion unit 6. That is, the output of the inverse quantization unit 8 generally includes the quantization error caused by the processing of the quantization unit 7. Needless to say, the smaller the quantization step, the smaller the quantization error.

The DCT coefficient obtained by the inverse quantizer 8 is input to the inverse transformer 9. The inverse transform unit 9 performs an operation that is the inverse of DCT (inverse DCT). As a result, an image signal of the same format as the image signal before being subjected to DCT conversion in the conversion unit 6 is obtained. Needless to say, this image signal
Generally, it is not the same as the image signal input to the conversion unit 6, and includes a quantization error.

The reconstructed image signal is stored as a reference image 13 in the prediction memory 11 on a frame-by-frame basis.
The reference image 13 stored in the prediction memory 11 is input to the motion vector detection unit 14. The current image 22 read from the frame memory reading unit 3 is also input to the motion vector detecting unit 14 at the same time. The motion vector detection unit 14 uses the current image 22 for each macroblock.
The portion of the reference image 13 closest to is searched. The searched part is referred to as a predicted image.

The motion vector detecting section 14 further sets, as a motion vector 24, a vector expressing the displacement from the predicted image to the macroblock on the current image 22 corresponding to the predicted image, and the variable length code. It is sent to the conversion unit 15. The prediction memory 11 stores the reference image 13 for each macroblock on the basis of the motion vector 24.
The predicted image 12 is extracted from the above and is sent to the difference circuit 4.

Here, a macroblock is a unit of an image in which the motion vector 24 is defined, and for example, 16 × 16 pixels for a luminance component of an image signal and 8 × 8 pixels for a color difference component. Composed of. 16 for luminance component
When a macro block is defined by x16 pixels, one macro block is 4 for the luminance component.
The DCT conversion process in the conversion unit 6 and the quantization process in the quantization unit 7 are performed in units of a block composed of divided 8 × 8 pixels.

The switching unit 5 is connected to a control unit (not shown), and selects one of the current image 22 and the difference image 23 based on an instruction from this control unit, as described above. Output to the conversion unit 6. Current image 2
When 2 is selected, intra-frame encoding, that is, intra-encoding is performed. The image thus obtained is called an I picture. When the difference image 23 output by the difference circuit 4 is selected, interframe predictive coding, which is a type of inter coding, is performed. The image thus obtained is called a P picture.

A mode switching unit 20 for switching the mode between intra / inter is inserted between the difference circuit 4 and the switching unit 5. Therefore, the difference image 23
Is not directly input to the switching unit 5, and the mode switching unit 2
Through 0, the mode selection image 10 output by the mode switching unit 20 is input to the switching unit 5. That is, the switching unit 5 directly selects and outputs either the current image 22 or the mode selection image 10.

The mode switching unit 20 selects one of the current image 22 and the difference image 23 in macro block units, and sends it to the switching unit 5 as a mode selection image 10. That is, the mode switching unit 20 functions to mix intra-coded I-pictures on a macroblock-by-macroblock basis in P-pictures that should originally be inter-coded.

The variable-length coding unit 15 executes a variable-length coding process such as Huffman coding on the input quantized coefficient, and at the same time, the motion vector detecting unit 1
The motion vector 24 sent out from No. 4 is added, and as a result, a coded signal is obtained. The obtained encoded signal is subjected to multiplexing processing in the subsequent multiplexing unit 16 and then temporarily stored in the transmission buffer 17.

The encoded signal accumulated in the transmission buffer 17 is sent to the transmission path 18 at a suitable time. From the transmission buffer 17, a status signal notifying that state, that is, that the buffer is overflowing or empty, is sent to the encoding parameter control unit 19. Since the encoded signal includes the motion vector 24, a decoding device that receives the encoded signal and reconstructs the current image 22 can perform decoding in consideration of the motion vector 24.

<1-2. Mode Switching Unit> Next, the mode switching unit 20 which is a feature of the present invention will be described. The mode switching unit 20 can realize a predetermined operation by including a memory having a program installed therein and a computer that operates according to the program.
The mode switching unit 20 can also be configured only by hardware including an integrated circuit without depending on a program.

First, for comparison and comparison, an example in which the mode switching section 20 operates based on "Test Model Near-Term 5" (hereinafter abbreviated as "TMN 5") will be described. "TMN 5" is a test model described by a research committee belonging to ITU-T (one of the permanent organizations of the International Telecommunication Union ITU). In "TMN 5", mode switching between intra / inter is performed as follows.

First, the average value mean of all luminance components (Y components) of one macroblock in the current image is calculated. That is, the mean value mean
Is calculated. Formula 1 assumes that one macroblock is composed of 16 × 16 pixels with respect to the luminance component. Then, the symbol Y _Ci (i = 1, ..., 4) in Equation 1
Represents the luminance components of all pixels in the i-th block (i = 1, ..., 4), which is one of four blocks obtained by dividing one macroblock into four equal parts.

[0030]

[Equation 1]

Next, the average value mean is subtracted from the luminance component for each block in the manner shown in the equation 2, and the sum A of the absolute values of the obtained differences is calculated.

[0032]

[Equation 2]

Next, the luminance component Y _Ci
The luminance component Y _pi and the luminance component Y _{Ci of} the macroblock corresponding to the motion vector are subtracted, and the sum sum of the absolute values of the differences is calculated.

[0034]

[Equation 3]

However, when both the x and y components (horizontal and vertical components on the screen) of the motion vector have the value 0, the sum sum of the absolute values is deducted by the value 100, as shown in Equation 4. Be done.

[0036]

[Equation 4]

Based on the sum sum thus obtained,
Mode selection is performed. That is, the intra mode is selected when the relationship of the equation 5 is true (true), and the inter mode is selected when the relationship is not true (false).

[0038]

[Equation 5]

The above is the intra / based on "TMN 5"
This is an algorithm for mode switching between inters.

On the other hand, in the mode switching unit 20 of this embodiment, mode switching is performed in the following manner. First, according to the procedure shown in Equation 6, for each of the four blocks,
The average value of the luminance component is calculated. In the formula 6, the symbol me
an (i) represents the average value of the luminance components in the i-th block (i = 1, ..., 4).

[0041]

[Equation 6]

Then, the average value mean (i) is subtracted from the luminance component for each block in the manner shown in Expression 7, and the sum A of the absolute values of the obtained differences is calculated.

[0043]

[Equation 7]

Next, the luminance component Y _Ci
The luminance component Y _pi and the luminance component Y _{Ci of} the macroblock corresponding to the motion vector are subtracted, and the sum sum of the absolute values of the differences is calculated.

[0045]

[Equation 8]

Based on the sum sum thus obtained,
Mode selection is performed. That is, the intra mode is selected when the relationship of the equation 5 is true (true), and the inter mode is selected when the relationship is not true (false).

In the test model "TMN 5", when there is a large difference between the average values of the brightness components in each block even if the variance of the brightness components is small among the blocks belonging to one macroblock. The mode with the larger code amount may be selected. For example, the case where the average value of the luminance components regarding each block included in one macroblock is 125, 36, 126, 38 and the variance is 8, 15, 9, 16 corresponds to this.

Unlike "TMN 5", the mode switching unit 20 of this embodiment does not calculate the average value of the luminance components over the entire macroblock, but calculates the average value for each block and then calculates the average value. The sum of the absolute value difference between the value and the luminance component in each block is used as a criterion for mode switching. Therefore, the mode switching unit 20 appropriately selects a smaller code amount to be generated. That is, there is an advantage that the code amount of the coded signal transmitted to the transmission path 18 can be suppressed low.

<2. Second Embodiment> The overall configuration of the image coding apparatus according to the second embodiment is shown in the block diagram of FIG.
In the second embodiment, the coding parameter control unit 19, which is the device portion that determines the quantization step, has a feature different from that of the first embodiment. That is, in the second embodiment, the coding parameter control unit 19 uses the image characteristic parameter calculated from the DCT coefficient of the image to be coded (coding target image) to achieve the target code amount. The optimum quantizing step instruction value (QUANT) is determined. The determined QUANT is sent to the quantization unit 7 in the form of a quantization step instruction signal 25 which is a signal expressing it.

The image characteristic parameter is a picture type such as an I / P picture to which an image to be coded belongs,
It is determined using a type adapted to the macroblock and a VLC (Variable Length Coding) table such as a Huffman table. QUANT determined by the above procedure
By performing the quantization by using, the encoding with the code amount within ± 5% with respect to the target code amount is achieved.

2 and 3 show the flow of encoding when the image to be encoded is an I picture. Also, FIG.
~ Fig. 6 shows the flow of encoding when the image to be encoded is a P picture. The encoding procedure will be described below with reference to these flowcharts.

As shown in FIG. 2, when encoding is started when the image to be encoded is an I picture, first,
In step S1, initialization is performed. That is,
Of the variables used for control such as rate control, all those that require initialization are initialized. Next, the processing of the first loop from steps S2 to S6 is repeatedly executed for each macroblock. First
In the loop, data for rate control is prepared.

First, in step S3, the conversion unit 6 performs DCT conversion. In subsequent step S4, the pseudo DC / AC coefficient prediction is performed with reference to the DCT coefficient values of adjacent macroblocks in the same image. The prediction mode is also pseudo determined by these data. Next, in step S5, image characteristic parameters are calculated. That is, the image characteristic parameter for performing rate control is calculated. The required data is all pseudo DC / AC of Y, Cr and Cb components.
It is the sum of absolute values of DCT coefficients after coefficient prediction.

When the first loop is iteratively executed for all macroblocks of the image to be coded, the process proceeds to step S7. In step S7, the I / P ratio is predicted and the code amount allocation is determined. That is, the ratio of the code amount of the I / P picture (I / P ratio) is estimated based on the code amount of the immediately preceding P picture and the predicted code amount obtained in the first loop.
And based on that, GOP (Group of Picture
s: code amount allocation to each picture forming a screen group composed of I, P pictures, etc. is determined. This processing is performed in the encoding parameter control unit 19, for example.

Next, as shown in FIG. 3, step S8
The processing of the second loop from to S18 is repeatedly executed for each macroblock. In the second loop, encoding is performed. First, in step S9, the remaining bits and QU
The ANT is updated. That is, the remaining bits and QUANT are calculated based on the amount of generated code up to the current target macroblock and the image characteristic parameter obtained in the first loop. This processing is executed by the encoding parameter control unit 19.

Next, in step S10, actual DC / AC coefficient prediction is performed. This DC / AC coefficient prediction is performed based on the locally decoded image signal. In the following step S11, quantization is performed. Quantization is executed by the quantizing unit 7 based on QUANT obtained in the previous step. Next, in step S12, variable length coding is performed. That is, the quantized coefficient obtained by the quantization is converted into a one-dimensional array, and then the variable length coding is performed. This processing is executed by the variable length coding unit 15.

Next, in step S13, the rate control parameter is updated. In the subsequent step S14, the quantization coefficient is inversely quantized. This process is executed by the inverse quantizer 8. Subsequently, in step S15, DC / AC coefficient compensation is performed. Then, in step S16, the inverse DC
T conversion is performed. This process is executed by the inverse conversion unit 9. Next, in step S17, the decoded image is stored in the prediction memory 11.

When the second loop is iteratively executed for all macroblocks of the image to be coded, the process then proceeds to step S19. Step S19
Then, the stuff bit is inserted. That is, when the generated code amount is smaller than the allocated bit amount, the stuff bit is inserted. By the above procedure, the I picture is encoded.

As shown in FIG. 4, when encoding is started when the image to be encoded is a P picture, first,
In step S21, initialization is performed. That is, of the variables used for control such as rate control, all those that require initialization are initialized. Next, the processing of the first loop of steps S22 to S30 is repeatedly executed for each macroblock. In the first loop, data for rate control is prepared.

First, in step S23, motion detection is performed. In a succeeding step S24, the intra / inter mode is judged. That is, either the intra mode or the inter mode is determined based on the result of the motion detection. Next, in step S25,
DCT conversion is performed. That is, the DCT transform is applied to the image to be encoded (current image for intra mode, difference image 23 for inter mode). This process is executed by the conversion unit 6.

Next, in step S26, it is determined whether or not the intra mode is set. Then, if it is the intra mode, the process proceeds to step S27, and if it is not the intra mode, the process proceeds to step S28. In step S27, pseudo DC / AC coefficient prediction is performed. The intra prediction mode is pseudo-determined. When step S27 ends, the process proceeds to step S28. In step S28, the prediction of the code amount that occurs other than the quantized DCT coefficient, such as the motion vector difference, is performed. This process is executed by the encoding parameter control unit 19, for example.

Next, in step S29, image characteristic parameters are calculated. That is, the image characteristic parameter for performing rate control is calculated. The data required differs between intra blocks and inter blocks. In intra block, Y, C
D after prediction of all pseudo DC / AC coefficients of r and Cb components
It is the sum of absolute values of CT coefficients. In the inter block, the sum of the absolute values of the AC components and the maximum value of the absolute values of the DCT coefficients are the maximum values of all the Y, Cr, and Cb components of the DCT coefficient after motion detection.

When the first loop is iteratively executed for all macroblocks of the image to be coded, thereafter, as shown in FIG. 5, the processes of the second loop of steps S31 to S40 are executed. It is repeatedly executed for each macroblock. In the second loop, encoding is performed.

When the second loop is started, first, in step S32, the remaining bits and QUANT are updated. That is, the remaining bits and QUANT are calculated based on the amount of generated code up to the current target macroblock and the image characteristic parameter obtained in the first loop. This process is performed by the encoding parameter control unit 19
Executed by

Next, in step S33, it is determined whether or not the macro block currently targeted is an intra block (macro block in intra mode). If it is an intra block, the process proceeds to step S34, and if not, step S34.
Leap to 35. In step S34, actual DC / AC coefficient prediction is performed based on the locally decoded image signal. After step S34, the process proceeds to step S35.

At step S35, quantization is performed.
Quantization is executed by the quantizing unit 7 based on QUANT obtained in the previous step. Next, in step S36, variable length coding is performed. That is,
The quantized coefficient obtained by the quantization is converted into a one-dimensional array, and then the variable length coding is performed. Step S36
The process of is executed by the variable length coding unit 15.

Next, in step S37, the rate control parameter is updated. In the following step S38, inverse quantization is performed on the quantized coefficient. This process is executed by the inverse quantizer 8. Subsequently, in step S39, it is determined whether or not the currently targeted macroblock is an intra block. If it is an intra block, the process proceeds to step S40, and if not, step S4.
Leap to 1.

In step S40, DC / AC coefficient compensation is performed. That is, in the intra block,
The coefficients are restored based on the intra prediction mode for each macroblock. When step S40 ends, the process proceeds to step S41 as shown in FIG. In step S41, inverse DCT conversion is performed. This process is executed by the inverse conversion unit 9.

Next, in step S42, it is determined whether or not the macro block currently targeted is an inter block (inter mode macro block). If it is an inter block, the process proceeds to step S43, and if not, the process jumps to step S44. In step S43, motion compensation is performed. That is, motion compensation is performed on the inter block based on the motion vector. Step S43
After that, the process proceeds to step S44.

In step S44, the locally decoded image is stored in the prediction memory 11. When the second loop is iteratively executed for all macroblocks of the image to be encoded, the process then proceeds to step S46.
Move to. In step S46, stuff bits are inserted. That is, when the generated code amount is smaller than the allocated bit amount, the stuff bit is inserted.
By the above procedure, the P picture is encoded.

In the above processing, the image characteristic parameter X _Ii of the intra block (i-th macro block) is
It is calculated based on Equation 9.

[0072]

[Equation 9]

Here, the code coeffs represents a coefficient, and the code bl
ks represents a block forming a macro block.

The image characteristic parameter X _Pi of the inter block (i-th macro block) is calculated based on the equation 10.

[0075]

[Equation 10]

Further, the image characteristic parameter X of the intra block (i-th macro block) in the P picture
_Ii is converted into an inter-block parameter X _Pi based on _Equation 11.

[0077]

[Equation 11]

QUANT for the i-th macroblock in the picture is calculated based on the formula 12 for the I picture and based on the formula 13 for the P picture.

[0079]

[Equation 12]

[0080]

[Equation 13]

In the prior art, standard deviation or SAD (sum of absolute differences) was used to determine QUANT. When the standard deviation was used, it was necessary to calculate the standard deviation separately from the calculation required for image coding. Further, when SAD is used, there is a problem that the prediction accuracy of the generated code amount is low. In addition, whichever is used, DC / A in the I-picture screen
There is a problem that the coding amount prediction corresponding to the C coefficient prediction cannot be performed.

On the other hand, in the device of the second embodiment,
As described above, DCT is used to determine QUANT.
The image characteristic parameter obtained from the coefficient is used. The image characteristic parameter has an advantage that it can be obtained from the DCT coefficient with a small amount of calculation. Further, since the image characteristic parameter is used, there is an advantage that the generated code amount can be predicted with high accuracy. Further, there is an advantage that the code amount prediction corresponding to the DC / AC coefficient prediction in the screen of the I picture can be performed.

<3. Third Embodiment> The overall configuration of the image coding apparatus according to the third embodiment is also shown in the block diagram of FIG. 1 similarly to the second embodiment. In the second embodiment, the processing is performed along two loops, that is, the first and second loops (FIGS. 2 to 6). On the other hand, the third embodiment is characteristically different from the second embodiment in that the two loops are configured to be integrated into a single loop. That is, the apparatus according to the third embodiment is based on the distribution history of image characteristic parameters of past images, the visual characteristics, and the generated code amount of the encoding target image that is being encoded, and is based on the current encoding target. The main feature is that the target code amount of the macroblock to be encoded is determined next to the macroblock.

7 and 8 show the flow of encoding when the image to be encoded is an I picture. In addition, FIG.
11 shows the flow of encoding when the image to be encoded is a P picture. The encoding procedure will be described below with reference to these flowcharts.

As shown in FIG. 7, when encoding is started when the image to be encoded is an I picture, first,
In step S51, initialization is performed. That is, of the variables used for control such as rate control, all those that require initialization are initialized. In the following step S52, the I / P ratio is estimated,
Also, the code amount allocation is determined. That is, the ratio of the coding amount of I picture / P picture (I / P ratio) is estimated from the latest coding amounts of I picture and P picture. Further, based on this, the code amount allocation to each picture forming the GOP is determined. This processing is performed in the encoding parameter control unit 19, for example.

Next, the processing of the loop from steps S53 to S66 is repeatedly executed for each macroblock. When the loop is started, first, in step S54, the conversion unit 6 performs DCT conversion. In subsequent step S55, pseudo DC / AC coefficient prediction is performed with reference to the DCT coefficient values of adjacent macroblocks in the same image. The prediction mode is also pseudo determined by these data.

Next, in step S56, the image characteristic parameters are calculated. That is, the image characteristic parameter for performing rate control is calculated. The required data is the sum of absolute values of DCT coefficients after prediction of all pseudo DC / AC coefficients of Y, Cr and Cb components. In the following step S57, the target code amount and QUAN
T is updated.

That is, the target code amount of the target macroblock is calculated based on the generated code amount up to the current target macroblock (target macroblock) and the distribution of the image characteristic parameters of the latest I picture. It Further, QUANT is calculated based on the target code amount and the image characteristic parameter calculated previously. The process of step S57 is executed by the encoding parameter control unit 19. Next, step S58.
At, the actual DC / AC coefficient prediction is performed. This DC / AC coefficient prediction is performed based on the locally decoded image signal.

As shown in FIG. 8, subsequent step S59.
Then, quantization is performed. Quantization is executed by the quantizing unit 7 based on QUANT obtained in the previous step. Next, in step S60, variable length coding is performed. That is, the quantized coefficient obtained by the quantization is converted into a one-dimensional array, and then the variable length coding is performed. This processing is executed by the variable length coding unit 15.

Next, in step S61, the rate control parameter is updated. In the following step S62, the quantized coefficient is inversely quantized. This process is executed by the inverse quantizer 8. Subsequently, in step S63, DC / AC coefficient compensation is performed. Then, in step S64, the inverse DC
T conversion is performed. This process is executed by the inverse conversion unit 9. Next, in step S65, the decoded (encoded) image is stored in the prediction memory 11.

When the loop is iteratively executed for all macroblocks of the image to be coded, the process proceeds to step S67. In step S67,
The stuff bit is inserted. That is, when the generated code amount is smaller than the allocated bit amount, the stuff bit is inserted. By the above procedure, the I picture is encoded.

As shown in FIG. 9, when encoding is started when the image to be encoded is a P picture, first,
In step S71, initialization is performed. That is, of the variables used for control such as rate control, all those that require initialization are initialized. Next, the processing of the loop from steps S72 to S93 is repeatedly executed for each macroblock.

When the loop is started, first, step S
At 73, motion detection is performed. Continued Step S
At 74, an intra / inter mode determination is made. That is, either the intra mode or the inter mode is determined based on the result of the motion detection. Next, in step S75, DCT conversion is performed.
That is, the DCT transform is applied to the image to be encoded (the current image 22 for the intra mode and the difference image 23 for the inter mode). This process
It is executed by the conversion unit 6.

Next, in step S76, it is determined whether or not the mode is the intra mode. If it is the intra mode, the process proceeds to step S77, and if it is not the intra mode, the process proceeds to step S78. In step S77, pseudo DC / AC coefficient prediction is performed. The intra prediction mode is pseudo-determined. When step S77 ends, the process proceeds to step S78. In step S78, the code amount generated by other than the quantized DCT coefficient, such as the motion vector difference, is predicted. This process is executed by the encoding parameter control unit 19, for example.

Next, in step S79, image characteristic parameters are calculated. That is, the image characteristic parameter for performing rate control is calculated. The data required differs between intra blocks and inter blocks. In intra block, Y, C
D after prediction of all pseudo DC / AC coefficients of r and Cb components
It is the sum of absolute values of CT coefficients. In the inter block, the sum of the absolute values of the AC components and the maximum value of the absolute values of the DCT coefficients are the maximum values of all the Y, Cr, and Cb components of the DCT coefficient after motion detection.

Next, as shown in FIG. 10, step S
At 80, the target code amount and QUANT are updated.
That is, the generated code amount up to the currently targeted macroblock (target macroblock) and the latest P
Based on the image characteristic parameter distribution of the picture,
The target code amount of the target macroblock is calculated. Further, QUANT is calculated based on the target code amount and the image characteristic parameter calculated previously. The process of step S80 is executed by the encoding parameter control unit 19.

Next, in step S81, it is determined whether or not the macro block currently targeted is an intra block. If it is an intra block, the process proceeds to step S82, and if not, the process jumps to step S83. In step S82, actual DC / AC coefficient prediction is performed based on the locally decoded image signal. After step S82, the process proceeds to step S83.

In step S83, quantization is performed.
Quantization is executed by the quantizing unit 7 based on QUANT obtained in the previous step. Next, in step S84, variable length coding is performed. That is,
The quantized coefficient obtained by the quantization is converted into a one-dimensional array, and then the variable length coding is performed. Step S84
The process of is executed by the variable length coding unit 15.

Next, in step S85, the rate control parameters are updated. In the following step S86, the quantized coefficient is inversely quantized. This process is executed by the inverse quantizer 8. Subsequently, in step S87, it is determined whether or not the currently targeted macroblock is an intra block. If it is an intra block, the process proceeds to step S88, and if not, step S8.
Leap to 9.

In step S88, DC / AC coefficient compensation is performed. That is, in the intra block,
Coefficients are restored based on the DC / AC coefficient prediction mode for each macroblock. When step S88 ends, the process proceeds to step S89 as shown in FIG. In step S89, inverse DCT conversion is performed. This process is executed by the inverse conversion unit 9.

Next, in step S90, it is determined whether or not the macro block currently targeted is an inter block. If it is an inter block, the process proceeds to step S91, and if not,
Jump to step S92. In step S91, motion compensation is performed. That is, motion compensation is performed on the inter block based on the motion vector.
After step S91, the process proceeds to step S92.

In step S92, the locally decoded image is stored in the prediction memory 11. When the loop is iteratively executed for all the macroblocks of the image to be encoded, the process proceeds to step S94. In step S94, stuff bits are inserted. That is, when the generated code amount is smaller than the allocated bit amount, the stuff bit is inserted. By the above procedure, the P picture is encoded.

In the above processing, the image characteristic parameter is calculated based on the same formula as in the second embodiment. Further, the target code amount for the i-th macroblock in the picture is calculated based on the equation 14 when the i-th macroblock is the I picture and based on the equation 15 when the i-th macroblock is the P picture.

[0104]

[Equation 14]

[0105]

[Equation 15]

QUANT for the i-th macroblock in the picture is calculated based on the equation 16 for the I picture and based on the equation 17 for the P picture.

[0107]

[Equation 16]

[0108]

[Equation 17]

In one of the prior arts, the analysis of the whole image to be encoded was performed prior to the determination of QUANT. As a result, although the accuracy of control of the generated code amount is high, there is a problem that the image to be encoded needs to be scanned twice, a large memory capacity is required for calculation, and preprocessing takes time. It was

In another conventional technique, QUANT is controlled based on the increase / decrease in the generated code amount.
Therefore, there is an advantage that the scanning of the image to be encoded only needs to be performed once, the memory capacity required for the calculation is small, and the preprocessing time is short. There is a problem that a delay occurs and the control accuracy of the generated code amount is low.

On the other hand, in the device of the third embodiment,
The image to be coded need only be scanned once, the memory capacity required for the calculation is small, and the preprocessing time is short, and there is an advantage that a high code amount control accuracy with respect to the calculation amount can be obtained.

<4. Fourth Embodiment> The overall configuration of the image coding apparatus according to the fourth embodiment is also shown in the block diagram of FIG. 1 similarly to the second embodiment. The fourth embodiment is the second embodiment.
Or allocation of target code amount used in 3, or QU
The main feature of ANT is that weight is added to each macroblock.

That is, in the device of the fourth embodiment, for each macroblock, for example, the central portion of the screen or the image characteristic parameter obtained from the DCT coefficient of the image to be coded Weighting parameters are calculated according to whether or not the macroblock is a visually important macroblock such as a portion where the image moves fast. Then, based on this weighting parameter, the target code amount is assigned to each macroblock or QUANT is weighted.

As an example of the calculation, first, the image position parameter P _ij for the (i, j) th macroblock is
It is calculated based on

[0115]

[Equation 18]

For other (i, j) not shown in the equation 18, the image position parameter P _ij is calculated by performing appropriate complementation. The image position parameter P _ij reflects the visual importance of the position on the screen.

Next, using the image characteristic parameter X _ij for the (i, j) th macroblock, the weighting parameter A _ij for the (i, j) th macroblock is calculated based on the equation (19).

[0118]

[Formula 19]

The weighting parameters A _ij calculated in this way are used for controlling the generated code amount described in the second and third embodiments. That is, the target code amount allocation and QUANT
, The weighting parameter A _ij is added as a product, so that the weighting correction according to the visual importance is added.

In the prior art, the standard deviation was used as the weighting parameter. As a result, there is a problem that the standard deviation needs to be calculated separately from the calculation necessary for image coding. Further, there is a problem that it is difficult to achieve both weighting and code amount control.

On the other hand, in the device of the fourth embodiment,
The image characteristic parameter can be calculated based on the DCT coefficient with a small amount of calculation, and further, the consistency with the generated code amount control described in the second and third embodiments is good, and compatibility can be easily obtained. .

<5. Fifth Embodiment> The overall configuration of the image coding apparatus according to the fifth embodiment is shown in the block diagram of FIG. The apparatus of the fifth embodiment is characterized mainly in that it is provided with a frame rate control unit 30 which is a device unit for controlling a frame rate, and a user image quality designation unit 31 attached to the frame rate control unit 30.

The frame rate control unit 30 firstly comprises the bit rate given from the outside and the user image quality designation unit 3
The initial frame rate is set according to the designated image quality selectively designated by the user through 1. Secondly, the frame rate control unit 30 determines the generated code amount and QUAN at the time of encoding.
The frame rate is automatically updated to the optimum value based on the history of T. The frame rate control unit 30 uses the third
In the process of encoding, the user image quality designation unit 3
The frame rate is updated according to the user's request through 1.

That is, when a high-definition image with a large amount of code per frame is to be transmitted, the frame rate is sacrificed, and conversely, when an image with smooth motion is to be transmitted by increasing the frame rate, each frame is transmitted. By sacrificing the definition of the image, the definition and the frame rate of the image of one frame, which are in a trade-off relationship within the restriction on the transmission capacity of the transmission path 18, can be selected.

The flow chart of FIG. 13 shows a flow of characteristic operations in the apparatus of FIG. As shown in FIG. 13, when the frame rate control is started, first, in step S101, the pattern is stored in the table. That is, the frame rate pattern that the device can adopt is stored in the reference table. At the same time, a possible frame rate change interval (= s seconds) is also given.
The table and a memory that stores the frame rate interval are built in the frame rate control unit 30, for example.

Next, in step S102, image quality parameters are determined. That is, the selection branches such as the width of the resolution and the interval that can be selected by the user are prepared as image quality parameters. For example, a value corresponding to the lower limit of the compression rate is set to 0, a value corresponding to the upper limit of the compression rate is set to 1, and a value in the range of 0 to 1 corresponding to the degree of resolution is prepared as a range selectable by the user. The lower limit of the compression rate corresponds to the lowest definition image and the highest frame rate, and the upper limit of the compression rate corresponds to the highest definition image and the lowest frame rate. The prepared image quality parameters are
It is stored in the reference table.

Next, in step S103, QUAN
The limit value of T is set. That is, the upper limit Qu of QUANT,
The lower limit value Ql, the effective upper limit value Qmu, and the effective lower limit value Qml are determined. QUANT must be in the range of lower limit Ql <QUANT <upper limit Qu, and effective lower limit Qml <QUANT <effective upper limit
It is desirable to be in the range of Qmu.

Next, in step S104, the bit rate is defined. Subsequently, in step S105, the image quality parameter corresponding to the frame rate pattern is stored in the reference table. Subsequently, in step S106, the image quality is designated by the user.
This designation is performed through the user image quality designation unit 31 configured as an input device.

Next, in step S107, the definition and the frame rate are searched. That is, the image quality parameter corresponding to the specified definition is searched from the reference table, and the frame rate corresponding to the value is searched from the reference table. Then, the obtained frame rate is set as the reference frame rate.

Next, in step S108, bit allocation and coding are performed. That is, bit allocation and encoding (encoding) of each frame are performed under the basic frame rate. Continue, step S1
In 09, the frame rate is automatically adjusted based on the encoding result. Subsequently, in step S110, when the image quality change is set by the user newly instructing the image quality, the process returns to step S107, and steps S107 to S107.
The processing of 109 is executed again.

14 and 15, step S109 is executed.
3 is a flowchart showing in detail the flow of the processing of FIG. Figure 1
As shown in FIG. 4, when the process of step S109 is started, it is determined in step S111 whether or not QUANT> upper limit Qu for the latest two frames.
If the determination result is “Yes”, the process is step S11.
On the contrary, if the result is “No”, the process proceeds to step S12.
Leap to zero.

In step S112, the frame rate is reduced by one step from the reference value. Next, in step S113, waiting is performed for s seconds. That is, the frame rate reduced in step S112 is maintained as it is for s seconds. The value of s seconds has been determined in the previous step S101.

In the following step S114, it is determined whether or not QUANT> upper limit Qu for the latest two frames. If the determination result is "Yes", the process proceeds to step S115, and if "No", the process proceeds to step S115.
Move to 116. In step S115, the frame rate is readjusted and the frame rate is lowered by one step from the current value. Then, the process is step S
Return to 113.

In step S116, it is determined whether or not QUANT> effective upper limit value Qmu for the latest two frames. If the determination result is "Yes", the process proceeds to step S117, and if "No", the process proceeds to step S117.
Transition to 118. In step S117, the frame rate is not changed and is kept at the current value. Then, the process returns to step S113.

On the other hand, in step S118, the frame rate is increased by one step from the current value. Subsequently, in step S119, it is determined whether or not the current frame rate matches the reference value. If the determination result is “No”, the process returns to step S113,
If “Yes”, the process of step S109 is completed.

Next, as shown in FIG. 15, step S
At 120, QUANT <for the latest two frames
It is determined whether or not the lower limit value Ql. If the determination result is "Yes", the process proceeds to step S121, and if "No", the process of step S109 is completed.

In step S121, the frame rate is raised by one step from the reference value. Next, in step S122, waiting is performed for s seconds. That is, the frame rate increased in step S121 is maintained as it is for s seconds.

In the following step S123, it is determined whether or not QUANT <lower limit value Ql for the latest two frames. If the determination result is “Yes”, the process proceeds to step S124, and if the determination result is “No”, step S124.
Move to 125. In step S124, the frame rate is readjusted, and the frame rate is increased by one step from the current value. Then, the process is step S
Return to 122.

In step S125, it is determined whether or not QUANT <effective lower limit Qml for the latest two frames. If the determination result is “Yes”, the process proceeds to step S126, and if the determination result is “No”, step S126.
Move to 127. In step S126, the frame rate is not changed and is kept at the current value. After that, the process returns to step S122.

On the other hand, in step S127, the frame rate is reduced by one step from the current value. Subsequently, in step S128, it is determined whether or not the current frame rate matches the reference value. If the determination result is “No”, the process returns to step S122,
If “Yes”, the process of step S109 is completed.

[0141]

In the apparatus of the first invention, the image characteristic parameter is calculated from the discrete cosine coefficient, so that the image characteristic parameter can be obtained with a small amount of calculation. Further, since the image characteristic parameter is used, it is possible to obtain the effect that the generated code amount can be predicted with high accuracy.

The apparatus of the second invention has the effect that the code amount prediction corresponding to the DC / AC coefficient prediction within the intra picture (I picture) screen can be performed.

In the apparatus of the third invention, the target code amount in the next macroblock to be coded is determined based on the distribution history of the image characteristic parameters of the past image and so on. Is required only once, the memory capacity required for the calculation is small, the preprocessing time is short, and a high code amount control accuracy can be obtained.

In the apparatus of the fourth invention, the image characteristic parameter is calculated based on the discrete cosine transform coefficient and the like, and the target code amount is assigned to each macroblock according to the weighting parameter calculated based on this. , And the quantization step is weighted. Therefore, the image characteristic parameter can be calculated with a small amount of calculation, and further, the consistency with the generated code amount control is good,
It is possible to obtain the effect that compatibility is easy.

[Brief description of drawings]

FIG. 1 is a block diagram of an apparatus according to first to fourth embodiments.

FIG. 2 is a flowchart showing a flow of operations of the apparatus according to the second embodiment.

FIG. 3 is a flowchart showing a flow of operations of the apparatus according to the second embodiment.

FIG. 4 is a flowchart showing a flow of operations of the apparatus according to the second embodiment.

FIG. 5 is a flowchart showing a flow of operations of the apparatus according to the second embodiment.

FIG. 6 is a flowchart showing a flow of operations of the apparatus according to the second embodiment.

FIG. 7 is a flowchart showing a flow of operations of the apparatus according to the third embodiment.

FIG. 8 is a flowchart showing a flow of operations of the apparatus according to the third embodiment.

FIG. 9 is a flowchart showing an operation flow of the device according to the third embodiment.

FIG. 10 is a flowchart showing a flow of operations of the apparatus according to the third embodiment.

FIG. 11 is a flowchart showing a flow of operations of the apparatus according to the third embodiment.

FIG. 12 is a block diagram of a device according to the fifth embodiment.

FIG. 13 is a flow chart showing a flow of operations of the apparatus according to the fifth embodiment.

FIG. 14 is a flowchart showing a flow of operations of the apparatus according to the fifth embodiment.

FIG. 15 is a flowchart showing a flow of operations of the apparatus according to the fifth embodiment.

[Explanation of symbols]

6 Converter 7 Quantizer 15 Variable length coding unit 19 Coding parameter control unit (control unit) 30 frame rate control unit (control unit)

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Takeshi Hashimoto             4-5-36 Miyahara, Yodogawa-ku, Osaka Stock Market             Company Megachips F-term (reference) 5C059 MA00 MA05 MA23 MC11 MC33                       MC35 ME01 NN01 SS06 TA46                       TB08 TC00 TC04 TC24 TC38                       TC41 UA02 UA33                 5J064 AA02 BA09 BA16 BB13 BC01                       BC08 BC09 BC14 BC16 BD01

Claims (4)

[Claims] 1. An image coding apparatus for coding an image, comprising: a transforming unit for performing a discrete cosine transform on the image to calculate a discrete cosine transform coefficient; and a quantizing step for the discrete cosine transform coefficient. Based on the image characteristic parameter, a quantizer that calculates the quantized coefficient by performing the quantization based on the image characteristic parameter from the discrete cosine transform coefficient, and a controller that determines the quantization step, An image encoding device comprising: 2. The image coding apparatus according to claim 1, further comprising a variable length coding unit that performs variable length coding on the quantized coefficient while referring to a variable length coding table, The image coding apparatus, wherein the control unit calculates the image characteristic parameter based on an image type, a macroblock type, and a method adapted to the variable length coding table. 3. The image encoding device according to claim 1, wherein the control unit further includes a distribution history of image characteristic parameters of past images, a visual characteristic, and occurrence during encoding. An image coding apparatus characterized by determining a target code amount in a next macroblock to be coded based on the code amount. 4. An image coding apparatus for coding an image, comprising: a transforming unit for performing a discrete cosine transform on the image to calculate a discrete cosine transform coefficient; and a quantizing step for the discrete cosine transform coefficient. Based on the quantization unit that performs the quantization based on the original calculation, and the position in the screen for each macroblock, and
An image characteristic parameter is calculated from the discrete cosine transform coefficient in the image to be encoded, and a weighting parameter for performing weighting according to the degree of visual importance is calculated based on the image characteristic parameter. An image coding apparatus, comprising: a control unit that allocates a target code amount and weights the quantization step for each macroblock based on a parameter.