JP4478480B2 - Video encoding apparatus and method - Google Patents

Description

  The present invention relates to a moving image encoding apparatus and method for encoding an input image to a predetermined target code amount using a weighting parameter in quantization processing in moving image encoding for encoding a moving image in a predetermined unit. Is.

  Due to dramatic progress in digital signal processing technology in recent years, recording of moving images on storage media and transmission of moving images via transmission paths, which have been difficult in the past, have been performed. In this case, each picture constituting the moving image is subjected to compression encoding processing, and the data amount is greatly reduced. As a typical compression coding process, for example, there is an MPEG (Moving Picture Experts Group) system.

  When compressing and encoding a series of pictures in a moving picture under a condition of a constant bit rate in accordance with the MPEG system, the spatial frequency characteristics of the scenes and pictures in the moving picture and the quantization scale (Q The amount of code varies greatly depending on the (scale) value. An important technique for minimizing coding distortion in realizing a compression coding apparatus having such coding characteristics is code amount control.

  Many algorithms for realizing the code amount control have been proposed so far. Among them, TM5 (Test Model Editing Commitee: "Test Model 5", ISO / IEC JTC / SC29 / WG11 / N0400 (Apr. 1993)) proposed in the process of standardization of the MPEG-2 encoding system. ) (Hereinafter, TM5: Patent Document 1) is well known.

<Prior art 1>
TM5 is composed of the following three steps (steps 1 to 3), and the Q (quantization) scale is controlled so that the bit rate is constant for each GOP (Group of Pictures). .

  [Step 1: Bit allocation]: The target code amount of the picture to be encoded next is calculated from the remaining code amount in the GOP.

  [Step 2: Code amount control]: The Q scale is calculated from the target code amount calculated in Step 1 according to the state of the virtual buffer.

  [Step 3: Adjustment of Q scale]: The final Q scale is determined based on the spatial activity of the macroblock.

  Of the three steps, detailed processing of step 1 that has the greatest influence on coding distortion will be described below.

[Step 1: Bit allocation]
Now, as shown in FIG. 14, the target code amount of the picture P3 is calculated prior to the encoding of the tenth picture P3 (P picture) in the current GOP. The process of step 1 is expressed by the following equation.

Where R _gop is the code amount allocated to the current GOP, N _i , N _p, and N _b are the number of remaining pictures in the current GOP of I, P, and B pictures, bits_rate is the target bit rate, and picture_rate is the picture -Represents the rate. Further, for each of the I, P, and B pictures, the picture complexity X _i , X _p, and X _b is calculated from the encoding result by the following equation.

Where R _i , R _p and R _b are code amounts obtained by encoding I, P and B pictures, respectively, and Q _i , Q _p and Q _b are all macroblocks in the I, P and B pictures, respectively. Is the average value of the Q scale. From the equations (1) and (2), the target code amounts T _i , T _p and T _b are calculated for the I, P and B pictures, respectively, using the following equations.

However, K _p = 1.0 and K _b = 1.4.
Based on the target code amount T _p of the picture P3 calculated from the above processing, the Q scale is calculated in step 2 and subsequent steps.

<Conventional technology 2>
As a code amount control method using a prefilter, Patent Document 1 discloses a “prefilter control method and apparatus in moving picture coding”. According to this method, the spatial frequency of each picture input to the encoding unit is controlled by an LPF (hereinafter referred to as a prefilter LPF), which is a pre-filter before the encoding unit, thereby reducing quantization distortion. is doing.

  For the control of the prefilter LPF, a balance function shown in FIG. 15 is defined to match the quantization distortion and the image sharpness deterioration. The two curves in FIG. 15 correspond to the following two functions F1 and F2, respectively.

F1 (motion amount, filter coefficient, Q scale, code amount)
F2 (filter coefficient, Q scale)
The intersection of the functions F1 and F2 is referred to as a balance point. At this point, the Q scale and the filter coefficient of the pre-filter LPF with the best matching of the code amount and the image quality are obtained.

<Conventional technology 3>
Similarly, Patent Document 2 proposes a “moving image encoding method” as another prior art of a code amount control method using a prefilter.

  According to this method, first, the encoding difficulty level Y is calculated using the function F for each of the I, P, and B pictures as follows.

Y = F (cumulative code amount, average Q scale)
Next, the filter coefficient parameter Z is calculated from the following equations from the encoding difficulty levels Y _i , Y _p and Y _b calculated for I, P and B, respectively.

Depending on the value of the filter coefficient parameter Z obtained by the equation (4), the actual filter coefficient S is selected from the preset values S0, S1 or S3 from the graph shown in FIG.
That is, by providing the filter coefficient Z corresponding to each filter coefficient S with a width, a sudden change in the filter coefficient Z is avoided.
Test Model 5 (Test Model Editing Commitee: "Test Model 5", ISO / IEC JTC / SC29 / WG11 / N0400 (Apr. 1993)) Japanese Patent No. 2894137 JP 2002-247576 A

  However, TM5 of prior art 1 shown in Non-Patent Document 1 has the following problems.

  In step 2 and step 3, when obtaining the final Q scale, only the difference between the target code amount of the picture and the code amount in the picture up to the current macroblock, and the spatial activity of the macroblock are used. .

  That is, TM5 attempts to adjust the visual characteristics while actually performing the encoding process after the target code amount of the picture to be encoded has already been determined. Therefore, there is a problem that the degree of quantitative deterioration of image quality and human visual characteristics are not sufficiently reflected.

  Prior art 2 disclosed in Patent Document 1 attempts to solve the problem of TM5 by using a prefilter LPF. However, in the function F1, a large-scale circuit for calculating the motion amount that is an argument is required. Further, the definition of the functions F1 and F2 and the balance point calculation method shown in FIG. 15 are not mentioned at all, and the control method and effect of the prefilter LPF are unclear.

  Furthermore, the prior art 3 shown by patent document 2 is trying to solve the subject of the prior art 2 by avoiding a rapid change, when changing a filter coefficient. However, since the filter coefficient is simply predicted only from the information of the accumulated code amount and the average Q scale, it is still difficult to say that the degradation degree of image quality and human visual characteristics are still taken into consideration.

  The present invention has been made in view of the above problems, and in consideration of the degradation of image quality and human visual characteristics, a moving image with an optimal code amount and encoding distortion amount under the condition of the assigned target code amount. It is an object of the present invention to provide a moving image encoding apparatus capable of generating image encoded data and a control method thereof.

In order to achieve the above object, a moving picture coding apparatus according to the present invention comprises the following arrangement. That is,
A moving image encoding apparatus for encoding an input image to a predetermined target code amount using a weighting parameter in quantization processing in moving image encoding for encoding a moving image in a predetermined unit, wherein the input image is distributed A variance calculating means for calculating
Filter means for performing a filtering process on the input image with a given filter characteristic;
Encoding means for performing quantization and encoding on the input image filtered by the filter means;
Decoding means for performing a decoding process on the encoded data output by the encoding means;
Detecting means for detecting a block distortion amount by the encoding means from an input image to the encoding means and a reconstructed image that is an output of the decoding means;
R−, which is the relationship between the code amount (R) and the coding distortion amount (D) in the coding means , using the variance of the input image to the coding means and the coding distortion amount generated by the coding means. An R-D model calculating unit that predetermines a defining formula that defines the D model and calculates a target code amount of the input image from the defining formula ;
A visual sensitivity model for evaluating visual sensitivity from an addition operation of a filter distortion amount generated by the filter means, an encoding distortion amount generated by the encoding means, and a block distortion amount detected by the detection means for an image immediately before the input image. A calculation means;
The input image target code amount , the coding distortion amount calculated by the visual sensitivity model calculating means, and the variance calculated by the variance calculating means correspond to the variance of the input image to the encoding means. Parameter calculating means for calculating the parameters;
The code calculated by the parameter calculation means from among dispersion characteristics between an input image to the filter means and an output image from the filter means in accordance with a change in a plurality of predetermined filter coefficients. A filter characteristic calculation unit that selects a parameter that corresponds to the variance of the input image to the conversion unit and a filter characteristic that is closest to the relationship of the variance calculated by the variance calculation unit;
RQ model calculation means for calculating a weighting parameter in the quantization process from the target code amount calculated by the RD model calculation means,
The parameter calculation unit is configured to obtain a variance of the input image filtered by the filter unit and an encoding distortion amount generated by the encoding unit, and a target code amount of the input image obtained from the prescribed expression. Is calculated in advance using a Lagrange undetermined multiplier method in which the evaluation formula for evaluating the visual sensitivity is maximized or minimized .

In order to achieve the above object, a moving picture coding method according to the present invention comprises the following arrangement. That is,
In moving picture coding for coding a moving picture in a predetermined unit, a moving picture coding method for coding an input picture to a predetermined target code amount using a weighting parameter in quantization processing,
A variance calculating step of calculating variance of the input image;
A filtering step for performing a filtering process on the input image with given filter characteristics;
An encoding step of performing quantization processing and encoding the input image filtered in the filtering step;
A decoding step of performing a decoding process on the encoded data output by the encoding step;
A detection step for detecting a block distortion amount due to the encoding step from an input image to the encoding step and a reconstructed image that is an output of the decoding step;
R−, which is the relationship between the coding amount (R) and the coding distortion amount (D) in the coding step , using the variance of the input image to the coding step and the coding distortion amount generated by the coding step. An R-D model calculating step of predetermining a defining formula for defining the D model , and calculating a target code amount of the input image from the defining formula ;
A visual sensitivity model for evaluating visual sensitivity from an addition operation of the filter distortion amount generated by the filtering step, the coding distortion amount generated by the encoding step, and the block distortion amount detected in the detection step for the image immediately before the input image. A calculation process;
A target code amount of the input image, and encoding distortion amount calculated by the visual sensitivity model calculation step, the dispersion and calculated by the variance calculation step, corresponding to the variance of the input image to the encoding step A parameter calculation step for calculating parameters;
The code calculated by the parameter calculation step from among dispersion characteristics between an input image to the filter step and an output image from the filter step according to a change in a plurality of predetermined filter coefficients. A filter characteristic calculation step of selecting a parameter corresponding to the variance of the input image to the conversion step and a filter characteristic closest to the relationship of the variance calculated by the variance calculation step;
An RQ model calculation step of calculating a weighting parameter in the quantization process from the target code amount calculated by the RD model calculation step,
In the encoding step, the parameter calculation step includes the variance of the input image filtered in the filtering step and the encoding distortion amount generated by the encoding step, and the target code amount of the input image is obtained from the prescribed expression. Is calculated in advance using a Lagrange undetermined multiplier method in which the evaluation formula for evaluating the visual sensitivity is maximized or minimized .

  According to the present invention, it is possible to generate moving image encoded data in which the code amount and the coding distortion amount are optimum under the condition of the allocated target code amount in consideration of the degradation of image quality and human visual characteristics. A moving image encoding apparatus and method thereof that can be provided can be provided.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  In the present invention, an example applied to an irreversible encoding method with encoding distortion will be described in detail.

  In the first embodiment, an example in which the present invention is applied to a general irreversible encoding method without limiting the encoding method is shown. In the second embodiment, an example applied to the MPEG encoding method is shown.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration of a moving picture coding apparatus according to Embodiment 1 of the present invention. FIG. 2 is a flowchart showing processing executed by the video encoding apparatus according to the first embodiment of the present invention.

  Hereinafter, the details of the operation of the moving picture coding apparatus 100 according to the first embodiment will be described with reference to FIGS. 1 and 2.

  It is assumed that the weighting parameter for the quantization process in the moving image coding apparatus is the Q scale.

  As shown in FIG. 1, the moving picture coding apparatus 100 is roughly composed of blocks of a prefilter unit 101, a coding unit 102, a local decoding unit 103, and a code amount control unit 104. Each of these blocks may be realized by hardware, or a part or all of the blocks may be realized as software by control using a CPU, RAM, and ROM.

  In explaining the operation of the moving picture coding apparatus 100, at the present time, as shown in FIG. 3, the coding process up to the picture I2 is completed, and then the picture I3 is coded. To do.

First, at step S200 in FIG. 2, the target code amount R _t of the picture I3 is set externally block (not shown). Method for calculating the R _t is not dependent on the present invention, for example, if the CBR scheme shown in the conventional example, which corresponds to step 1 of the processing of TM5.

In the moving picture coding apparatus 100, the Q scale of the encoder 102, rather than directly calculated from the set target code amount R _t, the encoding distortion amount estimated from the target code amount R _t, visual sensitivity Using the model calculation unit 107 and the RD model calculation unit 109, the pre-filter unit 101 and the encoding unit 102 are optimally divided.

In step S201, the variance S _i of the picture I3 is calculated by the variance calculation unit 105. For example, the variance S _i is calculated as follows.
When the coordinates (x, y), the picture size is M × N pixels, and the average of the picture of interest is AVE, the picture variance S _i is calculated by the following equation.

  Next, the RD model (RD definition formula) and the visual sensitivity model (visual sensitivity evaluation formula) of the encoding unit 102 used in steps S203 to S204 will be described.

The RD model R _c (S _f , MSE _c ) of the encoding unit 102 applied in the first embodiment is calculated using the following equation.

Here, I _c and Θ _c are constants, and when I _c = 1 and Θ _c = 0.5, in information theory, the relationship between the code amount known as the Rate Distortion theory and the coding distortion amount is represented. This is a known formula.
S _f is the variance of the input picture of the encoding unit 102, and corresponds to the variance of the output picture of the prefilter unit 101. The variance S _f is a variable that changes in accordance with the variance S _i of the input picture of the video encoding device 100 and the filter characteristics of the prefilter unit 101 in the first embodiment.

MSE _c is a coding distortion amount generated by the coding unit 102. MSE _c is a variable corresponding to the sum of squared differences between the input picture of the encoding unit 102 and the output picture of the local decoding unit 103.

I _c and Θ _i are defined as parameters depending on the encoding scheme of the encoding unit 102. In Embodiment 1, since it is assumed that the encoding method of the encoding unit 102 is not limited, I _c = 1 and Θ _c = 0.5 are applied.

Here, in FIG. 4, as a diagram representing the characteristic of the equation (5), the code amount R _c and the coding distortion amount MSE _c when S _f = 2300, I _c = 1 and Θ _c = 0.5 are calculated. Show the relationship.

The visual sensitivity model H _vs (S _f , MSE _c ) used in step S203 is defined as the following expression in the first embodiment.

Here, MSE _f is the amount of filter distortion generated in the pre-filter unit 101, B _cprev is the block distortion amount detected by the block distortion detection unit 106 during the encoding process of the immediately preceding picture, and S _cprev is the input immediately before the encoding unit 102 It is the variance S _f of the picture.
Further, the filter distortion amount MSE _f in the equation (6) is defined as the following equation.

Here, α is a constant depending on the type of filter of the pre-filter unit 101.
FIG. 5 shows a list of variables and constants used in the equations (5) to (7).

Next, the characteristics of the visual sensitivity model H _vs (S _f , MSE _c ) of the equations (6) and (7) used in the first embodiment are shown below.

Feature 1: Considering not only the encoding distortion amount MSE _c generated in the encoding unit 102 but also the filter distortion amount MSE _f (S _f ) generated in the prefilter unit 101, the distortion amount of the entire moving image encoding apparatus 100 It is possible to control the image quality with high accuracy.

Feature 2: By adding the block distortion amount B _cpfev as an evaluation amount, it is possible to evaluate image quality close to human visual sensitivity.

Using equation (6) and equation (7), in step S202, the variance S _i of the input picture of the video encoding device 100, the variance S _cprev of the immediately preceding input picture of the encoding unit 102, and the block distortion amount of the immediately preceding picture A visual sensitivity model H _vs (S _f , MSE _c ) is calculated from B _cprev .

Next, a method for calculating the variance S _f and the coding distortion amount MSE _c in the parameter calculation unit 108 in step S203 will be described.

In the first embodiment, the two visual sensitivity models H _vs (S _f , MSE _c ) and the Lagrange undetermined multiplier method are used under the constraint condition of the target code amount of the picture given to the video encoding device 100. A variance S _f and an encoding distortion amount MSE _c that optimize the relationship of the RD model R _c (S _f , MSE _c ) are calculated.

That is, if the target code amount of the picture given in the moving picture coding apparatus 100 and R _t, the constraint condition expression, expressed by the following equation.
[Constrained conditional expression]

  Furthermore, if the undetermined multiplier is defined as λ,

It becomes.
In addition, the following expression is defined as a necessary conditional expression.
[Required expression]

Therefore, in order to calculate the optimum solution of the variance S _f and the coding distortion amount MSE _c from the equations (10) and (8), the following equation is calculated in step S203.

However, in the first embodiment, I _c = 1 and Θ _c = 0.5.
Further, α is a coefficient depending on the type of the filter constituting the prefilter unit 101, and is a constant determined in advance when the moving picture coding apparatus 100 is constructed.

  Next, in step S204, the filter characteristic calculation unit 110 determines the filter characteristic of the pre-filter unit 101. In the first embodiment, filter characteristics are selected using changes in the variance of the input and output pictures of the prefilter unit 101.

The input picture variance S _i has been calculated in step S201, and the output picture variance S _f has been calculated in step S203.

Among the dispersion characteristics of the input and output pictures of the pre-filter unit 101 in accordance with the relationship between these two dispersions S _i and S _f and a plurality of predetermined filter coefficient changes, the filter coefficient having the closest characteristic is selected. Select one.

FIG. 6 shows the relationship between the variances S _i and S _f calculated from five curves indicating the dispersion characteristics of the input and output pictures of the pre-filter unit 101 corresponding to the predetermined filter coefficients C1 to C5. It shows that the nearest filter coefficient C2 is selected.

  The prefilter unit 101 receives the parameters C1 to C5 from the filter coefficient calculation unit 110, and changes the filter coefficient so that the corresponding filter characteristics are obtained.

In step S205, the RD model calculation unit 109 uses the encoded distortion amount MSE _c and the variance S _f obtained from the equation (11) to generate a code from the RD model R _c (S _f , MSE _c ). The target code amount R _c of the conversion unit 102 is calculated.

This is calculated by substituting the coding distortion amount MSE _c and the variance S _f into the RD model R _c (S _f , MSE _c ) of the equation (5).

In step S206, by using the target code amount R _c calculated in step S205, and calculates the Q scale of the encoding unit 102. For calculating the Q scale, the RQ model of the encoding unit 102 is used. In the first embodiment, the RQ model RQ _c (R _c , S _f ) of the encoding unit 102 is expressed by the following linear expression.

Here, R _c is the target code amount R _c calculated in step S205, and S _f is the variance S _f of the input picture of the encoding unit 102 calculated in step S203.
Β _c is a constant, and is obtained by substituting the values of R _c , S _i, and Q _c used in the immediately preceding picture into Equation (12) again. However, in the first embodiment, in order to improve the accuracy of calculating the Q scale, the RQ model RQ _c (R _c , S _f ) is updated in step S209 using the following equation.

However, n corresponds to the number of past pictures to be reflected in the RQ model RQ _c (R _c , S _f ).
As described above, after the processing from step S200 to step S206 is completed, the processing of the prefilter unit 101 and the encoding unit 102 is executed in step S207.

In parallel with the encoding process of the encoding unit 102, the block distortion detection unit 106 detects the block distortion amount B _cprev in step S208. The block distortion amount B _cprev uses the input picture of the encoding unit 102 and the output picture of the local decoding unit 103.

  Here, it is known that human visual sensitivity is very sensitive to block distortion. The cause of this block distortion is that orthogonal transformation and quantization processing are performed in units of square blocks of 8 × 8 pixels.

The method for detecting the block distortion amount B _cprev can be freely implemented without depending on the present invention. However, even if block distortion is detected for the same picture, the block distortion amount B _cprev is used as the detection method. Depends on different.

However, the difference may be obtained by taking into account the visual model H _vs (S _f , MSE _c ) of Equation (6) and multiplying B _cprev by a constant. This constant is a value that is uniquely determined when the moving picture coding apparatus 100 according to the first embodiment is configured if the detection method of the block distortion detection unit 106 is determined.

In the first embodiment, as a detection method of the block distortion detection unit 106, the calculation is performed using the ratio of the difference square sum MSE _blk of the 8 × 8 block boundary and the difference square sum MSE _all of the entire picture.

Here, it is assumed that the number of pixels in the horizontal direction of the input picture of the encoding unit 102 is x_size and the number of pixels in the vertical direction is y_size. The pixel value of the input picture of the encoding unit 102 with the horizontal coordinate J and the vertical coordinate I is CIN (J, I), and similarly, the pixel value of the output picture of the local decoding unit 103 is COUT. If (J, I), the block distortion amount B _cprev is calculated by the following equation (14).

Here, MSE _all is the sum of squared differences in the entire picture of CIN (J, I) and COU (J, I), and γ is a constant that depends on the detection method of the block distortion detection unit 106.
As described above, according to the first embodiment, the above-described processing in steps S200 to S209 is repeatedly performed every time a picture is input to the moving image coding processing apparatus 100, so that the degree of image quality degradation and human Control of the pre-filter unit 101 and the encoding unit 102 in consideration of visual characteristics can be realized.

  Therefore, it is possible to obtain encoded moving image data in which the code amount and the encoding distortion amount are optimal under the condition of the allocated target code amount.

<Embodiment 2>
As the second embodiment, an example in which the encoding unit is applied to the MPEG-4 encoding method will be described in detail.

  FIG. 7 is a block diagram showing the configuration of the moving picture coding apparatus according to the second embodiment of the present invention. FIG. 8 is a flowchart showing processing executed by the video encoding apparatus according to the second embodiment of the present invention.

  Here, the difference between each block constituting the moving picture coding apparatus 800 of the second embodiment shown in FIG. 7 and each block constituting the moving picture coding apparatus 100 of the first embodiment shown in FIG. One.

  Block Difference 1: The prefilter unit 101 in FIG. 1 corresponds to the Butterworth filter unit 801 in FIG.

  Block Difference 2: The encoding unit 102 in FIG. 1 corresponds to the MPEG encoding unit 802 in FIG.

  The internal block configuration of the code amount control unit 804 is the same as the internal block configuration of the code amount control unit 104 in FIG.

  Also, the MPEG encoding unit 802 includes a motion detection unit (ME) 805, a DCT unit 806, a quantization unit (QTZ) 807, and a variable length encoding unit (VLC) 808. Also, the local MPEG decoding unit 803 has blocks of a motion compensation unit (MC) 809, an inverse DCT unit (IDCT) 810, and an inverse quantization unit (IQTZ) 811.

  Each of these blocks may be realized by hardware, or a part or all of the blocks may be realized as software by control using a CPU, RAM, and ROM.

  Next, the difference between the flowchart showing the process executed by the video encoding apparatus according to the second embodiment in FIG. 8 and the flowchart showing the process executed by the video encoding apparatus 100 according to the first embodiment in FIG. The next two.

  Process Difference 1: The RD model used in the processes in steps S904 and 906 in FIG. 8 is different from the RD model used in steps S203 and 205 in FIG.

  Processing Difference 2: The filter characteristic selection method performed in step S905 in FIG. 8 is different from the filter characteristic selection method performed in step S204 in FIG.

  Hereinafter, in the overall processing in the MPEG-4 encoding method, the part corresponding to the processing of the moving image encoding apparatus 800 according to the second embodiment will be described, and then the difference between the two processes will be described in detail. To do.

[Corresponding part of the whole process]
In the second embodiment, as shown in FIG. 9, the entire stream is divided into a sequence composed of a plurality of pictures. In the code amount control, this sequence is performed as one unit, and encoding is performed so that each sequence has the same bit rate. For example, this sequence corresponds to Group_of_VideoObjectPlane () in the syntax of the MPEG-4 encoding system.

  FIG. 10 is a flowchart showing the processing of the MPEG-4 encoding method in one sequence. The number of pictures constituting the sequence and the target code amount of the sequence do not depend on the present invention.

For example, in step S1000, it is assumed that the target code amount of the sequence corresponds to R _gop in equation (1) of the prior art. In this case, in order to calculate the target code amount R _t of one picture constituting the sequence in step S1001, the conventional equations (2) and (3) can be used.

After the target code amount R _t of one picture constituting the sequence is calculated, in step S1002, the processing of FIG. 8 is repeated to encode all the pictures constituting the sequence.
[Processing difference 1]
Also in the second embodiment, the RD model R _c (S _f , MSE _c ) of the MPEG encoding unit 802 is defined as in the first embodiment. It is assumed that the picture types targeted by the moving picture coding processing apparatus 800 of the second embodiment are I picture and P picture.

The values of the two constants I _c and Θ _c of the RD model R _c (S _f , MSE _c ) in Expression (5) are used as the code amount R _c and the encoding distortion amount of the MPEG encoding unit 801 according to the second embodiment. It is defined to represent the relationship with MSE _c .

  In the coding of a P picture in the MPEG-4 coding method, unlike the coding of an I picture that uses information only within a picture, a difference calculation is performed using the correlation between adjacent pictures.

  This difference calculation is realized by two blocks of ME 805 for performing motion detection processing and MC 809 for performing motion compensation processing in FIG.

That is, even when the input picture of the MPEG encoding unit 802 is the same, the variance of the input picture of the DCT 806 that performs orthogonal transform processing differs depending on whether the I picture or the P picture is encoded. There arises a problem that the 802 RD model R _c (S _f , MSE _c ) cannot be expressed.

In order to solve this problem, the variance S _f of the input picture of the DCT 806 is calculated and defined as the variance S _f in the equation (5) at the time of encoding any of the I picture and the P picture. It ’s fine. However, in this case, it is necessary to newly define a distribution model considering the processing of the ME 805 and the MC 809.

Therefore, in the second embodiment, two RD models R _c (S _f , MSE _c ) corresponding to picture types are defined.

FIG. 11 is a diagram illustrating the relationship between the code amount R _{c of the} RD model R _ic (S _f , MSE _c ) and the encoding distortion amount MSE _c in the I picture of the MPEG encoding unit 801.

The curve indicated by “− ▲ −” in FIG. 11 is an RD model in which the two constants I _c and Θ _c in equation (5) are set to I _c = 1 and Θ _c = 0.5, respectively. , The curve indicated by “− ■ −” indicates that the I picture RD model R _ic corresponding to the I picture of the MPEG encoding unit 802 of the second embodiment in which I _c = 0.1 and Θ _c = 0.25. (S _f , MSE _c ) respectively. Furthermore, a curve indicated by “− ◆ −” indicates an actual measurement value when an I picture is actually encoded by the MPEG-4 encoding method.

In FIG. 11, in the region where the code amount R _c is 0.5 or more, there is a large divergence between the measured value curve and the curve of the I picture RD model R _ic (S _f , MSE _c ).

The bit rate corresponding to the code amount R _c = 0.5 is 6. when the image size of the input picture of the MPEG encoding unit 802 is VGA, the subsample 4-2-0, and the frame rate is 30 fps. This corresponds to a very high bit rate of 6 Mbps.

  Here, when the MPEG encoding unit 802 performs encoding at such a high bit rate, it is rare that block distortion occurs to a visually noticeable level. In the first place, in the Butterworth filter unit 801, There is no need for prefiltering to mitigate block distortion.

  That is, the process for controlling the filter characteristics of the Butterworth filter unit 801 from step S903 to step S906 can be omitted.

  Therefore, in step S902, if the target code amount of the picture given to the moving picture coding apparatus 800 is 0.5 bit / pixel or more, the process branches to step S907.

On the other hand, FIG. 12 shows the relationship between the coding amount R _c and the coding distortion amount MSE _c of the P picture RD model R _pc (S _f , MSE _c ) corresponding to the P picture of the MPEG coding unit 801.

The P picture R-D model R _pc (S _f , MSE _c ) corresponds to the curve indicated by “− ■ −” in FIG. 12, and the two constants I _c and Θ _c in the equation (5) are I _c = 0.15 and Θ _c = 0.15 respectively. “− ▲ −” corresponds to the RD model similar to FIG. 11, and the curve indicated by “− ◆ −” indicates the actual measurement when the P picture is actually encoded by the MPEG-4 encoding method. The value is shown.

In FIG. 12, as with the I picture RD model R _ic (S _f , MSE _c ), there is a large divergence from the actual measurement value in a region where the code amount R _c is 0.5 or more. The RD model R _pc (S _f , MSE _c ) is not used.

As described above, Steps S904 and 906 are different from Steps S203 and S205 of Embodiment 1 shown in FIG. 2 in that, instead of the RD model R _c (S _f , MSE _c ) of Embodiment 1, Depending on the picture type, only two I picture RD models R _ic (S _f , MSE _c ) and P picture RD model R _pc (S _f , MSE _c ) are used.

Therefore, in steps S904 and 906, if the processing of steps S203 and S205 shown in the first embodiment is defined according to the picture type, the constants I _c and Θ _c of equation (5) are defined as described above. good.

  Next, the process in step S905 will be described.

  In the moving image coding apparatus 800 according to the second embodiment, a Butterworth filter unit 801 having Butterworth characteristics is used as a prefilter unit.

  The Butterworth filter is known to have a maximum flat characteristic, and the frequency response characteristic is determined by the order.

  In the second embodiment, the filter characteristic of the Butterworth filter unit 801 is changed by changing the order of the Butterworth filter while fixing the cutoff frequency.

A graph showing the relationship between the frequency F _i of the input picture of the Butterworth filter unit 801 and the frequency F _f after passing through the filter when the order is changed from 1 to 5 is shown in FIG.

Using the relationship between the variance S _i of the input picture of the Butterworth filter unit 801 calculated in step S901 and the variance S _f of the output picture of the Butterworth filter unit 801 obtained in step S904, it is shown in FIG. From the curve representing the relationship between the frequencies F _i and F _f of the Butterworth filter unit 801 corresponding to the order, the order representing the relationship between the variances S _i and S _{f and} the relationship between the closest frequencies F _i and F _f is selected. It ’s fine.

  If the order is 0, the Butterworth filter function is turned off.

  As described above, according to the second embodiment, the same effect as that of the first embodiment can be obtained even in the MPEG-4 encoding method.

  As described above, according to the present invention, in the moving image encoding apparatus including the prefilter unit and the encoding unit, the prefilter unit and the encoding unit are considered in consideration of the degradation of image quality and human visual characteristics. By controlling the above, it is possible to obtain encoded moving image data in which the code amount and the encoding distortion amount are optimum under the condition of the assigned target code amount.

Specifically, a target code amount of a predetermined picture is set for the moving picture coding apparatus. Next, the variance S _i of the input picture to the moving picture coding apparatus is calculated. When encoding the immediately preceding picture, the block distortion amount B _cprev is calculated in advance from the input picture of the encoding unit and the output picture of the local decoding unit.

A visual sensitivity model evaluation formula is determined from the variance S _i and the block distortion amount B _cprev .

Using the determined evaluation formula of the visual sensitivity model and the defining formula (RD model) that defines the relationship between the coding amount of the coding unit and the coding distortion amount, the variance S _f of the picture after passing through the pre-filter unit and The encoding distortion amount MSE _c generated in the encoding unit is calculated as a solution of the Lagrange undetermined multiplier method with the target code amount of the input picture as a constraint condition.

The filter characteristics of the prefilter unit are determined using the variances S _i and S _f as parameters.

Furthermore, the target code amount R _c of the encoding unit is determined from the encoding distortion amount MSE _c and the RD model.

Using the determined target code amount R _c , a weighting parameter for the quantization process is calculated from a defining formula (RQ model) that defines the relationship between the code amount of the encoding unit and the weighting parameter for the quantization process.

Note that the visual sensitivity model is not limited to the evaluation formula (6) used in the first embodiment, and a variable corresponding to the coding distortion amount MSE _c of the RD model of the coding unit, The output picture variance S _f of the filter unit may be included as a variable.

  Furthermore, it goes without saying that the RQ model for calculating the Q scale from the target code amount of the encoding unit obtained from the RD model is not limited to the equation (12).

  Although the embodiments have been described in detail above, the present invention can take an embodiment as, for example, a system, an apparatus, a method, a program, or a storage medium, and specifically includes a plurality of devices. The present invention may be applied to a system that is configured, or may be applied to an apparatus that includes a single device.

  In the present invention, a software program (in the embodiment, a program corresponding to the flowchart shown in the drawing) that realizes the functions of the above-described embodiment is directly or remotely supplied to the system or apparatus, and the computer of the system or apparatus Is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  As a recording medium for supplying the program, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, the present invention includes a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on an instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

It is a block diagram which shows the structure of the moving image encoder of Embodiment 1 of this invention. It is a flowchart which shows the process which the moving image encoder of Embodiment 1 of this invention performs. It is a figure for demonstrating the picture used as the encoding object of embodiment of this invention. It is a figure which shows the characteristic of the RD model used in Embodiment 1 of this invention. It is a figure explaining the relationship between the parameter and coefficient used in Embodiment 1 of this invention. It is a figure for demonstrating selection of the characteristic of the pre filter of Embodiment 1 of this invention. It is a block diagram which shows the structure of the moving image encoder of Embodiment 2 of this invention. It is a flowchart which shows the process which the moving image encoder of Embodiment 2 of this invention performs. It is a figure for demonstrating the structure of the sequence in Embodiment 2 of this invention. It is a flowchart which shows the process of the MPEG-4 encoding system of Embodiment 2 of this invention. It is a figure which shows the characteristic of the I picture RD model used in Embodiment 2 of this invention. It is a figure which shows the characteristic of the I picture RD model used in Embodiment 2 of this invention. It is a figure which shows the relationship between the frequency of the input picture of Embodiment 2 of this invention, and the frequency after filter passing. It is a figure for demonstrating TM5 of a prior art. It is a figure which shows the graph explaining the balance function of a prior art. It is a figure which shows the graph which determines the filter coefficient of a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Moving image encoder 101 Prefilter part 102 Encoding part 103 Local decoding part 104 Code amount control part 105 Variance calculation part 106 Block distortion detection part 107 Visual sensitivity model calculation part 108 Parameter calculation part 109 RD model calculation part 110 Filter characteristic calculation unit 111 RQ model calculation unit

Claims (2)

A moving image encoding apparatus for encoding an input image to a predetermined target code amount using a weighting parameter in quantization processing in moving image encoding for encoding a moving image in a predetermined unit, wherein the input image is distributed A variance calculating means for calculating
Filter means for performing a filtering process on the input image with a given filter characteristic;
Encoding means for performing quantization and encoding on the input image filtered by the filter means;
Decoding means for performing a decoding process on the encoded data output by the encoding means;
Detecting means for detecting a block distortion amount by the encoding means from an input image to the encoding means and a reconstructed image that is an output of the decoding means;
R−, which is the relationship between the code amount (R) and the coding distortion amount (D) in the coding means , using the variance of the input image to the coding means and the coding distortion amount generated by the coding means. An R-D model calculating unit that predetermines a defining formula that defines the D model and calculates a target code amount of the input image from the defining formula ;
A visual sensitivity model for evaluating visual sensitivity from an addition operation of a filter distortion amount generated by the filter means, an encoding distortion amount generated by the encoding means, and a block distortion amount detected by the detection means for an image immediately before the input image. A calculation means;
The input image target code amount , the coding distortion amount calculated by the visual sensitivity model calculating means, and the variance calculated by the variance calculating means correspond to the variance of the input image to the encoding means. Parameter calculating means for calculating the parameters;
The code calculated by the parameter calculation means from among dispersion characteristics between an input image to the filter means and an output image from the filter means in accordance with a change in a plurality of predetermined filter coefficients. A filter characteristic calculation unit that selects a parameter that corresponds to the variance of the input image to the conversion unit and a filter characteristic that is closest to the relationship of the variance calculated by the variance calculation unit;
RQ model calculation means for calculating a weighting parameter in the quantization process from the target code amount calculated by the RD model calculation means,
The parameter calculation unit is configured to obtain a variance of the input image filtered by the filter unit and an encoding distortion amount generated by the encoding unit, and a target code amount of the input image obtained from the prescribed expression. A video encoding apparatus characterized in that, using a Lagrangian undetermined multiplier method that maximizes or minimizes the evaluation formula for evaluating the visual sensitivity, the moving picture coding apparatus is preliminarily calculated with a constraint that the amount of code is equal to the code amount . In moving picture coding for coding a moving picture in a predetermined unit, a moving picture coding method for coding an input picture to a predetermined target code amount using a weighting parameter in quantization processing,
A variance calculating step of calculating variance of the input image;
A filtering step for performing a filtering process on the input image with given filter characteristics;
An encoding step of performing quantization processing and encoding the input image filtered in the filtering step;
A decoding step of performing a decoding process on the encoded data output by the encoding step;
A detection step for detecting a block distortion amount due to the encoding step from an input image to the encoding step and a reconstructed image that is an output of the decoding step;
R−, which is the relationship between the coding amount (R) and the coding distortion amount (D) in the coding step , using the variance of the input image to the coding step and the coding distortion amount generated by the coding step. An R-D model calculating step of predetermining a defining formula for defining the D model , and calculating a target code amount of the input image from the defining formula ;
A visual sensitivity model for evaluating visual sensitivity from an addition operation of the filter distortion amount generated by the filtering step, the coding distortion amount generated by the encoding step, and the block distortion amount detected in the detection step for the image immediately before the input image. A calculation process;
A target code amount of the input image, and encoding distortion amount calculated by the visual sensitivity model calculation step, the dispersion and calculated by the variance calculation step, corresponding to the variance of the input image to the encoding step A parameter calculation step for calculating parameters;
The code calculated by the parameter calculation step from among dispersion characteristics between an input image to the filter step and an output image from the filter step according to a change in a plurality of predetermined filter coefficients. A filter characteristic calculation step of selecting a parameter corresponding to the variance of the input image to the conversion step and a filter characteristic closest to the relationship of the variance calculated by the variance calculation step;
An RQ model calculation step of calculating a weighting parameter in the quantization process from the target code amount calculated by the RD model calculation step,
In the encoding step, the parameter calculation step includes the variance of the input image filtered in the filtering step and the encoding distortion amount generated by the encoding step, and the target code amount of the input image is obtained from the prescribed expression. A moving picture coding method characterized in that it is calculated in advance using a Lagrangian undetermined multiplier method in which the evaluation formula for evaluating the visual sensitivity is maximized or minimized, with the equality of the code amount as a constraint .

Images