JP2009094645A - Moving image encoding apparatus and method for controlling the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To control a target encoding amount according to the degree of deterioration in picture quality caused by block distortion and visual statistic information on a picture. <P>SOLUTION: An encoding unit 105 performs encoding in units of blocks comprising a plurality of pixels to generate encoded data. An encoding amount detection unit 107 detects the amount of encoded data of the picture generated by the encoding unit 105. An encoding distortion detection unit 104 calculates, as a picture distortion amount, the amount of distortion in a block boundary position between a picture obtained by decoding the encoded data and the picture before the encoding. A statistic information calculation unit 101 calculates statistic information on properties affecting the distortion of the block boundary position when a picture of interest is encoded from the picture of interest. Then a first picture target encoding amount calculation unit 103 generates encoding parameters of a picture following the picture of interest on the basis of a sequence target encoding amount, the amount of the encoded data, the statistic information, and the picture distortion amount, and sets them to the encoding unit 105. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a moving image coding technique for performing real-time coding at a variable bit rate.

  Due to dramatic progress in digital signal processing technology in recent years, recording of moving images to a storage medium and transmission of moving images via a transmission path, which have been difficult in the past, are performed. In this case, each picture constituting the moving image is subjected to compression encoding processing, and the data amount is greatly reduced. One typical technique for this compression encoding processing is, for example, the MPEG (Moving Picture Experts Group) system.

  When a series of pictures are compression-encoded under the condition of a constant bit rate according to the MPEG system, depending on the scene composed of multiple pictures, the spatial frequency characteristics of pictures, the correlation between pictures, and the quantization scale value The code amount is greatly different. An important technique for minimizing coding distortion in realizing an apparatus having such coding characteristics is code amount control.

  Algorithms for realizing the code amount control can be broadly classified into two types: a fixed bit rate encoding method (hereinafter referred to as CBR method) and a variable bit rate encoding method (VBR method). In general, it is known that in the VBR system, codes are adaptively assigned according to the encoding difficulty level, so that the picture quality of decoded pictures is better than that in the CBR system. An adaptive code allocation method is realized, for example, by assigning a high bit rate to a scene with a high degree of encoding difficulty and assigning a low bit rate to a scene with a low degree of encoding difficulty.

  As the CBR system, TM5 (Test Model Editing Commitee: “Test Model 5”, ISO / IEC JTC / SC29 / WG11 / N0400 (Apr. 1993) proposed in the process of standardization of the MPEG-2 encoding system. ))) And Patent Document 1 are known.

  Patent Documents 2 and 3 are known as techniques for realizing the VBR method in real time (that is, in one pass). Further, as a technique for adaptively allocating a code amount according to a scene, Patent Document 4 that is attempted to be implemented using imaging control information is known. Next, each prior art will be described.

  In Patent Documents 2 and 3, as shown in FIG. 2, a coding difficulty level called a coding difficulty level calculation unit (201 and 202) is detected for a group of pictures and a picture to be coded. Use the means to do. This realizes a feed-forward VBR system. According to this method, a picture group consisting of a plurality of pictures is divided by the picture group dividing unit 200, and the encoding difficulty level of the picture group for the entire sequence is calculated by the encoding difficulty level information calculating unit 201. Variations in the picture quality of the decoded picture are suppressed by variably assigning the target code amount of the picture group according to the calculated encoding difficulty level.

In Patent Document 4, as shown in FIG. 3, the target code amount of a picture is variably assigned by using imaging control information to the encoding unit 302 in the imaging apparatus. In particular, according to this method, the microcomputer 304 confirms the focusing condition obtained from the focus information and the zoom position from the imaging control information calculation unit 301. The microcomputer 304 realizes the VBR method by assigning a large amount of picture code when the picture currently being photographed is at the wide end and the image is easily focused. According to this method, since the picture target code amount can be controlled using only the imaging control information, it can be realized more easily than the conventional VBR method.
Japanese Patent No. 3112035 Japanese Patent No. 3265818 Japanese Patent No. 3399472 JP 2003-18521 A

  However, Patent Documents 2 to 4 each have the following problems.

  First, according to Patent Document 2, the encoding difficulty level information calculation units 201 and 202 require encoding means similar to the encoding unit 205, and the processing load is very heavy.

  Further, Patent Document 3 discloses that a spatial activity is further used as an encoding difficulty level, but the spatial activity is insufficient to predict the encoding difficulty level in the encoding unit 205. Furthermore, since the picture target code amount is controlled only according to the encoding difficulty level, no visual information of the picture is considered, and adaptive code amount allocation is performed according to the scene. It ’s hard to say.

  Further, as disclosed in Patent Document 4, according to this, the code amount is adaptively allocated according to the scene from the zoom information. However, this proposal does not consider any visual information of the picture, and only increases the amount of code only when the image is easily focused at the wide end. Furthermore, the degree of difficulty in encoding is not considered at all, and it is difficult to perform encoding within a sequence target code amount determined from a target bit rate given at the start of imaging.

  The present invention has been made in view of the above problems. That is, the present invention considers the degree of image quality deterioration obtained from the degree of difficulty in encoding and the importance of the image quality of the scene obtained from visual statistical information of the picture. The present invention provides a technique for obtaining encoded moving image data with good image quality under a given target bit rate condition by controlling a target code amount for a scene.

In order to solve this problem, for example, a moving image encoding apparatus of the present invention has the following configuration. That is,
A moving image encoding apparatus that encodes continuously input pictures within a sequence target code amount determined from a target bit rate,
A dividing unit that divides a moving image composed of pictures arranged in a time axis into scenes composed of a plurality of preset pictures;
Encoding means for encoding an input picture in block units composed of a plurality of pixels and generating encoded data in accordance with an encoding parameter for determining a given quantization scale;
Code amount detection means for detecting the amount of encoded data of the picture generated by the encoding means;
Decoding means for decoding the encoded data obtained from the picture of interest;
A distortion amount calculating means for calculating a distortion amount at a boundary position of the block between a picture obtained by decoding by the decoding means and a picture before encoding as a picture distortion amount;
Statistical information calculating means for calculating statistical information of attributes that affect the distortion of the block boundary position when encoding the picture from the picture of interest;
Based on the sequence target code amount, the encoded data amount detected by the code amount detection unit, the statistical information calculated by the statistical information calculation unit, and the picture distortion amount calculated by the distortion amount calculation unit Setting means for generating coding parameters for a picture following the picture and setting the coding parameters in the coding means.

  According to the present invention, by controlling the target code amount for a scene according to the degree of deterioration in image quality due to block distortion and visual statistical information of a picture, a good image quality can be obtained under a given target bit rate condition. Encoded moving image data can be obtained.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

[First Embodiment]
FIG. 1 is a block configuration diagram of a moving image encoding apparatus that encodes a moving image composed of pictures arranged on a time axis in the embodiment. In the embodiment, MPEG-4 will be described as an example of an encoding method.

  The encoding unit 105 performs MPEG-4 encoding (DCT conversion, quantization, entropy encoding) on the input picture so that it is less than or equal to the picture target code amount Rp given as an encoding parameter. That is, the encoding unit 105 generates and outputs an encoded stream according to the given parameters. In general, since the control of the generated code amount depends on the quantization step value in the quantization process, the picture target code amount Rp can also be said to be a parameter for determining the quantization step (or quantization scale). . Also, the local decoding unit 106 performs MPEG-4 decoding using the encoded stream as an input, and outputs a locally decoded picture. Although details will become clear from the description to be described later, the DCT conversion is performed in units of blocks each composed of a plurality of pixels (8 × 8 pixels). For this reason, the local decoding unit 106 decodes the pixels at the boundary in the 8 × 8 pixel block (the 6 × 6 pixel decoding process one pixel inside the boundary is not performed).

  The code amount detection unit 107 detects the encoded data amount for one picture generated by the encoding unit 105 and outputs the detected result to a first picture target code amount calculation unit 103 described later.

  The coding distortion detection unit detects typical block distortion as coding distortion in the MPEG-4 coding method. The block distortion amount Bp, which is a scalar value indicating the degree of block distortion, is calculated using the pre-encoding picture supplied to the encoding unit 105 and the local decoded picture output from the local decoding unit 106 as follows. Calculate as follows.

Assume that the number of pixels in the horizontal direction of the input picture of the encoding unit 105 is x_siz, and the number of pixels in the vertical direction is y_siz. As shown in FIG. 4, when the horizontal coordinate is J and the vertical coordinate is I, the pixel value of the coordinate (I, J) before encoding is CIN (I, J). Similarly, the pixel value of the coordinates (I, J) in the decoded image obtained from the local decoding unit 106 is defined as COUT (I, J). Since the encoding unit 105 basically performs encoding in block units of 8 × 8 pixels, block distortion occurs at the boundary position of the block of 8 × 8 pixels. Therefore, the block distortion amount (picture distortion amount) Bp for the entire image can be obtained by the following algorithm.
for (I = 0; I <y_size -1; I ++) {
for (J = 0; J <x_size -1; J ++) {
if (J% 8 == 7) {
EDGEin = ABS (CIN (J, I)-CIN (J, I + 1));
EDGEout = ABS (COUT (J, I)-COUT (J, I + 1));
MSEblk ++ = POWER (EDGEin-EDGEout));}
else {
if (I% 8 == 7) {
EDGEin = ABS (CIN (J, I)-CIN (J + 1, I));
EDGEout = ABS (COUT (J, I)-COUT (J + 1, I));
MSEblk ++ = POWER (EDGEin-EDGEout));}
}}
Bp = MSEblk / MSEall; (1)
In the above, MSEall is the sum of squares of differences in the entire picture between CIN (J, I) and COUT (J, I). “X% Y” is a function that returns the remainder when the integer X is divided by the integer Y. Further, since the block distortion amount Bp is calculated by referring only to the pixel value at the block boundary, 6 × 6 pixels inside the block boundary are not referred to. As described above, it is for this reason that the local decoding unit 106 decodes the pixel value at the boundary in the block.

  Here, the algorithm will be briefly described. As described above, the block boundary position is a coordinate position that is an integral multiple of 8 in both the horizontal and vertical directions of the image. Since the coordinates of the upper left corner of the image are generally expressed as the origin (0, 0), the coordinate position of the pixel located at the boundary between two adjacent blocks is a coordinate whose remainder is 7 when the coordinates are divided by 8 And its coordinate is +1. According to the above algorithm, the difference between the two pixels located at the two block boundaries of the original (before encoding) image and the difference between the two pixels located at the block boundary after decoding is determined as two adjacent blocks. It can be said that it is an index value representing distortion. Since there are two types of adjacent blocks in the horizontal direction and the vertical direction, the block distortion amount Bp for the entire image can be calculated by accumulating the distortion values in each.

  Therefore, it is clear that if the block distortion amount Bp is large, it can be said that the decoded image has a significant image quality deterioration compared to the original image and the coding distortion is large. In this embodiment, since the MPEG-4 encoding method is targeted, the above algorithm calculates block distortion at coordinates that are an integral multiple of 8. However, if the block size is other than 8 × 8, the block distortion is determined accordingly. Find it.

  As described above, the coding distortion detection unit 104 calculates the block distortion amount Bp of the picture of interest. Then, the coding distortion detection unit 104 outputs the calculated block distortion length Bp to the coding parameter calculation unit 103.

The statistical information calculation unit 101 calculates statistical information of attributes that affect the distortion of the block boundary position. In the present embodiment, a pixel in a picture for each block consisting of 8 × 8 pixels, four statistics P _h as described below, P _s, P _y, and calculates the P _a. As a premise, it is assumed that each pixel of image data to be encoded is composed of three components of luminance (Y) and chroma (Cb, Cr). In the case of the MPEG-4 profile in which the subsample is 4-2-0, the chroma block is a 4 × 4 pixel block.

Hereinafter, information P _h constituting statistics statistics calculator 101 of the present embodiment is calculated, P _s, P _y, for P _a will be described.

[Statistical information P _h ]
Information P _h is the block is information indicating whether or not the skin color.

  It is known that human visual characteristics are very sensitive to the hue of skin color. Therefore, as one piece of information for calculating the picture quality importance in the picture quality importance calculation unit 102, the number of blocks corresponding to the flesh color hue in the input picture is calculated.

The determination of whether or not the skin color is used is realized by using the two-dimensional coordinates of Cb-Cr shown in FIG. If the average value of Cb in the block of the input picture is Cb ′, the average value of Cr is Cr ′, and the coordinates on FIG. 5 are Pbr (Cb ′, Cr ′), the hue Hθ of the block is It is obtained by the formula.
Hθ = tan (Cb / Cr) ⁻¹ (2)

  The skin color hue is known to be around 123 degrees in the Cb-Cr space, but in this embodiment, a section (angle range) of 100 to 150 degrees is defined in advance as the hue of the skin color. It is determined whether Hθ is an angle within the section.

The determination results and P _h, blocks P _h = 1 if the skin color, and P _h = 0 otherwise.

[Statistical information P _s ]
Information P _s is saturation information of the block. The saturation information P _s can also be calculated by using the Cb—Cr two-dimensional coordinates shown in FIG. Human visual characteristics are sensitive to block distortions in regions with relatively low saturation (regions close to achromatic colors). Therefore, the saturation information P _s of each block in the input picture is calculated as one piece of information for calculating the picture quality importance in the picture quality importance calculation unit 102.

For the saturation information P _s , the distance from the origin of the coordinate Pbr may be calculated.
Ps = √ (Cb ′ ² + Cr ′ ² ) (3)
[Statistical information P _y ]
Information _Py is an average value of the luminance information (Y) in the block. Human visual characteristics are sensitive to block distortion in regions where luminance Y is relatively high. Therefore, the average value P _y of the luminance Y of the block in the input picture is calculated as one piece of information for calculating the picture quality importance in the picture quality importance calculation unit 103. Since one block is 8 × 8 pixels, if the luminance of each pixel is Y _i (i = 0, 1, 2,..., 63), the average luminance value can be obtained by the following equation.
P _y = {ΣY _i } / 64 (4)

Statistics P _a]
Information P _a is the variance value information obtained from the values of the luminance information Y of each pixel in the block.

Human visual characteristics are sensitive to block distortion in regions where the spatial frequency is relatively low. Therefore, the variance value of the luminance Y in each block in the input picture is calculated as one piece of information for calculating the picture image quality importance in the picture image quality importance calculating unit 102. If the average luminance value of each pixel in the block is Y ′ and the luminance Y value of each pixel in the block is Yi (i = 0, 1, 2,..., 63), the following equation is obtained.
P _a = Σ (Y _i −Y ′) ² (5)
Strictly speaking, the variance is obtained by dividing the above equation (5) by the number of samples (8 × 8 = 64 in the embodiment). However, since it is sufficient to know the index value of the variance, the division is performed. Not.

  Heretofore, the four pieces of statistical information generated by the statistical information calculation unit 101 in the embodiment have been described.

For example, if the number of pixels x_size in the horizontal direction of the input picture in FIG. 4 is “640” and the number of pixels y_siz in the vertical direction is “480”, 4800 (= (640/8) × (480) in this image. / 8)) block exists. FIG. 6 shows an example of block division of a certain picture. If the number of the first block is 0 and the first block is represented as block # 0, the statistical information calculation unit 101 calculates the above four statistical information for block # 0 to block # 4799. The statistical information of each block can be expressed as an array P _h [N], P _s [N], P _y [N], P _a [N] (N = 0, 1, 2,..., 4799).

Next, the picture image quality importance calculation unit 102 will be described. The picture image quality importance calculation unit 102 includes statistical information P _h [N], P _s [N], P _y [N], P _a [N] input from the statistical information calculation unit 101, and FIG. The picture quality importance Pi of the input picture is calculated using a predetermined visual sensitivity table. The visual sensitivity table defines a weighting coefficient Cw [k] and a normalization coefficient Cd [k] (k = 0 to 3) for each of four pieces of statistical information. That is, the k = 0 weighting coefficient Cw [0] and the normalization coefficient Cd [0] are for the statistical information P _h [N]. The weighting coefficient Cw [1] and the normalization coefficient Cd [1] with k = 1 are for the statistical information P _s [N]. The weighting coefficient Cw [2] and the normalization coefficient Cd [2] for k = 2 are for the statistical information P _y [N]. K = 3 is the weighting coefficient Cw [3] and the normalization coefficient Cd [3] is for the statistical information P _a [N].

The picture image quality importance calculation unit 102 multiplies three pieces of statistical information excluding the statistical information Ph [N] and the normalization coefficient Cd [k] for each block in the picture, and clips it to a value of “1” or less. . It should be noted that the statistical information P _h [N] has already been normalized to binary values of 0 and 1. For example, with respect to the saturation information P _s [N], the saturation information Ps ′ [N] normalized by the following processing is obtained.
for (N = 0; N <48000; N ++) {
Ps' [N] = Ps [N] × Cd [1];
if (Ps'[N]> 1) {
Ps' [N] = 1;
}
} (6)

Similarly, P _y [N] and P _a [N] are also processed, and normalized statistical information P _y '[N] and P _a ' [N] are calculated.

Next, the picture image quality importance calculation unit 102 based on P _h [N] and the normalized statistical information Ps ′ [N], PY ′ [N], and Pa ′ [N] obtained by Expression (6), The picture quality importance Pi is obtained by performing the following processing. When obtaining the picture image quality importance Pi, Pi is initialized to “0”.
for (N = 0; N <48000; N ++) {
Pi = Pi + Ph [N] x Cw [0] + Ps' [N] x Cw [1] + PY '[N] x Cw [2] + Pa' [N] x Cw [3];
} (7)

  That is, the picture image quality importance level Pi can be said to be a sum value obtained by applying a weighting coefficient to each of the four normalized statistical information for each block of the target picture.

  Next, the first picture target amount calculation unit 103 will be described. Here, it is assumed that the target bit rate given to the moving picture encoding apparatus of the present embodiment is “Ts”. The first picture target amount calculation unit 103 uses the target bit rate Ts, the block distortion amount Bp input from the coding distortion calculation unit 104, and the picture quality importance Pi input from the picture quality importance calculation unit. Then, the picture target code amount Rp (encoding parameter) of the encoding unit 105 for the next picture (subsequent picture) is calculated. A method for calculating the picture target code amount Rp will be described with reference to FIG.

In the moving picture encoding apparatus of the present embodiment, a target code amount Rs of a scene composed of a plurality of preset pictures is obtained by the following equation. Note that one scene in the embodiment is assumed to be 15 pictures (Ns = 15). Here, the first picture in the scene is an I picture. If the frame rate of the input picture is 30 fps, the scene target code amount Rs (target code amount of 15 frames) is calculated according to the following equation (8).
Rs = Ts × 1/2 (8)

Furthermore, if the target picture input to the moving picture encoding apparatus is other than the first picture in the target scene (in the case of the second and subsequent pictures), for example, if it is picture P5 in FIG. The initial value Rp ′ of the picture target code amount of P5 is obtained from the following equation (9).
Rp ′ = (Rs−Rf) / Nr (9)
Here, Nr is the number of remaining pictures including the picture P5 in the sequence (number of uncoded pictures), and Rf is the total generated code amount from the picture P1 to the picture P4 in the scene.

  Here, the final picture target code amount Rp ′ is increased or decreased by the processing flow shown in FIG. 8 according to the block distortion amount Bp and the picture quality importance Pi according to the block distortion amount Bp. calculate.

  In the process of FIG. 8, Bp ′ and Pi ′ are the average values of the block distortion amounts Bp and the picture image quality importance levels Pi of all the pictures constituting the immediately preceding scene. In step S800 in FIG. 8, the block distortion amount Bp is compared with a predetermined threshold value Bmin. If the block distortion amount Bp is smaller than the threshold value Bmin as a result of the comparison in step S800, it is next determined in step S801 whether or not the initial value Rp ′ is to be decreased. The threshold value Bmin defines in advance a limit value of the block distortion amount that is difficult to recognize from the viewpoint of visual characteristics in the reconstructed picture. In step S801, the picture quality importance Pi of the picture P5 is compared with the picture importance Pi ′ which is the Pi average value of the immediately preceding scene. If the picture quality importance Pi is smaller, the initial value Rp ′ is set. Decrease in accordance with the formula in step S807. Β in the expression in step S807 is a constant including a predetermined decimal point of 0.0 or more.

On the other hand, if the block distortion amount Bp is greater than or equal to the threshold value Bmin as a result of the comparison in step S800, it is determined in steps S802 and S803 whether or not the initial value Rp ′ is to be increased. In step S802, the picture quality importance Pi of the picture P5 is larger than the picture importance Pi ′ that is the average value of Pi of the immediately preceding scene, and the block distortion amount Bp is the block distortion amount Bp that is the Bp average value of the immediately preceding scene. If “Re> 0” in step S804, the initial value Rp ′ is increased according to the equation in step S805. Here, “Re” subtracts the code amount obtained by increasing the initial value Rp ′ in the current scene from Rr corresponding to the surplus code amount for the target bit rate Ts obtained as a result of encoding up to the immediately preceding scene. Code amount. Rr and Re are obtained by calculating the following equation each time the encoding of the scene is completed.
Rr = Rr + Rs−Rs ′
Re = Rr (10)
Here, Rs ′ is a scene generation code amount generated in the scene. Re obtained by Expression (10) is set as an initial value used for the first picture of the scene, and is updated as needed in step S805.

  Next, a moving image encoding process according to this embodiment will be described with reference to FIG. FIG. 9 shows the transition of the picture target code amount Rp according to the present embodiment for three consecutive scenes from the beginning of the moving image.

  Since scene 0 is the first scene in the sequence, it is not possible to calculate the picture importance Pi ′, which is the Pi average value of the immediately preceding scene, and the block distortion amount Bp ′, which is the Bp average value of the immediately preceding scene. Therefore, only the process of step S806 in FIG. 8 is performed, and the initial value Rp ′ of the picture target code amount is directly supplied to the encoding unit 105 as the picture target code amount Rp.

  Next, in scene 1, in addition to step S806, the process of step S807 for reducing the code amount with respect to the initial value Rp ′ of the picture target code amount is performed. On the other hand, the process of step S805 for increasing the code amount is not performed in the scene 1. This is because the scene target code amount Rs is equal to the scene generation code amount Rs ′ and Rr is 0 in the encoding result of the scene 0 in FIG. Of course, in the scene 0, even if the code amount is not actively reduced in the process of step S807, the process of step S805 is executed in the scene 1 when it is determined that Rr> 0 according to the equation (10). Is possible. In the scene 1 in FIG. 9, it can be seen that the process of step S807 is performed in the pictures P2, P3 and P4.

  Next, in scene 2, in addition to steps S806 and S807, the process of step S805 is performed. This is because Rr> 0 is obtained from the calculation result of the equation (10) calculated prior to the encoding of the scene 2. This corresponds to the surplus code amount generated by the process of step S807 for the pictures P2, P3, and P4 in the scene 1. For the pictures P0, P1, and P2 of the scene 2, and further for the pictures P7, P8, and P9, the processing of step S805 causes the code amount to increase with respect to the initial value Rp ′ of the picture target code amount. It can be seen that the amount Rp is calculated.

  In the present embodiment, when obtaining the variables Pi ′ and Bp ′, the picture quality importance Pi and the block distortion amount Bp of the immediately preceding scene are used, but the picture quality importance Pi and block distortion of the immediately preceding multiple pictures are used. The amount Bp may be used. In this case, for example, when the input picture is P5, the picture quality importance Pi and the block distortion amount Bp of the pictures P5 to P13 of the current scene are used. It becomes.

[Second Embodiment]
A second embodiment will be described. FIG. 10 is a block diagram of the moving picture coding apparatus according to the second embodiment. The same reference numerals are assigned to the same configurations as those in the first embodiment. Accordingly, the encoding unit 105 according to the second embodiment performs an encoding process based on MPEG-4 as in the first embodiment.

  10 can be said to be obtained by adding a scene image quality importance calculation unit 1000, a scene dividing unit 1001, a scene target code amount calculation unit 1002, and an encoding parameter calculation unit 1003 to the configuration of FIG. In FIG. 10, except for the four newly added processing units, the same processing as in the first embodiment is performed, and detailed description thereof is omitted here.

  A comparison of the processing realized in the second embodiment and the first embodiment is shown in the table of FIG. In the second embodiment, the code amount control target is the scene target code amount Rs, which is different from the first embodiment in which the picture target code amount Ri is controlled.

  Processing of the scene image quality importance calculation unit 1000 will be described with reference to FIGS. 11 and 12. The scene image quality importance calculation unit 1000 calculates the scene image quality importance using the picture image quality importance Pi.

  First, the scene image quality importance calculation unit 1000 obtains the picture image quality importance Pi, which is a scalar value including a decimal point, obtained from the equation (7), from any of a plurality of class values according to the magnitude of the value. Classify. Although it is possible to select an optimum number of classes to be classified according to the embodiment, the description will be made assuming that the number of classes in this embodiment is “5”.

  In FIGS. 11A and 11B, the horizontal axis represents the picture quality importance Pi input from the picture quality importance calculation unit 102, and the vertical axis represents the divided picture quality importance class number Cp. It is. In order to divide into five classes, four threshold values T1 to T4 are defined in advance. Here, it is not always necessary to divide the picture quality importance Pi into equal intervals. FIG. 11A shows a setting example of threshold values T1 to T4 in the encoding mode when the image quality stability importance type is defined as the encoding mode of the moving image encoding apparatus of the present embodiment. FIG. 6B shows an example of setting the encoding mode thresholds T1 to T4 when the image quality sharpening type is defined.

  Further, the scene image quality importance calculation unit 1000 calculates the scene image quality importance Si using the divided picture image quality importance class number Cp. The scene image quality importance Si is obtained by averaging the picture image quality importance class numbers Cp of a plurality of pictures encoded in the past and the input picture after weighted addition. The number of past pictures to be referred to can be selected according to the embodiment. In this embodiment, the number of pictures is “5” for the sake of simplicity.

  FIG. 12 shows changes in scene image quality importance Ci and picture image quality importance class number Cp. Here, if the array storing the past five pictures of the picture quality importance class Cp is ArrayCp [N] (N = 0 to 4), the scene quality importance Si can be obtained by the following equation. However, N = 0 stores the picture quality importance class Cp of the previous picture, and N = 4 stores 5 pictures before.

Si = (C _wp0 × Ci + C _wp1 × ArrayPi [0] + C _wp2 × ArrayPi [1] + C _wp3 × ArrayPi [2] + C _wp4 × ArrayPi [3] + C _wp5 × ArrayPi [4]) / ( C _wp0 + C _wp1 + C _wp2 + C _wp3 + C _wp4 + C _wp5 )… (11)
However, C _{wp0 to} C _wp5 are predetermined weighting coefficients and are integers of 1 or more. In this embodiment, C _{wp0 to} C _wp4 are all set to the value “1”. The processing of Expression (11) is performed for each picture every time the picture quality importance Pi is input from the picture quality importance calculator 102. That is, the scene image importance Si obtained by the equation (11) is obtained by weighted addition of the tendency of the picture image importance Pi of a plurality of pictures encoded immediately before. In FIG. 12, the scene image quality importance is “1” for picture numbers 1 to 5. This is because all five previous pictures do not exist and the picture quality importance of picture number 0 is used in equation (11) until all five pictures are available.

  The scene division unit 1001 adaptively configures a scene from the scene image quality importance calculation unit 1000. In the second embodiment, code amount control is performed for the scene divided by the scene dividing unit 1001. Although a scene is composed of a plurality of pictures, the maximum number of pictures in the scene is defined in advance. The number Ns of pictures constituting the scene is set to be equal to or less than a preset Nmax (the number of pictures included in one scene is set to be equal to or less than the upper limit number). In the second embodiment, Nmax = 15 is set to simplify the description. In the scene dividing unit 1001, consecutive picture groups having the same scene image quality importance Si obtained by Expression (11) are set as pictures constituting one scene. Here, the scene image quality importance Si input from the scene image quality importance calculator 1000 is a scene image quality importance class Cs obtained by rounding the first decimal place to a scalar value including a decimal point. Consecutive pictures having the same scene image quality importance class Cs are configured as a scene. FIG. 13 shows a scene image importance class Cs and a scene configuration.

Next, the scene target code amount calculation unit 1002 will be described. The scene target code amount calculation unit 1002 receives scene quality importance Si input from the scene image quality importance calculation unit 1000, information indicating scene switching input from the scene division unit 1001, and input from the coding distortion calculation unit 104. The scene target code amount Rs is calculated from the block distortion amount Bp. The scene target code amount Rs is only calculated prior to encoding the first picture of the scene. When calculating the scene target code amount Rs, the CBR code amount Rcbr obtained from the target bit rate Ts and the frame rates Fr and Nmax shown by the following equation (12) is increased or decreased.
Rcbr = Ts × Nmax × 1 / Fr (12)

  The situation in which the scene target code amount is decreased from the CBR code amount Rcbr is a scene in which the block distortion amount Bp is small and the scene image quality importance Si is small. However, since the scene target code amount Rs needs to be calculated before the scene is encoded, it is necessary to predict the block distortion amount Bp and the scene image quality importance Si of the scene to be encoded. Therefore, in the second embodiment, this prediction is realized using the following method.

In the prediction of the block distortion amount Bp, the block distortion amount Bp of the scene is predicted by calculating the weighted average (weighted average) from the block distortion amounts Bp of a plurality of pictures encoded immediately before. The block distortion amount Bs to be predicted can be obtained by the following equation (13) if the array storing the block distortion amounts Bp of the past five pictures is ArrayBp [N] (N = 0 to 4).
Bs = (C _wb0 × ArrayBp [0] + C _wb1 × ArrayBp [1] + C _wp2 × ArrayBp [2] + C _wb3 × ArrayBp [3] + C _wp4 × ArrayBp [4]) / (C _wb0 + C _wb1 + C _wb2 + C _wb3 + C _wb4 )… (13)
However, C _{wb0 to} C _wb4 are predetermined weighting coefficients, and C _wb0 = 4 and C _{wb1 to} C _wb4 = 1.

  Next, the prediction of the scene image quality importance Si may be performed in accordance with the scene dividing method of the scene dividing unit 1001. That is, since the scene image importance class Cs obtained by rounding off the scene image importance Si is divided into scenes having the same scene image importance class Cs, the scene image importance Si of the scene may be set to the corresponding scene image importance class Cs.

  Next, a method for calculating the scene target code amount Rs will be described with reference to FIG.

  First, in step S1400, it is determined whether the predicted block distortion amount Bs is smaller than a predetermined constant Bmin. In step S1401, it is determined whether the predicted scene image quality importance class Cs is smaller than a predetermined constant CSmin. When the two determination results are true (Yes), the code amount reduced according to the scene image quality importance class Cs and the block distortion amount Bs predicted from the CBR code amount Rcbr is set as the scene target code amount Rs.

  On the other hand, if the block distortion amount Bs predicted in step S1402 is larger than a predetermined constant Bmin, Rr corresponding to the surplus code amount for the target bit rate Ts obtained as a result of encoding the immediately preceding scene in step S1403. When it is larger than 0, the code amount is increased with respect to the CBR code amount Rcbr to obtain the scene target code amount Rr.

  Note that γ in step S1406 and θ in step S1404 are predetermined constants.

  Finally, the processing of the second picture target code amount calculation unit will be described. With the scene target code amount Rs as an input, the picture target code amount Rp is output to the encoding unit 105. In the second embodiment, the process of calculating the picture target code amount Rp can be realized by using a conventional technique, and for example, realized by using the TM5 algorithm as follows.

For each of the I, P, and B pictures, the picture complexity X _i , X _p, and X _b (corresponding to the I, P, and B pictures, respectively) are obtained from the encoding result in the encoding unit 105 using the following equations.
X _i = RA _i × Q _i
X _p = RA _p × Q _p
X _b = RA _b × Q _b (14)
However, RA _i , RA _p , and RA _b indicate code amounts obtained as a result of encoding the I, P, and B pictures, respectively, and Q _i , Q _p, and Q _b are all in the I, P, and B pictures, respectively. The average value of the Q scale for the macroblocks. From the equation (14), using the following equation (15), the picture target code amounts T _i , T _p and T _b are obtained for each of the I, P and B pictures, and the picture type to be encoded by the encoding unit 5 is obtained. Accordingly, the picture target code amounts T _i , T _p and T _{b are} selected and output to the encoding unit.

ただし、Ｋp＝１．０、Ｋb＝１．４である。

However, Kp = 1.0 and Kb = 1.4.

[Third Embodiment]
FIG. 15 is a block diagram of a moving picture encoding apparatus according to the third embodiment. The moving picture coding apparatus in FIG. 15 is a moving picture coding apparatus according to the second embodiment in which an imaging control information calculation unit 1500 and a motion information calculation unit 1501 are newly added. Here, first, imaging control information (AE information _Pae , AF information _Paf ) calculated by the imaging control information calculation unit 1500 will be described.

_-Shooting control information: AE information _Pae
This is information for adjusting the exposure and shutter speed of the image pickup means that picks up an image from a subject (not shown) and gives an input picture to the moving picture coding apparatus of the third embodiment. The imaging control information calculation unit 1500 obtains AE information P _ae by calculating using the luminance (Y) of the input picture. Here, the AE information P _ae is P _ae > 0 when the imaging control information calculation unit 1500 determines that the input picture is underexposed or underexposed, and the degree of overexposure and underexposure. Is expressed as a number. It indicates that if the value of P _ae is greater exposure over and under is a violent situation. Otherwise, P _ae = 0 is output to the picture quality importance calculator 102.

Shooting control information: AF information P _af
This is information for adjusting the focal length by controlling the lens position of an imaging means (not shown). The imaging control information calculation unit 1500 obtains AF information P _af by calculating using the luminance (Y) of the input picture. If the imaging control information calculation unit 1500 determines that the input picture is a picture whose focal length is being adjusted, P _af = 1, otherwise P _af = 0 is sent to the picture image quality importance calculation unit 102. Output.

Next, the processing of the picture importance calculation unit 102 will be described. The right side of equation (6) and (7) shown in the first embodiment, the AE information P _ae and By adding AF information P _af, picture target code amount Rp shown in the first embodiment The sequence target code amount Rs shown in the second embodiment can be calculated in the same manner. The visual sensitivity table shown in the first embodiment is the weighting coefficient Cw [k] and the normalization coefficient Cd [k] (k = 0 to 3). The weighting coefficient Cw [4] and the normalization coefficient Cd [4] are respectively defined as values corresponding to the AE information _Pae , and the weighting coefficient Cw [5] and the normalization coefficient Cd [5] are respectively defined as the AF information P Define a value corresponding to _af .

  Furthermore, the weighting coefficient Cw [k] (k = 0 to 5) can be selected and changed from a plurality of predetermined combinations according to the imaging mode. An example of setting the weighting coefficient Cw [k] (k = 0 to 5) is shown in FIG. It is also possible to combine the picture quality importance class Cp calculation method corresponding to the image quality stability importance type and the image quality sharpness importance type shown in FIG. 11 described in the second embodiment.

  As described above, according to the present embodiment, the target code amount for a scene is controlled according to the degree of deterioration of image quality due to block distortion and the importance of the image quality of the scene obtained from the visual statistical information of the picture. As a result, it is possible to obtain encoded moving image data with good image quality under the condition of a given target bit rate.

  In addition, by controlling the target code amount for the scene according to the shooting situation in consideration of the imaging control information of the imaging means for the importance of the image quality of the scene, it is favorable under the condition of the given target bit rate. It is possible to obtain encoded moving image data with image quality.

It is a block block diagram of the moving image encoder in 1st Embodiment. It is a block block diagram of the conventional moving image encoder. It is a block block diagram of the conventional moving image encoder. 6 is a diagram for explaining the operation of a coding distortion detection unit 104. FIG. It is a figure for demonstrating the principle of the skin color detection using a Cb-Cr coordinate. It is a figure which shows the example which divided the image into blocks. Diagram explaining the structure of the scene It is a flowchart which shows the processing content of the 1st picture target code amount calculation part. It is a figure which shows transition of the picture target code amount. It is a block block diagram of the moving image encoder in 2nd Embodiment. It is a figure which shows the example of the class classification | category of the picture quality importance in 2nd Embodiment. It is a figure which shows transition of a picture image quality importance class. It is a figure which shows transition of a scene image quality importance class. It is a flowchart which shows the processing content of a scene target code amount calculation part. It is a block block diagram of the moving image encoder in 3rd Embodiment. It is a figure which shows the example of the visual sensitivity table in 1st Embodiment. It is a figure which shows the table which shows the correspondence of the information used for the encoding process of 1st, 2nd embodiment. It is a figure which shows the example of the table for the setting of the weighting coefficient Cw in 3rd Embodiment.

Claims (9)

A moving image encoding apparatus that encodes continuously input pictures within a sequence target code amount determined from a target bit rate,
A dividing unit that divides a moving image composed of pictures arranged in a time axis into scenes composed of a plurality of preset pictures;
Encoding means for encoding an input picture in block units composed of a plurality of pixels and generating encoded data in accordance with an encoding parameter for determining a given quantization scale;
Code amount detection means for detecting the amount of encoded data of the picture generated by the encoding means;
Decoding means for decoding the encoded data obtained from the picture of interest;
A distortion amount calculating means for calculating a distortion amount at a boundary position of the block between a picture obtained by decoding by the decoding means and a picture before encoding as a picture distortion amount;
Statistical information calculating means for calculating statistical information of attributes that affect the distortion of the block boundary position when encoding the picture from the picture of interest;
Based on the sequence target code amount, the encoded data amount detected by the code amount detection unit, the statistical information calculated by the statistical information calculation unit, and the picture distortion amount calculated by the distortion amount calculation unit A moving picture coding apparatus comprising: setting means for generating coding parameters for a picture following the picture and setting the coding parameters in the coding means.   The statistical information calculation means calculates information indicating whether the color is a skin color, saturation information, luminance information, and information indicating luminance dispersion for each block, and further calculates a picture for the entire picture from each calculated information. The moving image encoding apparatus according to claim 1, wherein information indicating importance is calculated. The setting means includes
For the first picture of the scene of interest, the encoding parameter is determined using the average value of the block distortion amount and the picture importance of each picture constituting the scene before the scene of interest,
For the second and subsequent pictures of the target scene, based on the preset scene target code amount, the encoded data amount of the already encoded picture in the target scene, and the number of unencoded pictures of the target scene, The moving picture coding apparatus according to claim 2, wherein the coding parameter is determined. A method for controlling a moving image encoding apparatus that encodes continuously input pictures within a sequence target code amount determined from a target bit rate,
A dividing step of dividing a moving image composed of pictures arranged in a time axis into a scene composed of a plurality of preset pictures;
An encoding step of encoding the input picture in units of blocks composed of a plurality of pixels and generating encoded data according to an encoding parameter for determining a given quantization scale;
A code amount detection step of detecting the amount of encoded data of the picture generated in the encoding step;
A decoding step of decoding the encoded data obtained from the picture of interest;
A distortion amount calculating step of calculating, as a picture distortion amount, a distortion amount at a boundary position of the block between a picture obtained by decoding in the decoding step and a picture before encoding;
A statistical information calculating step of calculating statistical information of an attribute that affects the distortion of the block boundary position when encoding the picture from the target picture;
Based on the sequence target code amount, the encoded data amount detected in the code amount detection step, the statistical information calculated in the statistical information calculation step, and the picture distortion amount calculated in the distortion amount calculation step And a setting step of generating an encoding parameter of a picture following the picture and setting the encoding parameter in the encoding step. A moving image encoding apparatus that encodes continuously input pictures within a sequence target code amount determined from a target bit rate,
Encoding means for encoding an input picture in block units composed of a plurality of pixels and generating encoded data in accordance with an encoding parameter for determining a given quantization scale;
Code amount detection means for detecting the code amount of the encoded data generated by the encoding means;
Decoding means for decoding the encoded data generated by the encoding means;
A distortion amount calculating means for calculating a distortion amount at a boundary position of the block between a picture obtained by decoding by the decoding means and a picture before encoding as a picture distortion amount;
Statistical information calculating means for calculating statistical information of attributes that affect the distortion of the block boundary position when encoding the picture from the picture of interest;
Based on the statistical information calculated by the statistical information calculation means, the importance level of the picture of interest is calculated, and the calculated importance level is compared with a plurality of preset threshold values to classify it into one of a plurality of class values. A picture quality importance calculating means for calculating a weighted average of class values of each of a plurality of previously set pictures of the current picture and the current picture as a picture quality importance;
Scene dividing means for dividing one or more pictures whose picture quality importance calculated by the picture quality importance calculating means is equal to or less than a preset upper limit and continuously the same as a scene;
Scene image quality importance calculating means for calculating the scene image quality importance of the scene from the picture image quality importance of each picture calculated by the picture image quality importance calculating means;
According to the sequence target code amount, the amount of encoded data detected by the code amount detection means, the picture distortion amount calculated by the distortion amount calculation means, and the scene image quality importance calculated by the scene image importance calculation means A scene target code amount calculating means for calculating a scene target code amount;
A moving image comprising: a setting unit configured to generate an encoding parameter of a picture subsequent to the picture of interest based on the scene target code amount calculated by the scene target code amount calculating unit, and to set the encoding parameter in the encoding unit Image encoding device.   The statistical information calculation means calculates information indicating whether the color is a skin color, saturation information, luminance information, and information indicating luminance dispersion for each block, and further calculates a picture for the entire picture from each calculated information. 6. The moving picture coding apparatus according to claim 5, wherein information indicating importance is calculated. Furthermore,
Imaging means;
Imaging control information calculating means for calculating imaging control information imaged by the imaging means,
6. The moving picture encoding apparatus according to claim 5, wherein the picture image quality importance calculating unit calculates the picture image quality importance according to the image capture control information calculated by the image capture control information calculating unit.   The picture quality importance calculating means calculates the picture quality importance by appropriately changing the imaging control information, the statistical information, and the weighting of the distortion amount according to the imaging mode of the imaging means. 8. The moving picture encoding apparatus according to claim 7, wherein A method for controlling a moving picture coding apparatus for coding pictures that are continuously input within a sequence target code amount determined from a target bit rate,
An encoding step of encoding the input picture in units of blocks composed of a plurality of pixels and generating encoded data according to an encoding parameter for determining a given quantization scale;
A code amount detection step of detecting a code amount of the encoded data generated in the encoding step;
A decoding step of decoding the encoded data generated in the encoding step;
A distortion amount calculating step of calculating, as a picture distortion amount, a distortion amount at a boundary position of the block between a picture obtained by decoding in the decoding step and a picture before encoding;
A statistical information calculating step of calculating statistical information of an attribute that affects the distortion of the block boundary position when encoding the picture from the target picture;
Based on the statistical information calculated in the statistical information calculation step, the importance of the picture of interest is calculated, and the calculated importance is compared with a plurality of preset threshold values, thereby classifying it into one of a plurality of class values. A picture quality importance calculating step for calculating a weighted average of class values of each of a plurality of preset pictures before the current picture and the current picture as a picture quality importance;
A scene division step of dividing one or more pictures that have the same or lower number of picture quality importance calculated in the picture quality importance calculation step into a scene that is equal to or less than a preset upper limit number;
A scene image quality importance calculating step of calculating a scene image quality importance of the scene from the picture image quality importance of each picture calculated in the picture image quality importance calculating step;
According to the sequence target code amount, the amount of encoded data detected in the code amount detection step, the picture distortion amount calculated in the distortion amount calculation step, and the scene image quality importance calculated in the scene image importance calculation step A scene target code amount calculating step for calculating a scene target code amount;
And a setting step of generating a coding parameter of a picture following the picture of interest based on the scene target code amount calculated in the scene target code amount calculation step and setting the encoding parameter in the encoding step. A method for controlling an image coding apparatus.

Images