JP2003153268A - Video encoding method and apparatus, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem of a conventional 2-path video encoder that cannot have maximally utilized the possibility of improvement of image quality by twice encoding because the encoding quantity assignment taking into account a difference from a visual property based on color and luminance is not available. SOLUTION: A video encoding amount assignment process 10 includes a process 121 that reads a visual class value by each second video unit calculated on the basis of a luminance value and a color difference value by each third video image in preliminary encoding and a process 122 that assigns a code quantity to the second video unit on the basis of a target code quantity of the first video unit assigned in a step 130 resulting from the target code quantity of the entire sequence and the encoding information of the first video unit and on the basis of encoding information and a visual class value in the second video unit.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video coding method for improving image quality in consideration of visual characteristics in a two-pass video encoder used for DVD authoring system for producing contents offline. The present invention relates to a code amount allocation method.

[0002]

2. Description of the Related Art In an authoring system based on the DVD-Video standard, information of a digitized video is compressed and recorded at an appropriate bit rate according to a scene, so that a limited disk capacity can be utilized to the maximum extent. It enables the production of high quality content. The video compression technology that enables this is the MPEG2 standard (ISO / IEC138) that defines the bit stream and the decoding device.
18-2).

First, the video coding based on the MPEG2 standard will be briefly described. FIG. 14 shows a block diagram of a video encoder based on MPEG2. First, with respect to the input video sequence 210, the frame rearrangement circuit 222 rearranges the frames in the encoding order according to the encoding type of the frame (or picture). The coding type includes an intra-frame coded frame (hereinafter, referred to as an I picture) and an inter-frame coded frame (hereinafter, P
There are three types: a picture) and a frame interpolation coded frame (hereinafter referred to as a B picture). An I picture is a picture that can be decoded using only the bitstream of that picture. A P picture is a picture that can be decoded using only the data of an I or P picture that was decoded before this picture and the data of this picture, and a B picture is two I or P pictures that are temporally preceding and following. It is a picture that can be decoded using the picture data and the picture data. In any type of picture, the picture is a luminance component Y, color difference components Cb, C
Each component of r is divided into fixed-size macroblocks having 16 × 16 pixels as a standard, and an encoding process is performed.

The motion estimation circuit 231 calculates the motion vector of the frame output from the frame rearrangement circuit 222. The motion estimation is performed by the frame rearrangement circuit 222.
You may go earlier. The calculated motion vector is used later in the motion compensation circuit 232. For the motion compensation, the data output from the differentiator 282 is DC-converted by the DCT circuit 241.
After the T-transform, the quantization circuit 251 quantizes, the inverse quantization circuit 252 performs inverse quantization, and the inverse DCT circuit 242 performs inverse DCT.
The value obtained by adding the previous motion compensation amount by the adder 281 to the data obtained by the conversion is performed. Motion compensation circuit 2
32 generates motion-compensated data from the data output from the adder 281 and the motion vector output from the motion estimation circuit 231 and outputs the data to the difference unit 282. The motion-compensated data is output to the adder 281 as the previous motion-compensated amount for the picture to be encoded next.

The differentiator 282 calculates the difference between the data output from the motion estimation circuit 231 and the data output from the motion compensation circuit 232. The video signal output from the differentiator 282 is an original image when the video signal input to the differentiator 282 is an I picture, and a differential image after motion compensation when the video signal is a P or B picture. In either case, the picture is divided into blocks of fixed size and the DCT circuit 24
1, the Discrete Cosine Transform (DCT) process, which is an important process for reducing the amount of information, is performed. The quantization circuit 251 performs a quantization process on the transform coefficient output from the DCT circuit 241, according to the value of the quantization width given by the quantization control circuit 263. The variable length coding circuit 261 converts the transform coefficient output from the quantization circuit 263 into a one-dimensional sequence, and performs variable length coding on the transformed sequence. The data after variable-length coding must be controlled by quantization control, which is a process of adaptively changing the quantization width for each macroblock due to the limitation of the coding bit rate. The quantization control is executed by the quantization control circuit 263. The data output as described above is VBV (Vide) for simulating the buffer state at the time of decoding.
Virtual buffer 2 called o Buffering Verifier)
Through 62, it is output as a video stream 290.

[0006] Next, a process when encoding is performed by software will be briefly described. FIG. 16 shows a block diagram of encoding processing based on MPEG2. First, the frame reading process 522 is performed. Here, the frames are rearranged in the coding order at the same time. Next, when it is not an I frame, a motion vector is calculated by the motion estimation processing 531 and a difference image is generated by the frame difference processing 582. In block division processing 510, the picture is divided into macroblocks.

For each divided macroblock, D
CT processing 541, quantization processing 551, and variable length coding processing 561 are performed. The data after variable length coding is subjected to VBV buffering and quantization control in a buffering and quantization control processing 562. When all the macroblocks have been encoded, a reconstruction process 532 is performed to generate a reconstructed image for prediction of the next frame.
When the coding of all frames is completed, the coding ends 5
90.

Generally, the higher the average bit rate of the output video stream, the better the image quality. However, in an application of a storage medium such as a DVD, the average bit rate is limited because the capacity for recording the stream is limited. Therefore, VBR (Variable Bit Rate) coding, which is a method of allocating a small bit rate to a scene that is easy to compress to reduce an unnecessary bit amount and a large bit rate to a scene that is difficult to compress, is effective. Is. By using VBR encoding, it is possible to improve the image quality as a whole within the range of the average bit rate allowed. However, in order to perform VBR encoding effectively, it is essential to collect and use information such as the difficulty level of the scene to be encoded in advance. for that reason,
For applications requiring real-time processing, a method is used in which video information is collected using an allowable delay time and used for encoding. Two-pass encoding is used in fields such as content production in which video quality is important and offline encoding is more desirable. Two
Path encoding is a method in which preliminary encoding is first performed to collect a feature amount such as image complexity, and in the second main encoding, encoding is performed based on the collected feature amount. .

An example of a 2-pass video encoder is shown in FIG.
Shown in. The input video signal 210 is input to the selector 113. In the first encoding, the selector 213 has the terminal 211.
Are connected, and the video signal is input to the video encoder 201 for preliminary encoding. The pre-encoding video encoder 201 performs the same encoding as the MPEG2 video encoder 201 described in FIG. However, since the first encoding is performed for the purpose of acquiring the encoding information 291 indicating the complexity of the image, the limitation of the buffer 262 in FIG. 14 is often not considered. The coded data 291 such as the coded data or the generated code amount output from the pre-encoding video encoder 201 is input to the video code amount assigner 300 for assigning the target code amount in the main encoding. The video code amount assigner 300 calculates a target code amount for each unit such as a frame or a slice from the obtained coding information and the coding conditions such as the target bit rate. It is also possible to calculate the quantization width in addition to the target code amount. A disk 350 for storing the result calculated as described above may be provided.

Formal encoding is executed by the main encoding video encoder 202 based on the above target code amount. In this case, the selector 213 is connected to the terminal 212, and the video signal exactly the same as in the pre-encoding is input to the main encoding video encoder 202. The main encoding video encoder 202 has the same configuration as the preliminary encoding video encoder 201, but the buffer 262 of FIG.
The capacity of must be enabled. The main encoding video encoder 202 performs formal encoding based on the target code amount calculated by the video code amount assigner 30, and outputs a video stream 290.

Further, FIG. 18 shows a two-pass video encoding process when it is realized by software. In the 2-pass video encoding process 400, first, a preliminary encoding process 401 is performed. Next, the video code amount allocation process 100 is performed based on the coding information obtained by the preliminary coding process 401. Finally, the formal encoding process 402 is performed based on the code amount allocation information obtained by the video code amount allocation process 43, the output video stream is obtained, and the process ends 490.

As described above, in the two-pass video coding, the coding information such as the generated code amount is obtained for the entire video sequence by the preliminary coding process, and the code amount allocation of the entire sequence is performed by the code amount allocation process. In the formal coding, since the coding is performed based on the optimized code amount allocation for the entire sequence, when the normal CBR coding is performed or when the VBR coding is performed with a certain delay allowed. It is possible to obtain a high image quality compared to.

[0013]

An example of the above-described two-pass video encoding is disclosed in Japanese Patent Laid-Open No. 11-1.
There are those described in Japanese Patent No. 36673 and Japanese Patent Laid-Open No. 11-262003. The methods described therein attempt to improve image quality by detecting phenomena such as scene changes and fades. However, in the actual encoding, not only the above-mentioned phenomenon but also the luminance value and the color difference value are simply numerically calculated, so that the deterioration of the image quality is apt to be noticeable due to the difference of the luminance and the color. For the purpose of obtaining high image quality, which is the greatest advantage of 2-pass video coding, there has been no method of allocating code amount based on such visual sensitivity of video.

There is a technique described in Japanese Unexamined Patent Publication No. 9-200760 that performs encoding based on visual sensitivity, but it is one-pass encoding, and the target code amount of the current frame is Since the prediction is performed from the immediately preceding frame, the accuracy of code amount allocation was not sufficient.

The present invention solves the above-mentioned conventional problems, and in a video encoding method having a function of performing a preliminary encoding for obtaining encoded information of a video, it is obtained by the preliminary encoding. By assigning the code amount based on the visual sensitivity obtained from the luminance value and color difference value of the pixel in addition to the encoded information such as the generated code amount, it is possible to increase the visual sensitivity based on the visual sensitivity at the time of formal encoding. The purpose is to obtain image quality.

[0016]

In order to achieve the above object, a video coding method of the present invention uses a luminance value and a color difference value of a pixel in a preliminary coding for obtaining video coding information. The visual class value is allocated for each second video unit based on the above, and when the code amount is allocated, the code amount is allocated based on the coding information such as the generated code amount obtained by the preliminary coding and the visual class value. .

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION The first invention of the present invention comprises a preliminary coding step for obtaining coding information of video,
Video code amount allocation step for performing code amount allocation for formal encoding based on the encoding information, and formal encoding based on the target code amount assigned in the code amount allocation step A video encoding method having a formal encoding step, wherein the preliminary encoding step determines a visual class value for each second video unit based on a luminance value and a color difference value for each third video unit. A visual class value calculating step of calculating, wherein the code amount allocating step includes the coding information of the second video unit obtained in the preliminary coding step and the visual class of each second video unit. A video encoding method comprising a second target code amount allocation step of allocating a target code amount to a second video unit based on a value, wherein the video encoding method comprises:
The visual class value is calculated for each second video unit (for example, macroblock) using the luminance value and the color difference value for each video unit (for example, pixel), and the calculated visual class value is calculated when the code amount is allocated. When assigning the target code amount of the second video unit from the target code amount of the video unit (for example, a frame), weighting is performed based on the visual class value, so that the color difference value in the first video unit (inside the frame) It is possible to absorb the difference in the visual sensitivity due to the difference in the luminance value and to provide a better encoded image quality in the formal encoding.

In a second aspect of the present invention based on the first aspect, the code amount allocating step is performed in the calculated second number.
A visual class representative value calculating step of aggregating visual class values for each video unit and assigning a visual class representative value to the first video unit; and a code of the first video unit obtained in the preliminary encoding step. A video encoding method, comprising: a first target code amount allocating step for allocating a target code amount to the first video unit based on the encoding information and the visual class value for each of the first video units. Yes, when assigning the target code amount to the first video unit from the total code amount, by weighting based on the visual class value, the visual difference due to the difference in the color difference value and the luminance value between the first video units (between frames) is performed. It is possible to reflect the difference in the target sensitivity on the target code amount and to provide a better coded image quality at the time of formal coding.

A third aspect of the present invention is a video encoding method which performs encoding only once, and has a certain finite length N (N> 1).
Pre-reading processing step of pre-reading and accumulating the data of the first video units, and for the N first video unit groups, based on the luminance value and the color difference value for each third video unit. Two
A visual class value assigning step of calculating a visual class value for each video unit, and a visual class representative value calculating processing step of totaling the calculated visual class values and assigning a visual class representative value to N first video units And a complexity estimation processing step of estimating encoding complexity for N first video units, and a target for the first video unit based on at least the complexity and the visual class representative value. A first target code amount allocation processing step of allocating a code amount, and a target code for a second video unit based on at least a target code amount of the first video unit and a visual class value for each of the second video units. A video encoding method, characterized by comprising a quantization control processing step for allocating an amount and performing a quantization control, wherein N finite length delays are applied to a first video unit (for example, a frame). Then, by performing pre-reading, the visual class value is calculated for each second video unit using the luminance value and the color difference value for each third video unit, and the visual class values for the N first video units are calculated. When assigning the representative value and assigning the target code amount of the first video unit and the target code amount of the second video unit, weighting is performed based on the visual class value, so that the first video unit (interframe) Also, it is possible to absorb a difference in visual sensitivity due to a difference between a color difference value and a luminance value inside the first video unit (within a frame), and to provide a better encoded image quality.

According to a fourth aspect of the present invention, a pre-encoding video encoder for acquiring video encoding information and a code amount allocation for formal encoding based on the encoding information. And a video coding amount allocator for performing formal coding based on the target code amount allocated by the code amount allocator, The encoder has a visual class value assigning circuit that calculates a visual class value for each second video unit based on a luminance value and a color difference value for each third video unit, and the code amount assigner performs the calculation. And a visual class value aggregator for totaling visual class values for each second video unit and assigning a visual class representative value to the first video unit, and a first visual unit for the first video unit obtained by the precoding encoder. Coded information and the above A first target code amount assigner that assigns a target code amount to a first video unit based on a class representative value, coding information of a second video unit output from the preliminary coding encoder, and the first video code unit. And a second target code amount assigner that assigns a target code amount to a second video unit based on a visual class value for each of the two video units. Quantity to first
When assigning the target code amount to each video unit, the weighting is performed based on the visual class value so that the difference in the visual sensitivity due to the difference in the color difference value and the luminance value between the first video units (between frames) causes the target code amount. Therefore, it is possible to provide a video encoding device that gives better encoded image quality during formal encoding.

Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1) In this embodiment, in a preliminary encoding process, a second video unit (eg, pixel) is generated based on a luminance value and a color difference value for each third video unit (eg, pixel). A visual class value for each macro block, etc. is calculated, and in the code amount allocation processing, the second visual unit is set on the basis of the coding information of the second visual unit and the visual class value for each second visual unit. A video encoding method characterized by allocating a target code amount will be described.

The video encoding method according to the first embodiment is
In the 2-pass video encoding process shown in FIG. 18, the flowchart shown in FIG. 2 is used as the preliminary encoding process 401, and the flowchart shown in FIG. 1 is used as the video code amount allocation process 100.

In FIG. 2, reference numeral 522 is a frame reading processing step, 110 is a visual class value allocation processing step, 583 is a selection step for judging whether or not the frame is an I frame, and 531 is executed when it is not an I frame. Motion estimation processing step, 582 is a frame difference processing step for obtaining a difference from the reconstructed image, 510 is a block division processing step, 541 is a DCT processing step, 551 is a quantization processing step, 561 is a variable length coding processing step, 571. Is a code amount measurement processing step for measuring the generated code amount of a macroblock, 584 is a selection step for determining the end of frame encoding, 532.
Is a reconstruction processing step for forming a reconstructed image for the next prediction frame, 585 is a selection step for judging the end of encoding of all frames, and 590 is an end.

The operation of the preliminary encoding process configured as above will be described below.

First, for the input frame read in the frame reading process 522, the visual class value assignment process 1
10 is executed. The visual class value assigning process is a process of assigning a high class value to a macroblock having high visual sensitivity and, conversely, assigning a low class value to a macroblock having low sensitivity.

The operation of the visual class assignment process 110 is shown in FIG.
Will be explained. FIG. 3 is a flowchart showing the flow of visual class assignment processing.

In FIG. 3, 1110 is an initialization step of a macroblock number B (B is an integer of 0 or more), 112
0 is the pixel number P (P is an integer of 0 or more) and the total value S
(S is an integer of 0 or more) is an initialization step. 113
1 is a processing step of converting the color difference value and luminance value of a pixel into a color space, 1132 is a processing step of calculating color sensitivity in the color space, 114 is a processing step of calculating luminance sensitivity from a pixel luminance value, 115 is a processing step of adding color sensitivity and luminance sensitivity to the total value S, 1121 is an adding step of the pixel number P, 1111 is an adding step of the macroblock number B, 1122 is processing for all pixels in the macroblock. 116 is a processing step for calculating the visual class value of the current macroblock, 1112 is a processing step for determining whether or not the processing has been completed for all macroblocks in the frame, 117 is a processing step for determining in the frame This is a process of outputting a visual class value.

First, the initialization step 1110 for the macro block number B and the initialization step 1120 for the pixel number P and the total value S are executed, and the processing is performed for the macro block B = 0 and the pixel P = 0. First, in the processing step 1131 for converting the color difference value and the luminance value of the pixel into the color space, the color difference values Cb (B, P) and C of the pixel are
The r (B, P) and the luminance value Y (B, P) are mapped in the color space.

Here, as the color space, a uniform color space (Uniform Color Space), which is created with the goal that the distance in the color space represents a visual color difference, is used. As a uniform color space, CIE1976L recommended by CIE
There are two types, * u * v and CIE1976L * a * b *, and the Munsell color system is also considered. Here, as a typical example, conversion to CIE1976L * u * v will be briefly described.

First, Y, Cb, and Cr are first converted into points in the RGB space, which is a basic color space. Next, the RGB space is converted into the CIE1931XYZ space. here,
u ′ and v ′ are defined by (Equation 1).

[0032]

[Equation 1]

Here, the brightness value Y of the reference white surface is set to Y ↓ 0,
If u'is u ↓ 0 and v'is v ↓ 0, L *, u *, v
* Is determined by (Equation 2).

[0034]

[Equation 2]

Since the L * u * v * space is a uniform color space, the visual color difference is given by the distance equation of (Equation 3).

[0036]

[Equation 3]

As described above, the color difference value and the luminance value of the pixel can be mapped in the L * u * v * space.

Next, a processing step 1132 for calculating the color sensitivity in the color space is performed. In the present embodiment, color sensitivity (chrominous) is applied to the L * u * v * space described above.
efficiency) is defined. Small changes dY, dCr, d
If the variations occurring in L *, u *, and v * with respect to Cb are defined as dL *, du *, and dv *, respectively, the color sensitivity E ↓ C
Can be defined by (Equation 4).

[0039]

[Equation 4]

In the above, processing step 1131 for converting into the color space and processing step 11 for calculating the color sensitivity.
32 is to be carried out once in order, but the repetitive sensitivity E for some slight changes dY, dCr, dCb.
Needless to say, it is possible to calculate ↓ C and use the average value or the representative value thereof. In addition, CI is used as a color space.
Although the E1976L * u * v * space has been described as an example, the color space is not limited to this space or the space described above. For example, a space based on the movement of an image or a point of interest is defined, or a macro is defined. Various methods can be applied as long as they define visual sensitivity, such as selecting a color space based on the size of activity in a block.

In the processing step 114 for calculating the luminance sensitivity from the luminance value, the luminance efficiency (luminous efficiency) is calculated from the luminance value Y (B, P) of the pixel. As a model showing the sensitivity of brightness, there is an incremental threshold curve published by Blackwell in 1963. Considering the range of the brightness of the object color used in the image on this curve, the region represented by the Weber ratio of (Equation 5) can be defined as a model of the sensitivity of brightness.

[0042]

[Equation 5]

Here, L is a brightness value, and ΔL is a brightness discrimination threshold value. ω is an experimental value, usually 0.01 to 0.0
It is about 2. Using this model, the brightness sensitivity E ↓ L
Can be defined by (Equation 6), where dL is a minute change in luminance.

[0044]

[Equation 6]

After the luminosity for the pixel is calculated as described above, a processing step 115 for adding the color sensitivity and the luminance sensitivity to the total value S is performed. In this step, the color sensitivity E ↓ C and the luminance sensitivity E ↓ L are reflected in the total sensitivity value S. Although various methods are conceivable for this processing, the simplest method is to perform threshold processing Th ↓ C (X) or Th ↓ L (X) on E ↓ C and E ↓ L according to the upper limit value and the lower limit value, and then perform normalization. This is to add the respective values of the conversion processing Nrm (X) to S. When generalized, using the color difference weight w (0 <w <1) and the luminance weight 1-w, the increment ΔS of S can be defined by (Equation 7).

[0046]

[Equation 7]

Since the luminance and the color difference are orthogonal to each other, using the color difference weight p (0 <p <∞) and the luminance weight 1 / p,
It is also possible to give it as a product as in (Equation 8).

[0048]

[Equation 8]

By repeating the above steps for all the pixels in the macroblock B, the sensitivity S of the macroblock can be calculated. It is also possible to perform the process of calculating the sensitivity according to the characteristics of the image. As an example, it represents the method of obtaining the average value of the luminance value and the color difference value in the macroblock, and not performing the sensitivity calculation for the pixels having values apart from the average value, and the characteristics of the macroblock. A method in which some representative pixels are calculated and the sensitivity is calculated only for the representative pixels can be considered.

The calculated sensitivity S is converted into the visual class value Cv in the processing step 116 for calculating the visual class value.
Map to (B). There are various possible mapping functions, but the sensitivity S is simply quantized Qc.
It can be given by (X) and (Equation 9).

[0051]

[Equation 9]

The number of representative levels of quantization is appropriately about 8 or 16 in consideration of acquiring information in macroblock units.

When the visual class values are calculated for all macroblocks in the frame by repeating the above processing,
Processing step 11 for outputting the visual class value in the frame
7 is executed and the processing ends 119.

When the visual class value assignment processing 110 of FIG. 2 is completed as described above, preliminary coding is performed in the same manner as the normal coding described with reference to FIG. In the preliminary encoding, the buffering and quantization control processing 562 shown in FIG. 16 is not normally performed. Instead, the macroblock encoded information output process 571 is performed. FIG. 10 shows an example of macroblock coding information. The macroblock coding information is information including at least a combination of a generated code amount and a quantization width, and may additionally include additional information such as a macroblock type.

When the frame coding is completed, a frame coded information output process 572 is performed. FIG. 11 shows an example of the frame coding information. The frame coding information includes at least the generated code amount and the picture coding type.
Further, characteristic information such as video brightness, movement, and activity may be included, and these can be used as parameters for code amount allocation. When the coding of all the frames is completed, the preliminary coding process 401 is completed.

Next, the operation of the video code amount allocation processing 100 will be described with reference to FIG. FIG. 1 is a flowchart showing the flow of a video code amount allocation process.

In FIG. 1, reference numeral 130 is a processing step for allocating a target code amount in frame units based on frame coding information, 180 is an initialization step of a frame number K (K is a natural number), 121 is a pixel in preliminary coding. A processing step of reading a visual class value for each macroblock calculated based on a luminance value and a color difference value for each macroblock; 122 is a macroblock based on the coding information and the visual class value of the second video unit in the Kth frame; , 181 is a step of incrementing K, 182 is a step of determining whether or not the allocation processing of all frames is completed, 142 is a processing step of outputting target code amount information in units of frames and macroblocks. Is.

The operation of the video code amount allocation processing configured as described above will be described below.

First, a processing step 130 of allocating a target code amount on a frame-by-frame basis based on the frame coding information is executed. In this step, the target code amount for each frame given at the formal encoding is calculated based on the frame encoding information including the target bit rate of the entire sequence and the generated code amount obtained by the preliminary encoding. To do.

Next, a series of steps for calculating the target code amount in macroblock units from the target code amount in frame units is performed. First, a processing step 121 of reading the visual class value for each macroblock calculated based on the luminance value and the color difference value for each pixel in the preliminary encoding is performed, and the visual class value Cv (B) for the Kth frame is performed. Read all. Next, in the Kth frame, the processing step 122 of allocating the code amount to the macroblock based on the coding information of the second video unit and the visual class value is executed. In this step, the complexity value calculated on the basis of the information of the generated code amount of the macroblock, the quantization width, and other additional information is modulated according to the visual class value. The target code amount of the frame is allocated to each macroblock based on the modulated complexity value.

The process of allocating the macroblock target code amount in the Kth frame will be described with reference to the example of FIG. The generated code amount D (B) of the B-th macroblock obtained by preliminary encoding (correctly D (K, B), but K is omitted in the following description) and the quantization width Q. From (B), the encoding complexity X of each macroblock
Find (B). For example, it can be defined by (Equation 10).

[0062]

[Equation 10]

The complexity X (B) is weighted by Cv (B) to calculate the visual complexity Xv (B). For example, with a and b as positive constants, they can be defined by a general expression such as (Equation 11).

[0064]

[Equation 11]

Using Xv_K, which is the result of summing up the complexity Xv (B) for all macroblocks, and the target code amount T (K) of the Kth frame, the target for each macroblock is calculated by (Equation 12). A code amount T (B) is assigned.

[0066]

[Equation 12]

In this way, the allocation result of FIG. 12 is obtained. It can be confirmed that the macroblock target code amount is weighted by the visual class value Cv (B). In the above process, the target code amount is obtained only based on the complexity X (B) of the macroblock and the visual class value Cv (B). However, the minimum guaranteed bit number such as the fixed code amount in the macroblock and the macroblock type are obtained. , Additional information such as block activity can be used.

When the process of allocating the target code amount to the macroblocks for all the frames is completed, the process step 142 of outputting the target code amount information in units of frames and macroblocks is executed and the process ends 190.

Finally, the formal encoding process 402 will be described with reference to FIG. The formal encoding process aims to generate a final stream in accordance with the target code amount assigned in the previous video code amount assignment process 100.

First, after the frame reading process 522 is performed, the target code amount reading process 573 is performed. The target code amount to be read is the frame target code amount and the macroblock target code amount. The macroblock target code amount is used to calculate the quantization width when performing the quantization control in the buffering and quantization control processing 562.

A method of calculating the quantization width will be described. One method is to use a table that associates the macroblock coding amount D (Q, X) with the set of the quantization width Q and the macroblock coding complexity X. The complexity X of a macroblock can be expressed by a value obtained by quantizing a statistical amount of a generally known activity into several levels. The table is created by changing the quantization width Q and the complexity X, encoding the test image, measuring the code amount, and the like, and held in advance by the encoder. A final quantization width is calculated by adding a weight by buffer control to the calculated quantization width.

The other steps are the same as in the case of the normal encoding described with reference to FIG.

In the above description, the target code amount of the macroblock is calculated in the video code amount allocation process 100. However, in the formal encoding process 402, the target code amount of the frame and the visual class value are read and the macroblock unit is read. It goes without saying that a configuration in which the division of roles, such as allocating the target code amount, is exchanged is also possible.

The advantage of this embodiment is that in a video coding method in which preliminary coding is performed to obtain image information, weighting is performed based on visual sensitivity in a frame. As a result, it becomes possible to obtain a visually good encoded image quality.

(Embodiment 2) In the present embodiment, in the code amount allocating step in Embodiment 1, the calculated class value for each second video unit is aggregated and the visual value is calculated for each first video unit. A class value is assigned, and the first video unit is targeted based on the coding information of the first video unit obtained in the preliminary coding step and the visual class value for each of the first video units. A video encoding method for allocating the code amount will be described.

In the present embodiment, in the video code amount allocation processing 100 of the first embodiment, weighting based on visual sensitivity can be applied to bit allocation between frames. FIG. 5 is a flowchart showing the video code amount allocation processing in the present embodiment.

In FIG. 5, reference numeral 121 denotes a processing step for reading the visual class value for each macroblock calculated based on the luminance value and the color difference value for each pixel in the preliminary encoding,
Reference numeral 131 is a processing step of totaling visual class values to calculate a visual class representative value for each frame, 132 is a processing step of allocating a target code amount to a frame based on frame coding information and a visual class representative value, and 180 is a frame number K Initialization step, 122 is a processing step of allocating the code amount to the macroblock based on the frame target code amount and the visual class value in the Kth frame, and 181 is K
Is an increasing step, 182 is a step of determining whether or not the allocation processing of all frames is completed, and 140 is a processing step of outputting target code amount information in units of frames and macroblocks.

First, a processing step 121 for reading the visual class value for each macroblock, which is calculated based on the luminance value and the color difference value for each pixel in the preliminary encoding, is executed. Next, the processing step 131 of totaling the visual class values and calculating the visual class representative value for each frame is executed. Here, the visual class representative value Cvr (K) can be obtained by (Equation 13) as an average value of the visual class Cv (K, B), for example.

[0079]

[Equation 13]

Here, MB_num is the number of macroblocks in the frame. Cvr (K) is Cv (K, B)
Unlike the above, it is not necessary to specifically quantize. The reason is that there is almost no restriction on the processing due to the large amount of information.

Visual class representative value Cvr (K) obtained
Is used to perform the processing step 132 of allocating the target code amount to the frame based on the frame coding information and the visual class representative value. By this processing, it becomes possible to perform distribution based on visual sensitivity in code amount allocation between frames.

The process of allocating the frame target code amount will be described with reference to the example of FIG. The coding complexity X (K) of each frame is obtained from the generated code amount D (K) of the Kth frame obtained by the preliminary coding and the average quantization width Qavg (K). For example, it can be defined by (Equation 14).

[0083]

[Equation 14]

The complexity X (K) is weighted by Cvr (K) to calculate the visual complexity Xv (K). For example, a and b can be defined by a general expression such as (Equation 15), where a and b are positive constants.

[0085]

[Equation 15]

The target code amount T (K) is assigned to each frame by (Equation 16) using Xv_all as a result of summing up the complexity Xv (K) for all frames and the target code amount Tall of the sequence. .

[0087]

[Equation 16]

The allocation result of FIG. 13 is obtained as described above. It can be confirmed that the frame target code amount is weighted by the visual class representative value Cvr (K).

When the target code amount of the frame is obtained, the subsequent processing is the same as that described with reference to FIG.

The advantage of this embodiment is that in a video coding method in which preliminary coding is performed to obtain image information, weighting is performed based on visual sensitivity between frames in addition to within frames. Therefore, it is possible to obtain visually good encoded image quality.

(Embodiment 3) In the present embodiment, when video is encoded only once, prefetching is performed by allowing a finite length of N delays for the first video unit (for example, frame). Therefore, the visual class value is calculated for each second image unit using the luminance value and the color difference value for each third image unit, and N
Video coding for assigning a visual class representative value to each of the first video units, and performing weighting based on the visual class value when allocating the target code amount of the first video unit and the target code amount of the second video unit The method will be described.

The present embodiment realizes the video coding method of the second embodiment by a single coding when a fixed delay amount N is allowed. FIG. 6 is a flowchart showing the video encoding process according to this embodiment.

First, the frame prefetch processing 523 for N (N> 1) sheets is performed and accumulated. Here, the frames are rearranged in the coding order at the same time. Next, the visual class value assignment processing 110 is performed on the N frames. The visual class value assignment processing 110 is as described with reference to FIG. 3 in the first embodiment. Then N
Visual class representative value calculation processing 13 for one frame
1 is performed. The visual class representative value calculation processing 131 is as described with reference to FIG. 5 in the second embodiment.

The complexity estimation processing 403 is performed on the accumulated N frames. The complexity estimation processing 403 estimates the complexity of video encoding for the subsequent N frames. Although various estimation methods can be considered, preliminary encoding such as 2-pass video encoding processing is performed by N.
It is possible to perform it for one frame to obtain coding information such as the quantization width and the generated code amount.

On the basis of the calculated frame complexity and the visual class representative value of the frame, a process 132 of allocating the target code amount from the N target code amounts to the current frame is performed. The target code amount for N frames can be calculated by using the bit rate, the picture coding type, and the bit storage surplus amount. It is also possible to predict the buffer state for the previous N frames from the current VBV buffer state and use it to calculate the target code amount of the current frame.

When the frame target code amount is calculated, the subsequent processing is almost the same as the normal video coding processing described with reference to FIG. The feature of this embodiment is that, after the variable length coding process 561, the quantization control process 123 based on the buffering and the visual class value is performed. In this quantization control processing 123, processing for calculating the quantization width for the next macroblock is performed from the target code amount of the frame, the macroblock complexity calculated in the complexity estimation processing 403, and the visual class value. To do.

In the present embodiment, the number of frames is N (N>
1), but in general applications GOP (Group Of Pic)
tures) It is practical to use an integer multiple of the number. The reason is that, regardless of which frame is the current frame, if frames that are an integral multiple of the number of GOPs thereafter are collected, the composition ratio of the included picture types becomes constant, and there is an advantage that code amount allocation is easy to perform. Is.

The advantage of this embodiment is that, when the delay of N frames is allowed, it is only necessary to execute one pass of video coding, and the visual sensitivity between frames as well as between frames can be improved. It is possible to perform weighting based on, which makes it possible to obtain visually better encoded image quality.

(Embodiment 4) In this embodiment, a video coding apparatus which realizes the coding of Embodiment 2 will be described.

The 2-pass video encoder of this embodiment is as already described with reference to FIG. The configuration of the pre-encoding encoder 201 is shown in FIG.

The pre-encoding encoder in FIG. 7 has the same configuration as that of the video encoder described in FIG. 14, and the description thereof will be omitted. 1101 is a visual class value assignment circuit, 270 is an encoded information output device, 2
Reference numeral 91 is output encoded information, and 292 is an output visual class value.

First, the input video 210 is input to the frame rearrangement circuit 222. Next, the visual class value assigning circuit 1101 performs a visual class value assigning process for each input frame in units of macroblocks. This process is similar to that described with reference to FIG. The obtained visual class value is output 292. The subsequent processing is basically the same as that of the video encoder described in FIG. From the output of the variable length coding circuit 261, the coding information output device 270 measures the generated code amount and outputs it as the coding information 291.

The video code amount assigner 300 in FIG.
The configuration of is shown in FIG. In FIG. 8, 310 is frame coding information, 311 is macroblock coding information, 3
12 is visual class value information, 381 is a frame target code amount, 382 is a macroblock target code amount, 383 is a visual class representative value, 321 is a frame code amount assigner for allocating the frame code amount 381, and 322 is a frame target code amount 381. A macroblock code amount assigner 323 that assigns a macroblock code amount 382 based on the following, a visual class aggregator that aggregates the visual class values 312 in frame units to calculate a visual class representative value 383, and a visual class aggregator 340 that calculates a target code amount. A memory for storing information, 330 is an allocation information output device that reads and outputs the allocated target code amount from the memory 340, 350 is a disk, and 390 is output target code amount information.

The operation of the video code amount assigner shown in FIG. 8 will be described below.

The visual class value information 312 is stored in the visual class value aggregator 323 in units of frames.
Calculation processing 3 is performed. This processing is the same as that described in step 131 of FIG. 5 in the second embodiment. The frame coding information 310 is input to the frame code amount assigner 321. The frame code amount assigner 321 determines the target bit rate of the entire sequence and the frame coding information 3
The frame target code amount 381 is assigned based on 10 and the visual class representative value 383. This processing is the same as that described in step 132 of FIG. 5 in the second embodiment. The calculated frame target code amount 381 is stored in the memory 34.
0, and is recorded on the disc 350 as needed.

The macroblock code amount assigner 322 is
Based on the macroblock coding information 311, the visual class value information 312, and the frame target code amount 381, a process of assigning the macroblock target code amount 382 is performed.
This processing is the same as that described in step 122 of FIG. 1 in the first embodiment. The allocated macroblock target code amount 382 is stored in the memory 340. When all the processing is completed, the allocation information output device 330 causes the memory 3
The frame target code amount 381 and the macroblock target code amount 382 are read from 40 and output as target code amount information 390.

The configuration of the main encoding video encoder 202 in FIG. 16 is shown in FIG. The video encoder for main encoding in FIG. 9 basically has the same configuration as the video encoder described in FIG. Reference numeral 273 is a target code amount setting circuit, and 274 is a memory.

First, the input video 210 is input to the frame rearrangement circuit 222. Next, the target code amount setting circuit 2
At 73, the frame target code amount and the macroblock target code amount for the input frame are set, and the memory 274
Memorized in. This processing is the same as that described in step 573 of FIG. 4 in the first embodiment. Subsequent processing,
Basically, it is similar to the video encoder described in FIG. The quantization control circuit 263 performs quantization control using the output of the buffer 262 and the target code amount stored in the memory 274. This processing is the same as that described in step 562 of FIG. 4 in the first embodiment. As described above, the final video stream 290 is obtained.

As described above, according to the present embodiment, it is possible to realize the coding processing described in the second embodiment with a specific hardware configuration.

[0110]

As is apparent from the above description, according to the present invention, video coding information is obtained in a video coding method for producing contents offline, which is used in a DVD authoring system or the like. In the preliminary encoding for, the visual class value is assigned to each second video unit based on the luminance value and the color difference value of the pixel, and when the code amount is assigned, the generated code amount obtained by the preliminary encoding, etc. By allocating the code amount based on the coding information and the visual class value of, the high image quality based on the visual sensitivity can be obtained at the time of formal coding.

[Brief description of drawings]

FIG. 1 is a flowchart showing a video code amount allocation process according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing the preliminary encoding process.

FIG. 3 is a flowchart showing the visual class value assignment processing.

FIG. 4 is a flowchart showing the formal encoding process.

FIG. 5 is a flowchart showing a video code amount allocation process according to the second embodiment of the present invention.

FIG. 6 is a flowchart showing video encoding processing according to the third embodiment of the present invention.

FIG. 7 is a block diagram showing the configuration of a video encoder for preliminary coding according to Embodiment 4 of the present invention.

FIG. 8 is a block diagram showing a configuration of the same video code amount assigner.

FIG. 9 is a block diagram showing the configuration of the same-coding video encoder.

FIG. 10 is a diagram showing an example of macroblock coding information according to Embodiment 1 of the present invention.

FIG. 11 is a diagram showing an example of the same frame encoded information.

FIG. 12 is a diagram showing an example of calculating the target code amount of the frame.

FIG. 13 is a diagram showing an example of calculating the macroblock target code amount.

FIG. 14 is a block diagram showing the configuration of a video encoder based on MPEG2.

FIG. 15 is a block diagram showing the configuration of a 2-pass video encoder.

FIG. 16 is a flowchart showing a video encoding process based on MPEG2.

FIG. 17 is a flowchart showing a 2-pass video encoding process.

[Explanation of symbols]

100 video code amount allocation processing 110 visual class value allocation processing 1101 visual class value allocation circuit 1131 conversion into color space 1132 color sensitivity calculation processing 114 luminance sensitivity calculation processing 115 color and luminance sensitivity addition processing 116 Visual class value calculation process 117 Visual class value output process 121 Visual class value reading process 122 Macroblock target code amount allocation process 123 Quantization control process based on buffering and visual class value 130 Frame target code amount allocation process 131 Visual class representative Value calculation process 132 Frame target code amount allocation process 140 Target code amount information output process 201 Precoding video encoder 202 Main coding video encoder 210 Input video 222 Frame rearrangement circuit 231 Motion estimation circuit 232 Motion compensation circuit 24 1 DCT circuit 242 Inverse DCT circuit 251 Quantization circuit 252 Inverse quantization circuit 261 Variable length coding circuit 262 Buffer 263 Quantization control circuit 270 Encoding information output device 273 Target code amount setting circuit 274 Memory 290 Video stream 291 Encoding information 292 Visual class value 300 Video code amount allocator 310 Frame coding information 311 Macroblock coding information 312 Visual class value information 321 Frame code amount allocator 322 Macroblock code amount allocator 323 Visual class value aggregator 340 Memory 350 Disk 381 Frame target code amount 382 Macroblock target code amount 383 Visual class representative value 390 Target code amount information 400 Video coding process 401 Preliminary coding process 402 Formal coding process 403 Complexity estimation process 510 Block division Processing 523 Frame prefetching processing 531 Motion estimation processing 532 Reconstruction processing 541 DCT processing 551 Quantization processing 561 Variable length coding processing 562 Buffering and quantization control processing 571 Macroblock coding information output processing 572 Frame coding information output processing 573 Target code amount reading process 582 Frame difference process

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5C053 FA24 GB28 GB38 KA04 KA30                 5C057 AA10 EM04 EM09 EM13 EM16                       GG01                 5C059 KK02 MA05 MA23 MC01 MC30                       ME01 SS13 SS20 TA60 TB04                       TB08 TC00 TC19 TD00 UA00                       UA02 UA38                 5J064 AA01 BA09 BA16 BB01 BB03                       BC01 BC08 BC16 BC21 BD03

Claims (19)

[Claims] 1. A video composed of a first video unit which is a unit of at least a frame or more, a second video unit which is a unit of at least a block or more, and a third video unit which is a unit of at least pixels or more. A video coding method for coding a signal, wherein a visual class value representing visual sensitivity is assigned to each second video unit based on a color difference value and a luminance value for each third video unit, Class value and first
The video coding method, wherein the target code amount is allocated for the second video unit based on the target code amount for the video unit. 2. A visual class representative value for each first video unit is calculated based on the visual class value for N (N is an integer of 1 or more) first video unit group, 2. The video encoding method according to claim 1, wherein the target code amount allocation of the first video unit is performed based on the visual class representative value and the target code amount of the first video unit group. 3. The first video unit is a frame,
The video encoding method according to claim 2, wherein the N is M times the GOP length (M is a natural number). 4. A code obtained by performing preliminary coding for obtaining information of a video sequence on a video sequence composed of the first video unit group and obtaining the code by the preliminary coding. A video encoding method having a function of allocating the code amount of the entire video sequence based on the encoding information, wherein in the preliminary encoding, the second encoding is performed based on the color difference value and the luminance value for each third image unit. A visual class value representing visual sensitivity is assigned to each video unit, and in the code amount allocation, a target code amount allocation of a second video unit is performed based on the visual class value and the target code amount of the first video unit. The video encoding method according to claim 1, wherein 5. The code amount allocation calculates a visual class representative value for each first video unit based on the visual class value, and based on the visual class representative value and a target code amount of the video sequence. 5. The video coding method according to claim 4, wherein the target code amount allocation for each of the first video units is performed. 6. The assignment of the visual class value converts at least the color difference value and the luminance value of a pixel into a color space representing color sensitivity, and calculates a color sensitivity which is a visual sensitivity of a point in the color space. The video encoding method according to claim 1, wherein the visual class value is obtained using at least the value obtained by summing up the color sensitivity for each second video unit. 7. The color space is CIE1976L * a *.
b * space and CIE1976L * u * v * space and a uniform color space including Munsell color system, and the calculation of the color sensitivity is performed on the color space caused by the minute change amount of the YCbCr space to which the video signal belongs. 7. The video coding method according to claim 6, wherein the video coding method is based on distance. 8. The allocation of the visual class value is performed by calculating a luminance sensitivity which is a visual sensitivity using a luminance sensitivity model from at least a luminance value of a pixel, and at least the luminance sensitivity is aggregated for each second image unit. 8. The video encoding method according to claim 1, wherein the visual class value is obtained using the value. 9. The luminance sensitivity model comprises a Weber ratio Δ
9. The video encoding method according to claim 8, which is based on L / L (L: luminance value, ΔL: threshold value for luminance discrimination). 10. A video composed of a first video unit which is a unit of at least a frame or more, a second video unit which is a unit of at least a block or more, and a third video unit which is a unit of at least pixels or more. A video coding device for coding a signal, wherein a visual class value for allocating a visual class value representing visual sensitivity for each second video unit based on a color difference value and a luminance value for each third video unit It has an allocating means and a second target code amount allocating means for allocating a target code amount of the second video unit based on the visual class value and the target code amount of the first video unit. Video coding device. 11. A first group of N (N is an integer equal to or greater than 1) first image unit group based on the visual class value.
The visual class representative value calculating means for calculating the visual class representative value for each image unit, and the target code amount of the first image unit based on the visual class representative value and the target code amount of the first image unit group. 11. The video coding apparatus according to claim 10, further comprising a first target code amount allocation unit that performs allocation. 12. The video encoding apparatus according to claim 11, wherein the first video unit is a frame, and the N is M times a GOP length (M is a natural number). 13. A preliminary coding means for acquiring information of a video sequence for a video sequence composed of the first video unit group, and the preliminary coding means. A video encoding device having a code amount assigning means for assigning a code amount of an entire video sequence based on encoding information, wherein the preliminary encoding means is a third color difference value and a luminance value for each video unit. A visual class value allocating means for allocating a visual class value representing visual sensitivity for each second video unit based on the above, and the code amount allocating means has the visual class value and the target code of the first video unit. 11. The video encoding apparatus according to claim 10, further comprising a second target code amount assigning unit that assigns a target code amount for each second video unit based on the amount. 14. The code amount assigning means calculates a visual class representative value for each first image unit based on the visual class value, a visual class representative value calculating means, the visual class representative value and the image. A first code amount allocation for the first video unit based on the target code amount of the sequence
14. The video encoding device according to claim 13, further comprising: a target code amount assigning unit. 15. The visual class value assigning means transforms at least a color difference value and a luminance value of a pixel into a color space representing a color sensitivity, and a visual sensitivity of a point of the color space, that is, Color sensitivity calculation means for calculating color sensitivity,
15. The video encoding device according to claim 10, further comprising a visual class value calculating unit that obtains a visual class value using at least the value obtained by summing up the color sensitivity for each second video unit. 16. The color space is CIE1976L * a.
An * b * space, a CIE1976L * u * v * space, and a uniform color space including the Munsell color system, wherein the color sensitivity calculation means is on the color space caused by a minute change amount of the YCbCr space to which the video signal belongs. The video encoding device according to claim 15, wherein the video encoding device calculates the distance based on the distance. 17. The visual class value assigning means calculates a visual sensitivity, that is, a luminance sensitivity using a luminance sensitivity model from at least a luminance value of a pixel, and at least the luminance sensitivity in a second image. The video encoding device according to any one of claims 10 to 16, further comprising: a visual class value calculating unit that obtains a visual class value using a value aggregated for each unit. 18. The video encoding device according to claim 17, wherein the luminance sensitivity model is based on a Weber ratio ΔL / L (L: luminance value, ΔL: threshold value for luminance discrimination). 19. A recording medium on which a stream generated by using the video encoding method or apparatus according to claim 1 is recorded.