WO2010130190A1

WO2010130190A1 - Rate control method and device for multi layer coding

Info

Publication number: WO2010130190A1
Application number: PCT/CN2010/072588
Authority: WO
Inventors: 张鹏; 高云超; 梁丽杰
Original assignee: 华为技术有限公司
Priority date: 2009-05-11
Filing date: 2010-05-11
Publication date: 2010-11-18
Also published as: CN101562741B; CN101562741A

Abstract

Embodiments in the present invention provide a rate control method and device for multi-layer coding, wherein the method includes: allocating a target bit number to an access unit of the current Group Of Pictures (GOP) according to an obtained available bit number (101); allocating a corresponding base layer bit number and a corresponding enhancement layer bit number to the base layer and the enhancement layer of the access unit respectively according to the target bit number (102); performing the base layer encoding according to the base layer bit number, and performing the enhancement layer encoding according to the enhancement layer bit number (103). By using the schemes provided by the embodiments in the present invention, under the condition of satisfying a total bit rate constraint, the proportion of the target bit number occupied by the base layer and the enhancement layer can be adjusted rationally, and the quality of the bit stream outputted by an encoder is improved, thus the decoded image quality is improved.

Description

Multi-layer coded rate control method and device

This application claims priority to Chinese Patent Application No. 200910138431.3, entitled "Multi-layer Coded Rate Control Method and Apparatus", filed on May 11, 2009, the entire contents of which are incorporated by reference. In this application.

Technical field

Embodiments of the present invention relate to the field of video coding, and in particular, to a code rate control method and apparatus for multi-layer coding.

Background technique

Currently, Scalable Video Coding (SVC) technology is one of the key technologies to solve video adaptation. SVC provides many good features based on the advantages of efficient encoding of H.264 and Advance Video Coding (AVC), such as: Time domain scalability based on level B image structure. , enhance the ability of airspace to scale, and so on.

Rate control technology is an important technology in video coding. A good rate control method enables the decoded image quality of the video bit stream output by the encoder to be matched or approximated to distortion optimization under bandwidth-limited conditions. The rate control technology includes two modes: Variable Bit Rate (VBR) and Constant Bit Rate (CBR). The VBR mode is an open-loop processing mode. The input is a video source and a fixed quantization parameter (QP). Since the image complexity in the actual video sequence is constantly changing, the bit rate is also corresponding. Changes, resulting in unstable image quality. The CBR mode is a closed-loop processing mode. The input is the video source and the target bit. According to the estimation of the complexity of the video source, the occupancy of the decoder buffer, and the channel bandwidth, the QP is dynamically adjusted to obtain a code that meets the requirements. rate. For the CBR mode, the rate control technique mostly uses the buffer smoothing mechanism and is implemented by adjusting the QP.

Unlike single-layer coding, SVC supports encoding of multiple spatial layers, including the base layer (the referenced image layer can be independently decoded without relying on other layers) and the enhancement layer (refer to the image layer of other layers when decoding), where the base layer Independent and rate control techniques are used respectively for the enhancement layer. The inventor found in the research process that the basic layer and the enhancement layer adopt independent rate control technology to reduce the quality of the encoder output bit stream, thereby reducing the decoded image quality. Summary of the invention Embodiments of the present invention provide a multi-layer code rate control method and apparatus, which improve the quality of an encoder output bit stream, thereby improving decoded image quality.

The code rate control method for the multi-layer coding provided by the embodiment of the present invention includes:

Allocating a target number of bits to an access unit of the current image group according to the number of available bits; assigning a corresponding base layer number and an enhancement layer to the base layer and the enhancement layer of the access unit respectively according to the target number of bits Number of bits

The base layer is encoded according to the number of base layer bits, and the enhancement layer is encoded according to the enhancement layer bit number.

The multi-layer code rate control device provided by the embodiment of the invention includes:

a target bit allocation module, configured to allocate a target number of bits to an access unit of the current image group according to the obtained number of available bits;

a layer bit allocation module, configured to allocate a corresponding base layer number and an enhancement layer bit number to the base layer and the enhancement layer of the access unit according to the target number of bits;

And an encoding module, configured to encode the base layer according to the number of base layer bits, and encode the enhancement layer according to the enhanced layer bit number.

In the embodiment of the present invention, the number of the target bits is allocated to the access unit according to the available number of bits, and the number of bits is allocated to the base layer and the enhancement layer of each access unit, and the total bit rate constraint is satisfied by the solution provided by the embodiment of the present invention. Under the condition, the proportion of the base layer and the enhancement layer in the target bit number can be reasonably adjusted, and the quality of the encoder output bit stream is improved, thereby improving the decoded image quality.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other embodiments may be obtained by those skilled in the art without departing from the drawings. 1A is a schematic diagram of a correspondence between a basic layer and an enhancement layer of an SVC according to an embodiment of the present invention;

FIG. 1B is a schematic diagram showing another correspondence between the basic layer and the enhancement layer of the SVC to which the embodiment of the present invention is applied;

2 is a flowchart of a code rate control method for multi-layer coding according to an embodiment of the present invention; 3 is a flowchart of a code rate control method for multi-layer coding according to Embodiment 2 of the present invention;

FIG. 4 is a schematic structural diagram of a code rate control apparatus for multi-layer coding according to an embodiment of the present invention.

detailed description

The technical solutions of the embodiments of the present invention are further described in detail below with reference to the accompanying drawings and embodiments.

The basic layer and the enhancement layer of the SVC may have a corresponding relationship, for example, as shown in FIG. 1A, which is a schematic diagram of a correspondence between a basic layer and an enhancement layer of the SVC, and the basic layer and the enhancement layer have a hierarchical time domain layer; As shown in FIG. 1B, which is another schematic diagram of the corresponding relationship between the base layer and the enhancement layer of the SVC, the base layer has a level 2 time domain layering, and the enhancement layer has a level 3 time domain layer. It should be noted that the foregoing two basic layer and enhancement layer correspondence diagrams are merely examples of the relationship between the base layer and the enhancement layer. The code rate control method and apparatus for multi-layer coding provided by the embodiments of the present invention are not limited to The above two application scenarios.

FIG. 2 is a flowchart of a code rate control method for multi-layer coding according to an embodiment of the present invention. As shown in FIG. 2, the embodiment specifically includes the following steps:

Step 101: Assign a target number of bits to an access unit (Access Unit, hereinafter referred to as AU) of a current group of pictures (hereinafter referred to as GOP) according to the number of available bits. Step 102: According to the number of target bits, respectively Allocating a corresponding base layer bit number and an enhancement layer bit number to the base layer and the enhancement layer of the access unit;

Step 103: Encode the base layer according to the number of base layer bits, and encode the enhancement layer according to the enhancement layer bit number.

In this embodiment, the target number of bits is allocated to the AU according to the number of available bits, and the number of bits is allocated to the base layer and the enhancement layer of each AU. The scheme provided in this embodiment can be reasonably adjusted under the condition of satisfying the total bit rate constraint. The ratio of the base layer and the enhancement layer in the target number of bits improves the quality of the encoder output bit stream, thereby improving the decoded image quality.

FIG. 3 is a flowchart of a method for controlling a code rate of a multi-layer coding according to Embodiment 2 of the present invention. As shown in FIG. 3, the embodiment specifically includes the following steps:

Step 201: Calculate the available number of bits of the remaining images in the current GOP according to the number of AUs included in the current GOP, the target bit rate of the image, the virtual buffer occupancy rate, and the frame rate of the video sequence.

This step can be performed according to a preset bit rate condition, for example, according to the initial input value of the code rate control in the encoder - the target bit rate. Assuming that the current GOP is the i-th GOP, after the j-1th image of the i-th GOP is encoded, the remaining map of the GOP except the first j-1 images The number of available bits of the image _; /) can be calculated by: x ._ ) · ₌₁

Where ? _; /) is the target bit rate of the jth image in the i-th GOP (if the initial input value of the code rate control module in the encoder - the target bit rate is a variable amount over time, then here The target bit rate is the target bit rate of the jth image coding time), which is the virtual buffer occupancy rate of the jth picture in the i-th GOP, f is the frame rate of the video sequence, N _; is the i-th GOP The number of AUs included is the actual number of bits used for the j-1th image.

Step 202: Allocate a target number of bits to each AU of the current GOP according to the available number of bits, the target bit rate, the frame rate of the video sequence, and the virtual buffer occupancy rate.

Further, in this embodiment, while referring to the available number of bits, the target bit rate, the frame rate of the video sequence, and the virtual buffer occupancy rate, the complexity of the image and the structure of the time domain layer may also be referred to, and each of the current GOPs is given. One AU allocates the target number of bits. For example, different allocation principles may be used to allocate the target number of bits. Specifically, the allocation principle of the target number of bits may include: The higher the complexity of the image, the more bits need to be allocated; the higher the level of the time domain hierarchy to which the AU belongs , the fewer bits that can be allocated.

This embodiment provides a method for allocating a target number of bits, which is specifically described as follows. Among them, current

The GOP is the i-th GOP. For the convenience of description, the subscript i of each specific parameter is ignored or not, for example, /) is _; /), and other parameters are similar.

After the end of the first AU encoding of the current GOP, the virtual buffer target occupancy rate <S(2) = V{2), and the virtual buffer target occupancy rate of the remaining time domain base layer AU in the current GOP can be expressed as :

S(2) W (j)x(L + l)xR(j) R(i) , S(j + 1) = S(j) - - + ^p — ~ = ~-^--^ - (2 ,)

N _p -l fx(W _p U) + W _b U)xL) f where /) is the average complexity of the base layer, ^ /) is the average complexity of the enhancement layer, and ^ is the number of P frames in the current GOP L is the number of unreferenced images (such as B frames) between two consecutive reference pictures (such as I frames or P frames).

Considering the virtual buffer target occupancy S /) and the actual occupancy rate (_ / ·), and the target bit rate i? /) and frame rate constraints, the number of bits required for the j-th AU r /) is: ^(i)=^+rx^(i)-^(i)] (3) where y is a constant, and its value can be determined in the application according to the actual situation.

According to the number of available bits C/) of the remaining images in the current GOP and the number of remaining AUs, the number of bits available for the jth AU can be obtained. ^ /) is:

, W(jl)xB(j) , _Λ , τ ( j) = _ ί ( 4 ) where , and N _Z are the remaining frames of the time domain base layer and the enhancement layer, respectively, which is the j-1th AU The correlation weighting coefficient of the base layer complexity is the correlation weighting coefficient of the enhancement layer complexity of the j-1 AU.

The target bit number 7^ /) that can finally be assigned to the jth AU is the weighted average of τ (β and τ), as shown in the following equation:

Where "sum is constant," the value depends on whether the AU contains a base layer. When the AU contains a base layer, "can take a value of 1, when the AU does not contain a base layer," can be between 0 and 1. Value; for? (·) weight. The specific value of "and can be determined in the application according to the actual situation.

Step 203: For one of the AUs, allocate a corresponding base layer number and an enhancement layer bit number to the base layer and the enhancement layer of the AU according to the target number of the AU.

In this step, if there is only an enhancement layer in the AU, the number of enhancement layer bits T _EL U)-T _AU (j) and the number of base layer bits ?k /)=0 are allocated.

If the AU includes an enhancement layer and a base layer, the allocated base layer bit number r _ffi /) and the enhancement layer bit number 7^ /) satisfy:

^(i) = ^(i) + 7k(i) ( 6 ) Since the encoding process of the enhancement layer needs to refer to information of the base layer, such as: intra prediction information, motion vector, and residual coefficient, therefore,

The output of the bitstream is of quality, which in turn affects the quality of the decoded image. In order to ensure the quality of the decoded image, the present embodiment can allocate and r /) in the following two ways.

The first method: According to the degree of distortion and the number of target bits, the multi-turn coding method is used to obtain the satisfaction rate. Distortion optimized r _ffi /) and

The combined multi-turn coding method makes the distortion degree D of the coded image and the number of bits R actually used for coding satisfy the minimum rate-optimized Lagrangian operator, that is, min +li?}.

(TBL EL)

The second method: According to the model of distortion and target bit number (ie, rate-distortion RD model), the T _BL sum that satisfies the rate-distortion optimization is calculated and obtained.

^T BL U) and T _EL (J) satisfy the following formula: f(T _BL , T _EL ) = g(D(j)) (7) where, for the distortion of the image of the jth AU, the current image can be used The mean absolute error MAD) or the error squared sum SSD(j) or absolute error and SAD (J, et al.

Further, the above formula (7) may be specifically as follows:

* ( ^T EL(J) - h _EL (J)) * (T _BL {j) - h _BL (j)) = _{kt A} ( ₈ ) * (W) - h _EL U)) - * (T _BL { j) - h _BL (j)) ⁴ where is the experimentally obtained constant. ^ and the number of bits of the header information encoded for the base layer and the enhancement layer image respectively, k ₂ , k ₃ , k ₄ are the show, the number, which is a constant between 4 and 8, and the value method and H.264 The method for finding the first-order coefficients in the RQ model is similar.

According to equations (6) and (7), solve the model shown in the following equation:

You can get the value of (and the value of ().

It should be noted that: Equation (8) is a specific implementation of Equation (7). Of course, there are other implementations that can be combined with Equation (6) to obtain values of ⁷ ^W and ⁷ ^ that satisfy the rate-distortion optimization. The implementation manners are all claimed in the embodiments of the present invention.

Step 204: Calculate a base layer quantization parameter according to the number of base layer bits, and calculate an enhancement layer quantization parameter according to the enhancement layer bit number.

In this embodiment, the RQ model may be used to calculate the base layer quantization parameter and the enhancement layer quantization parameter respectively. Further, the RQ model may be a first-order model used by a Moving Picture Experts Group (hereinafter referred to as MPEG)-2 test model (TM) -5 or H. The second-order model used by 264/AVC, for example:

Where 7X/) is the number of bits, which is the quantization parameter, _A /) is the header information of the base layer or enhancement layer and the number of bits of the motion vector, and is a constant. In actual coding, M4D /) can be obtained from the prediction of the previous frame or the base layer. Specifically, when calculating the base layer quantization parameter, it may be obtained according to the number of base layer bits, the header information of the base layer, and the number of bits of the motion vector, and the model shown in the formula (10); when calculating the enhancement layer quantization parameter, according to The number of enhancement layer bits, the header information of the enhancement layer, and the number of bits of the motion vector and the model shown in equation (10) are obtained.

Step 205: Encode the base layer according to the base layer bit number and the base layer quantization parameter, encode the enhancement layer according to the enhancement layer bit number and the enhancement layer quantization parameter, and obtain and output the multiplexed scalable bit stream.

Step 206: Update the parameters of the rate control according to the number of coded bits and the degree of distortion obtained by the coding.

After the encoding of the base layer and the enhancement layer of the current AU is completed, the parameters of the rate control are updated according to the number of bits and distortion actually obtained by the coding, and the parameters that need to be updated include parameters related to the current coded image in each of the above formulas, for example. Mouth: S /), V(j) , R /), 5 ( , W _p (j) , W _b (JN _{LZ r} , N _LNZ , _r , /) ( and ^ (etc. Then, jump to the step 203. Continue processing the next AU; if the current AU is the last AU, the encoding process ends.

When an AU encoding is finished, the encoder's virtual buffer occupancy can be updated by:

j = 2, 3,..., N _i

f

In this embodiment, the number of the target bits is allocated to each AU according to the number of available bits, and the number of bits is allocated to the base layer and the enhancement layer of each AU, and the solution provided by this embodiment can satisfy the total bit rate constraint. Reasonably adjusting the proportion of the base layer and the enhancement layer in the target bit number improves the quality of the encoder output bit stream, thereby improving the decoded image quality; further, the embodiment can perform multi-turn coding or RD model establishment. The method obtains an optimized combination of bit allocation of the base layer and the enhancement layer satisfying the rate distortion optimization, thereby obtaining the output bit of the multi-layer coding optimized for rate distortion. The stream improves the coding quality; and, in this embodiment, an appropriate QP is selected according to the number of bits allocated to each layer, so that the encoder can make full use of the inter-layer correlation and improve the coding efficiency.

FIG. 4 is a schematic structural diagram of a code rate control apparatus for multi-layer coding according to an embodiment of the present invention. As shown in FIG. 4, the embodiment specifically includes: a target bit allocation module 11, a layer bit allocation module 12, and an encoding module 13. The target bit allocation module 11 allocates a target number of bits to the AU of the current GOP according to the obtained number of available bits. The layer bit allocation module 12 allocates corresponding base layer bits to the base layer and the enhancement layer of the AU according to the target number of bits. The number and the enhancement layer bit number; the encoding module 13 encodes the base layer according to the number of base layer bits, and encodes the enhancement layer according to the enhancement layer bit number.

Further, the embodiment may further include an available bit calculation module 14 that calculates according to the number of AUs included in the current GOP, the target bit rate of the image, the virtual buffer occupancy rate, and the frame rate of the video sequence. The number of available bits. Specifically, the available bit calculation module 14 can calculate the number of available bits according to the equation (1).

The target bit allocation module 11 may be specifically configured to give each AU of the current GOP according to the available bit number, the target bit rate, the frame rate of the video sequence, the virtual buffer occupancy rate, the complexity of the image, and the structure of the time domain layer. The target bit number is allocated. Specifically, the target bit allocation module 11 can allocate the target bit number to the AU according to the equations (2) - (5).

The layer bit allocation module 12 may be specifically configured to obtain a base layer bit number and an enhancement layer bit number satisfying rate distortion optimization according to the distortion degree and the target bit number by using a multi-turn coding method. The layer bit allocation module 12 can also be specifically configured to calculate the base layer bit number and the enhancement layer bit number that satisfy the rate distortion optimization according to the model of the distortion degree and the target bit number.

The foregoing encoding module 13 may be specifically configured to calculate a base layer quantization parameter according to the number of base layer bits, calculate an enhancement layer quantization parameter according to the enhancement layer bit number, and base layer according to a base layer bit number and the base layer quantization parameter. Encoding is performed, and the enhancement layer is encoded according to the enhancement layer bit number and the enhancement layer quantization parameter. The encoding module 13 can calculate the base layer quantization parameter and the enhancement layer quantization parameter using the R-Q model. Further, the R-Q model may be a first-order model adopted by MPEG-2 TM-5 or a second-order model adopted by H.264/AVC.

The embodiment may further include an update module 15 that updates the parameters of the code rate control according to the number of coded bits and the degree of distortion obtained by the coding, and the parameters that need to be updated include parameters related to the current coded image in each of the above formulas. .

In this embodiment, each AU is allocated a target number of bits according to the number of available bits, and then each is given The basic layer and the enhancement layer of the AU allocate the number of bits. With the scheme provided in this embodiment, under the condition of satisfying the total bit rate constraint, the proportion of the base layer and the enhancement layer in the target number of bits can be reasonably adjusted, and the encoder is improved. The quality of the bit stream is output, thereby improving the quality of the decoded image. Further, in this embodiment, the optimal combination of the bit layer of the base layer and the enhancement layer satisfying the rate distortion optimization can be obtained by multi-turn coding or the method of establishing the RD model. Obtaining a multi-coded output bit stream optimized by rate distortion improves the coding quality; and, in this embodiment, selecting an appropriate QP according to the number of bits allocated to each layer, so that the encoder can make full use of inter-layer correlation and improve The coding efficiency.

Each of the above embodiments is a multi-layer coding method applicable to SVC, and the embodiment of the present invention is also applicable to other coding techniques using inter-layer prediction, such as scalable coding techniques in MPEG-2 and MPEG-4.

A person skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by using hardware related to the program instructions. The foregoing program may be stored in a computer readable storage medium, and the program is executed when executed. The steps of the foregoing method embodiments are included, and the foregoing storage medium includes: a medium that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

It should be noted that the above embodiments are only used to explain the technical solutions of the embodiments of the present invention, and are not limited thereto; although the embodiments of the present invention are described in detail with reference to the foregoing embodiments, those skilled in the art should understand The technical solutions described in the foregoing embodiments may be modified, or some of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the technical solutions of the embodiments of the present invention. The spirit and scope of the programme.

Claims

Rights request

A code rate control method for multi-layer coding, comprising:

2. The multi-layer code rate control method according to claim 1, further comprising: according to the number of access units, the target bit rate, the virtual buffer occupancy rate, and the video included in the current image group. The frame rate of the sequence, the number of available bits is calculated.

The multi-coded rate control method according to claim 1, wherein the assigning the target number of bits to the access unit of the current image group comprises: according to the available number of bits, the target bit rate, and the video The frame rate of the sequence, the virtual buffer occupancy, the complexity of the image, and the structure of the time domain layering allocate the target number of bits to the access units of the current image group.

The multi-layer code rate control method according to any one of claims 1 to 3, wherein the base layer and the enhancement layer of the access unit are respectively allocated corresponding base layer bit numbers and enhancement layer bits. The number includes: according to the degree of distortion and the number of target bits, the number of base layer bits and the number of enhancement layer bits are obtained by using a multi-turn coding method.

The multi-layer code rate control method according to any one of claims 1 to 3, wherein the base layer and the enhancement layer of the access unit are respectively allocated corresponding base layer bit numbers and enhancement layer bits. The number includes: calculating the base layer bit number and the enhancement layer bit number according to a model of the distortion degree and the target bit number.

The multi-layer code rate control method according to claim 5, wherein the calculating the base layer bit number and the enhancement layer bit number according to the model of the distortion degree and the target bit number includes: a model shown by the equation, calculating the number of base layer bits and the number of enhancement layer bits;

Wherein, the number of base layer bits is indicated, indicating the number of enhancement layer bits, and 7^ /) indicates the target bit number, indicating the degree of distortion.

7. The multi-layer code rate control method according to claim 1, wherein: Encoding the base layer according to the number of base layer bits, and encoding the enhancement layer according to the enhancement layer bit number, including:

Calculating a base layer quantization parameter according to the number of base layer bits, and calculating an enhancement layer quantization parameter according to the enhancement layer bit number;

The base layer is encoded according to the base layer bit number and the base layer quantization parameter; the enhancement layer is encoded according to the enhancement layer bit number and the enhancement layer quantization parameter.

The code rate control method for multi-layer coding according to any one of claims 1 to 3, wherein the base layer is coded according to the number of bits of the base layer, according to the number of bits of the enhancement layer. After the enhancement layer is encoded, the method further includes: updating the parameters of the rate control according to the number of coded bits and the degree of distortion obtained by the coding.

9. A multi-layer code rate control apparatus, comprising:

10. The multi-layer code rate control apparatus according to claim 9, further comprising a usable bit calculation module, configured to: according to the number of access units included in the current image group, a target bit rate, and a virtual The buffer occupancy rate and the frame rate of the video sequence are calculated, and the number of available bits is calculated.

The multi-layer code rate control apparatus according to claim 9, wherein the target bit allocation module is specifically configured to: according to the available number of bits, a target bit rate, a frame rate of a video sequence, and a virtual buffer The area occupancy rate, the complexity of the image, and the structure of the time domain layering assign the target number of bits to the access unit of the current picture group.

The multi-layer code rate control apparatus according to any one of claims 9 to 11, wherein the layer bit allocation module is specifically configured to obtain a multi-turn coding method according to the distortion degree and the target bit number. The base layer number of bits and the number of enhancement layer bits.

The multi-layer code rate control apparatus according to any one of claims 9 to 11, wherein the layer bit allocation module is specifically configured to calculate the base layer according to a model of distortion degree and target bit number. Number of bits and number of enhancement layer bits.

The multi-layer coded rate control apparatus according to any one of claims 9 to 11, wherein the encoding module is specifically configured to calculate a base layer quantization parameter according to the base layer bit number, according to the enhancement. The layer bit number calculates an enhancement layer quantization parameter, and the base layer is encoded according to the base layer bit number and the base layer quantization parameter, and the enhancement layer is encoded according to the enhancement layer bit number and the enhancement layer quantization parameter.

The multi-layer code rate control apparatus according to any one of claims 9-11, further comprising an updating module, configured to perform parameter control parameters according to the number of coded bits and the degree of distortion obtained by the encoding. Update.