FR3041851A1 - ASSOCIATED FLOW ALLOCATION METHOD, DEVICE, ENCODER AND COMPUTER PROGRAM - Google Patents

Abstract

The invention relates to a rate allocation method between a first layer, called the base layer and at least a second layer, called the enhancement layer, of a scalable coder able to code a sequence of digital images, to a overall target bit rate. According to the invention, the method comprises the following steps, implemented following the coding of a group of images of the sequence (GOP (i)): obtaining (E1) a measurement of the coding quality of the enhancement layer for the current image group from a predetermined metric; obtaining (E2) an effective encoding cost of the base layer and an effective encoding cost of the at least one enhancement layer for the current image group; computing (E3) a rate ratio between a said enhancement layer and the overall rate from the estimated coding costs; estimating (E6) a performance function at least from the calculated throughput ratio, the quality metric obtained for the current GOP and a predetermined curve model; and determining (E8) an optimum rate ratio in a predetermined ratio range, from the estimated performance function and a quality-maximizing criterion of said at least one enhancement layer.

Description

Flow allocation method, device, encoder and associated computer program 1. Field of the invention

The field of the invention is that of the coding of digital image sequences, in particular of the flow regulation between several layers of a scalable coding scheme (for "scalable", in English). The invention may especially, but not exclusively, apply to the transmission of coded picture sequences on transmission channels of various capacities. 2. Presentation of the prior art

Consider a scalable encoder, which allows to encode content in several hierarchical levels, the decoding of a level requiring the prior decoding of the underlying levels. This dependence is explained by the fact that the scalable coding exploits the existing correlations between the various signal variations in order to reduce the redundancies and to improve the compression. In the particular case of 2D scalable video coding, the input signal is declined in several versions whose characteristics improve for each layer (for example: definition, frame-rate, SNR (for "Signal to Noise Ratio", in English), Color-Gamut, Dynamics, etc ...).

FIG. 1 illustrates an example of a video scalable coder C, known to those skilled in the art, making it possible to encode the source signal S in two quality levels S-0 and S-1 respectively corresponding to the layers L-0 and L-1. The encoder must adhere to CD-0 rate guidelines for layer L-0 and CD-I for layer L-1. These two added target rates constitute an overall bit rate DG and are associated with a bit rate ratio RD = CD-0 / DG.

A first processing unit T makes it possible to transform the signal S into as many input signals S-0, S-1 as there are scalability layers available in the scalable encoder. In this case, the processing block T consists of a first block T-1 which generates the highest level S-1, then a second block T-0 which then drifts the lowest level S-0. These input signals are adapted to the characteristics of the scalability layers. For example, for spatial scalability, a source signal in UHD ("Ultra High Definition") format will be pre-sampled by a first preprocessing block T-0 to create an input signal HD (for "High High Definition"). Definition ", in English) S-0 adapted to the L-0 base layer. For temporal scalability, a source signal at 120 frames per second will be reduced to 60 frames per second. For a color gamut scalability, a source signal placed in a BT.2020 color space will be declined into a Rec.709 version using a color-mapping algorithm. For a scalability in bit-depth, a signal whose channels are represented on the bits will be declined into an 8 bits version. For dynamic scalability, an HDR signal will be transformed into an SDR signal using a tone-mapping operator. It is understood that these pretreatments, known to those skilled in the art, are adapted to the type of scalability implemented by the coder.

The input signals S-0 and S-1 respectively supply a coder C-0 of the layer L-0 and the coder C-1 of the layer L-1. These encoders are controlled by two RC-0 rate control units for the L-0 layer coder and RC-1 for the L-1 layer coder. The encoders and their rate control units communicate by exchanging IRC-0 coding information for the L-0 and IRC-1 layer for the L-1 layer. During the compression of the L-0 layer coding information IC-0 of the L-0 layer are extracted and transmitted to a processing block IT-0 which will process in order to make them exploitable IC-1 by the coder C-1. Such coding information includes, for example, residue distribution information, quantization parameters, a Lagrangian multiplier, rate information and / or coded data quality, reference images from the lower layer or movement.

The signal S-0 of the layer L-0, called the base layer, is first encoded by a coder C-0 which produces an independently decodable bit stream Bs-0. During this first encoding, inter-layer information IC-0 is extracted (eg information on the prediction modes used, the motion vectors, the local coded-decoded image, etc.) and then eventually set to scale, processed and finally transmitted to the coder C1 of the upper layer L1, said enhancement layer or enhancement.

The encoder C-1 of the upper layer, in turn, compresses the signal S-1 corresponding to the layer L-1 by exploiting this inter-layer information in addition to the information normally used. In the same way, IC-1 inter-layer information is extracted, processed and transmitted to the upper layer encoder, as appropriate. The encoder C-1 produces a bit stream Bs-1. The bitstream Bs-1 can only be correctly decoded if the inter-layer information has been well reconstructed at the decoder, therefore only if the lower layer has been previously decoded. The two bit streams BS-0, Bs-1 may be multiplexed by the block M to form a global bit stream BsT.

This encoding process and decoding dependency is repeated for all existing upper layers. This type of encoder implicitly operates at a fixed rate ratio, since the CD-I and CD-0 target rates imposed on the layers are associated with a certain ratio CD-0 / (CD-0 + CD-1).

This encoding method avoids the separate, therefore non-optimal, coding of a signal into several levels of information. The bit streams thus produced can be sent as needed and the decoding and display capacity of the terminal.

International patent application published under the number W02006 / 119436 discloses a multi-layer and multi-pass coding method which adjusts the coding quality of each of the layers so that the coding rate of the base layer and that of the enhancement layer satisfy a mathematical relationship, such as for example a flow ratio value. A first pass makes it possible to collect characteristic parameters of the data to be coded and then to estimate the bit rates to be allocated to each layer from the parameters collected. The second pass implements the estimated flow rates. A fixed rate ratio is imposed between the layers. For example, it is arbitrarily chosen equal to 1: 1. 3. Disadvantages of prior art

A disadvantage of this solution is that it does not explain how to determine the rate ratio. At best, it is constrained by an arbitrary value fixed a priori. 4. Objectives of the invention The invention improves the situation. The invention particularly aims to overcome these disadvantages of the prior art.

More precisely, an object of the invention is to propose a solution that makes it possible to distribute a global target bit rate optimally between the scalability layers, so as to maximize the quality of the coded picture sequence. 5. Presentation of the invention

These objectives, as well as others that will appear later, are achieved by means of a flow allocation method between a first layer, called the base layer and at least a second layer, called the improvement layer. of a scalable coder capable of coding a sequence of digital images at a target overall rate.

The method according to the invention is particular in that it comprises the following steps, implemented following the coding of a group of images of the sequence: obtaining a measure of quality of coding of the improvement layer for the current image group from a predetermined metric; obtaining an effective coding cost of the base layer and an effective encoding cost of the at least one enhancement layer for the current image group: calculating a flow ratio between a said layer of improvement and overall throughput from estimated coding costs; estimating a performance function at least from the calculated throughput ratio, the quality metric obtained for the current GOP and a predetermined curve model; and Determining an optimum rate ratio in a predetermined ratio range from the estimated performance function and a quality-maximizing criterion of said at least one enhancement layer.

With the invention, it is estimated from at least one group of images that has just been encoded, the evolution of the performance of the encoder as a function of a flow ratio between the layers and is determined, from of this estimate and for the coding of the next group of images, the rate ratio that optimizes the quality of the coding for the overall fixed bit rate.

Thus, the invention is based on an entirely new and inventive approach to bit rate allocation that does not a priori set the rate ratio between the coding layers, but models the coding performance as a function of the rate ratio obtained. on groups of images previously encoded and deduces the optimal value to implement for the next group of images.

According to an advantageous characteristic of the invention, the step of estimating a performance function comprises a determination of parameters of the predetermined curve model from the calculated flow rate ratio and the quality measurement obtained for the group of current images and a plurality of pairs of values of quality measurements and flow rate previously obtained.

For example, the parameters of the curve model are computed according to a least squares error minimization method. The curve model used can be polynomial of order 1 or higher. Advantageously, the curve model can be chosen according to results obtained on a representative set of test sequences.

The plurality of pairs of values can be derived from the encoding of previous groups of images or from a correspondence table, previously constructed from a representative set of test sequences.

According to another aspect of the invention, the rate allocation method further comprises a step of determining at least one allowed ratio interval. The range of allowed ratios can be fixed, for a simple implementation of the invention.

According to another aspect of the invention, the method comprises a step of updating a remaining bit rate from the coding costs obtained and the target overall bit rate, and the determining step determines a first ratio interval as a function of quality-flow constraints on the base layer and the calculated remaining flow rate. The range of allowed bit rates may vary during encoding. Advantageously, the invention takes into account flow and quality constraints on the base layer and the remaining target flow rate to determine the most suitable authorized flow rate range.

According to another aspect of the invention, the method comprises a step of updating a remaining bit rate from the coding costs obtained and the overall target bit rate (DG), and the determining step determines a second slot. ratio as a function of quality flow constraints on a said improvement layer and the calculated remaining flow rate.

The procedure is the same for the layer or layers of improvement.

According to another aspect of the invention, the determining step determines an intersection of the first and second intervals. The allowed total rate ratio interval thus obtained takes into account the constraints of all scalability layers.

According to another aspect of the invention, the step of determining a ratio interval comprises: estimating a rate-quality function from a predetermined curve model, and at least the quality measurements. and the coding costs obtained for the current group of images; a calculation of the flow rate associated with a said quality constraint from the estimated quality flow function; and a calculation of the flow rate ratio corresponding to said constraint as a function of the calculated remaining flow rate. At a quality constraint, a corresponding flow rate is matched using a run-of-river quality-flow function and the corresponding limit of the allowed throughput ratio interval is derived.

According to another aspect of the invention, the estimation of a rate-quality function comprises a determination of parameters of the predetermined curve model from the measurements of flow and quality obtained for the current group of images and a plurality of pairs of quality and flow measurement values previously obtained.

Advantageously, the estimation of the quality bit rate function, like that of the performance function, exploits a plurality of quality and bit rate values previously obtained, either following the encoding of previous groups of images of the sequence, or from a representative set of test sequences, or both. It is refined as and progressively the encoding progress. The collected quality and rate values can be stored in a buffer of a predetermined size or in a correspondence table. One or the other are updated as and when encoding.

According to yet another aspect of the invention, the method comprises a step of updating a remaining bit rate from the coding costs obtained and the overall target bit rate and a step of calculating a target bit rate of the bit layer. base and a target rate of said at least one enhancement layer for a next image group according to the remaining updated rate and the calculated optimal flow rate ratio.

In a single-pass scheme, the determined optimal ratio is used to calculate the flow instructions to be applied when encoding the next GOP.

According to yet another aspect of the invention, the method comprises a step of calculating a target bit rate of the base layer and a target bit rate of said at least one enhancement layer for the next picture group by a function of the flow rate remaining updated following a first encoding of the next image group and the optimal ratio of calculated flow rate.

In a multi-pass scheme, during the second pass, the optimal ratio calculated from the quality / flow values obtained following the encoding of the current image group makes it possible to calculate the flow instructions to be applied to re-encode the GOP. current.

The method which has just been described in its various embodiments is advantageously implemented by a flow allocation device between a first layer, called the base layer and at least a second layer, called the improvement layer, a scalable coder capable of coding a sequence of digital images at a target overall rate.

According to the invention, such a device comprises the following units, which can be implemented following the coding of a group of images of the sequence: evaluation of a measurement of quality of coding of the improvement layer for the group of images running from a predetermined metric; obtaining an effective coding cost of the base layer and an effective encoding cost of the at least one enhancement layer for the current image group: calculating a flow ratio between a said layer of improvement and overall throughput from estimated coding costs; estimating a performance function from the calculated throughput ratio, the quality metric obtained for the current image group and a predetermined curve model; and determining an optimum rate ratio in a predetermined ratio range, from the estimated performance function and a quality-maximizing criterion of said at least one enhancement layer.

Of course, the flow allocation device according to the invention can be arranged to implement, independently or in combination, all the embodiments described above for the flow allocation method. .

Correlatively, the invention also relates to a scalable coder in layers, comprising a first coder capable of encoding a first layer, said base layer of a digital image sequence at a first rate and at least a second encoder capable of encoding a second layer, said enhancement layer of said sequence, at a second rate, said encoder comprising a first rate control module able to control the first rate consumed by the base layer and at least one second rate control module capable of to control the second rate consumed by a said improvement layer, characterized in that it further comprises a flow allocation device according to the invention adapted to allocate the first rate to the first module and the second rate to the second module . The invention also relates to a computer program comprising instructions for implementing the steps of a flow allocation method as described above, when this program is executed by a processor.

This program can use any programming language. It can be downloaded from a communication network and / or recorded on a computer-readable medium. The invention finally relates to a recording medium, readable by a processor, integrated or not integrated with the flow allocation device according to the invention, possibly removable, respectively storing a computer program implementing the method of flow allocation as previously described. 6. List of Figures Other advantages and features of the invention will appear more clearly on reading the following description of a particular embodiment of the invention, given as a simple illustrative and non-limiting example, and attached drawings, among which: Figure 1 (already described) schematically shows a scalable coder with two layers according to the prior art; Figure 2 shows schematically the steps of a rate allocation method according to the invention;

Figure 3 illustrates the overhead provided by a scalable coding scheme with respect to a non-scalable coding scheme according to a particular metric; Figure 4 schematically shows a scalable two-layer encoder according to a first embodiment of the invention; Figure 5 schematically shows the steps of a rate allocation method according to a second embodiment of the invention; Figures 6A to 6D show overall quality versus logarithmic flow curves for three test sequences; Figure 7 schematically shows a scalable two-layer encoder according to a second embodiment of the invention; Figure 8 schematically shows a scalable two-layer encoder according to a third embodiment, multi-pass; FIG. 9 shows an example of a LUT correspondence table of a L-1 layer coding quality measurement as a function of the bit rate according to a fourth embodiment of the invention; FIGS. 10A and 10B show examples of correspondence tables of each of the scalability layers, between a scalability quality measurement of a scalability layer and a measurement of the bit rate used according to a fifth embodiment of the invention; Figure 11 illustrates results obtained by the rate allocation method according to the first embodiment of the invention in a set of 10 HD sequences; Figure 12 illustrates results obtained by the rate allocation method according to the second embodiment of the invention in a set of 10 HD sequences; Figure 13 illustrates the rate and quality constraints imposed on the scalable encoder according to the second embodiment of the invention to obtain the results of Figure 12; and Figure 14 schematically shows an example of a hardware structure of a rate allocation device according to the invention. 7. Description of a particular embodiment of the invention

The general principle of the invention is based on the estimation of a performance function of the coding of the enhancement layer of a scalable coder for at least one group of encoded GOP images. This function links the obtained coding quality to a given bit rate ratio between the base layer and the enhancement layer. From this function, the invention determines, for the next group of images to be encoded, the ratio which, according to the evolution of the estimated performance function, will contribute to maximizing the level of performance of the coding.

In order to illustrate as simply and clearly as possible the various embodiments of the proposed invention, the case of a scalable 2D video coding with two layers is considered.

Of course the invention is not limited to this example and concerns any scalable coder, regardless of the number of layers used.

We consider a source signal S representative of a sequence of M digital images, with M nonzero integer. The sequence is divided into groups of images (GOP for "Group of Pictures" in English).

In the remainder of the description, a group of GOP (i) images of the signal S, before its encoding by the scalable encoder C.sub.GOP (i, l) and GOP (i, 2), are considered to be the variations of this GOP in the input signals corresponding to both layers.

In relation with FIG. 2, a method is described for estimating a target bit rate ratio for encoding a sequence of images, following the encoding of at least one group of images of this sequence. according to a first embodiment of the invention. This method is advantageously implemented by the coder C of FIG.

The encoder must therefore provide two bitstreams that respect a global bit rate DG while having the degree of freedom to choose, when encoding, rate ratios in the range IR = IF. A global bit rate DG allocated to the scalable coder C is considered for encoding the sequence S. This coder comprises two layers L-0 and L-1. The overall flow rate is therefore defined as the sum of the flow rate obtained in the two layers: DG = DL + DL_X (1) with D1_o flow rate reached in the L-0 layer and D ^ flow rate reached in the L-1 layer.

A flow rate ratio is defined as follows:

(2)

The IR ratio interval to be respected when encoding the L-0 and L-1 layers is specified as follows:

(3)

The value of the ratio between the L-0 and L-1 layers must therefore belong to this interval. In this first embodiment, the interval is fixed. For example, it is specified as follows: IF = [0.35 - 25%, 0.35 + 25%] (4)

The value 0.35 is close to the average value found by using the test conditions ("common test conditions" in English) specified by the MPEG ("Moving Picture Coding Expert Group") standardization group. In addition, it is an arbitrary value commonly used in industry to approximate HD / UHD throughput ratio. Here the encoder is given a degree of freedom of + -25% compared to this ratio to optimize the compression.

In previous work described in an article titled "Toward Optimal Bitrate Allocation in the Scalable HEVC Extension: Application to U HDTV," by Biatek et al, published in the Proceedings of the ICCE-Berlin 2015 Conference, held in Berlin from 6 to 9 September 2015, the inventor highlighted the impact of the ratio on the coding performance of a scalable 2D video schema and the fact that the judicious choice of the ratio could have a large impact on coding performance. Indeed, it is possible to observe in Figure 3 the additional cost provided by the scalable HD / UHD scheme compared to the simple HEVC UHD encoding measured with the Bjôntegaard metric (BD-BR and BD-PSNR) as a function of the flow rate ratio. , on a set of 10 sequences. These curves illustrate the potential for gain that a rate ratio adjustment can bring. For example, for the "StudioDancer" sequence, choosing a ratio of 0.3 results in an additional flow cost of about 20% while a ratio of 0.5 would reduce this extra cost to around 10%.

Following the encoding of the current image group GOP (i), El is obtained with information Q1 representative of a coding quality measurement of the current group of images for the L-1 layer. For example, this measurement has been evaluated according to a given metric (for example the PSNR, SSIM, etc ...), known to those skilled in the art. This measurement can be made locally by the rate allocation device implementing the method according to the invention or obtained from another module of the scalable coder, such as for example the coder C-1.

During a step E2, a measurement of a coding cost B0 of the current GOP by the coder C-0 and a measurement of a coding cost B1 of the current GOP by the coder C-1 are obtained. These coding costs are expressed in the number of bits consumed.

In E3, the effective ratio of the current GOP is calculated as follows:

(5)

In E4, the DR rate to be applied to the remaining images is calculated which will make it possible to respect the overall flow constraint DG. This rate is calculated according to the number of bit already consumed BC, the overall cost B0 + B1 generated by the current GOP, the frame rate F, the total number of images to be encoded NT and the number of remaining images NI.

(6)

In E5, the number of bits to be consumed for the GOP following BG is calculated according to DR and the size of this GOP NG (i + 1).

(7)

In E6, a Q (R) coding performance function is estimated, intended to provide a quality measurement Q of the coding of the L-1 layer as a function of the effective rate ratio R 1.

Many more or less complex methods can be used to estimate this function. In this first embodiment, a simple way of modeling this function is to use the quality and bit rate measurements obtained after the encoding of the current image group and the preceding groups.

For the i-th coded group of images, we therefore consider the effective quality Qi and the effective ratio Ri, calculated as follows:

Qi = Ql (8)

Ri = B0 / (B0 + B1) (9)

If a buffer MP of K pairs of samples (Ri, Qi) with non-zero integer K is stored in memory, for example equal to 6 for a buffer of 1 second and a frequency of 48 images per second, updated in E60 after the encoding of the current image group, it is possible to model the performance function Q (R) based on this buffer.

Advantageously, a simple way of modeling this function is to use a model of polynomial curve, of degree 1 for example: Q (R) = ax R + b (10)

By using a total least squares method on the measurements stored in memory, it is possible to estimate a and b by solving the following system:

(U)

(12)

Several methods can be used to solve this system, for example using a Gaussian elimination or invert the matrix A and calculate A ~ XY.

During a step E8, the performance function obtained Q (R) is used to determine the point of the curve that maximizes the coding quality of the L-1 layer in the permitted IR ratio interval and recovered at E7. This optimal ratio Ro is selected as follows:

(13)

The ratio estimation method is repeated for each new group of encoded images. By proceeding in this way, the model described in Equation 10 can be updated after the encoding of a new group of images, along the water, and the optimal ratio Ro to be applied to the next group of images GOP (i + l) is calculated via Equation 13.

It is understood that the encoding of a new group of images provides a new pair of values (Ri, Qi), for example stored in the MP buffer, which contributes to refining the estimate of the performance function and therefore the determination of the optimal ratio Ro to be applied during the encoding of the GOP according to GOP (i + 1).

In E9, the JRC-0 and JRC-1 target bit rate or target bitrate values of the GOP (i + 1) image group are calculated for each of the L-0 and L-1 layers. The budgets allocated to each layer are first calculated by multiplying the total budget allocated for the next GOP BG, obtained from the overall bit rate DG allocated for the coding of the sequence and the bit rate already consumed for the preceding image groups , by the estimated optimal ratio Ro to obtain the budget allocated to the layer L-0 and by (1-Ro) to obtain the budget allocated to the layer L1. Then, these budgets are multiplied by the image frequency F and divided by the number of NG (i + 1) images of the GOP (i + 1) in order to obtain the JRC-0 and JRC-1 bit rates: (14) (15)

These two rate targets are transmitted to the RC-0, RC-1 rate control modules of each of the layers.

In relation with FIG. 4, there is shown a scalable encoder C comprising a joint flow control unit JRC implementing the rate ratio estimation method according to the first embodiment of the invention. Overall throughput to meet DG and ratio interval

Fixed IF are specified as instructions for the JRC Joint Flow Control Unit. The JRC unit communicates with the two flow control units, RC-0 for the L-0 layer and RC-1 for the L-1 layer via the JRC-0, DO, JRC-1, Q1 and D1 information.

An advantage of this first embodiment is that it makes it possible to optimize the ratio of flow between the layers of the encoder for a range of values of rate ratios fixed a priori. One possible application is, for example, coding of the "best possible effort" type (or "best-effort coding" in English) for scalability where it is desired to ensure the best coding performance possible, without imposing a minimum quality on the base layer. The rate ratio interval is then set according to the degree of freedom that one wishes to leave during the coding.

According to a second embodiment of the invention, the authorized rate ratio interval is no longer fixed a priori, but it is subject to quality and speed constraints for each of the scalability layers.

These different quality and flow constraints for the two layers take, for example, the form of permitted ranges of values IQO, IQ1, IDO and ID1. Note that these constraints can vary dynamically over time. For example, in case of partial failure on the broadcast network bit trains, they can be relaxed temporarily.

The constraints are specified in the following way: IQO = [Qojnin 'Qo_max] (16) IQ1 = [Qljnin> Ql_max] (17) IDO = [Djjnin, D0_max] (18) ID1 - [Dx min> Di max] (19) )

In the same way as for the first embodiment, a performance estimation function is constructed. Step E1 obtains measurements of the Q1 coding quality obtained in the L-1 layer for the current image group and step E10 obtains a QO coding quality measurement of the L-0 base layer. Step E2 collects the coding costs BO and B1. These measurements and costs obtained are intended to be exploited by subsequent steps of the method according to the invention, such as the steps E60, E70, E71, E12 and E13 which are going to be detailed below.

As for the first embodiment, the effective ratio is always calculated in E3 (Equations 8 and 9). A buffer MP is also updated in E60 and the performance function Q (R) is estimated from this buffer at E6 (Equation 11 and 12). The bit rate DR to be imposed for the coding of the remaining images in order to respect the global bit rate constraint DG is always calculated in E4. This rate is calculated as a function of the number of bits already consumed, the encoding costs BO and B1 (collected in E2) of the current GOP GOP (i), the image frequency F, and the number of remaining images NI. In E5, the number BG of bits to be consumed for the GOP according to GOP (i + 1) is calculated according to the remaining bit rate DR and the size NG (i + 1) of this GOP.

In this embodiment, the overall ratio ratio IR must respect both the constraints of the L-0 layer and the L-1 layer. Let IRO and IR1 be the intervals respectively defined by the stresses of the layers L-0 and L-1, then the global interval IR is defined as the intersection of these two intervals: IR = IRO n IRl (20)

The intervals of the two layers IRO and IR1 can also be calculated as the intersection of sub-intervals related to quality and flow constraints: / RO = SIQOnSIDO (21) IR1 - SIQ1 nSID1 (22)

The calculation of IRO is done in E70 and the calculation of IR1 in E71. It is assumed that F0 (D) and F1 (D) are the quality-flow functions of the two layers linking the quality Q evaluated for a given metric, for example: PSNR, SSIM, etc. at flow D. It is possible to express the subintervals as follows: siqo = [R0 Qmin 'Ro_Qmax] (23) SIQ1 - | R | (24) SIDO - [R0Dmin> R0_Dmax] (25) SiDi - [RiDmin> R1_Dmax] (26)

Before the encoding of a group of images following GOP (i + l), the intervals IQO ', IQ1', IDO 'and ID1' to respect in flow and quality are recalculated according to the quality of the images already coded in each layer for the preceding groups of images, associated effective rates and initially fixed constraints IQO, IQ1, IDO and ID1. The new bounds of the quality and throughput intervals constrained for further encoding, in particular for the GOP group (i + l), are calculated as follows: (27) (28) (29) (30) (31) (32) (33) (34)

With QMO, QM1 the average qualities achieved at this stage of the coding for the L-0 and L-1 and BCO and BC1 layers the bits already consumed in the layers L-0 and L-1. This step of updating the stress intervals is carried out in E70 for the L-0 layer and in E71 for the L-1 layer. These ranges are therefore: (35) (36) (37) (38)

Once these intervals are updated, it is possible to calculate the bounds of the rate ratio sub-ranges specified in Equations 21 to 22 as follows: (39) (40) (41) (42) (43) (44) (45)

(46) with Q0 = FO (D), Q1 = F1 (D) functions that link the quality to the flow rate for the L-0 and L-1 layers.

In the same way as for the performance function Q (R), the functions F0 (D) and F1 (D) must be estimated during the encoding. For example, if PSNR is considered as a quality metric, many more or less complex methods and models can be used to estimate this function.

Advantageously, a simple way of proceeding is to keep in memory two buffers MO and M1 of K pairs of samples (Q0it, D0it) and (Qi,;, Oi ,;) respectively associated with the layers L-0 and L1 with K whole. , for example equal to 6 for a buffer of 1 second and a frequency of 48 images per second: (47) (48) (49) (50) where QO, BO, Q1 and B1 designate the quality measurements, flow obtained for the current GOP GOP (i) in layers L-0 and L1 and NG (i) the number of images of the current GOP.

These two buffers are constructed respectively at E12 and E13 for the layers L-0 and L-1, thanks to the information collected in E1 and E2. Based on these two buffers, it is possible to estimate the flow-distortion functions of the two layers F0 (D) and F1 (D). Exponential and quadratic modeling can be used. An easy way to do this is to estimate these functions as being affine over a given interval in logarithmic scale.

In Figure 6, we can observe the precision of an affine estimate over 5Mbps intervals, for three Ultra-High Definition (UHD) 2D video sequences.

A simple approach is therefore to define the quality flow functions as follows: F0 (D) = a0 x log {p) + b0 (51) F1 (D) =% x log {D) + bx (52)

In the same way as for the performance function, the parameters of the models used can be estimated using the total least squares method by solving the following systems of equations:

(53) (54) (55) (55)

The estimates of the functions F0 (D) and F1 (D) are carried out in the steps E14 and E15. The rate-distortion functions defined in Equations 51 and 52 can be easily inverted and used to determine the SIQO, SIDO, SIQ1, SID1 subintervals defined in Equations 23 to 25, then the IRO, IR1 layer intervals defined in the Equations. 21 and 22 and finally the global interval IR defined in Equation 20. Once this global interval is estimated, one proceeds in the same way as in the first embodiment, that is to say that one determines First, in E8, the rate ratio Ro that maximizes the performance function Q (R) over this interval IR and then applies it to the allocated budget BG in E9 (Equation 14 and 15). This gives the JRC-0, JRC-1 rate setpoints for the GOP (i + 1) image group.

In relation to FIG. 7, a scalable encoder is presented whose joint flow control unit J RC implements the second embodiment of the invention. According to this mode, the JRC unit inputs, in addition to the target DG target bit rate, the IQO, IDO, IQ1, ID1 quality and flow constraints associated with each of the scalability layers.

An advantage of the second embodiment is that it makes it possible to impose quality and / or flow constraints on one or more of the scalability layers. It is possible to specify constraints on the encoder and thus to match the bitstreams of data it generates to the target configuration.

For example, if one wishes to fill a storage space of a volume having a capacity of B bits with a video of N images declined in two versions HD (L-0) and UHD (Ll) with a frequency of F images per second, guaranteeing a minimum quality Q on the HD service and a maximum quality of the UHD service, the method according to the invention will specify the overall bit rate parameters DG = (B * F / N) and of minimum quality for the layer of Qomin-Q base, in order to obtain an optimized encoding respecting these constraints. In particular, the enhancement layer L-1 will occupy all the space left available after the coding of the L-O base layer.

In other cases of hybrid uses for which the broadcasting of bitstreams is done in part on a conventional broadcast network type TNT (for Digital Terrestrial Television) for the HD layer (L-0) and via the Internet for the UHD layer (Ll), the invention allows to introduce other constraints. For example, when the service operator has the ability to guarantee a consistent quality of service for the HD, between Qmin / Qmax, while restricting the UHD throughput in a Dmin / Dmax bandwidth in order to touch the most wide audience possible via the Internet.

The embodiments which have just been presented propose an estimation of the flow rate ratio "on the water", that is to say which relies on information resulting from the encoding of at least one previous group of images. It is understood that the modeling of the performance function is refined as more samples of quality measurement are acquired as a function of the actual flow ratio. On the other hand, during a change of scene, a reset takes place and it takes several successive GOPs to optimize the modeling of the performance function.

In relation to FIG. 8, a method of estimating a rate ratio according to a third embodiment of the invention is now described. This third embodiment is based on a multi-pass approach. In other words, a first implementation of the method according to the invention AD (i + 1) is carried out following the encoding of a group of images GOP (i) and for the encoding of a group of images. images according to GOP (i + 1) according to the second embodiment which has just been described. Following the encoding of the current image group GOP (i), the intervals relating to the stresses of the layer L-0 and L-1 are firstly calculated with the current flow-quality functions. Then the intersection of these two intervals is calculated and produces the global IR interval which respects all the constraints. The optimal ratio Ro is then determined by minimizing the function of estimating the current performance over the overall interval. The flow instructions JRC-0 for the layer L-0 and JRC-1 for the layer L-1 are determined and transmitted to the flow control units RC-0, RC-1 of the two layers. The group of images following GOP (i + 1) is then pre-coded by the coders C-0 and C-1 in accordance with the flow instructions JRC-0, JRC-1. The number of bits consumed BO and B1 for the first encoding pass of each layer of the group GOP (i + 1) is evaluated as well as the quality of the encoding Q0, Q1. These values are exploited during a second implementation AD (i + 1) 'of the rate allocation method according to the second embodiment of the invention to update the estimation steps of the functions F0 ( D) and F1 (D) linking the quality constraint intervals to the flow rate, as well as the performance function Q (R). We understand that in this way, we can model these functions more precisely. Advantageously, the buffers MP, MO and M1 are updated with the new values Q0, BO1, Q1 and B1 obtained during the first encoding pass of the GOP (i + 1). The optimal rate ratio R0 'is then reevaluated from the updated performance function. Flow instructions JRC-0 ', JRC-1' are calculated. The group of images GOP (i + 1) is encoded a second time, starting from the new calculated instructions.

An advantage of this multi-pass mode is that it does not require initialization of the model since the correct parameters are calculated for the current GOP during the first pass and applied during the second pass. This mode is therefore more robust to scene changes.

In relation to FIG. 9, a fourth embodiment of the invention is described. This mode aims to improve the accuracy of the estimation of the performance function, in particular in the case of a single-pass mode.

In this fourth embodiment, the Q (R) function is no longer estimated from the pairs of flow ratio values, quality of a buffer MP, but from a correspondence table or LUT (for "Look" Up Table, in English). In this mode, the LUT is used to link a finite number of rate ratio values to the corresponding quality values in the L-1 layer.

For example, the values stored in the LUT at the beginning of the encoding were previously computed from the encoding statistics of a representative set of test sequences. One way of proceeding is, for example for a set of encoded sequences, to establish statistics of image group rates according to quality, and to use them to then choose the function model (and its parameters) on closer to these statistics, all sequences combined.

In particular, one advantage is to ensure that the samples (Qi, Ri) are evenly distributed over a much wider range of values of values, thus including the allowed ratio range IR. By estimating the Q (R) function over this ratio range, we obtain a much more accurate estimate of the actual behavior of the function. After the encoding of a group of GOP (i) images, the effective ratio Ri is used to calculate the index of the correspondence table, then the measured effective quality Qi replaces the old value located at this index. Based on this new sample set, the performance estimation function can be updated. In other words, it is rescaled to the current sequence.

In this embodiment, a quadratic model described in Equation 47 is used, unlike that of embodiments 1 and 2. This quadratic model allows a more accurate modeling of the performance function and is more suitable. to an estimate based on a wider set of samples. Q (R) a = axR2 + bxR + c (57)

In the same way as in the embodiment 1, the parameters of the quadratic model can be estimated by solving the matrix system described in the equations 58 and 59. (58) (59)

This fourth embodiment makes it possible to model the performance function over a much wider range of flow rate ratios, and thus to make a much finer modeling that is faithful to the real curve that one wishes to estimate. By using this modeling, the estimation of the ratio will be more precise.

According to a fifth embodiment of the invention, the functions F0 (D) and D1 (D) are no longer estimated from the buffers MO and M1, but from two correspondence tables LUT-0, LUT-1, illustrated respectively in Figures 10A and 10B. In this mode, the LUTs link a finite number of flow rate values to the corresponding quality values in each of the L-0 and L-1 layers. In particular, one advantage is to ensure that samples (Q0, t.D0u, and (.Qu'Du) are evenly distributed over a much broader range of flow values by estimating the functions F0 (D) and F1 ( D) on these ranges of extended flows, one thus obtains estimates much more faithful to the real behavior of these functions The values present at the beginning of encoding in these LUTs can be established for example from the statistics of coding of a set representative of test sequences.

After the encoding of each group of images, the effective qualities of the L-0 and L1 layers, QO and Q1 are used to calculate the indexes of the LUTO correspondence tables, LUT1, then the actual binary costs BO and B1. The new values obtained replace the old values located at this index. Based on these new sample sets, Flow-Quality functions can be updated. In this embodiment, a 3rd order model described in Equations 60 and 61 rather than the

model of the second embodiment, because this kind of model, more complex but more accurate, corresponds to that used to model the flow-quality functions used by the Bj0ntegaard metric. F0 (D) = a0x log (D) 3 + b0 x log (D) 2 + c0 x log (D) + d0 (60) F1 (D) =% x log (D) 3 + bt x log (D) 2 + cx x log (D) + (61)

In the same way as in the embodiment 1, the parameters of the two models can be estimated by solving the matrix systems described in the equations 62 to 65. (62) (63) (64) (65)

This fifth embodiment makes it possible to model flow-quality functions over a much wider range of flow rates, and thus to have a much finer modeling and faithful to the real curve. By using this modeling, the estimation of a flow associated with a quality will then be more precise.

It will be noted that the invention which has just been described can be implemented by means of software and / or hardware components. In this context, the terms "module" and "entity", used in this document, may correspond either to a software component, or to a hardware component, or to a set of hardware and / or software components, capable of implementing perform the function (s) described for the module or entity concerned.

In relation with FIG. 11, the performance of the first embodiment is presented on a set of UHD sequences (EBU UHD-1 data set). Here scalable compression SHVC (for Scalable High Efficiency Video Coding) is compared to the simple HEVC encoding of the highest layer (here UHD). In this case, the scalable compression provides additional cost, because of the additional service transmitted via the base layer. The increase of flow brought by our method in the sense of the BD-BR is noted garc. The increase in bit rate provided by the fixed ratio encoding is noted GRef. The loss reduction provided by our method is calculated as follows:

(56)

Finally the rate reduction of our method compared to the fixed ratio encoding is noted Gx. It is observed that the method proposed in embodiment 1 makes it possible to reduce on average the losses by 20%, with a BD-BR gain of 4.25%. The method is more effective than the fixed ratio 9 times out of 10 sequences tested.

In connection with FIG. 12, the performance of Embodiment 2 improved by Embodiments 4 and 5 (based on LUTs) is presented. In this case, we set the constraints described in figure 13. In this case of use, a minimum quality is fixed on the layer 0 and a range of flow is constrained in the layer 1. The same indicators of performances as those in relationship with Figure 11, and previously described, are used. The chosen fixed ratio encoding is the first whose encoding makes it possible to respect all these constraints. We observe in Figure 12 that our method allows, in this case of use, to reduce losses by about 26% compared to the fixed-ratio encoding, while respecting the same constraints. The BD-BR gain found is about 10%. Our method is also more efficient 9 times out of 10 sequences.

In relation to FIG. 14, an example of a simplified structure of a device 100 for allocating the bit rate between the layers of a scalable coder according to the invention is now presented. The device 100 implements the rate allocation method according to the invention which has just been described in relation to FIGS. 2 and 5.

For example, the device 100 comprises a processing unit 110, equipped with a processor p1, and driven by a computer program Pgl 120, stored in a memory 130 and implementing the method according to the invention.

At initialization, the code instructions of the computer program Pgx 120 are for example loaded into a RAM before being executed by the processor of the processing unit 110. The processor of the processing unit 110 sets implement the steps of the method described above, according to the instructions of the computer program 120.

In this exemplary embodiment of the invention, the device 100 comprises at least one GET Q unit for obtaining information Q0, Q1 representative of GOP coding quality measurements for each of the L-0 and L1 layers, a unit GET B for obtaining flow measurement B0, B1 for the encoding of the current GOP by the layers L-0 and L1, a unit for obtaining an interval IR of authorized flow rates, a unit for determining the ratio η of the effective bit rate obtained following the coding of the current GOP (i), an estimation unit EST Q (R) of a performance function Q (R) and a unit DET Ro for determining the optimal ratio Ro from the estimated function. Advantageously, the device 100 further comprises a CALC unit JRC-0, JRC-1 for calculating the bit rates JRC-0 and JRC-1 to be allocated to the layers L-0 and L1 for the coding of the GOP according to GOP (i + 1). .

The device 100 further comprises a first storage unit MP, for example of the buffer type, a plurality N of encoded image groups and at least a second storage unit MO, M1 of the correspondence tables implemented in the fourth and fifth embodiments of the invention.

These units are driven by the μΐ processor of the processing unit 110.

Advantageously, such a device 100 may be integrated with a scalable encoder 10. Such a scalable encoder may itself be integrated with a terminal user or server type equipment AND. The device 100 is then arranged to cooperate at least with the following modules of the scalable coder: a coder C-0 of the first layer L-0; a coder C-1 of the second layer L-1; an RC-0 rate control module allocated to the L-0 layer; an RC-1 rate control module allocated to the L-1 layer. The invention can find many applications and scenarios. For example, to optimize the scalable compression of a video sequence intended to be stored on a medium of a given dimension. By setting as the total throughput the rate corresponding to the capacity of the storage, and by setting the desired minimum levels of quality on the layers we can use the proposed method. In this case, the invention makes it possible to fill the storage medium optimally. The invention can be used in the case of a broadcast type of use, or it is desired to introduce a new service of higher quality while maintaining compatibility with existing receivers. In this case, it is desired to limit the additional cost introduced by adding a scalable service, while maximizing the quality of the new service, and maintaining the quality of the service to be transmitted to the existing fleet. In this case, the proposed invention makes it possible to respond to these problems optimally, without requiring the implementation of a simulcast encoding, more expensive.

We can also think of the case of HD / UHD hybrid broadcasting for example. In this case the HD service is transmitted via the lowest level layer, on a conventional channel (TNT, Satellite). The UHD service is provided by the scalability, and in particular by the highest level layer that would be transmitted via the IP network. If we consider that a minimum quality of service wants to be delivered to non-premium users (HD), and that the broadcaster wishes to reach the widest possible audience via a "broadband" type of telecommunication network (average bandwidth 80% of its customers for example), the invention can then be used to optimize the compression of services offered.

Finally, the invention can also be implemented in the case of video conferencing, on certain networks or a fixed capacity communications channel is allocated, then disseminated to potentially smaller capacity networks that can not handle all the layers. In this case, the proposed invention can take into account all these constraints, coupled with quality constraints to provide an optimized scalable video conferencing service delivering the best possible service, regardless of the client's bandwidth capacity.

It goes without saying that the embodiments which have been described above have been given for purely indicative and non-limiting purposes, and that many modifications can easily be made by those skilled in the art without departing from the scope. of the invention. They can advantageously be combined with each other. For example, following a change of plane of the image sequence, the invention makes it possible to perform a pre-coding for a certain duration, so as to estimate the performance function on x seconds, then to make a second pass a once the correct parameters have been determined over a sufficient flow ratio range. It will be understood that the bit rate allocation will only be multi-pass for a determined duration following the detection of a change of plan, and then it will return to one-pass at the end of this period.

In addition, the rate allocation method just described can be applied to a scalable coder comprising more than two scalability layers. If, for example, the encoder comprises three scalability layers L-0, L1, L-2, then the performance function becomes multivariable and links the highest quality of the L-2 layer to the rate ratios R0 = D0 / DG and R1 = D1 / DG of the two lower layers. The modeling of this function is carried out using a curve model and a set of measured samples. The calculation of the ratio interval remains unchanged, but can be expressed in two ways depending on the scalability layer in which one places oneself to calculate the ratio: IRL0 = IRO n IR1 n IR2 if one places oneself in the layer L-0 (57) IRL1 = IRO n IR1 n IR2 when placed in the layer L1 (58)

The subintervals still exist, and their representation also differs according to the layer in which one is placed. The optimization problem to be solved would then become:

(62)

Claims (14)

A method for allocating a bit rate between a first layer (L-0), said base layer and at least one second layer (L1), called the enhancement layer, of a scalable coder able to code a sequence of digital images, according to a global target bit rate (DG), characterized in that it comprises the following steps, implemented following the coding of a group of images of the sequence (GOP (i)): (E1, E1) a measure of quality of coding (Q1) of the enhancement layer for the current group of images from a predetermined metric; obtaining (E2) an effective encoding cost of the base layer and an effective encoding cost of the at least one enhancement layer for the current image group: computing (E3) a ratio of rate between a said enhancement layer and the overall rate from the estimated coding costs; estimating (E6) a performance function at least from the calculated throughput ratio, the quality metric obtained for the current GOP and a predetermined curve model; and determining (E8) an optimum rate ratio (Ro) in a predetermined ratio interval (IF, IR), from the estimated performance function and according to a quality-maximizing criterion of said at least one layer improvement. Flow rate allocation method according to claim 1, characterized in that the step of estimating a performance function comprises determining parameters of the predetermined curve model from the calculated flow rate ratio and the measurement. of quality obtained for the current image group and a plurality of pairs of previously obtained quality measurement and flow rate values. 3. Rate allocation method according to one of the preceding claims, characterized in that it further comprises a step (E7) for determining at least one allowed ratio interval. 4. Rate allocation method according to one of the preceding claims, characterized in that it comprises a step (E10) of obtaining a coding quality measurement (QO) of the base layer, a step (E4) for updating a remaining bit rate (DR) from the obtained coding costs and the overall target bit rate (DG), and in that the determining step (E7) determines a first ratio interval by This is a function of quality throughput constraints on the base layer, quality measurements, coding costs and calculated remaining throughput. 5. Rate allocation method according to one of the preceding claims, characterized in that it comprises a step (E4) for updating a remaining bit rate (DR) from the coding costs obtained and the bit rate. target (DG), and in that the determining step determines a second ratio interval as a function of quality flow constraints on a said improvement layer and the calculated remaining flow rate. 6. Rate allocation method according to claims 4 and 5, characterized in that the determining step determines an intersection of the first and second intervals. Flow rate allocation method according to one of Claims 4 to 6, characterized in that the step of determining (E7) a ratio interval comprises an estimate (E14, E15) of a bit rate function. quality from a predetermined curve model and at least the quality measurements (QO, Ql) of the layers, the coding costs obtained for the current group of images, a calculation of the flow rate associated with a said constraint of quality from the estimated quality rate function and a calculation of the flow ratio corresponding to said constraint as a function of the calculated remaining flow rate (DR). Flow rate allocation method according to claim 7, characterized in that the estimate (E14, E15) of a rate-quality function comprises a determination of parameters of the predetermined curve model from the measurements of flow and quality obtained for the current image group and a plurality of pairs of previously obtained flow quality measurement values. 9. Rate allocation method according to one of the preceding claims, comprising a step (E4) for updating a remaining bit rate (DR) from the coding costs obtained and the overall target bit rate (DG), characterized in that it further comprises a step (E9) of calculating a target rate (JRC-0) of the base layer and a target rate (JRC-1) of said at least one layer of enhancement for a next image group (GOP (i + 1) based on the updated flow rate (DR) and the optimal ratio (R0) of calculated flow rate. 10. Rate allocation method according to one of the preceding claims, characterized in that it comprises a step (E4 ') for calculating a target rate of the base layer and a target bit rate of said least one enhancement layer for the next image group as a function of the flow rate remaining updated following a first encoding of the next image group and the optimal ratio (R0 ') of calculated flow rate. 11. Device (100) for allocating a bit rate between a first layer, called a base layer and at least a second layer, called an enhancement layer, of a scalable coder capable of coding a sequence of digital images (S) , at a global target bit rate (DG), characterized in that it comprises the following units, which can be implemented following the coding of a group of images of the sequence (GOPi): - evaluation (GET Q) an improvement layer coding quality measurement for the current image group from a predetermined metric; estimating (GET B) an effective encoding cost of the base layer and an effective encoding cost of the at least one enhancement layer for the current image group: - calculation (CALC) of a rate ratio between a said improvement layer and the overall bit rate from the estimated coding costs; estimating (EST Q (Ri)) a performance function at least from the calculated throughput ratio, the measured quality and a predetermined curve model; and determining (DET RJ) an optimum rate ratio in a predetermined ratio interval, based on the estimated performance function and according to a criterion for maximizing the quality of said at least one improvement layer. 12. A layer-scalable encoder (10) comprising a first encoder (C-0) capable of encoding a first layer, called a base layer of a digital image sequence, at a first rate and at least a second encoder ( C1) capable of encoding a second layer, referred to as the enhancement layer of said sequence, at a second rate, said encoder comprising a first rate control module (RC-0) able to control the first rate consumed by the base layer. and at least one second flow control module (RC-1) capable of controlling the second flow rate consumed by a said improvement layer, characterized in that it further comprises a flow-rate allocation device (100) according to claim 11 adapted to allocate the first rate to the first module and the second rate to the second module. 13. Computer program (PgJ comprising instructions for implementing the rate allocation method according to one of claims 1 to 10, when executed by a processor. 14. A recording medium readable by a processor, characterized in that it is able to store the computer program according to claim 13.