FR2851109A1 - Model parameters determination method for estimating image sequence distortion, involves assigning current image sequence of model parameters with reference image sequence selected based on representation information - Google Patents

Abstract

The method involves comparing information representative of current image sequences with information representative of some of the sequences of images for reference. The image sequence for reference is selected if the representative information is closest to those of the current image sequences. The current image sequences of the values of the model parameters are assigned with the selected reference sequence. Independent claims are also included for the following: (a) a device for determining the parameters of a model of estimate of the distortion adapted to a current image sequences code (b) an apparatus for communication with a device for parameter determination (c) a computer storage with instructions of a computer program for execution for the stages of the process of determination of parameters (d) a computer program loaded in a programmable apparatus.

Description

<Desc / Clms Page number 1>

 The present invention relates to a method and a device for determining parameters of a distortion estimation model that are adapted to a sequence of current images to be encoded.

 It should be noted that the invention generally applies to a sequence of images that may contain one or more images.

 It is known that digital image sequences require a relatively high memory capacity during storage.

 For this reason, many studies have been conducted for several years in order to optimally compress such digital data.

 In general, data compression methods aim to establish a trade-off between preserving acceptable visual quality and reducing the size of stored data.

 To do this, it has for example sought to optimize the distortion generated by the coding of the images under a coding rate constraint.

 This approach is usually based on bit rate / distortion modeling of the data source (video encoder or still image encoder).

 It is for example known to use such an approach in the flow control algorithm TMN8 described in the article "Rate control for low-delay video communications", J. Ribas-Corbera, S. Lei, Technical Report ITUTelecommunications Standardization video coding expert group, Document Q15-A-20, June 1997 of the H263 video coding standard version 2 (Draft text of recommendation H. 263 version 2 (H. 263+) for decision, Garry Sullivan, technical report ITU- T, DRAFT 21, February 1998).

 Such bit rate / distortion models make it possible to obtain for each possible coding option an estimation of the bit rate and the distortion generated without having to perform precoding of the images which proves to be expensive in terms of calculation.

<Desc / Clms Page number 2>

 Optimization of the distortion generated by image encoding under a rate constraint is then achieved by determining encoding options that are compatible with the rate constraint and that minimize distortion.

 The use of such models therefore makes it possible to significantly reduce the complexity of flow control algorithms.

 The flow / distortion models are very important in flow control algorithms since they make it possible to consider regulating the coding bit rate of a video sequence in real time without having to perform expensive pre-encodings of the video signal. sequence to determine the distortion and throughput values for each possible coding option.

 It is therefore interesting to use rate regulation based on bit rate and distortion models for applications such as videoconferencing where, for interactivity issues between users, these applications do not support any latency in the coding. .

 Flow control algorithms very often use models that give the rate of an image or macro-block of the image as a function of the distortion or, conversely, which makes it possible to perform a coding with a level of distortion or fixed rate.

où Rq correspond au débit d'un macro-bloc de l'image en fonction de la distorsion D, Xq est une constante du modèle pour un paramètre de quantification Q donné et Qd est un coefficient de ce modèle. A model for estimating the bit rate of a macroblock of an image is known from US 6 256 423 in the name of Sarnoff Corporation and which is written:

where Rq corresponds to the bit rate of a macroblock of the image as a function of the distortion D, Xq is a constant of the model for a given quantization parameter Q and Qd is a coefficient of this model.

 This model is used in a rate control loop that is implemented in a video coder based on the H. 263 version 2 standard defined in the document entitled "Draft text of recommendation H. 263 version 2 (H. 263+) for decision "by Gary Sullivan, ITU-T Technical Report, DRAFT 21, February 1998.

<Desc / Clms Page number 3>

 In the basic version of this standard it is assumed that only the first image of the image sequence is encoded in Intra (I) encoding mode and that it will not undergo flow control.

 Under these conditions, if a rate constraint is imposed on the encoder, it will have to pass, without coding, a certain number of images after the first coded image in Intra mode in order to compensate for a possible exceeding of the constraint of debit.

 After this image jump, the following images are coded in P encoding mode and undergo rate regulation based on the rate estimation model provided above.

 The authors of the aforementioned H. 263+ standard propose to use the idle time of the encoder during the image jump to initialize the two Xqet Qd parameters. For this purpose, one of the images that have been discarded during the image jump, in iterative manner with each possible quantization parameter, is pre-encoded in encoding mode P. Given the number of quantization parameters considered by the standard, this process generates thirty-one pre-coding phases.

 This procedure appears to be expensive in terms of calculations and can not be adapted to an encoder in which no image jump is possible.

 On the other hand, this approach implies that all the constituent macro-blocks of the P-images are coded in Inter coding mode. However, this hypothesis is contradictory with the standard which offers three possible modes for the macro-blocks of the images P (Intra, Inter and non-coded). If these three modes were to be taken into account, the number of pre-coding steps needed should then be at least doubled.

 Another method of flow control adapted to the H.263 version 2 standard and which is proposed in the document US Pat. No. 6,119,991 in the name of Sharp Laboratories of America is not known elsewhere.

 This document provides a model for estimating the bit rate of a macroblock of an image according to its variance and quantization parameter assigned to it and this model is written:

<Desc / Clms Page number 4>

où Q est le paramètre de quantification pour le macro-bloc considéré, S est la variance du macro-bloc, A est une constante égale au nombre de pixels dans un macro-bloc (16*16) et K et C sont deux paramètres du modèle.

where Q is the quantization parameter for the macro-block considered, S is the variance of the macro-block, A is a constant equal to the number of pixels in a macro-block (16 * 16) and K and C are two parameters of the model.

 This model is adaptive in the sense that K and C are updated at each macroblock during coding.

 These two parameters are initialized at the level of the first image P with two given values which do not take into account the sequence of images to be encoded.

 Subsequently, at the beginning of each image the parameters K and C are initialized to the last value taken by these two parameters in the previous image. It should be noted that at the beginning of the image sequence the parameters are unsuitable, which reduces the efficiency of the estimation.

 In addition, by initializing at the beginning of the image the parameters with the last values taken in the preceding image, it is hypothesized that the successive images have homogeneous characteristics.

 Therefore, a very marked change between two images in a video sequence, such as a change in scenery, should lead to poor flow estimates.

 The initialization of the model parameter values represents a real disadvantage.

 Moreover, in this model, no distinction is made between the macroblocks coded in Intra mode and those coded in Inter mode. It then seems difficult to predict the flow of these two modes with a single model as the rates generated by each of the two coding modes are different.

 In addition, in some cases, it is not possible to re-evaluate the model parameters at the macro-block level during coding.

 Certain methods of optimizing the quantization parameter of the macroblocks must moreover evaluate the bit rate and the distortion of the macroblocks before the coding of the image.

<Desc / Clms Page number 5>

The model proposed above would lose much of its efficiency if it could not be updated at a macro-block level.

 However, the updating of the model at the level of a macro-block is binding.

 Moreover, the Applicant has found that when adapting a distortion estimation model to a sequence of images to be encoded, it is necessary to have a sufficient number of real data samples of the sequence. .

 Indeed, modeling the behavior of a small number of samples and / or unrepresentative samples may be of poor quality.

 In practice, for example, the number of available real data samples may be insufficient, which is for example the case at the beginning of the image sequence, when a small number of images have been coded: Distortion will then be unreliable due to the presence of little or no representative samples.

 Thus, the Applicant has found that it would be advantageous to obtain a satisfactory adaptation of the parameters of a distortion estimation model to an image sequence to be encoded when the number of actual data samples available for the adaptation is unrepresentative and / or insufficient.

 The present invention aims to remedy at least one of the aforementioned drawbacks by proposing a method for determining parameters of a distortion estimation model adapted to a sequence of current images to be encoded, characterized in that, at each sequence images of a set of reference image sequences are associated information representative of the image sequence and the parameter values of the distortion model adapted to this sequence of images, the method comprising the following steps: information representative of the current image sequence, - comparing information representative of the current image sequence with information representative of at least some of the reference image sequences of the set,

<Desc / Clms Page number 6>

 depending on the result of the comparison, selecting the reference image sequence whose representative information is closest to that of the current image sequence, assigning to the current image sequence the parameter values of the distortion model associated with the selected reference image sequence.

 Correlatively, the invention provides a device for determining parameters of a distortion estimation model adapted to a sequence of current images to be encoded, characterized in that, at each sequence of images of a set of sequences reference images being associated with information representative of the image sequence considered and the parameter values of the distortion model adapted to this sequence of images, the device comprises: means for obtaining information representative of the sequence of current images, means for comparing information representative of the current image sequence with information representative of at least some of the reference image sequences of the set, means for selecting the sequence reference images whose representative information is closest to that of the current image sequence according to the result of the comparison reason.

 means for assigning to the current image sequence the parameter values of the distortion model associated with the selected reference image sequence.

 Thus, when the number of observations needed to adapt the model parameters to a current sequence of images is insufficient, the invention is particularly useful since it makes it possible to obtain a valid and efficient distortion model for the sequence common.

 Indeed, one of a given set of reference image sequences searches for the one that most closely resembles the current image sequence to be encoded, and then the values of

<Desc / Clms Page number 7>

 model parameters that were previously determined for the most similar reference sequence.

 There is therefore no need to have a large number of samples of the current sequence to code.

 In order to find the nearest reference sequence of the current sequence, for example, information representative of a reference sequence is compared with that representative of the current sequence.

 This information representative of a sequence of images has the function of describing the content of the sequence.

 This is, for example, information representative of the spatial activity and / or the temporal activity of a sequence of images.

 Reference image sequences are sequences of drive images, i.e., test image sequences that form a sufficiently heterogeneous array to represent substantially all types of image sequences encountered in convenient.

 Thus, in such a set of reference sequences, one can find a sequence that resembles or even corresponds in some cases to virtually any current sequence to be encoded.

 Moreover, the proposed invention is not very complex and independent of the distortion model used.

 In addition, the invention avoids having to update the distortion model at a macro-block.

 In addition, the invention makes it possible to take into account different coding modes applied to macroblocks of an image.

 According to one characteristic, the comparison step provides for measuring a distance between the current image sequence and at least some of the reference image sequences of the set in a n-dimensional space, where each image sequence is represented in this space by n coordinates each corresponding to information of different types which are representative of the sequence in question.

<Desc / Clms Page number 8>

 Measuring such a distance has the advantage of being simple and quick to perform.

 According to another characteristic, the sequence of reference images whose content is closest to that of the current image sequence is that which minimizes the distance measured between the current image sequence and the reference image sequences. considered.

 According to one characteristic, the information representative of the spatial activity is the variance of the image sequence.

 Such information reliably represents the spatial activity of the sequence.

 According to one characteristic, the information representative of the temporal activity is an average value E of the different mean squared error values calculated between several successive image pairs of the image sequence.

 Such information reliably represents the temporal activity of the sequence.

For example, the distance Dn measured between, on the one hand, the current image sequence represented by a variance Vcurr and a squared mean error Ecurr and, on the other hand, a reference image sequence represented by a variance Vn and a quadratic mean error In expressed by the formula:

This distance has the advantage of being particularly simple and quick to calculate.

 In addition, the distance Dn can be approximated relatively reliably by the expression #Vn - Vcurr # +! En - Ecurrl, which has the advantage of accelerating the processing.

 According to one characteristic, the distortion estimation model comprises at least one logarithmic operator weighted by at least one exponential operator.

 Such a model is particularly advantageous insofar as, by weighting the logarithmic operator with an exponential operator, it reduces

<Desc / Clms Page number 9>

 the complexity of the model, without losing the precision in estimating the distortion values provided by this model.

 Moreover, reducing the complexity of the model facilitates the adaptation of this model to a sequence of images and / or a coding or transcoding method modifying the spatial and / or temporal resolution of the sequence.

 With such a model, the modeling of a sequence of images is more efficient than in the prior art.

 It will be noted that when speaking of coding of an image sequence within the meaning of the present invention, the term "coding" also encompasses the notion of transcoding.

 The invention also relates to a communication apparatus comprising a device as briefly described above.

 According to another aspect, the invention also aims at: a computer-readable information storage means or a microprocessor comprising code instructions of a computer program for executing the steps of the method according to the invention as briefly set forth above, and - a partially or fully removable computer-readable information storage medium or microprocessor having code instructions of a computer program for executing the steps of process according to the invention such as that briefly described above.

 According to yet another aspect, the invention relates to a computer program loadable in a programmable apparatus, comprising sequences of instructions or portions of software code to implement steps of the method according to the invention as briefly discussed below. above, when said computer program is loaded and executed on the programmable apparatus.

 The features and advantages relating to the device, to the communication device comprising such a device, to the information storage means and to the computer program being the same as those

<Desc / Clms Page number 10>

 exposed above concerning the process according to the invention, they will not be recalled here.

 Other characteristics and advantages of the present invention will emerge more clearly on reading the description which follows, made with reference to the appended drawings, in which: FIG. 1 schematically represents a server type communication apparatus in which the invention can be implemented; FIG. 2a represents a coding algorithm for video data; FIG. 2b represents a video data decoding algorithm; FIG. 3 represents a data processing algorithm according to the invention; FIG. 4 is an algorithm detailing the operations performed in step E302 of FIG. 3; FIG. 5 is an algorithm detailing the operations (distance determination) performed in step E409 of FIG. 4; FIG. 6 schematically illustrates the selection of the reference sequence closest to the current sequence; FIG. 7 represents an algorithm for determining the parameters of a distortion model that are adapted to a current sequence; FIG. 8 is an algorithm detailing the operations of the search step E708 (FIG. 7) of the reference sequence closest to the current sequence; Fig. 9 is an embodiment of a programmable apparatus embodying the invention.

 The invention will be presented hereinafter in the context of a flow control algorithm which is described in the article "Real Time Implementation of Rate-distortion Optimized Coding Mode Selection for H.263 Video Coders" by KH Yang. , A. Jacquin, Proceeding of ICIP 98,1998.

<Desc / Clms Page number 11>

où QP1 et M, sont respectivement le paramètre de quantification pris dans l'ensemble Q=[1,31] et le mode de codage pris dans l'ensemble M={INTER,INTRA, NON CODEE} pour le macro-bloc donné d'indice i, D,(.) et Rl (. ) représentent respectivement la distorsion et le débit du macro-bloc i, et # et # sont respectivement l'ensemble des modes de codage et l'ensemble des paramètres de quantification choisis pour les n macro-blocs. In this algorithm adapted to the H.263 version 2 video coding standard, the authors of the article select the coding modes and the quantization parameters for a set of n macroblocks. To do this, they rely on an optimization of the distortion under constraint of a flow and which is written in a Lagrangian form:

where QP1 and M, are respectively the quantization parameter taken from the set Q = [1.31] and the coding mode taken from the set M = {INTER, INTRA, NOT CODED} for the given macro-block 'index i, D, (.) and Rl (.) respectively represent the distortion and the rate of the macro-block i, and # and # are respectively the set of coding modes and the set of quantization parameters chosen for the n macro-blocks.

où D représente la distorsion normalisée des macro-blocs de l'image codés en mode de codage Inter en fonction de leur variance normalisée S, la normalisation étant effectuée par rapport au paramètre de quantification QP du macro-bloc considéré. The rate and distortion information is obtained using, in part, the bit rate and distortion model proposed in KH Yang's "Normalized Rate-distortion Model for H.263-compatible Coded and Its Application to Quantizer Selection". AT . Jacquin, NS Jayant, proc. ICIP 97. This distortion estimation model is written:

where D represents the normalized distortion of the macroblocks of the image encoded in Inter coding mode according to their normalized variance S, the normalization being carried out with respect to the quantization parameter QP of the macroblock considered.

It should also be noted that the normalized variance S of the macroblocks is deduced from the variance S of these using the following formula:

The distortion of macroblocks for a given quantization parameter QP as a function of the normalized distortion is provided by the equation

<Desc / Clms Page number 12>

next :

où a, ss, y sont les paramètres du modèle, S représentant la variance d'un bloc de données de l'image qui est, dans l'exemple considéré, un macro-bloc de données. From this known model, the invention proposes to weight the logarithmic operator of this model by at least one exponential operator as indicated in the formula below:

where a, ss, y are the parameters of the model, S representing the variance of a data block of the image which is, in the example considered, a macro-block of data.

 By thus weighting by an exponential operator the known distortion estimation model involving a logarithmic operator, we reduce the complexity of this model since only three parameters are necessary to define the model instead of seven for the known model.

 This model is valid for coded macroblocks in Intra coding mode and Inter coding mode.

 However, as will be seen later, the coding process of these two modes is very different and a single n-tuple of parameters (in this case it is a triplet) can not suffice.

 Thus, for a given sequence of images and a given coding method, two triplets of parameters in the example considered will be necessary: one of the triplets will be specific to the Intra coding mode ([alpha] IN TR A'ssIN TR A '[gamma] IN TR A), the other triplet being specific to the Inter coding mode ([alpha] INTER, ssINTER' [gamma] INTER).

 It will be seen later in the description how the above model involving the parameters a, ss, y will be adapted to a given video sequence and a given coding method.

 One of the advantages of the distortion model proposed above is its adaptability: it can indeed be adapted to a given sequence of images, which will increase the efficiency of the estimation of the distortion.

<Desc / Clms Page number 13>

 To obtain a valid model, it is necessary to have encoded a sufficient number of images, so as to obtain a sufficient number of observations of the data to be modeled.

 However, as we have seen above, it is not always possible, in practice, to obtain sufficient data to observe.

 The invention makes it possible precisely to determine a distortion model adapted to a current sequence of images without, however, requiring obtaining a minimum number of observations on the data to be modeled.

 The description of the method according to an exemplary embodiment of the invention breaks down, in a general manner, as follows: # Constitution of a set of reference image sequences: - search for values of the parameters of the adapted distortion model to each reference sequence of the set, - obtaining at least one descriptor of each reference sequence of the set (information representative of the content of the sequences).

Analysis of the current image sequence to be coded: obtaining at least one descriptor of the current sequence; comparing the current sequence with at least some of the reference sequences of the set via the descriptors; selecting a reference sequence similar to the current sequence in that its content is the closest to that of the other reference sequences; - assigning to the current sequence the values of the parameters of the selected reference sequence.

 Note that the constitution of the set of reference image sequences is performed before the analysis of the current image sequence, and not in real time, when it comes to processing this current sequence to code , that is, to adapt the distortion model to it.

 The method according to the invention is, for example, implemented on the apparatus of FIG.

<Desc / Clms Page number 14>

 In this figure is shown a first communication device 1 which is, for example, a server connected, not shown, to one or more other communication devices (client machines), via a communication network and connections considered established.

 This network is for example a wireless local communication network inside a home.

 Note that the image data stored in the server may have been received from the environment outside the home, for example, by another communication network such as the Internet.

 In the exemplary embodiment considered, only video data will be considered.

 The device according to the invention makes it possible to adapt a model for estimating distortion to a sequence of current images to be encoded.

 The device 2 according to the invention is for example integrated with the server 1 of FIG. 1 and may comprise electronic elements and / or software elements.

 However, the device according to the invention could, for example, be confused with the server 1.

 The server comprises a digital data storage unit 3 such as video sequences.

 It should be noted that the videos are created, for example, by a digital video camera or any other means of data acquisition or are transmitted by an external communication network and are stored in the unit 3 which may or may not be included in the device. device 2 according to the invention.

 This unit can be, for example, a local or distributed database.

 The device comprises units 4 and 5 which respectively make it possible to encode and decode a video sequence.

 The algorithms of the coding and decoding methods will be described later respectively with reference to FIGS. 2a and 2b.

<Desc / Clms Page number 15>

 The device also comprises a storage unit 6 for temporarily storing digital data and coded video, and a descriptor recovery unit 7 referred to above from the storage unit 3.

 The device 2 further comprises a unit 8 for determining the variances and distortions, a unit 9 for searching the parameters of the distortion model, a unit 10 for storing the various data and variables (variances, real distortions of the macroblocks , optimal parameters, minimum distances ...) obtained during the execution of various algorithms described below.

 The device 2 according to the invention also comprises a control unit 11 of the different operations performed.

 It will be noted that the different units that make up the device according to the invention and, more generally, the server, reside on the same machine.

 However, it may be envisaged to distribute these units on several server machines and to establish communications between these different units through the communication network.

 This distribution of units does not change the implementation of the invention, we will consider later, for reasons of simplicity, all units are located on the same machine that we call server.

 Returning now to the description of the process according to the invention.

sites http:llkbs.cs.tu-berlin.dehstewelvceglsequences.htm et http://standards.pictel.com/ftp/video-site/sequences/ ou d'autres séquences connues de l'homme du métier. Ces séquences possèdent des caractéristiques suffisamment hétérogènes pour constituer un ensemble représentatif des différentes séquences existantes que l'on rencontre en pratique. To constitute a set of reference image sequences, it is possible to use the so-called training sequences which are available on the

sites http: llkbs.cs.tu-berlin.dehstewelvceglsequences.htm and http://standards.pictel.com/ftp/video-site/sequences/ or other sequences known to those skilled in the art. These sequences have sufficiently heterogeneous characteristics to constitute a representative set of the various existing sequences that are encountered in practice.

 At each sequence of the set of selected sequences will be associated, on the one hand, two triplets of parameters of the model of

<Desc / Clms Page number 16>

 distortion and, on the other hand, one or more descriptors of the sequence, as will be seen later in the presentation.

 The search process of the parameter triplets (a, ss, y) of the distortion model for each of the training sequences of the set of selected sequences and for each coding mode envisaged will now be described.

 This will help explain how the distortion model chosen is adapted to each training video sequence and to a given coding method.

 This search begins with the actual coding of the desired training sequence that is to be modeled and the parameters of the distortion estimation model which will then be determined will be adapted to this sequence and to the coding method envisaged.

 The coding of a video sequence, as well as its decoding, will be described with reference to FIGS. 1a and 1b.

 The objective of coding is to obtain a representative number of observations of the distortion as a function of the variance. Each of the observations corresponding to a macro-block, we will code thirty images of the training sequence considered. If the images are, for example, in CIF format, then 30 * 396 observations will be obtained.

 Figures 2a and 2b respectively illustrate the coding and decoding of a color video sequence in YUV format.

 This sequence is a training sequence of all the selected training sequences.

 According to a preferred embodiment of the invention, the video decoder used for the implementation of this invention is in accordance with the visual part of the MPEG-4 standard (Information technology- Generic coding of audiovisual objects = part2: visual, ISO / IEC JTC1 / SC 29 / WG11 No. 3056, December 2000).

 In the example discussed here, the video will be considered as a single rectangular object.

<Desc / Clms Page number 17>

 Therefore, only a limited set of compression tools that allow texture decoding and motion compensation will be taken into account. The tools needed to decode the arbitrary form of an object are therefore not used here.

 It should be noted that the texture of an image is given by the values of the pixels of the image which are expressed on three components.

 The set of training video sequences are stored on the server 1 in uncompressed form.

 For example, the 4: 2: 0 YUV format is used, which means that the chrominance components (U, V) have four times fewer samples (pixels) than the luminance components (Y).

 FIG. 2a illustrates the coding method implemented by the coding unit 4 of FIG. 1 on each image of the video sequence in question.

 It will be noted that each image of the video sequence is divided into data blocks of 8x8 pixels.

 Each group of four blocks forms a macro-block of 16x16 pixels.

 The coding of an image is performed on each of the macroblocks according to two distinct coding modes: one denoted 1 for the Intra coding mode and the other coded P for the predictive coding mode or Inter.

 A sequence of 1 second images will be coded as follows: IPPIPPIPPIPPIPPIPPIPPIPPIPPIPP ...

 This means that the first image is coded according to the Intra mode, the next two according to the Inter mode, the next according to the Intra mode, the following two according to the Inter mode, etc.

 Thus, for a sequence of 30 frames per second, 10 images will be encoded in Intra mode and 20 images in Inter mode.

 In Intra mode, the coding is carried out block by block according to the steps E110 to E113 of FIG. 2a.

 In this encoding mode, each block constitutes an elementary unit of data which is encoded independently of the other blocks.

<Desc / Clms Page number 18>

 It will be recalled that the values of the pixels of each image of the image sequence are, for example, stored temporarily in the temporary storage unit 6 of FIG. 1 and constitute a video which is called the original video.

 These values can also be stored in the storage unit 3.

 The method for coding an Intra mode block starts with a step E110 during which the discrete cosine transform is calculated according to the known method described in the article entitled "Fast Algorithms for the Discrete Cosine Transform", E. Feig and S. Winograd, IEEE Trans. On Signal Proc., Vol.

40, No. 9, September 1992.

 Step E110 is followed by a step Et11 which provides for quantifying the transformed coefficients obtained during step E110.

 Step E111 is followed by a step E112 in which the quantized coefficients are ordered so that the variable length coding method used in the subsequent step E113 is as efficient as possible.

 This is done by using the scanning method described in the standard Information technology-Generic coding of audio-visual objects = part2: visual, ISOIIEC JTC 1 / SC 29 / WG11 N3056, December 2000.

 Step E112 is followed by a step E113 in which the statistical coding of the ordered coefficients is carried out according to the variable length coding method described in the standard Information technology- Generic coding of audio-visual objects = part2: visual, ISOIIEC JTC 1 / SC 29 / WG11 N3056, December 2000.

 At the end of step E113, the statistically coded coefficients are stored in the temporary storage unit 6 and are concatenated with the other coding information of the current image.

 In predictive or Inter mode, the coding is macro-block by macroblock where each macroblock constitutes an elementary unit of data which is coded independently of the others.

 Each of the macroblocks can be coded either in Intra mode or Inter mode.

<Desc / Clms Page number 19>

 If the macroblock is coded in Intra mode, then the coding is performed according to the steps E110 to E113 described above.

 If the macroblock is coded in Inter mode, the coding is performed according to the steps E115 to E117 which will be explained below.

 In step E115 a conventional motion estimation method is applied by comparison of macroblocks.

 The algorithms used are based on the simplifying assumption that the pixels of the same macroblock are animated by the same movement.

 The motion vector of a macroblock of a current image is then determined by finding the most resembling macroblock in the previous image in a preselected search window.

 If the dimensions of the search window have an influence on the calculation time of the motion estimation, the major problem related to the motion estimation which is carried out by comparison of macroblocks remains the determination of the distance between the macroblocks.

 A motion estimation method described in the article "Online motion estimation", L. Difdier, R., will preferably be used.

Kamdem, LIM University of Provence, Dec. 96.

 This method makes it possible to obtain a motion vector with two components of translational movement.

 To obtain a binary train conforming to the MPEG-4 standard, this motion vector is coded differentially with respect to a prediction made from the motion vectors of the already coded macroblocks of the immediate neighborhood of the current macroblock.

 Step E115 is followed by a step E116 during which the previously determined differential motion vector is statistically coded.

 The coefficients thus obtained are stored in the temporary storage unit 6 and are concatenated with the other coding information of the current image.

<Desc / Clms Page number 20>

 Once the movement for a macro-block has been estimated during the execution of the step E115, then, according to the step E117, a predicted macro-block is constructed from the macro-block of the previous image to which the motion vector associated with this macroblock is applied.

 Then, a so-called difference macro-block is calculated between the macroblock of the current image and the predicted macroblock.

 The variance of this macro-difference block is then measured.

 If the variance is greater than a predetermined threshold, then the macroblock of the current image is encoded in Intra mode.

 This macro-block will be coded in Inter mode otherwise.

 Step E117 is followed by a step E118 during which either the difference macro-block is coded according to the Intra mode described above, or the current macro-block is coded according to the Intra mode.

 The coefficients resulting from the Intra coding mode are stored in the temporary storage unit 6 and are concatenated with the other coding information of the macroblock of the current image.

 The parameters used during the quantization and variable length coding steps are defined for each macroblock, whether in the Intra or Inter mode.

 For this purpose, a conventional flow control method is used during a step E114.

 This method is used to control the rate allocation for each macroblock so as not to exceed the total bit rate allocated to the transmission of the compressed video.

 In the example described, the maximum flow rate value is set by the BP value.

 The quantization parameters will be set at the start of coding to a value between the lower and upper values of the quantization parameters set by the MPEG-4 standard.

 For example, a parameter QP equal to 20 will be used.

<Desc / Clms Page number 21>

 However, due to the distortion model normalization property, the model parameters will remain valid regardless of the QP value.

 The data coded and temporarily stored in the unit 6 are put in the MPEG-4 format, as indicated in the standard Information technology- Generic coding of audio-visual objects = part2: visual, ISO / IEC JTC 1 / SC 29 / WG11 N3056, December 2000.

 During coding, the control unit 11 of FIG. 1 uses the calculation unit 8 which calculates, for each macroblock of the image, its variance before the quantization operation.

 For macro-blocks coded in Intra mode, the variance calculation is performed directly from the macroblocks of the original sequence before step E111.

 For macroblocks encoded in Inter mode, the variance is calculated on the difference macroblocks in step E117.

 The variance information as well as the coding mode used for each macroblock are stored in the storage unit 10 of FIG.

 FIG. 2b illustrates the decoding method implemented by the decoding unit 5 of the device 2 of FIG. 1.

 FIG. 2b illustrates the decoding of a coded macroblock either in Intra mode, as indicated by steps 122 to 126, or in Inter or predictive mode as indicated by steps 127 to 132.

 The decoding algorithm starts with a step E119 during which the coding mode of the current macro-block is recovered.

 During step E120, a test is made to know if the macroblock has been coded in Intra mode.

 If so, the step E120 is followed by a step E121 which provides for recovering, from the storage unit 6, the statistical coefficients (variable length codes) related to this macroblock.

 Step E121 is followed by a step E122 which decodes these coefficients according to the coefficient table described in the standard.

<Desc / Clms Page number 22>

 Information technology- Generic coding of audio-visual objects = part2: visual, ISO / IEC JTC 1 / SC 29 / WG11 N3056, December 2000.

 The coefficients thus obtained form a set of two-dimensional data denoted QF [u] [v].

 During the next step E123, an inverse quantization operation is performed on the values of the data set QF [u] [v] to obtain the two-dimensional array F "[u] [v] according to the quantization tables described. in the standard Information technology- Generic coding of audiovisual objects = part2: visual, ISO / IEC JTC 1 / SC 29 / WG11 N3056, December 2000.

 In the next step E124, a saturation method is applied to the values of the coefficients of the two-dimensional array F "[u] [v] according to the standard Information technology- Generic coding of audio-visual objects = part2: visual, ISO / IEC JTC 1 / SC 29 / WG11 N3056, December 2000, so as to obtain the two-dimensional array F '[u] [v] whose values belong to the interval [-2 bits per pixel + 3, 2 bits per pixel + 3 - 1].

 The number of bits per pixel is preferably 8.

During the next step E125, a control operation of the saturated coefficients is performed so as to obtain the DCT coefficients
F [u] [v].

 During the next step E126, the values of the pixels of the current macro-block are reconstructed as a function of the DCT coefficients F [u] [v] using the decoding method described in the article "Fast Algorithms for the Discrete Cosine" Transform, E. Feig and S. Winograd, IEEE Trans. On Signal Proc., Vol. 40, No. 9, September 1992.

 These reconstructed values are stored in the storage unit 6.

 The decoding algorithm of a macro-block is given below.

 The values of the coefficients dc ~ scaler, quantizer scale and matrix W [l] [u] [v] used in this algorithm are values defined by the standard Information technology- Generic coding of audio-visual objects = part2: visual, ISO / IEC JTC 1 / SC 29 / WG11 N3056, December 2000. for (v = 0; v <8; v ++) {

<Desc / Clms Page number 23>

for (u=0; u<8;u++) { if (QF[u][v] == 0) F"[u][v] 0; else if ( (u==0) && (v==0) && (macroblockjntra) ) { F'1u][v] dç scaler * QF[u][v]; } else if ( macroblockjntra ) { F'1u][v] = ( QF[u][v] * W[0][u][v] * quantiser~scale )/32; } else F'1u][v] = ( ( ( QF[u][v] * 2 ) + Sign(QF[u][v]) ) * W{1][u][v] * quantiser~scale ) / 32; } } } } sum = 0; for (v=0; v<8;v++) { for (u=0; u<8;u++) { if ( F'1u][v] > 2 bits~per~pixel + 3 - 1 ) { F1u][v] 2 bits~per~pixel + 3 ~ .

for (u = 0; u <8; u ++) {if (QF [u] [v] == 0) F "[u] [v] 0; else if ((u == 0) && (v == 0) && (macroblockjntra)) {F'1u] [v] dc scaler * QF [u] [v];} else if (macroblockjntra) {F'1u] [v] = (QF [u] [v] * W [0] [u] [v] * quantize ~ scale) / 32;} else F'1u] [v] = (((QF [u] [v] * 2) + Sign (QF [u] [v ])) * W {1] [u] [v] * quantize ~ scale) / 32;}}}} sum = 0; for (v = 0; v <8; v ++) {for (u = 0; u <8; u ++) {if (F'1u] [v]> 2 bits ~ per ~ pixel + 3 - 1) {F1u] [v] 2 bits ~ per ~ pixel + 3 ~.

} else {if (F "[u] M <-2bits-per-pixel + 3) F1u] [v] = -2 bits ~ per ~ pixel + 3.

} else {F1u] [v] F'1u] [v]; }} sum sum + F1u] [v]; F [u] [v] F1u] [v]; }}

<Desc / Clms Page number 24>

avec u, v = 0, 1, 2, N-1, où x, y sont les coordonnées spatiales, u, v sont les coordonnées dans le domaine transformé, et

The method of discrete cosine transformation of a pixel f (x, y) of a block of size N x N is given by the following equation:

with u, v = 0, 1, 2, N-1, where x, y are the spatial coordinates, u, v are the coordinates in the transformed domain, and

Each pixel is represented by n bits (to represent one of the components Y, U or V) and the transformed coefficients are represented by (n + 4) bits. The amplitude of the DCT coefficients is in the range [- 2n + 3: + 2n + 3-1].

 Preferably, a fast method of DCT coding / decoding is used.

 This method, described in the article "Fast Algorithms for the Discrete Cosine Transform", E. Feig and S. Winograd, IEEE Trans. On Signal Proc., Vol.

où Y est le vecteur des coefficients DCT de dimension 8 etC8la matrice 8*8 de coefficients DCT. 40, No. 9, September 1992, allows to code / decode macroblocks of size 2m (here m = 3) according to the following equation:

where Y is the vector of the DCT coefficients of dimension 8 and C8 the matrix 8 * 8 of DCT coefficients.

<Desc / Clms Page number 25>

 This matrix can be factorized and, after some algebraic manipulations, this matrix is equal to C8 = P8 * K8 * B8, where P8 is a matrix of permutation of the signs of dimension 8 * 8, K8 and B8 are matrices of dimension coefficients 8 * 8 which have been defined to obtain a fast encoder / decoder.

 Following step E126, step E133 provides for storing and viewing the decoded macroblocks. The current image is reconstructed once all the macroblocks of that image are decoded and viewed.

 Returning to step E120, if the result of the test is negative, it means that the macroblock has been coded in Inter or predictive mode.

 In the next step E127, information is then retrieved from the storage unit 6 of FIG.

 This information is specific to the coding of the macroblock in Inter mode and is, for example, the statistical coefficients (variable length codes) related to this macroblock.

 Step E127 is followed by a step E128 which provides for decoding these statistical coefficients so as to obtain the differential motion vector.

 During the next step E129, the motion vector is reconstructed from the differential motion vector and the motion vector (Px, Py) of the macroblock of the preceding image disposed at the same spatial position.

The algorithm used is the following: r size = vop ~ fcode - 1 f = 1 r high = (32 * f) - 1; low = ((-32) * f); range = (64 * f); if ((f == 1) Il (horizontal ~ mv ~ data == 0))
MVDx = horizontal ~ mv ~ data;

<Desc / Clms Page number 26>

else MVDx = ( ( >4>s(horizontal~mv~data) - 1 ) * f ) + horizontaLmv~residual + 1; if (horizontal mv~data < 0) MVDx = - MVDx; if ( (f ==1 ) p (vertical mv data == 0) ) MVDy = vertical mv~data; else MVDy = ( ( /4bs(verticaL'T'v~data) 1 ) * f ) + vertical mv~residual + 1; if (vertical mv data < 0) MVDy = - MVDy; MVx = Px + MVDx; if ( MVx < low ) MVx = MVx + range; if (MVx > high) MVx = MVx - range; MVy = Py + MVDy; if ( MVy < low ) MVy = MVy + range; if (MVy > high) MVy = MVy - range;
Les paramètres du train binaire sont tels que les composantes MVDx et MVDy du vecteur de mouvement différentiel, appartiennent à l'intervalle [low; high].

else MVDx = ((>4> s (horizontal ~ mv ~ data) - 1) * f) + horizontaLmv ~ residual + 1; if (horizontal mv ~ data <0) MVDx = - MVDx; if ((f == 1) p (vertical mv data == 0)) MVDy = vertical mv ~ data; else MVDy = ((/ 4bs (verticaL'T'v ~ data) 1) * f) + vertical mv ~ residual + 1; if (vertical mv data <0) MVDy = - MVDy; MVx = Px + MVDx; if (MVx <low) MVx = MVx + range; if (MVx> high) MVx = MVx - range; MVy = Py + MVDy; if (MVy <low) MVy = MVy + range; if (MVy> high) MVy = MVy - range;
The parameters of the bitstream are such that the MVDx and MVDy components of the differential motion vector belong to the interval [low; High].

 The extreme values of this interval are chosen during the implementation of the decoder.

 Moreover, the components of the reconstructed motion vector, MVx and MVy, also belong to this interval. Values that

<Desc / Clms Page number 27>

 define this interval for motion vectors are defined by the standard Information technology- Generic coding of audio-visual objects = part2: visual, ISO / IEC JTC 1 / SC 29 / WG11 N3056, December 2000.

 Note that r ~ size, f, MVDx, MVDy, high, low and range are variables, while the horizontal data ~ mv-data, vertical ~ mv ~ data, horizontal ~ mv ~ residual and vertical ~ mv ~ residual are extracted from the binary train.

 The variable vop ~ fcode refers to the prediction mode used which is extracted from the bit stream.

 During this same step E129, the macroblock of the previous image is compensated in motion, that is to say that the macroblock is displaced from the motion vector that has just been reconstructed.

 During the next step E130, the information specific to the coding of the macro-block in Intra mode and, in particular, the statistical coefficients which represent the macro-block of difference are recovered.

 During the next step E131, the difference macro-block is reconstructed from these statistical coefficients according to the Intra decoding method described above, during the execution of the steps E122 to E126.

 Step E131 is followed by a step E132 which calculates the sum of the compensated macroblock of step E129 and the difference macroblock obtained in step E131.

 The step E132 is followed by a step E133 which organizes the storage of the decoded macroblocks in the unit 6.

 Once an image is decoded, the control unit 11 of Figure 1 uses the unit 8 which calculates for each macro-block of the image its real distortion. This is obtained by calculating the mean squared error EQM between the macroblocks of each image of the original video sequence and the macroblocks of the corresponding image of the coded / decoded sequence.

 The distortion information of each macroblock is then stored in the storage unit 10.

 As soon as a sufficient number of images of the training sequence in question have been coded / decoded according to the abovementioned algorithms, the unit

<Desc / Clms Page number 28>

 11 of FIG. 1 executes the algorithm for searching the parameters of the model for estimating the distortion.

 In the embodiment which follows, we consider thirty images of a video sequence that have been coded / decoded and we will look for two triplets of parameters (a, ss, y) of the model, the one specific to the macro- blocks coded in Inter mode and the other specific to macro-blocks coded in Intra mode.

 Except for the coding mode of the macro-blocks, which will be specific according to the macro-blocks envisaged, the process of searching for the parameters will be the same. In the following, we will consider the example of macroblocks coded in Inter mode, the principle is easily deduced for macroblocks coded in Intra mode.

 The values of the variance and the distortion of each macro-block which have respectively been obtained during the above-mentioned coding and decoding operations are stored in a table TAB forming a set of points which will subsequently be called a point cloud.

 The scatter plot represents the actual measured distortion of a set of image data (blocks of data that are here macroblocks) as a function of a measure of their energy (eg their variance).

 The adaptation of the model to the original data sequence under consideration and to the coding method envisaged consists in determining the triplet of parameters that will best represent the relationship between the distortion and the variance.

 In the example considered, the triplet of parameters that is sought is that which will make it possible to minimize the distance between, on the one hand, the scatterplot representing the measured real distortion of a set of data blocks (macro- blocks) of the image according to their variance and, on the other hand, the set of distortion / variance points obtained by applying the distortion estimation model.

The search for this minimum distance can be written as follows:

<Desc / Clms Page number 29>

 where D (S1, QP1) is the distortion provided by the model for a variance Sl and a quantization parameter QP1, D, (S ,, QP,) is the actual distortion for a given macro-block and Np is the number of macro-blocks considered in the minimization process (number of points distortion / variance).

 Since all the macroblocks are quantized in the same way, the quantization parameter QPi is common to all the macroblocks and is equal to QP.

 Although this is not mandatory, it is possible to reduce the complexity of the process of minimizing the distance by limiting the search space of the parameter triplet.

It is thus possible to determine a search space delimited by terminals for each of the three parameters a, ss, y which will then be searched for in the following intervals:

The determination of these three intervals is made from the set of training image sequences mentioned above and which are representative of the heterogeneity of the sequences to be processed.

 The set of training sequences comprises, for example, sequences with little movement and others characterized by a large movement, slightly textured sequences and others more textured ...

 It should be noted that this step of previously determining a limited search space will be performed only once outside the process of searching for parameters for a given video sequence.

 The process of limiting the search space will be described with reference to FIG.

 FIG. 3 illustrates an algorithm comprising different instructions or portions of software code corresponding to steps of the method according to the invention.

<Desc / Clms Page number 30>

 The computer program that is based on this algorithm is stored in the apparatus 1 of Figure 1 and in the programmable apparatus of Figure 9 which will be described later.

 The execution of this computer program thus makes it possible to implement the method according to the invention.

 The algorithm of FIG. 3 comprises a first step E300 during which each search interval is initialized for each of the three parameters considered in the interval [0, 100].

 During the next step E301, a variable k is initialized to the value 0, this variable serving to count the K training sequences envisaged for the aforementioned set.

 In the next step E302, the parameter triplet (a, ss, y) is searched for the training sequence of order k.

 The process of searching for these parameters will be explained later with reference to FIGS. 4 and 5.

 The algorithm then comprises a step E303 for storing the parameters thus determined.

 The next step E304 is to test the value of the variable k to see if all the training sequences have been taken into consideration.

 If not, the step E304 is followed by a step E305 which increments the variable k of the training sequences by one unit and this step is then followed by the step E302 already mentioned above.

 On the other hand, when all the training sequences have been taken into consideration in this algorithm, then the step E304 is followed by a step E306 in which the validity of the limits of the search intervals for the parameters considered is checked.

 The terminals are considered valid if none of the parameters that were determined in step E302 have reached one of these terminals.

 In this case, the step E306 is followed by a step E307 during which storage is carried out in the storage unit 10 of the

<Desc / Clms Page number 31>

#mm = min #k' k #max = max ak, Nmm - min k, Nmax - mk X k. FIG. 1, thus validated terminals which take the following values:

#mm = min #k 'k #max = max ak, Nmm - min k, Nmax - mk X k.

Ymm = mm Yk, Ymax = max Ykk
On the contrary, when the bounds are considered invalid, then step E306 is followed by a step E308 in which new bounds are determined.

* In this step, each terminal that has been reached is incremented by 1 if it is an upper bound of the range or decreased by a value of 1 if it is a lower bound.

 Step E308 is then followed by step E301 already described above.

 The algorithm of FIG. 4 comprises different instructions or portions of software code corresponding to steps of the method according to the invention. More particularly, this algorithm relates to the determination of the three parameters a, ss mentioned in step E302 of the algorithm of FIG.

 The computer program which is based on this algorithm is stored in the apparatus 1 of FIG. 1 and in the programmable apparatus of FIG. 9.

 The execution of this program makes it possible to implement the method according to the invention.

 The algorithm comprises a first step E400 of initialization of the parameter a to the lower limit of its search interval, namely, in the example considered, the value amax / 100.

 In the next step E401, a test is made to determine if the parameter a is less than or equal to the upper limit, [alpha] max, of its search interval.

 If not, step E401 is followed by step E402 which sets

<Desc / Clms Page number 32>

 end to the algorithm.

 On the contrary, when a is less than or equal to the upper bound of its search interval, then the next step E403 provides for initializing the parameter p to the value of the lower bound of its search interval, namely, in the example considered, the value pmax / 100.

 In the next step E404, a test is made to determine if the parameter p is less than or equal to the upper bound of its search interval, pmax.

 If not, go to the next step E405, which plans to increment the value of parameter a.

 In the example considered, the step of incrementing this parameter is fixed at the value [alpha] max / 100.

 Step E405 is then followed by step E401 already described above.

 On the contrary, when the parameter is less than or equal to the upper bound of its search interval, then step E404 is followed by a step E406. During this step, it is expected to initialize the parameter y to the value of the lower limit of its search interval, namely, in the example considered, the value [gamma] max / 100.

 In the next step E407, it is intended to perform a test to determine whether the parameter y is less than or equal to the upper bound of its search interval, ymax.

 If not, step E407 is followed by a step E408 in which the parameter ss is incremented according to an incrementing step which, in the example in question, is set to the value pmax / 100.

 This step is then followed by step E404 already described above.

 On the other hand, if the parameter y is less than or equal to the upper bound of its search interval, then step E407 is followed by a step E409. During this step, the distance between the points of the cloud of points mentioned above and the set of points obtained is determined by applying the model for estimating the distortion according to the invention.

<Desc / Clms Page number 33>

This step will be detailed below with reference to FIG.
Subsequent to step E409, step E410 provides for incrementing the parameter y according to an incrementing step which, in the example under consideration, is set to the value ymax / 100.

This step is then followed by step E407 already described above.
The operations performed in step E409 of the algorithm of FIG. 4 will now be described with reference to FIG. 5. This algorithm illustrates the process implemented for determining the distance between the points representing sub-clouds. and the average distortion / average variance points obtained by the distortion estimation model according to the invention, for a given parameter triplet.

 The algorithm of FIG. 5 comprises different instructions or portions of software code corresponding to steps of the method according to the invention.

 The computer program based on this algorithm is stored in the apparatus 1 of FIG. 1 and in the programmable apparatus of FIG. 9.

 The execution of this program thus makes it possible to implement the method according to the invention.

 The algorithm starts with an initialization step E500 of the variables n and Icumul at the value 0.

 The variable n will be used to count the N, points that will be processed in this algorithm and the variable turns will correspond to the sum of the distances between the points.

 Note however that instead of calculating the distance between each point of the point cloud at each point obtained by the distortion model according to the invention, it is possible to simplify the operations by dividing the point cloud real distortion / actual variance in several sub-clouds as provided in step E501 of the algorithm.

 This additional step is an improvement over the minimum distance calculation process that would not include this step.

 To do this, the number of sub-clouds Ni is set according to a criterion

<Desc / Clms Page number 34>

 reduction of computational complexity.

 In the example considered, this number is for example equal to 20.

 Note that each sub-cloud contains the same number of points Np / N ,,, where the number Np represents the number of points distortion / variance of the point cloud considered.

 The sub-clouds are obtained by traversing all the Np points represented by distortion coordinates, variance in the direction of the increasing variances.

 When a number Np / N, of points have been found, then an undernum is constituted and one starts counting again the points of the cloud of points remaining from the current variance in order to constitute the following undernature.

 Each sub-cloud of points is then represented by the average variance and the average distortion calculated on the points of this sub-cloud and these mean values then correspond, for each sub-cloud, to a point called representative point of this sub-cloud. .

 Thus, it will be determined later, as will be seen in the subsequent step E504, the distance between, on the one hand, the points representing the N, obtained sub-clouds and, on the other hand, the set of points distortion / variance obtained by the model and corresponding to the averages of the energies of each subnuage.

 Thus, the algorithm for searching the parameters is less complex to implement than if an exhaustive calculation of the distance of each point of the cloud distortion / variance at each point obtained by the model had to be performed.

 In the next step E502, a test is performed to determine if the variable n is less than the value Ni.

 If not, this step is followed by a step E503 which ends the execution of the algorithm of FIG. 5 and allows the change of one or more parameters, as provided for in step E410 and the steps following of Figure 4.

 When the result of the test performed in step E502 is positive, then

<Desc / Clms Page number 35>

 this step is followed by a step E504 during which the standard distortion model according to the invention is applied to the variance Sn of the point underling considered.

 We then determine the distance In given by the following formula: ln = #D (QP, Sn) -Dn (QP, Sn) #, where D (QP, Sn) and Dn (QP, Sn) are respectively the modeled distortion and the distortion of the representative of the sub-point cloud for the average variance Sn of this sub-point cloud.

 In the next step E505, the sum of the distance Icumul and the distance In which has been determined in the previous step is made.

 In the next step E506, a comparison is made of the current cumulative distance Icumul obtained in step E505 with the minimum value of the cumulative distance that has been determined with a previous parameter triple.

 In the case where the current cumulative distance is greater than the minimum cumulative distance Imin, and where [alpha] # / 100 or ssmax / 100 or [gamma] # [gamma] max / 100, then step E506 is followed by a step E507 that ends the determination of the distance and makes it possible to take into consideration the following triplet of parameters.

a = amax/100etp = Pmax/100ety = ymax/100 (première itération de l'algorithme), alors l'étape E506 est suivie d'une étape E508. On the contrary, if Icumul is inferior to Imin or if

a = amax / 100etp = Pmax / 100ety = ymax / 100 (first iteration of the algorithm), then the step E506 is followed by a step E508.

 During this step, a test is performed to check whether the cumulative distance has been determined on the Ni points representing the sub-clouds.

 If not, step E508 is followed by step E509 which increments variable n by one to account for the next sub-cloud representative point.

 This step is then followed by step E502 already described above.

<Desc / Clms Page number 36>

 If, on the other hand, the Ni representative points have been taken into account in this algorithm, then we go to step E510 which plans to store the values of the current parameters a, ss, y in the variables aopt, ssopt and yopt.

 During this step, the minimum value Inim of the cumulative distance Icumul obtained with the preceding parameter triple takes the value of the current cumulative distance which has just been determined.

 This ends the algorithm of Figure 5.

 It should be noted that this process is then performed again for the coded macroblocks according to the Intra coding mode. Thus, at the end of the search for the parameters making it possible to minimize the distance between the cloud of real points and the cloud of modeled points, two triplets of parameters will be available, one specific to the Intra coding mode and the other one specific to the Intra coding mode. Inter coding mode.

 As described with reference to FIGS. 3 to 5, two parameter triplets (a, ss, y) of the distortion model have been determined for a given drive sequence of all the K drive sequences.

 The algorithms of FIGS. 3 to 5 are performed iteratively for each of the K drive sequences.

 Once the two parameter triplets have been determined for a training sequence, it is intended to obtain from this sequence information representative of its content (descriptors).

 It is necessary to define relevant descriptors capable of representing the essential information of the sequence. It is proposed, for example, to use two descriptors: the first, the average variance of the sequence, will represent the spatial activity of the sequence while the second, the mean squared error between two successive images, denoted EQM, will represent the temporal activity of this sequence.

 Note that other descriptors can be used and that the number of descriptors can be increased without changing the principle of the invention.

<Desc / Clms Page number 37>

 Moreover, a single descriptor may, in certain cases, be sufficient to be representative of a sequence of images.

 To obtain the average variance information, the macro-block variance information is used which is stored in the storage unit 10 of FIG. 1. An average of the variance will then be computed on all the coded macroblocks of FIG. the sequence. This information constitutes the first descriptor of the training sequence considered and is stored in the storage unit 3 of FIG.

 The mean squared error (MSE) information is computed between two successive images of the original sequence during encoding provided that the current image is not the first one. At the end of the coding, the NI -1 average quadratic error information obtained is averaged.

This information then constitutes the second descriptor of the training sequence considered and is stored in the storage unit 3 of FIG.

 These operations are repeated for each training sequence of all the K training sequences.

 As explained above, all the training sequences (reference sequences) have thus been constituted by associating with each of the sequences, on the one hand, parameter values of the distortion model adapted to the sequence under consideration ( two triplets of parameters) and, secondly, two descriptors (mean variance and mean squared error) of this sequence.

 It was thus possible to classify the training sequences of the set according to their content.

 Each sequence is represented in a two-dimensional space: the first dimension is given by the average variance of the sequence and the second dimension is given by the average MSE between two successive images of the sequence.

 The objective of the description of the method according to the invention which follows is to search among the training sequences of the set, that which is closest to the current sequence to be coded. We will then consider

<Desc / Clms Page number 38>

 that a good approximation of the parameters of the model of the current sequence can be given by the parameters of the selected sequence.

 A graphic representation of this selection procedure is given in FIG. 6. In this figure, the sequence 11 has been selected as being closest to the current sequence among a set of eleven training sequences.

 The analysis of the current image sequence according to the invention and which aims to determine the parameters of the distortion model adapted to it is implemented during the coding of the current sequence and is illustrated by the algorithm of FIG. 7. For example, the current sequence is coded under a rate constraint Rc, expressed in bits / s, and which is transformed into an image constraint RI = Rc / Fr, where Fr is the temporal frequency of the sequence .

 It is assumed that the current sequence is stored in the storage unit 6 of FIG. 1 in YUV format and that its temporal frequency is 30 Im / s. It is also assumed that the coding process will retain the time frequency of the sequence to be encoded and the rate constraint RI will be that used in the rate control algorithm.

 The algorithm of FIG. 7 comprises various instructions or portions of software code corresponding to steps of the method according to the invention which are carried out in real time, unlike the algorithms of FIGS. 3 to 5 (search parameters for the training sequences ) and obtaining the descriptors of the training sequences.

 The computer program noted "Prog" which is based on this algorithm is stored in the search unit 9 of Figure 1 and executed by the unit 9 under the control of the control unit 11, which allows to put implement the method according to the invention.

 This program is also stored in the apparatus of Figure 9.

 The algorithm comprises a first step E700 for initializing the nframe variable to 0. This variable is used to count the coded images.

<Desc / Clms Page number 39>

 Step E700 is followed by step E701 which extracts the nframe @th image from the sequence.

 Step E701 is followed by step E702 which verifies the number of coded pictures. If, for example, a sufficient number of images have been encoded (this number is determined empirically and is, for example, equal to 30), it is considered that the number of distortion / variance observations obtained is sufficient to implement search for parameters adapted to the current sequence.

 In this case, the step E702 is then followed by the step E703 which adapts the model to the current sequence by applying the algorithms of FIGS. 4 and 5 described above.

 Step E703 is followed by step E704 which encodes the current image.

The flow control algorithm uses the model parameters defined in step E704. Note that the step E704 is optional and will only be implemented if the apparatus performing the coding has sufficient resources to perform the search parameters as provided for the algorithms of Figures 4 and 5. In the opposite case, the step E702 will always be followed by step E705.

 If the number of coded pictures is smaller than Nframe, step E702 is followed by step E705 which checks the number of the current picture.

 If this number is 0, we go directly to the coding of the current image in step E704 and, for example, there will be no flow control for this image. This will be coded in I mode with a quantization parameter equal to 13.

 If, on the contrary, the number of the coded picture is different from 0, then the step E705 is followed by the step E706 which calculates the variance Vcurr of the current picture.

 Step E706 is followed by step E707 which calculates the mean squared error, Ecurr, between the current image and the image that precedes it.

 Steps E706 and E707 aim to obtain information representative of the current sequence.

<Desc / Clms Page number 40>

 Step E707 is followed by step E708 which searches for the training sequence of the set of K drive sequences that is closest to the current sequence from the point of view of the content.

 The detail of this step is provided by the description which follows, made with reference to FIG. 8 which illustrates an algorithm completing that of FIG. 7.

 The algorithm of FIG. 8 comprises a first initialization step E800 of the variables Dmun 'n and Idopt which respectively represent the minimum distance between a training sequence of the set and the current sequence, the number of the sequence of which is compared with the current sequence and the identifier of the training sequence minimizing the distance to the current sequence.

où Vn et En sont respectivement la variance et l'erreur quadratique moyenne de la séquence d'entraînement (séquence testée) à laquelle la séquence courante est comparée et qui sont stockées dans l'unité de stockage 3 de la figure 1. Note that in the exemplary embodiment of the invention, the distance metric used is the distance L2 calculated according to the following formula:

where Vn and En are respectively the variance and the mean squared error of the training sequence (tested sequence) to which the current sequence is compared and which are stored in the storage unit 3 of Figure 1.

 Step E800 is followed by step E801 which calculates, according to the above formula, the distance Dn between the tested training sequence and the current sequence.

Step E801 is followed by step E802 which checks if distance Dn is less than Dmun #
If so, step E802 is followed by step E803 in which the variable Idopt takes the value n and Dmun takes the value of Dn just determined.

 Step E802 and step E803 are both followed by step E804 which tests whether the K drive sequences have been tested.

 If not, step E804 is followed by step E805 which increments variable n by one unit to proceed to the next training sequence.

<Desc / Clms Page number 41>

 Step E805 is then followed by step E801 already described.

 If all the sequences have been tested, the search algorithm of FIG. 8 is terminated by a step E806.

 During this step, the parameters of the model of the training sequence which minimized the distance Dmun (sequence identified by the variable Idopt) are assigned to the current sequence.

 The parameters of the distortion estimation model are thus determined in a manner adapted to the current sequence when there is a number of observations on the current sequence to be coded that is insufficient or even nil.

 Returning to FIG. 7, step E708 is followed by step E704 already described above which proceeds to coding the current image. In this case, the rate control algorithm uses the distortion model with the parameters defined in step E708 (FIG. 8).

 In any case, step E704 is followed by step E709 which verifies the number of coded pictures.

 If all the images have been encoded, the algorithm ends in step E710.

 In the opposite case, step E709 is followed by step E711 which increments the Nframe variable by one unit to proceed to the next image.

 Step E711 is then followed by step E701 already described above.

 With reference to FIG. 9, an example of a programmable apparatus embodying the invention is described. This device is suitable for processing image sequences.

 The server of FIG. 1 takes, for example, the shape of the apparatus of FIG. 9 which includes a device according to the invention, that is to say having all the means necessary for the implementation of Embodiments of the invention (means for obtaining information representative of a current sequence, means for comparing information, means for selecting the nearest training sequence, means for assigning parameter values ), or itself constitutes such a device according to the invention.

 According to the embodiment chosen and shown in FIG. 9, an apparatus embodying the invention is for example a microcomputer 500

<Desc / Clms Page number 42>

 or a workstation connected to different peripherals, for example a digital camera 501 (or a scanner, or any means of acquiring or storing an image) connected to a graphics card and supplying the device with video data to be modeled and to code. The apparatus 500 comprises a communication bus 502 to which are connected: a central processing unit 503 (microprocessor), which performs the function of the control unit 11 of FIG. 1, a read-only memory 504, which can comprise the program "Prog" which is based on the algorithms of FIGS. 2a, 2b, 3 to 5.7 and 8, - a random access memory 506, comprising registers 507 adapted to record variables created and modified during the execution of the program above, and in particular the variables a, ss, y, aopt, ssopt, [gamma] opt, turn Imin, n, amine [alpha] max, ssmin, ssmax, [gamma] min, Ymax, Ni, TAB, D (QP, Sn), Dn (QP, Sn), QP, Sn, Dmin, Dn, Vn, En, Vcurr, Ecurr, nframe, Nframe, K, Idopt and Nmax mentioned with reference to the preceding figures, - a screen 508 making it possible to visualize the data to be processed and / or to act as a graphical interface with the user who can interact with the program according to the invention, using a keyboard 510 or any other means such as a pointing device, not shown, such as for example a mouse or an optical pen, a hard disk 512 which may comprise the aforementioned "Prog" program, a floppy disk drive 514 adapted to receive a floppy disk 516, and read or write data to it, - a communication interface 518 connected to a communication network 520, for example the Internet network, the interface being able to transmit and receive data.

 The communication bus allows communication and interoperability between the various elements included in the microcomputer 500 or connected to it. The representation of the bus is not limiting and, in particular, the central unit is able to communicate instructions to any element

<Desc / Clms Page number 43>

 of the microcomputer 500 directly or via another element of the microcomputer 500.

 The executable code of the program denoted "Prog" allowing the programmable apparatus to implement the methods for modeling the distortion of the training sequences (FIGS. 3 to 5), of adapting the model to a sequence of images current (FIGS. 7 and 8), coding / decoding (FIGS. 2a and 2b) according to the invention, can be stored for example in the hard disk 512 or in the read-only memory 504 as represented in FIG. 9.

 Although only one program is identified, it is possible to have several programs or sub programs to implement the invention.

 According to one variant, the diskette 516 may contain compressed and stored data as well as the executable code of the program or programs according to the invention which, once read by the apparatus 500, will be stored in the hard disk 512.

 In the second variant, the executable code of the program or programs may be received via the communication network 520, via the interface 518, to be stored identically to that described above.

 Floppies can be replaced by any information medium such as, for example, a compact disc (CD-ROM) or a memory card. In general, an information storage means, readable by a computer or by a microprocessor, integrated or not, possibly removable, is adapted to store a program whose execution allows the implementation of the method according to the invention.

 More generally, the program can be loaded into one of the storage means of the device 500 before being executed.

 The central unit 503 will control and direct the execution of the instructions or portions of software code of the program or programs according to the invention, which instructions are stored in the hard disk 512 or the read only memory 504 or else in the other storage elements. supra. When powering up, the program (s) stored in a memory

<Desc / Clms Page number 44>

 nonvolatile, for example the hard disk 512 or the ROM 504, are transferred into the RAM RAM 506 which will then contain the executable code of the program or programs according to the invention, as well as registers for storing the variables necessary for the implementation in work of the invention.

 It should be noted that the data processing apparatus comprising a device according to the invention may also be a programmed apparatus.

 This device then contains the code of the computer program or programs for example frozen in a specific application integrated circuit (ASIC).

 It will be noted that the apparatus of FIG. 9 which contains the program "Prog" previously mentioned may be the device 2 of FIG.

Claims (26)

A method of determining parameters of a distortion estimation model adapted to a current image sequence to be encoded, characterized in that, at each image sequence of a set of reference image sequences associated information representative of the image sequence considered and the distortion model parameter values adapted to this sequence of images, the method comprising the following steps: obtaining information representative of the current image sequence, comparing information representative of the current image sequence with information representative of at least some of the reference image sequences of the set, according to the result of the comparison, selecting the image sequence reference whose representative information is closest to that of the current image sequence, - assignment to the current image sequence Te distortion model parameter values associated with the selected reference image sequence.  2. Method according to claim 1, characterized in that the reference image sequences are sequences of training images, that is to say sequences of test images which form a sufficiently heterogeneous set for represent substantially all types of image sequences encountered in practice.  3. Method according to claim 1 or 2, characterized in that the comparison step provides for measuring a distance between the current image sequence and at least some of the reference image sequences of the set in a test space. dimension n, where each image sequence is represented in this space by n coordinates each corresponding to information of different type which are representative of the sequence in question. <Desc / Clms Page number 46>  4. Method according to claim 3, characterized in that the reference image sequence whose representative information is closest to that of the current image sequence is that which minimizes the distance measured between the current image sequence and the reference image sequences considered.  5. Method according to one of claims 1 to 4, characterized in that the information representative of a sequence of images are information representative of the spatial and / or temporal activity of the sequence.  6. Method according to claim 5, characterized in that the information representative of the spatial activity is the variance of the image sequence.  7. Method according to claim 5, characterized in that the information representative of the temporal activity is an average value E of the different mean squared error values calculated between several pairs of successive images of the image sequence.  8. Method according to claims 3 and 5 to 7, characterized in that the distance Dn measured between, on the one hand, the current image sequence represented by a variance Vcurr and a squared mean error Ecurr and on the other hand , a reference image sequence represented by a variance Vn and a mean squared error En is expressed by the formula:

9. Method according to one of claims 1 to 8, characterized in that the distortion estimation model comprises at least one logarithmic operator weighted by at least one exponential operator.  The method according to claim 9, characterized in that the weighted distortion estimation model applies to information representative of the variance of a data block of the image.  11. The method as claimed in claim 10, characterized in that the model for estimating the weighted distortion is written as follows:

 where a, ss, y are the parameters of the model and S is the variance of a block of <Desc / Clms Page number 47>  where QP represents a quantization parameter of the data block under consideration.

5 10 15 20 25 30 image data, S being the normalized variance and with:  12. Device for determining parameters of a distortion estimation model adapted to a current image sequence to be encoded, characterized in that, at each image sequence of a set of reference image sequences associating information representative of the image sequence considered and the parameter values of the distortion model adapted to this sequence of images, the device comprises: means for obtaining information representative of the current image sequence means for comparing information representative of the current image sequence with information representative of at least some of the reference image sequences of the set; means for selecting the sequence of image images; reference whose representative information is closest to that of the current image sequence based on the result of the comparison.  means for assigning to the current image sequence the parameter values of the distortion model associated with the selected reference image sequence.  13. Device according to claim 12, characterized in that the reference image sequences are sequences of training images, that is to say sequences of test images which form a sufficiently heterogeneous set for represent substantially all types of image sequences encountered in practice.  Device according to Claim 12 or 13, characterized in that the comparison means provide for measuring a distance between the current image sequence and at least some of the reference image sequences of the set in a dimension space. n, where each sequence <Desc / Clms Page number 48>  of images is represented in this space by n coordinates each corresponding to information of different type which are representative of the sequence in question.  Device according to claim 14, characterized in that the reference image sequence whose representative information is closest to that of the current image sequence is that which minimizes the distance measured between the current image sequence. and the reference image sequences considered.  16. Device according to one of claims 12 to 15, characterized in that the information representative of a sequence of images are information representative of the spatial and / or temporal activity of the sequence.  17. Device according to claim 16, characterized in that the information representative of the spatial activity is the variance of the image sequence.  18. The device as claimed in claim 16, characterized in that the information representative of the temporal activity is an average value E of the various mean squared error values calculated between several successive image pairs of the image sequence.  19. Device according to claims 14 and 16 to 18, characterized in that the distance Dn measured between, on the one hand, the current sequence of images represented by a Vcurr variance and an Ecurr average squared error and on the other hand , a reference image sequence represented by a variance Vn and a mean squared error En is expressed by the formula:

20. Device according to one of claims 12 to 19, characterized in that the distortion estimation model comprises at least one logarithmic operator weighted by at least one exponential operator.  21. Device according to claim 20, characterized in that the weighted model for estimating the distortion applies to information representative of the variance of a data block of the image. <Desc / Clms Page number 49>  22. Device according to claim 21, characterized in that the model for estimating the weighted distortion is written:

 where a, ss, y are the parameters of the model and S is the variance of a block of data of the image, S being the standardized variance and with:

 where QP represents a quantization parameter of the data block under consideration.  23. Communication device, characterized in that it comprises a device for determining parameters according to one of claims 12 to 22.  A computer-readable or microprocessor-readable information storage means having code instructions of a computer program for performing the steps of the method of determining parameters according to one of claims 1 to 11.  25. Removable, partially or completely readable information storage medium by a computer or a microprocessor comprising code instructions of a computer program for carrying out the steps of the method of determining parameters according to one of the claims. 1 to 11.  Computer program loadable in a programmable apparatus, characterized in that it comprises sequences of instructions or portions of software code to implement the steps of the method of determining parameters according to one of claims 1 at 11, when this computer program is loaded and executed by the programmable apparatus.