EP1468573A1

EP1468573A1 - Method for the synchronization of two digital data flows with identical content

Info

Publication number: EP1468573A1
Application number: EP03709901A
Authority: EP
Inventors: Alexandre Joly
Original assignee: Telediffusion de France ets Public de Diffusion
Current assignee: Telediffusion de France ets Public de Diffusion
Priority date: 2002-01-24
Filing date: 2003-01-23
Publication date: 2004-10-20
Also published as: US7043684B2; CA2474197C; CA2474197A1; FR2835124A1; WO2003063514A1; FR2835124B1; US20050166101A1

Abstract

The invention relates to a method for synchronizing two digital data flows with identical content, consisting of the following steps: a) at least two characteristic numbers which express at least one characteristic parameter of the content are generated at given intervals for each of the two digital data flows S1 and S2; b) points D1 and D2 which represent at least one of said characteristic parameters in an at least two-dimensional space are generated from said numbers for each of the two flows S1 and S2. Point D1 corresponds to flow S1 while point D2 corresponds to flow S2. Said points are located in a chronological range T defining paths that are representative of data flows S1 and S2 which are to be synchronized; c) the chronological ranges of duration T assigned to the digital data flows S1 and S2 are offset from each other by calculating an overlay criterion of said paths, an optimal value of which represents the targeted synchronization; d) said offset between the chronological ranges, which corresponds to said optimal value, is selected as the value which is representative of the synchronization.

Description

METHOD FOR SYNCHRONIZING TWO DIGITAL DATA STREAMS OF THE SAME CONTENT.

The invention relates to a method for synchronizing two digital data streams of the same content, and in particular a reference stream corresponding for example to a transmission by a remote transmission system, and a stream received with possible degradations. , usable in particular for an evaluation of the quality of the transmission.

In the field of broadcasting audiovisual signals, the introduction of digital offers new perspectives and makes it possible to offer a greater number of services to users.

During the various stages necessary for broadcasting, the signals are modified. In fact, the technical constraints imposed, in terms of bit rate or bandwidth for example, imply the appearance of characteristic degradations during difficult transmission conditions. To be able to ensure quality of service, it is necessary to develop tools and instruments which make it possible to measure the quality of the signals and to estimate the extent of the degradations introduced if necessary. Many measurement methods have been developed for this purpose. The majority of these are based on a comparison of the signal (called reference signal) present at the input of the system to be studied, with the signal (called degraded signal) obtained at the output of the system. Certain so-called "reduced reference" methods use a comparison between numbers calculated on the reference and on the degraded signal instead of directly using the samples of the signal. In both cases, it is necessary to time synchronize the signals to perform the quality assessment by comparison. Figure 1 illustrates the general principle used by these methods. If the synchronization of the signals can be easily carried out in simulation or when the system to be studied is reduced (case of an encoder-decoder or "coded" for example) and not distributed geographically, this is very different in the case of a system complex, especially in the case of the supervision of a broadcasting network. Thus, this synchronization step is often a critical step in the quality algorithms.

Beyond the quality measurement applications in a broadcasting network, the method presented in this report is applicable whenever a time synchronization between two audio signals and / or video should be made, especially in the context of a distributed and extended system.

Various techniques can be used to synchronize digital signals in time. The objective is to match a part of the degraded signal SD with a part of the reference signal SR. Figure 2 illustrates this problem in the case of two audio signals. This involves determining a DEC offset allowing the signals to be synchronized.

In the case of an audio signal, the part (or element) to be matched is a time window, that is to say an arbitrary duration T of the signal.

We can divide existing methods into three classes:

• Correlation approach in the time domain I: This is the most common approach. It consists in comparing the samples of the two audio signals SR and SD to be synchronized, based on their content. Thus, the use of the standardized intercorrelation function for example, between SR and SD, makes it possible to search for the maximum resemblance over a given time range T, for example of more or less 60 ms, that is to say 120 ms. The synchronization precision obtained is potentially to the nearest sample.

• Correlation approach in the time domain using marker signals: the methods which use this principle aim to overcome the need for significant variations in the signal. For this, a specific marker signal is inserted into the audio signal SR. This signal is designed to allow robust synchronization. Thus, an intercorrelation procedure identical to the previous one can be applied between the marker signals extracted from the SR and SD signals to be synchronized, which in principle allows robust synchronization whatever the content of the audio signal. In order to be applicable, the operation of inserting the marker signal must be such that the content of the audio signal is modified in the least perceptible manner possible. Several techniques can be used, including "watermarking", to insert these marker signals or specific patterns.

• Synchronization by time marks: The methods of this class can only be used in the case where the signals are associated with time marks. So the process is based on the searches, for each reference mark of the reference signal, the one which is closest in the sequence of time reference marks associated with the degraded signal.

An efficient signal synchronization process is characterized by a compromise between: its precision, ie the maximum error made by synchronizing the two signals. In particular, the method can be sensitive to the content of the signals, its computation complexity, and finally the volume of data necessary to perform synchronization.

The main disadvantage of the most conventionally used techniques (the correlation approach mentioned above) is the computing power required. Indeed, it becomes very important as soon as the search range T increases (Figure 2). In addition, the other major drawback is the need for content that is constantly evolving significantly. Depending on the type of signals analyzed, this assumption is not always true. The content of the signals therefore has a direct influence on the performance of the method. In addition, to use this type of approach on complete time signals, it is necessary to have the two signals SR and SD at the point of comparison: it is a very strong constraint, which it is not possible to satisfy for certain applications such as monitoring an operational broadcasting network.

The second approach (correlation with marker signals) has the characteristic of modifying the content of the audio signal in order to insert marker signals, without guaranteeing the impact of this operation on the quality: the measurement process therefore influences the measurement herself. Whatever the performance for the synchronization of two signals, this approach is not always suitable for a real quality assessment application. Finally, in the case of synchronization by time marks, it is the need to have time marks which is the major drawback. Only a few applications can use such a technique. Indeed, the accuracy of the time marks is not always satisfactory. Within the framework of the surveillance of a broadcasting network and due to the multiple constraints undergone by the transported signals, and multiple devices crossed (encoders, multiplexers, trans-multiplexers, decoders, ...), there is no very strict relationship between audio signals and time marks. This solution then does not reach the precision necessary for a quality measurement application with reference. The object of the present invention is to define a method making it possible to achieve synchronization, of a level of precision which can be chosen, of a lower complexity than the existing methods, by combining the advantages of several approaches. According to the invention, “rough” synchronization delimits a range of error, the duration of which is compatible with the subsequent use of conventional “fine” synchronization methods if extreme precision is required.

The originality of the proposed method is to achieve synchronization from at least one characteristic parameter, calculated on the signals SD and S _R , defining a multidimensional trajectory, to deduce therefrom a synchronization of the signals themselves. Since the method uses the content of the signals, this temporal content must vary continuously to ensure optimal synchronization, as does the temporal correlation of the state of the prior art. The advantage of the method is to make a correlation by means of a multidimensional trajectory, obtained in particular by combining several characteristic parameters, which makes it possible to obtain a reliability greater than that of known methods.

A fundamental advantage of the method proposed by the invention is that it requires only little data to perform synchronization, which is very useful in the context of the supervision of a broadcasting network. Indeed, in this context, it is generally not possible to have the two complete signals SR and S _D in the same place. Therefore, it is not possible to use a conventional time correlation approach. In addition, in the context of a qualitometry application, the second approach by correlation with marker signals is not easily applicable since it acts on the quality of the signals. On the contrary, the synchronization method according to the invention is compatible with qualitometry techniques based on the comparison of parameters calculated on the signals. The data representative of this or these characteristic parameters are easily transportable by a digital link to the comparison points. This digital link advantageously uses the same transmission channel as the audio signal; alternatively, a digital link dedicated can be chosen. In a particular embodiment for a quality measurement application, the data used to perform synchronization are thus derived from one or more quality measurement parameters. Furthermore, a so-called coarse synchronization is carried out from data D1 and D2 calculated at intervals of Δ = 1024 audio samples. Fine synchronization can be achieved with data D1 calculated at intervals of Δ = 1024 audio samples, and with data D2 calculated at intervals of r <Δ, for example r = 32 audio samples. Thus, the method allows in this case to perform a fine synchronization 32 times more precise than the transmission interval of the quality measurement parameters.

The process is therefore naturally integrated into a digital television quality monitoring system on an operational broadcasting network. However, it is applicable whenever a time synchronization between two signals must be carried out. The proposed method thus makes it possible to achieve synchronization with a precision that can be chosen and thus obtain a very reduced range of uncertainty. Advantageously, it will use at least part of the parameters already calculated to study the quality of the signal. Being able to start from an extended search range is also an advantage, especially since the robustness of synchronization increases with the width of this starting range.

The proposed method therefore does not require having external time cues to the audio signals. The signal to be synchronized also does not need to be changed, which is an important requirement in a quality measurement application.

The invention thus relates to a method of synchronization between two digital data streams of the same content characterized in that it implements: a) for each of the two digital data streams Si and S ₂ , generate at given intervals of at least two characteristic numbers expressing at least one parameter characteristic of their content; b) for each of the two flows Si and S ₂ , generate from said numbers of points Di and D ₂ associated respectively with each of said flows and representing at least one said characteristic parameter in a space with at least two dimensions, the points Di d 'on the one hand and D ₂ on the other hand, which belong to a time range T, defining trajectories representative of the data flows Si and S ₂ to be synchronized; c) shifting, with respect to each other, the time periods of duration T assigned to the digital data streams Si and S ₂ by calculating a criterion for superimposing said trajectories, an optimal value of which represents the synchronization sought; d) choose as the value representative of the synchronization, the offset between the time ranges corresponding to said optimal value. Advantageously, the method is characterized in that one of the digital data streams is a reference stream Si and in that the other data stream is a stream S ₂ received through a transmission system and in that the numbers characteristics of the reference flow Si are transmitted with the latter, whereas the numbers characteristic of the received flow S ₂ are calculated on reception.

According to a first variant, the method is characterized in that c) implements: d) calculating a distance D between on the one hand a first trajectory represented by the points Di belonging to a first time range of duration T, and d ' on the other hand, a second trajectory represented by the points D ₂ belonging to a second time range of duration T, said distance D constituting said superposition criterion; c2) shifting, with respect to each other, said first and second time periods of duration T until a minimum of the distance D which constitutes said optimum value is obtained;

The distance D can be an arithmetic mean of the distances d, for example Euclidean distances, between the corresponding points Di, D ₂ of the two trajectories.

According to a second variant, the method is characterized in that c) implements: d) calculating a correlation function between the corresponding points (Di, D ₂ ) of the two trajectories, said correlation function constituting said superposition criterion; c2) shifting said first and second time periods of duration T relative to each other until a minimum of the correlation function which constitutes said optimal value is obtained. According to a third variant, the method is characterized in that c) implements: d) transforming each trajectory into a series of angles between successive segments defined by the points of the trajectory. c2) shifting, with respect to each other, said first and second time periods of duration T until a minimum is obtained between the differences between the values of angles obtained for homologous segments of the two trajectories, said minimum constituting said value optimal. The method can be characterized in that c) implements: d) transforming the two trajectories into a series of surfaces intercepted by the successive segments defined by the points of said trajectories, the total surface intercepted constituting said superposition criterion. c2) shifting the time periods of duration T relative to each other until a minimum of said total intercepted surface is obtained, which constitutes said optimal value.

A said given interval can be equal to Δ for one of the two data streams and to r <Δ for the other data stream and this in order to improve the precision of the synchronization.

The method can be characterized in that the generation of said characteristic numbers implements for a reference audio stream and for the transmitted audio stream, the following steps: a) calculating for each time window the power spectral density of the audio stream and apply a filter representative of the attenuation of the inner and middle ear to obtain a filtered spectral density, b) calculate from the filtered spectral density the individual excitations using the frequency spreading function in the scale basilar, c) determine from said individual excitations the compressed loudness using a function modeling the non-linear frequency sensitivity of the ear, to obtain basilar components, d) separate the basilar components into n classes (by example with n <5), preferably in three classes, and calculate for each class a number C representing the sum of the fr quences of this class, the characteristic numbers being constituted by the numbers C. Alternatively, there are <<characteristic numbers which are generated from said numbers C. n is chosen significantly less than the number of audio samples of a time window, for example less than 0.01 times this number of samples.

The method can be characterized in that the generation of a said characteristic number implements, for a reference audio stream and for a transmitted audio stream, the following steps: a) calculating N coefficients of a prediction filter by a autoregressive modeling, b) determining in each time window the maximum of the residue by difference between the predicted signal using the prediction filter and the audio signal, said maximum of the prediction residue constituting a said characteristic number. The method can be characterized in that the generation of said characteristic numbers implements for a reference audio stream and for a transmitted audio stream, the following steps: a) calculating for each time window the power spectral density of the audio stream and apply a filter representative of the attenuation of the inner and middle ear to obtain a frequency spreading function in the basilar scale, b) calculate individual excitations from the frequency spreading function in the scale basilar, c) obtain from said individual excitations the compressed loudness from a function modeling the non-linear frequency sensitivity of the ear, to obtain basilar components, d) calculate from said basilar components N 'prediction coefficients a prediction filter by an autoregressive modeling, e) generate for each time window at at least one characteristic number from at least one of the N 'prediction coefficients.

The characteristic numbers can consist of between 1 and 10 of said prediction coefficients, and preferably between 2 and

5 of said coefficients. For a digital audio signal, a characteristic number can be instantaneous power and / or power spectral density and / or bandwidth.

For a digital video signal, a characteristic number can be the continuous coefficient of the transform of at least one area of an image belonging to the data stream by a linear and orthogonal, block or global transform, and / or the contrast d '' at least one area of the image, and / or the spatial activity SA of at least one area of an image or its temporal activity (the latter being defined by comparison with a previous image), and / or the average brightness of at least one area of an image.

Points can be generated from at least two characteristic numbers from a single characteristic parameter.

Alternatively, said points can be generated from at least two characteristic numbers originating from at least two characteristic audio and / or video parameters.

The method can be characterized in that the data stream comprises video data and audio data and in that it implements a first video synchronization from points D'i and D ' ₂ associated with at least one parameter video characteristic corresponding to said video stream, and a second audio synchronization from points D "ι and D" ₂ associated with at least one audio characteristic parameter corresponding to said audio stream.

It can then be characterized in that it has a step of determining the synchronization trigger between the video stream and the audio stream by difference between said offsets obtained for the video stream and for the audio stream.

Other characteristics and advantages of the invention will appear better on reading the description, in conjunction with the appended drawings in which:

- Figure 1 illustrates the architecture of a system for measuring the quality of an audio signal according to the prior art;

- Figure 2 illustrates the problem of synchronizing audio signals; - Figure 3 illustrates an increase in the precision of synchronization that can be obtained in the context of the present invention;

- Figure 4 gives an example of two two-dimensional trajectories of audio signals to be synchronized, in the case where r = Δ 12;

- Figures 5 and 6 illustrate two variants of synchronization between two paths assigned to the two data streams;

- Figure 7 is a block diagram of a synchronization method according to the invention (synchronization by trajectory); - Figures 8 to 10 illustrate a synchronization according to the invention, the significant parameter being an audio perceptual parameter, Figures 10a and 10b illustrating the situation before and after synchronization of two trajectories.

FIG. 11 illustrates an implementation of the method using as a characteristic parameter an autoregressive modeling of the signal, with linear prediction coefficients.

The first step of the method corresponds to the calculation of at least two characteristic numbers from one or more characteristic parameters over all the time windows of the signals to be synchronized, over the desired synchronization range: each number is therefore calculated all the durations Δ (Cf. Figures 2, and 3), which gives N = T / Δ parameters. This or these numbers should be, if possible, simple to calculate so as not to require too much computing power. Each characteristic parameter can be of any kind and can be represented for example by a single number. For example, for an audio signal a parameter characteristic of the content of the signal may be the bandwidth.

Having the parameters only all the durations Δ makes it possible to greatly reduce the amount of data originating from the reference signal S _R necessary for synchronization. However, the precision of the synchronization which will be achieved is necessarily limited: the uncertainty with respect to an ideal synchronization, that is to say to the nearest signal sample, is ± Δ / 2. If this uncertainty is too great, an alternative is to reduce the period Δ: however, this modification is rarely possible because on the one hand the calculation of the characteristic number (s) is called into question, and on the other hand it leads increase the amount of data needed for synchronization. In the particular embodiment where the parameters also serve to evaluate the quality by comparison of the parameters Pi and

P'-i, any synchronization error greater than the value r _0, called resolution of the parameter, will not allow the degradations introduced to be estimated (Situation A in Figure 3).

In order to obtain an arbitrary synchronization precision, with an uncertainty value r which may for example be less than Δ 2, while not increasing the quantity of data extracted from the reference signal, the method can implement a calculation of the characteristic numbers with a finer temporal resolution. For this, the parameters are calculated all the durations r <Δ, on the second signal to be synchronized called "degraded", which corresponds to Δ / r parameters Pi 'for a parameter P- |. The calculation complexity increases, going from T / Δ to T / r calculation windows, but only on the received signal. Situation B Figure 3 illustrates the method used. For example, r is a sub-multiple of Δ.

Notations:

T: synchronization search range. T is a multiple of Δ. ro: maximum admissible synchronization error / uncertainty e: synchronization error

Δ period for calculating the parameters on the signal.

P _k : parameter calculated on the first so-called "reference" signal SR. k is a time index identifying which calculation period Δ corresponds to Pk.

P'k: parameter calculated on the second so-called "degraded" signal SD- k is a time index identifying which calculation period Δ corresponds

pk

P'k ': parameter calculated on the second so-called "degraded" signal SD-k is a time index identifying which calculation period Δ corresponds to P _k . i is a time sub-index identifying a number of durations r within the period p between 1 and Δ / r.

Note: All times correspond to an integer number of samples of the audio or video signal. To define one or more coordinates, the second step applies a treatment to the parameters. A set of β coordinates is calculated for each set of parameters Pk or P'k 'obtained on the window k of duration Δ corresponding for example to 1024 samples, of the reference or degraded signal respectively.

• The primary goal of this step is to obtain relevant coordinate values to achieve synchronization, and with given limits and limits. Thus, each coordinate is obtained from a combination of the characteristic numbers available. In addition, this step makes it possible to reduce the number of dimensions and therefore to simplify the subsequent operations. In a preferred embodiment, two coordinates must be obtained (β = 2). For example, if two characteristic parameters are used, each of them can be used to determine a coordinate. Alternatively, a higher number of characteristic numbers can be used; processing can be carried out to pass a lower number of numbers, for example at two coordinates and these are then interpreted as a projection of a space with as many dimensions as characteristic numbers towards a space with for example 2 coordinates .

The third step corresponds to the construction of the trajectory (Figure 4). The trajectory defines a signature of a segment of the audio signal over the duration T by a series of points in a space with as many dimensions as there are coordinates. The use of a space with two or more dimensions makes it possible to construct a very particular trajectory, which makes it possible to obtain for synchronization a high reliability and precision. After these three stages, the synchronization of the signals comes down to the synchronization of two trajectories (or curves parameterized by time) in a space with two dimensions or more: the first trajectory is defined by the points R _k , obtained from the numbers significant P calculated all the durations Δ over the time range T. There are N = T / Δ points R. the second trajectory is defined by the points D _k = D _k ¹ , obtained from the significant numbers P _k = P _k ¹ calculated all the durations Δ on the range T. There are N '= N = T / Δ points D _k .

When a duration r <Δ is used to calculate the parameters P'k ¹ , the trajectory is defined by the points D _k ', the number of N' = T / r points. To this end, a criterion of resemblance between two trajectories of N points (or of N and N 'points) is implemented. The following methods are given as an example:

The first method proposed is a minimization of a distance between the two trajectories.

The basic idea is to calculate, over a portion of the trajectory, a distance. Depending on the maximum possible desynchronization range of the curves, corresponding to the audio or video signals, an appropriate portion of each trajectory is selected. On these portions, a cumulative Diff of the distances d between the vertices R and D + deita or Dk + deita 'of the curves is calculated respectively by the relations (1) and (2) below, by applying successive delta shifts in order to find the offset minimizing the distance Diff between trajectories.

Figure 4 illustrates the calculation on an example, with points defined by two coordinates, in a space with β = 2 dimensions. On the so-called “degraded” signal, the parameters are calculated every time r = Δ / 2, that is to say with a resolution twice as fine as on the first signal

The Diff distance gives the difference between the two trajectories. The arithmetic mean of the vertex to vertex distances is a preferred embodiment, but another distance calculation is also applicable.

with d (A, B) distance between two points or vertices. This distance d (A, B) can also be arbitrary. In a particular embodiment, the Euclidean distance is used:

d (A, B) = where αd - 1..co (2) where aj and bj are the coordinates of points A and B and β denotes the number of coordinates of each point.

The delta offset giving the minimum distance Diff corresponds to the desynchronization of the curves, and consequently of the starting signals. In the example proposed (Figure 4), the offset found will be 2, ie twice the period Δ of calculation of the initial parameter. The synchronization range will therefore be between:

t + 2 * Δ_- _{and + 2} * Δ + - (3)

2 2.

The second proposed criterion is a maximization of a correlation between the two trajectories

This criterion works in a similar way to the previous one, except that it consists in maximizing the Correl value. Relations (1) and (2) are replaced by the following two.

Correl (delta) = D _k * R _{k + delta (4)} where the operation * is the dot product defined by:

where θj and bj are the coordinates of points A and B.

The methods below are particularly suitable for the case where β = 2 coordinates

Other techniques make the method more robust to the presence of significant differences between the signals to be synchronized, due for example to degradations during a broadcast, namely.

• distance between the successive angles of the trajectories, This method consists in transforming the two-dimensional trajectory into a series of angles measured between successive segments defined by the points of the trajectory. Figure 5 illustrates the definition of angles Δφ.

The criterion used to synchronize the two trajectories is the minimization of the following relationship: Diff (delta) - φ _k ï- _λ v delta (6)

• surface intercepted between the two curves,

This method consists in transforming the two-dimensional trajectory into a series of surfaces intercepted by the successive segments defined by the points of the trajectory. Figure 6 illustrates the definition of intercepted surfaces S.

The criterion used to synchronize the two trajectories is the minimization of the following relationship:

SSrï- _o o _t t _a a _i l _e e == SS sSoOmmmmee

• Finally, the simultaneous use of several criteria is possible. Once the delta value of the desynchronization offset between the two signals has been determined by one of the preceding methods, the two signals can be resynchronized by applying the delta offset to one of the signals. Synchronization is carried out, with a precision determined by the rate of calculation of the characteristic numbers.

The synoptic of a synchronization process is given in Figure 7. When the desired precision is not reached, that is to say that the synchronization is too "rough" for the intended application, a last step can consist in refine the previous result.

A procedure of the prior art can be applied to the synchronization uncertainty range Δ or r, which is now sufficiently reduced to be of acceptable complexity. A time domain correlation approach, preferably with marker signals, can be used for example.

However, this step should only be used in certain specific cases. In fact, in the type of targeted quality measurement application, a refinement of the synchronization is generally not necessary because the precision obtained is sufficient. In addition, the techniques of the prior art require, as explained above, to have available signal data which are not easily transportable in a complex and distributed system.

One of the particular embodiments corresponds to the implementation of the invention in the context of an application for monitoring the audio quality in a digital television broadcasting network. In this context, a major advantage of the invention is to use the data used for the quality evaluation to achieve synchronization, since this avoids or minimizes the need to transmit data specific to synchronization. Various characteristic numbers (reduced reference), intended for the estimation of the importance of the degradations introduced during the broadcasting of the signals, are calculated on the reference signal at the input of the network. These PR reference numbers are transmitted via a data channel to the quality measurement point. Characteristic numbers P are calculated on the degraded signal at this measurement point. The quality estimation will thus be done by comparing the parameters PR and P - For this, they must be synchronized, from the characteristic parameter or parameters used for the reference.

The quality estimation will be done by comparing the PR and PM parameters. For this, they must be synchronized.

The principle of objective perceptual measurements is based on the transformation of the physical representation (sound pressure, level, time and frequency) into the psychoacoustic representation (sound force, masking level, time and critical bands or barks) of two signals (the signal of reference and the signal to be evaluated) in order to compare them. This transformation takes place thanks to a modeling of the human auditory system (generally, this modeling consists of a spectral analysis in the Barks domain followed by spreading phenomena).

The example below of implementing the method according to the invention implements a perceptual characteristic parameter called “Perceptual Account Gap”. The original idea for this parameter is to establish a measure of uniformity of a window of the audio signal. Thus, a sound signal whose frequency components are stable is considered to be uniform. Conversely, "perfect" noise corresponds to a signal which uniformly covers all the frequency bands (flat spectrum). This type of parameter therefore makes it possible to characterize the content of the signal. This capacity is reinforced by its perceptual character, that is to say taking into account characteristics of the human auditory system known by psychoacoustics.

Several steps are implemented to take into account psychoacoustics. They are applied to the reference signal and to the degraded signal. These steps are as follows:

• Windowing of the temporal signal in blocks, then, for each block, calculation of the excitation induced by the signal using a hearing model. This representation of the signals takes into account the phenomena of psychoacoustics, and provides a histogram whose accounts are the values of the basilar components. This makes it possible to take into consideration only the audible components of the signal and therefore to be limited to useful information. To obtain this excitation, classical modeling can be used: attenuation of the outer and middle ear, integration according to critical bands and frequency masking. The time windows chosen are approximately 42 ms (2048 points at 48 kHz) with an overlap of 50%. This provides a temporal resolution of the order of 21 ms.

Several stages are involved in this modeling. For the first step, the attenuation filter for the outer and middle ear is applied to the power spectral density, obtained from the signal spectrum. This filter also takes into account the absolute hearing threshold. The notion of critical bands is modeled by a transformation of the frequency scale into a basilar scale. The following stage corresponds to the calculation of individual excitations to take account of masking phenomena, thanks to the spreading function in the basilar scale and to a non-linear addition. The last step makes it possible to obtain the compressed loudness, by a power function, to model the non-linear frequency sensitivity of the ear, by a histogram comprising the 109 basilar components.

• The histogram accounts obtained are then periodically vectorized into three classes to obtain a representation along a trajectory. It is this trajectory which makes it possible to visualize the evolution of the structure of the signals and is used for synchronization. This also makes it possible to obtain a simple and concise characterization of the signal and therefore to have a reference parameter (or characteristic parameter). Several strategies exist to fix the limits of these three classes: The simplest is to separate the histogram into three zones of equal sizes. Thus, the 109 basilar components, which represent 24 Barks, can be separated by the following indices: ISi = 36 or z = - * 36 = 7.927 Barks (8)

109

IS ₂ = 73 or z = - * 73 = 16,073 Barks (9)

109 '

The second strategy takes into account the scaling areas of BEERENDS. This corresponds to gain compensation between the excitation of the reference signal and that of the signal to be tested by considering three zones on which the ear would perform this same operation. Thus, the fixed limits are the following:

24 ISi = 9 or z = - * 9 = 1,982 Barks (10)

109

24 IS ₂ = 100 or z = - * 100 = 22.018 Barks (11) 109

The trajectory is then represented in a triangle, called the frequency triangle. For each block we obtain three accounts Ci, C ₂ and C ₃ , therefore two Cartesian coordinates according to the following formulas:

r = C ₂ / N * sin (; r / 3) (13) with C _? : sum of excitations for high frequencies

(above S ₂ ) C ₂ : account associated with the medium frequencies (components between Si and S ₂ ) and N = Cι + C + C3: Total sum of the values of the components.

A point (X, Y) is therefore obtained for each time window of the signal. Each of the X and Y coordinates constitutes a characteristic number. Alternatively, Ci, C ₂ and C ₃ can be taken as characteristic numbers. For a complete sequence, the associated representation is thus a trajectory parameterized by time, as shown in Figure 8. Among various possible methods for synchronizing the trajectories, the technique chosen by way of example is that based on minimizing the distance between the points of the trajectories.

It is important to note that the calculation of the parameter used in this case for synchronization remains complex, but this parameter can also be used to estimate the quality of the signal. It must therefore in any case be calculated, and it is therefore not an additional load added to the calculation during the comparison, especially since the calculation relating to this parameter is performed locally only for the digital stream received. FIG. 9 summarizes the method used to synchronize the signals within the framework of the supervision of the quality of broadcast signals, with the characteristic parameter above.

The following example illustrates the case of a reference file (R1), coded then decoded MPEG2 Layer2 at 128 kbits / s to obtain a degraded file (R2). The desynchronization introduced is 6000 samples. The offset found is 6 windows, i.e. 6 * 1024 = 6144 samples. The error (144) is much less than the period (1024) of the characteristic parameter used. Figures 10a and 10b show the trajectories before and after synchronization: Before synchronization (Figure 10a), there is no point-to-point correspondence between the two trajectories. After synchronization (Figure 10b), the correspondence between the two trajectories is optimal within the meaning of the distance criterion (Cf. relation (1).

It is generally not necessary to do a finer synchronization. This is particularly the case if the uncertainty given by the procedure exposed here is less than the maximum synchronization error admissible by the quality measurement parameter. For the most demanding quality parameters, the necessary resolution r ₀ is of the order of 32 samples. In the case of FIG. 10a, the initial range is of the order of

120 ms, i.e. 5760 samples at 48 kHz. By using only the characteristic numbers available for the quality evaluation (every 1024 samples, ie the duration Δ), a first synchronization is carried out with an uncertainty of 1024 samples, ie a factor greater than 5 compared to 5760, for very limited computing power devoted to synchronization. However, for example in a second step, the more frequent calculation of the quality parameters on the second (degraded) signal (r <Δ) makes it possible, if desired, to further reduce the synchronization error to r samples. Another characteristic parameter implements an autoregressive modeling of the signal.

The general principle of linear prediction is to model the signal as being a combination of its past values. The idea is to calculate the N coefficients of a prediction filter by an autoregressive modeling (all pole). With this adaptive filter, it is possible to obtain a predicted signal from the actual signal. Prediction errors or residuals are calculated by difference between these two signals. The presence and amount of noise in a signal can be determined by analyzing these residues.

The comparison of the residues obtained on the reference signal and those calculated from the degraded signal, and therefore of the noise levels, makes it possible to estimate the importance of the modifications and defects inserted.

The reference to be transmitted corresponds to the maximum of υ residues over a time window of given size. It is in fact not interesting to transmit all the residues if the bit rate of the reference wants to be reduced.

To adapt the coefficients of the prediction filter, two methods are given below by way of example:

- LEVINSON-DURBIN's algorithm which is described for example in the work of M. BELLANGER - Digital signal processing - Theory and practice (MASSON ed. 1987) p. 393 to 395. To use it, it is necessary to have an estimate of the autocorrelation of the signal on a set of No samples. This autocorrelation is used to solve the Yule-Walker system of equations and thus obtain the coefficients of the predictor filter.

Only the first N values of the autocorrelation function can be used, where N denotes the order of the algorithm, that is to say the number of coefficients of the filter. On a window of 1024 samples, we keep the maximum of the prediction error.

The gradient algorithm which is described for example in the aforementioned work of M. BELLANGER p. 371 and following. The main drawback of the previous parameter is the need, in the case of an implementation on DSP, to store the N ₀ samples to estimate autocorrelation, have the coefficients of the filter then calculate the residuals. This second parameter makes it possible to avoid this by using another algorithm making it possible to calculate the coefficients of the filter: the algorithm of the gradient. This uses the error made to update the coefficients. The filter coefficients are changed in the direction of the gradient of the instantaneous quadratic error, with the opposite signal.

Once the residuals obtained by difference between the predicted signal and the real signal, only the maximum of their absolute values, over a time window of given size T, is preserved. The reference vector to be transmitted can thus be reduced to a single number.

After transmission then synchronization, the comparison consists of a simple calculation of the distance between the maxima of the reference and of the degraded signal, for example by difference.

The main advantage of the two parameters is the flow required to transfer the reference. This allows the reference to be reduced to a value for 1024 signal samples.

However, no model of psychoacoustics is taken into account.

Another characteristic parameter implements an autoregressive modeling of the basilar excitation.

Compared to the classical linear prediction, this method allows to take into account the phenomena of psychoacoustics, in order to obtain an evaluation of the perceived quality. For this, the calculation of the parameter goes through a modeling of various hearing principles. A linear prediction models the signal as a combination of its past values. Analysis of the residuals (or prediction errors) makes it possible to determine and estimate the presence of noise in a signal. The major drawback when using these techniques is the fact that there is no consideration of the principles of psychoacoustics. Thus, it is not possible to estimate the amount of noise actually perceived.

The process takes up the general principle of classical linear prediction. It also incorporates the phenomena of psychoacoustics to adapt it to the non-linear sensitivity in frequency (loudness) and intensity (tone) of the human ear. We modify the spectrum of the signal, via an audition model, before calculating the coefficients of the linear prediction by an autoregressive modeling (all pole). The coefficients thus obtained make it possible to model the signal in a simple way while taking account of psychoacoustics. It is these prediction coefficients that will be transmitted and will serve as a reference when comparing with the degraded signal. The first part of the calculation of this parameter corresponds to the modeling of the principles of psychoacoustics using classical hearing models. The second part is the calculation of the linear prediction coefficients. The last part corresponds to the comparison of the prediction coefficients calculated for the reference signal and those obtained for the degraded signal. The different steps of this method are therefore as follows:

- Temporal windowing of the signal then calculation of an internal representation of the signal by modeling the phenomena of psychoacoustics. This step corresponds to the calculation of the compressed loudness, which is in fact the excitation induced by the signal at the level of the inner ear. This representation of the signals takes into account the phenomena of psychoacoustics, and is obtained from the spectrum of the signal, using conventional models: attenuation of the outer and middle ear, integration according to critical bands and frequency masking. This calculation step is identical to the parameter described above.

- Autoregressive modeling of this compressed loudness in order to obtain the coefficients of a RIF prediction filter, just like in a classic linear prediction. The method used is that of autocorrelation, by solving the Yule-Walker equations. The first step in obtaining the prediction coefficients is therefore the calculation of the signal autocorrelation.

By considering the compressed loudness as a filtered spectral power, it is possible to calculate the autocorrelation of the perceived signal by inverse Fourier transformation. One of the methods to solve this system of equations of

Yule-Walker and thus obtaining the coefficients of a predictor filter is the use of the Levinson-Durbin algorithm.

These are the prediction coefficients which constitute the reference vector to be transmitted up to the point of comparison. The transformations used during the final calculation on the degraded signal, are the same as for the initial phase on the reference signal. - Estimation of degradations by calculating a distance between the vectors from the reference and the degraded signal. It is a comparison of the vectors of coefficients defining points obtained for the reference and for the audio signal transmitted, which makes it possible to estimate the degradations introduced during the transmission. This must be done on an adapted number of coefficients. The larger the number, the more precise the calculations can be, but the higher the bit rate required for transmitting the reference. Several distances can be used to compare the vectors of coefficients. The relative importance of the coefficients can for example be taken into account.

The principle of the method can be summarized according to the following diagram (Figure 11).

The modeling of the phenomena of psychoacoustics makes it possible to obtain 24 basilar components. The order N of the prediction filter is 32. From these, 32 autocorrelation coefficients are estimated, which gives 32 prediction coefficients of which only 5 to 10 coefficients are kept as an indicator vector of quality, for example the first 5 to 10 coefficients.

The main advantage of the parameter comes from taking into account the phenomena of psychoacoustics. To do this, it was necessary to increase the bit rate required to transfer the reference to 5 or

10 values for 1024 signal samples (21 ms.), I.e. a rate of 7.5 to 15 kbits / s.

The characteristic parameter P can generally be any quantity obtained from the content of the digital signals and for example in video:

- the brightness of the image or of an area thereof as given by the continuous coefficient F (0.0) of the discrete cosine transform TCD of the image, or of any other block transform, linear and orthogonal, block or global,

- and / or the contrast of the image or of an area thereof, given for example by the application of a Sobel filter,

- And / or the activity of the SA image of the image as defined for example in the PCT Application WO 99/18736 of the Applicant, and obtained by a transformation by linear and orthogonal blocks (TCD, transformed from FOURIER, HAAR, HADAMARD, SLANT, transformed into wavelets etc.),

- the average of the image, and in audio: - the power,

- And / or the power spectral density as defined in the French patent application FR 2 769 777 filed on October 13, 1997, and / or one of the parameters described above.

It will be noted that the parameter P is liable to be subjected to degradations due to transmission, but it is observed in practice that at the degradation levels generally observed in transmission networks, synchronization can be carried out using the method according to the invention.

In general, once synchronization has been acquired, the method makes it possible to verify that it is well preserved, so as to be able to remedy as necessary disturbances such as interruption of bit stream, change of bit stream, change decoder, etc., by resynchronizing the two digital signals E and S.

The method described is applicable whenever it is necessary to synchronize two digital streams. The method makes it possible to obtain a first synchronization range of width sufficiently reduced to allow the use of conventional fine synchronization methods, and this in real time.

The method advantageously exploits one or more characteristic parameters of the signals to be synchronized which are represented by at least two characteristic numbers, instead of all of the signals. According to a preferred embodiment, the combined use of several parameters makes it possible to ensure greater synchronization reliability than the prior techniques. In addition, the invention makes it possible to achieve synchronization of a level of precision which can be chosen, and of a lower complexity than the existing methods. This synchronization delimits a range of error, of a duration subsequently authorizing the use of conventional “fine” synchronization methods if higher precision is required. The measuring equipment implementing the method of the invention finds its application in particular in the field of broadcasting networks digital audiovisual signals to enable monitoring of the quality of the signals delivered by the network.

The invention also makes it possible to achieve, for a data stream incorporating audio and video data, synchronization for sound and for image. For this purpose, a synchronization is carried out for the video, by calculating a synchronization offset for the video, and a synchronization is carried out for the audio, by calculating a synchronization offset for the audio. In addition, by comparing the value of the two offsets, it can be known whether, for example during a transmission, there has been an offset between the sound and the image.

Claims

1. Method of synchronization between two digital data streams of the same content characterized in that it implements: a) for each of the two digital data streams Si and S ₂ , a generation at given intervals of at least two characteristic numbers expressing at least one parameter characteristic of their content; b) for each of the two flows Si and S ₂ , the generation, from said numbers, of points Di and D ₂ representing at least one said characteristic parameter in a space with at least two dimensions, the points Dj corresponding to the flow Si d on the one hand and D ₂ corresponding to the flow S ₂ on the other hand which are situated in a time range T defining trajectories representative of the data flows Si and S ₂ to be synchronized; c) shifting, with respect to each other, the time periods of duration T assigned to the digital data streams Si and S ₂ by calculating a criterion for superimposing said trajectories, an optimal value of which represents the synchronization sought; d) choose as the value representative of the synchronization, the offset between the time ranges corresponding to said optimal value.

2. Method according to claim 1, characterized in that one of the digital data streams is a reference stream Si and in that the other data stream is a stream S ₂ received through a transmission system and in that that the characteristic numbers of the reference stream Si are transmitted with the latter, while the characteristic numbers of the received stream S ₂ are calculated on reception.

3. Method according to one of claims 1 or 2, characterized in that c) implements: d) calculating a distance D between on the one hand a first trajectory represented by the points Di belonging to a first time range of duration T, and on the other hand, a second trajectory represented by the points D ₂ belonging to a second time range of duration T, said distance D constituting said superposition criterion; c2) shifting, with respect to each other, said first and second time periods of duration T until a minimum of the distance D which constitutes said optimum value is obtained;

4. Method according to claim 3, characterized in that the distance D is an arithmetic mean of the distances d between the corresponding points Di, D ₂ of the two trajectories.

5. Method according to claim 4, characterized in that said distance d between the points is a Euclidean distance.

6. Method according to one of claims 1 or 2, characterized in that c) implements: d) calculating a correlation function between the corresponding points Di, D ₂ of the two trajectories, said correlation function constituting said superposition criterion; c2) shifting said first and second time periods of duration T relative to each other until a minimum of the correlation function which constitutes said optimal value is obtained.

7. Method according to one of claims 1 or 2, characterized in that c) implements: d) transforming each trajectory into a series of angles between successive segments defined by the points of the trajectory. c2) shifting, with respect to each other, said first and second time periods of duration T until a minimum is obtained between the differences between the values of angles obtained for homologous segments of the two trajectories, said minimum constituting said value optimal.

8. Method according to one of claims 1 or 2, characterized in that c) implements: d) transforming the two trajectories into a series of surfaces intercepted by the successive segments defined by the points of said trajectories, the total intercepted surface constituting said overlap criterion. c2) shifting the time periods of duration T relative to each other until a minimum of said total intercepted surface is obtained, which constitutes said optimal value.

9. Method according to one of the preceding claims, characterized in that a said given interval is equal to Δ for one of the data streams Si and to r <Δ for the other data stream S ₂ .

10. Method according to one of the preceding claims, characterized in that it implements a first synchronization in choosing the same given interval Δ for the two data streams, then a second synchronization for which the given interval is equal to Δ for one of the data streams S1 and to r <Δ for the other data stream S2.

11. Method according to one of the preceding claims, characterized in that the generation of said characteristic numbers implements for a reference audio data stream and for a transmitted audio data stream, the following steps: a) calculating for each time window the power spectral density of the audio stream and apply to it a filter representative of the attenuation of the inner and middle ear to obtain a filtered spectral density, b) calculate from the filtered spectral density the individual excitations using the frequency spread function in the basilar scale, c) determine from said individual excitations the compressed loudness using a function modeling the non-linear frequency sensitivity of the ear, to obtain basilar components, d) separate the basal components into n classes (with n significantly less than the number of audio samples in a window temporal), preferably in three classes, and calculate for each class a number C representing the sum of the frequencies of this class, these characteristic numbers being the number of n and constituted by a said number C or being the number of n '<n and being generated from said numbers C.

12. Method according to one of the preceding claims, characterized in that the generation of a said characteristic number implements, for a reference audio data stream and for a transmitted audio data stream, the following steps: a) calculate N coefficients of a prediction filter by autoregressive modeling, b) determining in each time window the maximum of the residue by difference between the signal predicted using the prediction filter and the audio signal, said maximum of the prediction residue constituting undit characteristic number.

13. Method according to one of the preceding claims, characterized in that the generation of said characteristic numbers brings work for a reference audio data stream and for a transmitted audio data stream, the following steps: a) calculate for each time window the power spectral density of the audio stream and apply to it a filter representative of the attenuation of the inner and middle ear, to obtain a frequency spreading function in the basilar scale, b) calculating individual excitations from the frequency spreading function in the basilar scale, c) obtaining from said individual excitations the loudness compressed from a function modeling the non-linear frequency sensitivity of the ear, to obtain basilar components, d) calculating from said basilar components N 'prediction coefficients of a prediction filter by an autoregressive modeling, e ) generate for each time window at least one characteristic number from at least N 'pred coefficients iction.

14. Method according to claim 13, characterized in that the characteristic numbers are constituted by, between 1 and 10 of said prediction coefficients, and preferably between 2 and 5 of said coefficients.

15. Method according to one of the preceding claims, characterized in that a characteristic number is the instantaneous power of an audio signal.

16. Method according to one of the preceding claims, characterized in that a characteristic number is the power spectral density of an audio signal.

17. Method according to one of the preceding claims, characterized in that a characteristic number is the bandwidth of an audio signal.

18. Method according to one of the preceding claims, characterized in that a characteristic number is the continuous coefficient of the transform by a linear and orthogonal transform of at least part of an image belonging to the data stream, this transform being produced by blocks or globally.

19. Method according to one of the preceding claims, characterized in that a characteristic number is the contrast of at least one area of an image belonging to the data stream.

20. Method according to one of the preceding claims, characterized in that a characteristic number is the spatial or temporal activity of at least one area of an image.

21. Method according to one of the preceding claims, characterized in that a characteristic number is the average brightness of at least one area of an image.

22. Method according to one of the preceding claims, characterized in that said points Di and D ₂ are generated from at least two characteristic parameters.

23 Method according to claim 22, characterized in that said characteristic parameters are audio parameters.

24. The method of claim 22, characterized in that said characteristic parameters are video parameters.

25 Method according to one of the preceding claims, characterized in that the data stream comprises video data and audio data and in that it implements a first video synchronization from points D'i and D ' ₂ associated with at least one video characteristic parameter corresponding to said video stream, and a second audio synchronization from points D "ι and D" ₂ associated with at least one audio characteristic parameter corresponding to said audio stream.

26. The method of claim 25, characterized in that it has a step of determining the synchronization trigger between the video stream and the audio stream by difference between said offsets obtained for the video stream and for the audio stream.