EP1021805B1 - Method and apparatus for conditioning a digital speech signal - Google Patents

Description

The present invention relates to techniques digital speech signal processing.

Many representations of the signals of speech take into account the harmony of these signals resulting from the way they are produced. In the in most cases this translates into determination of a tone frequency of the speech signal.

Digital processing of speech signals have recently experienced significant developments in various fields: speech coding for transmission or storage, speech recognition, decreased noise, echo cancellation ... Very often, these treatments involve an estimate of the frequency tonal and specific operations in connection with estimated frequency.

Many methods have been designed to estimate the tone frequency. A commonly used method used is based on a linear prediction by which we evaluate a prediction delay inversely proportional to the tone frequency. This delay may be expressed as a whole or fractional number of times digital signal sample. Other methods directly detect attributable signal breaks at closures of the speaker's glottis, the intervals of time between these breaks being inversely proportional to the tone frequency.

Another method, offering a combination of prediction, interpolation and resampling in the spectral domain, is disclosed in US-A-522608.

When a transformation in the frequency domain, such as a discrete Fourier transform, is performed on the digital speech signal, we are led to consider a discrete spectrum of the speech signal. The discrete frequencies considered are those of the form (a / N) × F _e , where F _e is the sampling frequency, N the number of samples of the blocks used in the discrete Fourier transform, and has an integer ranging from 0 to N / 2-1. These frequencies do not necessarily include the estimated tone frequency and / or its harmonics. This results in an imprecision in the operations carried out in connection with the estimated tonal frequency, which can cause distortions of the processed signal by affecting its harmonic character.

A main object of the present invention is to propose a way to condition the speech signal which makes it less sensitive to the above drawbacks.

The invention therefore proposes a process as indicated in the claim 1 and a device as claimed in claim 9.

The invention thus provides a method of conditioning of a digital speech signal processed by successive frames, in which an analysis is carried out harmonic of the speech signal to estimate a frequency tonal of the speech signal on each frame where it presents voice activity. After estimating the frequency of the speech signal on a frame, we condition the speech signal of the frame by oversampling it to a multiple oversampling frequency of the estimated tone frequency.

This provision allows, in processing performed on the speech signal, to favor frequencies closest to the estimated tone frequency compared to other frequencies. We therefore preserve better the harmonic character of the speech signal. For calculate spectral components of the speech signal, we distribute the conditioned signal by blocks of N samples subject to transformation in the field frequency, and we choose the ratio between the frequency oversampling and the estimated tone frequency as a divisor of the number N.

• on estime des intervalles de temps entre deux ruptures consécutives du signal attribuables à des fermetures de la glotte du locuteur intervenant pendant la durée de la trame, la fréquence tonale estimée étant inversement proportionnelle auxdits intervalles de temps ;
• on interpole le signal de parole dans lesdits intervalles de temps, afin que le signal conditionné résultant de cette interpolation présente un intervalle de temps constant entre deux ruptures consécutives.

The previous technique can be further refined by estimating the tonal frequency of the speech signal on a frame as follows:

• estimating time intervals between two consecutive breaks in the signal attributable to closings of the glottis of the speaker intervening during the duration of the frame, the estimated tone frequency being inversely proportional to said time intervals;
• the speech signal is interpolated in said time intervals, so that the conditioned signal resulting from this interpolation has a constant time interval between two consecutive breaks.

This way of proceeding constructed artificially a signal frame on which the speech signal has ruptures at constant intervals. We take thus taking into account possible variations in frequency tonal over the duration of a frame.

An additional improvement is that that after processing each frame, we keep, among samples of the denoised speech signal provided by this processing, a number of samples equal to an integer multiple of times the ratio between the frequency sampling frequency and estimated tone frequency. This avoids distortion problems caused by phase discontinuities between frames, which are not generally not fully corrected by techniques overlap-add classics.

The fact of having conditioned the signal by the oversampling technique provides good measure of the degree of voicing of the speech signal on the frame, from a calculation of the entropy of the autocorrelation of the spectral components calculated on the basis of the conditioned signal. The higher the spectrum disturbed, that is to say the more it is seen, the higher the values of entropy are low. Signal conditioning of speech accentuates the irregular aspect of the spectrum and therefore variations in entropy, so that the latter is a measure of good sensitivity.

In the remainder of this description, we illustrate the packaging process according to the invention in a denoising system of a speech signal. We will understand that this process can find applications in many other types of digital processing of speech: coding, recognition, echo cancellation ...

• la figure 1 est un schéma synoptique d'un système de debruitage ;
• les figures 2 et 3 sont des organigrammes de procédures utilisées par un détecteur d'activité vocale du système de la figure 1 ;
• la figure 4 est un diagramme représentant les états d'un automate de détection d'activité vocale ;
• la figure 5 est un graphique illustrant les variations d'un degré d'activité vocale ;
• la figure 6 est un schéma synoptique d'un module de surestimation du bruit du système de la figure 1 ;
• la figure 7 est un graphique illustrant le calcul d'une courbe de masquage ;
• la figure 8 est un graphique illustrant l'exploitation des courbes de masquage dans le système de la figure 1 ;
• la figure 9 est un schéma synoptique d'un autre système de débruitage mettant en oeuvre la présente invention ;
• la figure 10 est un graphique illustrant une méthode d'analyse harmonique utilisable dans un procédé selon l'invention ; et
• la figure 11 montre partiellement une variante du schéma synoptique de la figure 9.

Other features and advantages of the present invention will appear in the description below of nonlimiting exemplary embodiments, with reference to the appended drawings, in which:

• Figure 1 is a block diagram of a noise reduction system;
• Figures 2 and 3 are flowcharts of procedures used by a voice activity detector of the system of Figure 1;
• FIG. 4 is a diagram representing the states of a voice activity detection automaton;
• FIG. 5 is a graph illustrating the variations of a degree of vocal activity;
• Figure 6 is a block diagram of a noise overestimation module of the system of Figure 1;
• FIG. 7 is a graph illustrating the calculation of a masking curve;
• FIG. 8 is a graph illustrating the use of the masking curves in the system of FIG. 1;
• FIG. 9 is a block diagram of another denoising system implementing the present invention;
• FIG. 10 is a graph illustrating a harmonic analysis method usable in a method according to the invention; and
• FIG. 11 partially shows a variant of the block diagram of FIG. 9.

The denoising system shown in FIG. 1 processes a digital speech signal s. A windowing module 10 puts this signal s in the form of successive windows or frames, each consisting of a number N of digital signal samples. Conventionally, these frames can have mutual overlaps. In the following of this description, it will be considered, without this being limiting, that the frames consist of N = 256 samples at a sampling frequency F _e of 8 kHz, with a Hamming weighting in each window, and 50% overlap between consecutive windows.

The signal frame is transformed in the frequency domain by a module 11 applying a conventional fast Fourier transform (TFR) algorithm to calculate the module of the signal spectrum. The module 11 then delivers a set of N = 256 frequency components of the speech signal, denoted S _{n, f} , where n denotes the number of the current frame, and f a frequency of the discrete spectrum. Due to the properties of digital signals in the frequency domain, only the first N / 2 = 128 samples are used.

To calculate the noise estimates contained in the signal s, the frequency resolution available at the output of the fast Fourier transform is not used, but a lower resolution, determined by a number I of frequency bands covering the band [0 , F _e / 2] of the signal. Each band i (1 ≤ i ≤ I) extends between a lower frequency f (i-1) and a higher frequency f (i), with f (0) = 0, and f (I) = F _e / 2 . This division into frequency bands can be uniform (f (i) - f (i-1) = F _e / 2I). It can also be non-uniform (for example according to a barks scale). A module 12 calculates the respective averages of the spectral components S _{n, f} of the speech signal in bands, for example by a uniform weighting such that:

This averaging reduces the fluctuations between the bands by averaging the noise contributions in these bands, which will decrease the variance of the estimator of noise. In addition, this averaging allows a large reduction of the complexity of the system.

The averaged spectral components S _{n, i} are addressed to a voice activity detection module 15 and to a noise estimation module 16. These two modules 15, 16 operate jointly, in the sense that degrees of vocal activity γ _{n, i} measured for the different bands by the module 15 are used by the module 16 to estimate the long-term energy of the noise in the different bands, while these long-term estimates B and _{n, i} are used by module 15 to carry out a priori denoising of the speech signal in the different bands to determine the degrees of vocal activity γ _{n, i} .

The operation of modules 15 and 16 can correspond to the flowcharts represented in the figures 2 and 3.

In steps 17 to 20, the module 15 proceeds a priori to denoising the speech signal in the different bands i for the signal frame n. This a priori denoising is carried out according to a conventional process of non-linear spectral subtraction from noise estimates obtained during one or more previous frames. In step 17, the module 15 calculates, with the resolution of the bands i, the frequency response Hp _{n, i} of the a priori denoising filter, according to the formula: Hp or = S or - α ' not -τ1, i · B not -τ1, i S not -τ2, i where τ1 and τ2 are delays expressed in number of frames (τ1≥1, τ2≥0), and α '/ n, i is a noise overestimation coefficient whose determination will be explained later. The delay τ1 can be fixed (for example τ1 = 1) or variable. The lower the confidence in the detection of voice activity.

où βp_i est un coefficient de plancher proche de 0, servant classiquement à éviter que le spectre du signal débruité prenne des valeurs négatives ou trop faibles qui provoqueraient un bruit musical. In steps 18 to 20, the spectral components pp _{n, i} are calculated according to:

where βp _i is a floor coefficient close to 0, conventionally used to prevent the spectrum of the denoised signal from taking negative or too low values which would cause musical noise.

Steps 17 to 20 therefore essentially consist in subtracting from the signal spectrum an estimate, increased by the coefficient α '/ n -τ1, i , of the noise spectrum estimated a priori.

In step 21, the module 15 calculates the energy of the a priori denoised signal in the different bands i for the frame n: E _{n, i} = Êp 2 / n, i. It also calculates a global average E _{n, 0} of the energy of the a priori denoised signal, by a sum of the energies per band E _{n, 1} , weighted by the widths of these bands. In the notations below, the index i = 0 will be used to designate the overall band of the signal.

Quant à l'énergie à long terme E _n,i, elle peut être calculée à l'aide d'un facteur d'oubli B1 tel que 0<B1<1, à savoir E _n,i = B1. E _{n -1, i} + (1-B1).E_{n , i} .In steps 22 and 23, the module 15 calculates, for each band i (0≤1≤I), a quantity ΔE _{n, i} representing the short-term variation of the energy of the noise-suppressed signal in the band i, as well as long-term value E _{n, i} of the energy of the denoised signal in band i. The quantity ΔE _{n, 1} can be calculated by a simplified derivation formula:

As for long-term energy E _{n, i} , it can be calculated using a forgetting factor B1 such that 0 <B1 <1, namely E _{n, i} = B 1. E _{n -1, i} + (1- B 1). E _{n , i} .

After calculating the energies E _{n , i} of the noise-suppressed signal, its short-term variations ΔE _{n, i} and its long-term values E _{n, i} in the manner indicated in FIG. 2, the module 15 calculates, for each band i (0 i i I I), a value ρ _i representative of the evolution of the energy of the denoised signal. This calculation is carried out in steps 25 to 36 of FIG. 3, executed for each band i between i = 0 and i = I. This calculation uses a long-term estimator of the noise envelope ba _i , an internal estimator bi _i and a noisy frame counter b _i .

In step 25, the quantity ΔE _{n, l} is compared with a threshold ε1. If the threshold ε1 is not reached, the counter b _i is incremented by one unit in step 26. In step 27, the long-term estimator ba _i is compared to the value of the smoothed energy E _{n, i} . If ba _l ≥ E _{n, i} , the estimator ba _i is taken equal to the smoothed value E _{n, i} in step 28, and the counter b _i is reset to zero. The quantity ρ _i , which is taken equal to the ratio ba _i / E _{n, 1} (step 36), is then equal to 1.

If step 27 shows that ba _i < E _{n, i} , the counter b _i is compared with a limit value bmax in step 29. If b _j > bmax, the signal is considered to be too stationary to support vocal activity. The aforementioned step 28, which amounts to considering that the frame contains only noise, is then executed. If b _i ≤bmax in step 29, the internal estimator bi _i is calculated in step 33 according to: bi i = (1- Bm ). E or + B m . ba i In this formula, Bm represents an update coefficient between 0.90 and 1. Its value differs depending on the state of a voice activity detection automaton (steps 30 to 32). This state δ _n-1 is that determined during the processing of the previous frame. If the automaton is in a speech detection state (δ _n-1 = 2 in step 30), the coefficient Bm takes a value Bmp very close to 1 so that the noise estimator is very slightly updated in the presence of speech. Otherwise, the coefficient Bm takes a lower value Bms, to allow a more significant update of the noise estimator in the phase of silence. In step 34, the difference ba _i -bl _i between the long-term estimator and the internal noise estimator is compared to a threshold ε2. If the threshold ε2 is not reached, the long-term estimator ba _l is updated with the value of the internal estimator bi _l in step 35. Otherwise, the long-term estimator ba _l remains unchanged . This avoids that sudden variations due to a speech signal lead to an update of the noise estimator.

After having obtained the quantities ρ _i , the module 15 proceeds to the voice activity decisions in step 37. The module 15 first updates the state of the detection automaton according to the quantity ρ ₀ calculated for l of the signal band. The new state δ _n of the automaton depends on the previous state δ _n-1 and on ρ ₀ , as shown in Figure 4.

Four states are possible: δ = 0 detects silence, or absence of speech; δ = 2 detects the presence of voice activity; and the states δ = 1 and δ = 3 are intermediate states of ascent and descent. When the automaton is in the state of silence (δ _n-1 = 0), it remains there if ρ ₀ does not exceed a first threshold SE1, and it goes into the state of ascent otherwise. In the rising state (δ _n-1 = 1), it returns to the state of silence if ρ ₀ is smaller than the threshold SE1, it goes into the speaking state if ρ ₀ is greater than a second threshold SE2 greater than the threshold SE1, and it remains in the rising state if SE1≤ ρ ₀ ≤SE2. When the automaton is in the speech state (δ _n-1 = 2), it remains there if ρ ₀ exceeds a third threshold SE3 smaller than the threshold SE2, and it goes into the descent state in the case opposite. In the descending state (δ _n-1 = 3), the automaton returns to the speaking state if ρ ₀ is greater than the threshold SE2, it returns to the silent state if ρ ₀ is below a fourth threshold SE4 smaller than the threshold SE2, and it remains in the descending state if SE4≤ ρ ₀ ≤SE2.

In step 37, the module 15 also calculates the degrees of vocal activity γ _{n, i} in each band i≥1. This degree γ _{n, i} is preferably a non-binary parameter, that is to say that the function γ _{n, i} = g (ρ _i ) is a function continuously varying between 0 and 1 depending on the values taken by the magnitude ρ _i . This function has for example the appearance shown in FIG. 5.

Module 16 calculates the band noise estimates, which will be used in the denoising process, using the successive values of the components S _{n, i} and the degrees of voice activity γ _{n, i} . This corresponds to steps 40 to 42 of FIG. 3. In step 40, it is determined whether the voice activity detection machine has just gone from the rising state to the speaking state. If so, the last two estimates B and _{n -1, i} and B and _{n- 2 , i} previously calculated for each band i≥1 are corrected in accordance with the value of the previous estimate B and _{n -3, i} . This correction is made to account for the fact that, in the ascent phase (δ = 1), the long-term noise energy estimates in the voice activity detection process (steps 30 to 33) may have been be calculated as if the signal contained only noise (Bm = Bms), so that they risk being tainted with error.

ou λ_B désigne un facteur d'oubli tel que 0<λ_B<1. La formule (6) met en évidence la prise en compte du degré d'activité vocale non binaire γ_n,i.In step 42, the module 16 updates the noise estimates per band according to the formulas:

where λ _B denotes a forgetting factor such as 0 <λ _B <1. Formula (6) shows how the degree of non-binary vocal activity γ _{n, i is} taken into account.

As indicated previously, the long-term noise estimates B and _{n , l} are overestimated, by a module 45 (FIG. 1), before proceeding to denoising by nonlinear spectral subtraction. Module 45 calculates the overestimation coefficient α '/ n, i previously mentioned, as well as an increased estimate B and' / n, i which essentially corresponds to α '/ n, i . B and _{n , i} .

The organization of the overestimation module 45 is shown in FIG. 6. The enhanced estimate B and '/ n, i is obtained by combining the long-term estimate B and _{n, i} and a measure ΔB max / n, i the variability of the noise component in band i around its long-term estimate. In the example considered, this combination is essentially a simple sum made by an adder 46. It could also be a weighted sum.

The overestimation coefficient α '/ n, i is equal to the ratio between the sum B and _{n , i} + Δ B max / n, i delivered by the adder 46 and the delayed long-term estimate B and _{n -τ3, i} (divider 47), capped at a limit value α _max , for example α _max = 4 (block 48). The delay τ3 is used to correct, if necessary, in the rise phases (δ = 1), the value of the overestimation coefficient α '/ n, i , before the long-term estimates have been corrected by steps 40 and 41 in Figure 3 (for example τ3 = 3).

The increased estimate B and '/ n, i is finally taken equal to α' / n, i . B and _{n-τ 3, i} (multiplier 49).

The measure ΔB max / n, i of the noise variability reflects the variance of the noise estimator. It is obtained as a function of the values of S _{n, i} and of B and _{n, i} calculated for a certain number of previous frames on which the speech signal does not present any vocal activity in the band i. It is a function of the deviations | S _{nk , i} - B and _{nk, i} | calculated for a number K of frames of silence (nk≤n). In the example shown, this function is simply the maximum (block 50). For each frame n, the degree of vocal activity γ _{n, i} is compared to a threshold (block 51) to decide whether the deviation | S _{n , i} - B and _{n, i} |, calculated in 52-53, may or may not be loaded into a queue 54 of K locations organized in first in first out (FIFO) mode. If γ _{n, i} does not exceed the threshold (which can be equal to 0 if the function g () has the form of FIG. 5), the FIFO 54 is not supplied, while it is in the opposite case. The maximum value contained in FIFO 54 is then provided as a measure of variability ΔB max / n, i .

The measure of variability ΔB max / n, i can, as a variant, be obtained as a function of the values S _{n, f} (and not S _{n, i} ) and B and _{n , i} . We then proceed in the same way, except that FIFO 54 does not contain | S _{n - k , i} - B and _{nk , i} | for each of the bands i, but rather

Thanks to independent estimates of long-term fluctuations in noise B and _{n, i} and its short-term variability Δ B max / n, i , the enhanced estimator B and '/ n, i provides excellent robustness to the musical noises of the denoising process.

où τ4 est un retard entier déterminé tel que τ4≥0 (par exemple τ4=0). Dans l'expression (7), le coefficient β 1 / i représente, comme le coefficient βp_i de la formule (3), un plancher servant classiquement à éviter les valeurs négatives ou trop faibles du signal débruité.A first phase of the spectral subtraction is carried out by the module 55 shown in FIG. 1. This phase provides, with the resolution of the bands 1 (1 i i 1 1), the frequency response H 1 / n, i of first denoising filter, as a function of the components S _{n, i} and B and _{n, i} and the overestimation coefficients α '/ n, i . This calculation can be performed for each band i according to the formula:

where τ4 is a determined integer delay such as τ4≥0 (for example τ4 = 0). In expression (7), the coefficient β 1 / i represents, like the coefficient β p _i of formula (3), a floor conventionally used to avoid negative or too low values of the denoised signal.

As is known (EP-A-0 534 837), the overestimation coefficient α _{n, i} could be replaced in formula (7) by another coefficient equal to a function of α '/ n, i and of an estimate of the signal-to-noise ratio (for example S _{n, i} / B and _{n, i} ), this function decreasing according to the estimated value of the signal-to-noise ratio. This function is then equal to α '/ n, i for the lowest values of the signal-to-noise ratio. Indeed, when the signal is very noisy, it is a priori not useful to reduce the overestimation factor. Advantageously, this function decreases to zero for the highest values of the signal / noise ratio. This protects the most energetic areas of the spectrum, where the speech signal is most significant, the amount subtracted from the signal then tending towards zero.

This strategy can be refined by applying it selectively to frequency harmonics pitch of the speech signal when it has voice activity.

Thus, in the embodiment shown in FIG. 1, a second denoising phase is carried out by a module 56 for protecting harmonics. This module calculates, with the resolution of the Fourier transform, the frequency response H 2 / n, f of a second denoising filter as a function of the parameters H 1 / n, i , α '/ n, i , B and _{n, i} , δ _n , S _{n, i} and the tone frequency f _p = F _e / T _p calculated outside the phases of silence by a harmonic analysis module 57. In the phase of silence (δ _n = 0) , the module 56 is not in service, that is to say that H 2 / n, f = H 1 / n, i for each frequency f of a band i. The module 57 can apply any known method of analysis of the speech signal of the frame to determine the period T _p , expressed as an integer or fractional number of samples, for example a linear prediction method.

The protection provided by the module 56 may consist in carrying out, for each frequency f belonging to a band i:

Δf = F _e / N represents the spectral resolution of the Fourier transform. When H ² _{n , f} = 1, the quantity subtracted from the component S _{n, f} will be zero. In this calculation, the floor coefficients β 2 / i (for example β 2 / i = β 1 / i express the fact that certain harmonics of the tonal frequency f _p can be masked by noise, so that it doesn ' is not worth protecting them.

This protection strategy is preferably applied for each of the frequencies closest to the harmonics of f _p , that is to say for any arbitrary integer.

c'est-à-dire remplacer la condition (9) ci-dessus par :

Cette façon de procéder (condition (9')) présente un intérêt particulier lorsque les valeurs de η peuvent être grandes, notamment dans le cas où le procédé est utilisé dans un système à bande élargie.If we designate by δf _p the frequency resolution with which the analysis module 57 produces the estimated tonal frequency f _p , that is to say that the real tonal frequency is between f _p -δf _p / 2 and f _p + δf _p / 2, then the difference ink the η-th harmonic of the real tonal frequency is its estimate η × f _p (condition (9)) can go up to ± η × δ _p / 2. For high values of η, this difference can be greater than the spectral half-resolution Δf / 2 of the Fourier transform. To take account of this uncertainty and to guarantee the good protection of the harmonics of the real tonal frequency, we can protect each of the frequencies of the interval

i.e. replace condition (9) above with:

This way of proceeding (condition (9 ')) is of particular interest when the values of η can be large, in particular in the case where the method is used in a broadband system.

For each protected frequency, the corrected frequency response H 2 / n, f can be equal to 1 as indicated above, which corresponds to the subtraction of a zero quantity in the context of spectral subtraction, that is to say ie full protection of the frequency in question. More generally, this corrected frequency response H 2 / n, f could be taken equal to a value between 1 and H 1 / n, f depending on the degree of protection desired, which corresponds to the subtraction of an amount less than which would be subtracted if the frequency in question was not protected.

The spectral components S 2 / n, f of a denoised signal are calculated by a multiplier 58: S 2 not , f = H 2 not , f . S not , f

This signal S 2 / n, f is supplied to a module 60 which calculates, for each frame n, a masking curve by applying a psychoacoustic model of auditory perception by the human ear.

The masking phenomenon is a principle known from functioning of the human ear. When two frequencies are heard simultaneously, it is possible that one of the two is no longer audible. We say then that it is hidden.

où les indices q et q' désignent les bandes de bark (0 ≤ q, q' ≤ Q), et S 2 / n,q, représente la moyenne des composantes S 2 / n,f du signal excitateur débruité pour les fréquences discrètes f appartenant à la bande de bark q'.There are different methods for calculating masking curves. One can for example use that developed by JD Johnston ("Transform Coding of Audio Signals Using Perceptual Noise Criteria", IEEE Journal on Selected Area in Communications, Vol. 6, No. 2, February 1988). In this method, we work in the frequency scale of the barks. The masking curve is seen as the convolution of the spectral spreading function of the basilar membrane in the bark domain with the excitatory signal, constituted in the present application by the signal S 2 / n, f . The spectral spreading function can be modeled as shown in Figure 7. For each bark band, we calculate the contribution of the upper and lower bands convoluted by the spreading function of the basilar membrane:

where the indices q and q 'denote the bark bands (0 ≤ q, q' ≤ Q), and S 2 / n, q , represents the mean of the components S 2 / n, f of the excitatory signal denoised for the discrete frequencies f belonging to the bark band q '.

où SFM représente, en décibels, le rapport entre la moyenne arithmétique et la moyenne géométrique de l'énergie des bandes de bark, et SFM_max=-60 dB.The masking threshold M _{n, q} is obtained by the module 60 for each bark band q, according to the formula: M n, q = C n, q / R q where R _q depends on the more or less voiced character of the signal. As is known, a possible form of R _q is: 10.log 10 (R q ) = (A + q) .χ + B. (1-χ) with A = 14.5 and B = 5.5. χ designates a degree of voicing of the speech signal, varying between zero (no voicing) and 1 (strongly voiced signal). The parameter χ can be of the known form:

where SFM represents, in decibels, the ratio between the arithmetic mean and the geometric mean of the energy of the bark bands, and SFM _max = -60 dB.

The denoising system also includes a module 62 which corrects the frequency response of the denoising filter, as a function of the masking curve M _{n, q} calculated by the module 60 and of the increased estimates B and '/ n, i _, calculated by the module 45. The module 62 decides the level of denoising which must really be reached.

By comparing the envelope of the estimate increased by the noise with the envelope formed by the masking thresholds M _{n, q} , it is decided to denoise the signal only insofar as the estimate increased B and '/ n, i exceeds the masking curve. This avoids unnecessarily removing noise masked by speech.

The new response H 3 / n, f , for a frequency f belonging to the band i defined by the module 12 and to the banae of bark q, thus depends on the relative difference between the increased estimate B and '/ n, i of the corresponding spectral component of the noise and the masking curve M _{n, q} , as follows:

In other words, the quantity subtracted from a spectral component S _{n, f} , in the process of spectral subtraction having the frequency response H 3 / n, f , is substantially equal to the minimum between on the one hand the quantity subtracted from this spectral component in the spectral subtraction process having the frequency response H 2 / n, f , and on the other hand the fraction of the increased estimate B and '/ n, i of the corresponding spectral component of the noise which, if if necessary, exceeds the masking curve M _{n, q} .

FIG. 8 illustrates the principle of the correction applied by the module 62. It schematically shows an example of masking curve M _{n, q} calculated on the basis of the spectral components S 2 / n, f of the noise-suppressed signal, as well as the estimation plus B and '/ n, i of the noise spectrum. The quantity finally subtracted from the components S _{n, f} will be that represented by the hatched areas, that is to say limited to the fraction of the increased estimate B and '/ n, i of the spectral components of the noise which exceeds the curve masking.

This subtraction is carried out by multiplying the frequency response H 3 / n _, f of the denoising filter by the spectral components S _{n, f} of the speech signal (multiplier 64). A module 65 then reconstructs the denoised signal in the time domain, by operating the inverse fast Fourier transform (TFRI) which reverses samples of frequency S 3 / n _, f delivered by the multiplier 64. For each frame, only the N / 2 = 128 first samples of the signal produced by module 65 are delivered as final denoised signal s ³ , after reconstruction by addition-overlap with the N / 2 = 128 last samples of the previous frame (module 66).

FIG. 9 shows a preferred embodiment of a denoising system implementing the invention. This system comprises a certain number of elements similar to corresponding elements of the system of FIG. 1, for which the same reference numbers have been used. Thus, modules 10, 11, 12, 15, 16, 45 and 55 provide in particular the quantities S _{n, i} , B and _{n , i} , α '/ n, i , B and' / n, i and H 1 / n, f to perform selective denoising.

The frequency resolution of the fast Fourier transform 11 is a limitation of the system of FIG. 1. In fact, the frequency subject to protection by the module 56 is not necessarily the precise tonal frequency f _p , but the frequency closest to it in the discrete spectrum. In some cases, it is then possible to protect harmonics relatively far from that of the tone frequency. The system of FIG. 9 overcomes this drawback thanks to an appropriate conditioning of the speech signal.

In this conditioning, the sampling frequency of the signal is modified so that the period 1 / f _p covers exactly an integer number of sample times of the conditioned signal.

Many harmonic analysis methods that can be implemented by the module 57 are capable of providing a fractional value of the delay T _p , expressed in number of samples at the initial sampling frequency F _e . A new sampling frequency f _{e is} then chosen so that it is equal to an integer multiple of the estimated tone frequency, ie f _e = pf _p = pF _e / T _p = KF _e , with p integer. In order not to lose signal samples, f _e should be greater than F _e . One can in particular impose that it is between F _e and 2Fe (1 K K 2 2), to facilitate the implementation of the packaging.

Of course, if no voice activity is detected on the current frame (d _n = 0), or if the delay T _p estimated by the module 57 is whole, it is not necessary to condition the signal.

So that each of the frequency harmonics tonal also corresponds to a whole number samples of the conditioned signal, the integer p must be a divider of the size N of the signal window produced by module 10: N = αp, with α integer. This size N is usually a power of 2 for putting implementation of the TFR. It is 256 in the example considered.

The spectral resolution Δf of the discrete Fourier transform of the conditioned signal is given by Δf = pf _p / N = f _p / α. It is therefore advantageous to choose p small so as to maximize α, but large enough to oversample. In the example considered, where F _e = 8 kHz and N = 256, the values chosen for the parameters p and α are indicated in table I. 500 Hz <f _p <1000 Hz 8 <T _p <16 p = 16 α = 16 250 Hz <f _p <500 Hz 16 <T _p <32 p = 32 α = 8 125 Hz <f _p <250 Hz 32 <T _p <64 p = 64 α = 4 62.5 Hz <f _p <125 Hz 64 <T _p <128 p = 128 α = 2 31.25 Hz <f _p <62.5 Hz 128 <T _p <256 p = 256 α = 1

This choice is made by a module 70 according to the value of the delay T _p supplied by the harmonic analysis module 57. The module 70 provides the ratio K between the sampling frequencies to three frequency change modules 71, 72, 73 .

The module 71 is used to transform the values S _{n, i} , B and _{n , i} , α '/ n, i , B and ' / n, i and H 1 / n, f , relating to the bands i defined by the module 12 , in the scale of modified frequencies (sampling frequency f _e ). This transformation consists simply in dilating the bands i in the factor K. The values thus transformed are supplied to the module 56 for protecting harmonics.

This then operates in the same manner as above to provide the frequency response H 2 / n, f of the denoising filter. This response H 2 / n, f is obtained in the same way as in the case of FIG. 1 (conditions (8) and (9)), except that in condition (9), the tonal frequency f _p = f _e / p is defined according to the value of the entire delay p supplied by the module 70, the frequency resolution Δf being also provided by this module 70.

The module 72 performs the oversampling of the frame of N samples provided by the windowing module 10. Oversampling in a rational K factor (K = K1 / K2) consists of first performing a oversampling in the integer factor K1, then a subsampling in the integer factor K2. These oversampling and subsampling in whole factors can be done classically at using polyphase filter banks.

The conditioned signal frame supplied by the module 72 includes KN samples at the frequency f _e . These samples are sent to a module 75 which calculates their Fourier transform. The transformation can be carried out from two blocks of N = 256 samples: one consisting of the first N samples of the frame of length KN of the conditioned signal s', and the other of the last N samples of this frame. The two blocks therefore have an overlap of (2-K) x100%. For each of the two blocks, we obtain a set of Fourier components S _{n, f} . These components S _{n, f} are supplied to the multiplier 58, which multiplies them by the spectral response H 2 / n, f to deliver the spectral components S 2 / n, f of the first denoised signal.

These components S 2 / n, f are addressed to the module 60 which calculates the masking curves in the manner previously indicated.

Preferably, in this calculation of the masking curves, the quantity χ designating the degree of voicing of the speech signal (formula (13)) is taken from the form χ = 1-H, where H is an entropy of the autocorrelation of the spectral components S 2 / n, f of the denoised conditioned signal. The autocorrelations A (k) are calculated by a module 76, for example according to the formula:

A module 77 then calculates the normalized entropy H, and supplies it to module 60 for the calculation of the masking curve (see SA McClellan et al: “Spectral Entropy: an Alternative Indicator for Rate Allocation?”, Proc. ICASSP'94 , pages 201-204):

Thanks to the conditioning of the signal, as well as to its denoising by the filter H , 2 / n, f the normalized entropy H constitutes a measurement of voicing very robust to noise and variations in the tonal frequency.

The correction module 62 operates in the same way as that of the system in FIG. 1, taking into account the overestimated noise B and '/ n, i rescaled by the frequency change module 71. It provides the response in frequency H 3 / n, f of the final denoising filter, which is multiplied by the spectral components S _{n, f} of the signal conditioned by the multiplier 64. The components S 3 / n, f which result therefrom are brought back into the time domain by the TFRI 65 module. At the output of this TFRI 65, a module 80 combines, for each frame, the two signal blocks resulting from the processing of the two overlapping blocks delivered by the TFR 75. This combination can consist of a weighted sum Hamming of samples, to form a denoised conditioned signal frame of KN samples.

The denoised conditioned signal supplied by the module 80 is subject to a change in sampling frequency by the module 73. Its sampling frequency is reduced to F _e = f _e / K by the operations opposite to those performed by module 75. Module 73 delivers N = 256 samples per frame. After the reconstruction by addition-recovery with the N / 2 = 128 last samples of the previous frame, only the N / 2 = 128 first samples of the current frame are finally kept to form the final denoised signal s ³ (module 66).

In a preferred embodiment, a module 82 manages the windows formed by the module 10 and saved by the module 66, in such a way that a number M of samples is saved equal to an integer multiple of T _p = F _e / f _p . This avoids the problems of phase discontinuity between the frames. Correspondingly, the management module 82 controls the windowing module 10 so that the overlap between the current frame and the next one corresponds to NM. This recovery of NM samples will be required in the recovery sum carried out by the module 66 during the processing of the next frame. From the value of T _p supplied by the harmonic analysis module 57, the module 82 calculates the number of samples to be saved M = T _p × E [N / (2T _p )], E [] designating the part whole, and command modules 10 and 66 accordingly.

In the embodiment which we have just describe, the tone frequency is estimated in an average way on the frame. However the tonal frequency can vary some little over this period. It is possible to take into account these variations in the context of the present invention, in conditioning the signal so as to obtain artificially a constant tone frequency in the frame.

For this, we need the analysis module 57 harmonic provides the time intervals between the consecutive breaks in speech signal due to closures of the glottis of the intervening speaker for the duration of the frame. Usable methods to detect such micro-ruptures are well known in the area of harmonic signal analysis lyrics. In this regard, we can consult the articles following: M. BASSEVILLE et al., “Sequential detection of abrupt changes in spectral characteristics of digital signed ”, IEEE Trans. on Information Theory, 1983, Vol. IT-29, No. 5, pages 708-723; R. ANDRE-OBRECHT, "A new statistical approach for the automatic segmentation of continuous speech signals ”, IEEE Trans. on Acous., Speech and Sig. Proc., Vol. 36, No. 1, January 1988; and C. MURGIA et al., "An algorithm for the estimation of glottal closure instants using the sequential detection of abrupt changes in speech signals ", Signal Processing VII, 1994, pages 1685-1688.

où e 0 / m et σ 2 / 0 représentent le résidu calculé au moment de l'échantillon m de la trame et la variance du modèle à long terme, e 1 / m et σ 2 / 1 représentant de même le résidu et la variance du modèle à court terme. Plus les deux modèles sont proches, plus la valeur w_m du test statistique est proche de 0. Par contre, lorsque les deux modèles sont éloignés l'un de l'autre, cette valeur w_m devient négative, ce qui dénote une rupture R du signal.The principle of these methods is to perform a statistical test between two models, one in the short term and the other in the long term. Both models are adaptive linear prediction models. The value of this statistical test w _m is the cumulative sum of the posterior likelihood ratio of two distributions, corrected by the Kullback divergence. For a distribution of residuals having a Gaussian statistic, this value w _m is given by:

where e 0 / m and σ 2/0 represent the residue calculated at the time of the sample m of the frame and the variance of the long-term model, e 1 / m and σ 2/1 likewise representing the residue and the variance of the short term model. The closer the two models are, the more the value w _m of the statistical test is close to 0. On the other hand, when the two models are distant from each other, this value w _m becomes negative, which indicates a break R of the signal.

FIG. 10 thus shows a possible example of evolution of the value w _m , showing the breaks R of the speech signal. The time intervals t _r (r = 1.2, ...) between two consecutive breaks R are calculated, and expressed in number of samples of the speech signal. Each of these intervals t _r is inversely proportional to the tone frequency f _p , which is thus estimated locally: f _p = F _e / t _r over the r-th interval.

We can then correct the temporal variations of the tone frequency (that is to say the fact that the intervals t _r are not all equal on a given frame), in order to have a constant tone frequency in each of the frames d 'analysis. This correction is carried out by modifying the sampling frequency over each interval t _r , so as to obtain, after oversampling, constant intervals between two glottal breaks. The duration between two breaks is therefore modified by oversampling in a variable ratio, so as to lock in on the largest interval. In addition, care is taken to comply with the conditioning constraint that the oversampling frequency is a multiple of the estimated tone frequency.

FIG. 11 shows the means used to calculate the conditioning of the signal in the latter case. The harmonic analysis module 57 is produced so as to implement the above analysis method, and to provide the intervals t _r relative to the signal frame produced by the module 10. For each of these intervals, the module 70 (block 90 in FIG. 11) calculates the oversampling ratio K _r = p _r / t _r , where the integer p _r is given by the third column of table I when t _r takes the values indicated in the second column . These oversampling reports K _r are supplied to the frequency change modules 72 and 73, so that the interpolations are carried out with the sampling ratio K _r over the corresponding time interval t _r .

The largest T _p of the time intervals t _r supplied by the module 57 for a frame is selected by the module 70 (block 91 in FIG. 11) to obtain a torque p, α as indicated in table I. The frequency d the modified sampling is then f _e = pF _e / T _p as before, the spectral resolution Δf of the discrete Fourier transform of the conditioned signal being always given by Δf = F _e /(α.T _p ). For the frequency change module 71, the oversampling ratio K is given by K = p / T _p (block 92). The module 56 for protecting the harmonics of the tone frequency operates in the same manner as above, using for condition (9) the spectral resolution Δf provided by the block 91 and the tone frequency f _p = f _e / p defined according to the value of the entire delay p supplied by block 91.

This embodiment of the invention also involves an adaptation of the window management module 82. The number M of samples of the denoised signal to be saved on the current frame here corresponds to an integer number of consecutive time intervals t _r between two glottal breaks (see FIG. 10). This arrangement avoids the problems of phase discontinuity between frames, while taking into account the possible variations of the time intervals t _r on a frame.

Claims (9)

1. Method of conditioning a digital speech signal processed by successive frames, characterized in that harmonic analysis of the speech signal is performed to estimate a pitch frequency (f_p) of the speech signal over each frame in which it features vocal activity, and in that, after estimating the pitch frequency of the speech signal over one frame, the speech signal of the frame is conditioned by oversampling it at an oversampling frequency (f_e) which is an integer multiple of the estimated pitch frequency.
2. Method according to claim 1, wherein spectral components (S_n,f) of the speech signal are computed by distributing the conditioned signal (s') into blocks of N samples transformed into the frequency domain, N being a predetermined integer, and wherein the ratio (p) between the oversampling frequency (f_e) and the estimated pitch frequency is a factor of the number N.
3. Method according to claim 2, wherein the number N is a power of 2.
4. Method according to claim 2 or 3, wherein a degree of voicing (χ) of the speech signal is estimated over the frame from a computation of the entropy (H) of the autocorrelation of spectral components (S 2 / n,f) computed on the basis of the conditioned signal (s').
5. Method according to claim 4, wherein the degree of voicing (χ) is measured on the basis of a normalised entropy H of the form:

   

   where A(k) is the normalised autocorrelation defined by:

   

   S 2 / n,f designating said spectral component of rank f computed on the basis of the oversampled signal.
6. Method according to any one of the preceding claims, wherein, after processing each conditioned signal frame, a number (M) of signal samples supplied by such processing is retained which is equal to an integer multiple of the ratio (T_p) between the sampling frequency (F_e) and the estimated pitch frequency (f_p).
7. Method according to any one of claims 1 to 5, wherein the estimation of the pitch frequency of the speech signal over a frame includes the steps of:

   estimating time intervals (t_r) between two consecutive breaks (R) of the signal which can be attributed to glottal closures of the speaker occurring during the frame, the estimated pitch frequency being inversely proportional to said time intervals;

   interpolating the speech signal in said time intervals, so that the conditioned signal (s') resulting from such interpolation has a constant time interval between two consecutive breaks.
8. Method according to claim 7, wherein, after processing each frame, a number (M) of samples of the speech signal supplied by such processing is retained which corresponds to an integer number of estimated time intervals (t_r).
9. A device for conditioning a digital speech signal (s), comprising processing means arranged to implement a method according to any one of the preceding claims.

Images