CA2596411A1 - Method and system for providing an acoustic signal with extended bandwidth - Google Patents
Method and system for providing an acoustic signal with extended bandwidth Download PDFInfo
- Publication number
- CA2596411A1 CA2596411A1 CA002596411A CA2596411A CA2596411A1 CA 2596411 A1 CA2596411 A1 CA 2596411A1 CA 002596411 A CA002596411 A CA 002596411A CA 2596411 A CA2596411 A CA 2596411A CA 2596411 A1 CA2596411 A1 CA 2596411A1
- Authority
- CA
- Canada
- Prior art keywords
- signal
- broadband
- bandwidth
- bandwidth limit
- acoustic signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000000295 complement effect Effects 0.000 claims abstract description 38
- 229920000136 polysorbate Polymers 0.000 claims abstract description 6
- 230000003595 spectral effect Effects 0.000 claims description 60
- 238000001228 spectrum Methods 0.000 claims description 57
- 230000007774 longterm Effects 0.000 claims description 40
- 230000005284 excitation Effects 0.000 claims description 38
- 230000000875 corresponding effect Effects 0.000 claims description 31
- 239000000306 component Substances 0.000 claims description 14
- 230000000694 effects Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 claims description 2
- 239000013598 vector Substances 0.000 description 58
- 239000011159 matrix material Substances 0.000 description 13
- 238000005070 sampling Methods 0.000 description 12
- 230000006978 adaptation Effects 0.000 description 8
- 238000012549 training Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 230000003190 augmentative effect Effects 0.000 description 4
- 238000009499 grossing Methods 0.000 description 4
- 206010001497 Agitation Diseases 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004040 coloring Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 210000001260 vocal cord Anatomy 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 229910052729 chemical element Inorganic materials 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000003455 independent Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 229940086255 perform Drugs 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/06—Determination or coding of the spectral characteristics, e.g. of the short-term prediction coefficients
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/08—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L2019/0001—Codebooks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Quality & Reliability (AREA)
- Cable Transmission Systems, Equalization Of Radio And Reduction Of Echo (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Reduction Or Emphasis Of Bandwidth Of Signals (AREA)
Abstract
The invention is directed to a method for providing an acoustic signal with extended bandwidth, comprising automatically determining a current upper and a current lower bandwidth limit of a received acoustic signal, automatically determining at least one complementary signal to complement the received acoustic signal between a prede-fined lower broadband bandwidth limit and the current lower bandwidth limit and/or be-tween the current upper bandwidth limit and a predefined upper broadband bandwidth limit, wherein the predefined lower broadband bandwidth limit is smaller than the cur-rent bandwidth limit and the predefined upper broadband bandwidth limit is larger than the current upper bandwidth limit, automatically assembling the at least one comple-mentary signal and the received acoustic signal to obtain an acoustic signal with ex-tended bandwidth.
Description
Method and System for Providing an Acoustic Signal with Extended Bandwidth The invention is directed to a method and a system for providing an acoustic signal, in particular a speech signal, with extended bandwidth.
Acoustic signals transmitted via an analog or digital signal path usually suffer from the drawback that the signal path only has a restricted bandwidth such that the transmitted acoustic signal differs considerably from the original signal. For example, in the case of conventional telephone connections, a sampling rate of 8 kHz is used resulting in a maximal signal bandwidth of 4 kHz. Compared to the case of audio CD's, the speech and audio quality is significantly reduced.
Furthermore, many kinds of transmissions show additional bandwidth restrictions. In the case of an analog telephone connection, only frequencies between 300 Hz and 3.4 kHz are transmitted. As a result, only 3.1 kHz bandwidth are available.
In principle, the bandwidth of telephone connections could be increased by using broadband or wideband digital coding and decoding methods (so-called broadband codecs). In such a case, however, both the transmitter and the receiver have to sup-port corresponding coding and decoding methods which would require the implementa-tion of a new standard.
As an alternative, systems for bandwidth extension can be used as described, for ex-ample, in P. Jax, Enhancement of Bandlimited Speech Signals: Algorithms and Theo-retical Bounds, Dissertation, Aachen, Germany, 2002 or E. Larsen, R. M. Aarts, Audio Bandwidth Extension, Wiley, Hoboken, NJ, USA, 2004. These systems are to be im-plemented on the receiver's side only such that existing telephone connections do not have to be changed. In these systems, the missing frequency components of an input signal with small bandwidth are estimated and added to the input signal.
An example of the structure and the corresponding signal flow in such a state of the art bandwidth extension system is illustrated in Fig. 6. In general, both the lower and the upper frequency ranges are re-synthesized.
At block 601, an incoming or received acoustic signal x(n) in digitized form is proc-essed by sub-sampling and block extraction so as to obtain signal vectors x(n). Here, the variable ndenotes the time. In this Figure, it is assumed that the incoming signal x(n) has already been converted to the desired bandwidth by increasing the sampling rate. In this conversion step, no additional frequency components are to be generated which can be achieved, for example, by using appropriate anti-aliasing or anti-imaging filter elements. In order to not amend the transmitted signal, the bandwidth extension is performed only within the missing frequency ranges. Depending on the transmission method, the extension concerns low frequency (for example from 0 to 300 Hz) and/or high frequency (for example 3400 Hz to half of the desired sampling rate) ranges.
In block 602, a narrowband spectral envelope is extracted from the narrowband signal, the narrowband signal being restricted by the bandwidth restrictions of the telephone channel. Via a non-linear mapping, a corresponding broadband envelope signal is es-timated from the narrowband envelope. The mappings are based, for example, on codebook pairs (see J. Epps, W. H. Holmes, A New Technique for Wideband En-hancement of Coded Narrowband Speech, IEEE Workshop on Speech Coding, Con-ference proceedings, pages 174 to 176 June 1999) or on Neural Networks (see J.-M.
Valin R. Lefebvre, Bandwidth Extension of Narrowband Speech for Low Bit-Rate Wideband Coding, IEEE Workshop on Speech Coding, Conference Proceedings, pages 130 to 132, September 2000). In these methods, the entries of the codebooks or the weights of the neural networks are generated using training methods requiring large processor and memory resources.
Furthermore, in block 603, a broadband or wideband excitation signal having a spec-trally flat envelope is generated from the narrowband signal. This excitation signal cor-responds to the signal which would be recorded directly behind the vocal cords, i.e. the excitation signal contains information about voicing and pitch, but not about form and structures or the spectral shaping in general. Thus, to retrieve a complete signal, such as a speech signal, the excitation signal has to be weighted with the spectral envelope.
For the generation of excitation signals, non-linear characteristics (see U.
Kornagel, Spectral Widening of the Excitation Signal for Telephone-Band Speech Enhancement, IWAENC 01, Conference Proceedings, pages 215 to 218, September 2001) such as two-ray rectifying or squaring, for example, may be used.
Acoustic signals transmitted via an analog or digital signal path usually suffer from the drawback that the signal path only has a restricted bandwidth such that the transmitted acoustic signal differs considerably from the original signal. For example, in the case of conventional telephone connections, a sampling rate of 8 kHz is used resulting in a maximal signal bandwidth of 4 kHz. Compared to the case of audio CD's, the speech and audio quality is significantly reduced.
Furthermore, many kinds of transmissions show additional bandwidth restrictions. In the case of an analog telephone connection, only frequencies between 300 Hz and 3.4 kHz are transmitted. As a result, only 3.1 kHz bandwidth are available.
In principle, the bandwidth of telephone connections could be increased by using broadband or wideband digital coding and decoding methods (so-called broadband codecs). In such a case, however, both the transmitter and the receiver have to sup-port corresponding coding and decoding methods which would require the implementa-tion of a new standard.
As an alternative, systems for bandwidth extension can be used as described, for ex-ample, in P. Jax, Enhancement of Bandlimited Speech Signals: Algorithms and Theo-retical Bounds, Dissertation, Aachen, Germany, 2002 or E. Larsen, R. M. Aarts, Audio Bandwidth Extension, Wiley, Hoboken, NJ, USA, 2004. These systems are to be im-plemented on the receiver's side only such that existing telephone connections do not have to be changed. In these systems, the missing frequency components of an input signal with small bandwidth are estimated and added to the input signal.
An example of the structure and the corresponding signal flow in such a state of the art bandwidth extension system is illustrated in Fig. 6. In general, both the lower and the upper frequency ranges are re-synthesized.
At block 601, an incoming or received acoustic signal x(n) in digitized form is proc-essed by sub-sampling and block extraction so as to obtain signal vectors x(n). Here, the variable ndenotes the time. In this Figure, it is assumed that the incoming signal x(n) has already been converted to the desired bandwidth by increasing the sampling rate. In this conversion step, no additional frequency components are to be generated which can be achieved, for example, by using appropriate anti-aliasing or anti-imaging filter elements. In order to not amend the transmitted signal, the bandwidth extension is performed only within the missing frequency ranges. Depending on the transmission method, the extension concerns low frequency (for example from 0 to 300 Hz) and/or high frequency (for example 3400 Hz to half of the desired sampling rate) ranges.
In block 602, a narrowband spectral envelope is extracted from the narrowband signal, the narrowband signal being restricted by the bandwidth restrictions of the telephone channel. Via a non-linear mapping, a corresponding broadband envelope signal is es-timated from the narrowband envelope. The mappings are based, for example, on codebook pairs (see J. Epps, W. H. Holmes, A New Technique for Wideband En-hancement of Coded Narrowband Speech, IEEE Workshop on Speech Coding, Con-ference proceedings, pages 174 to 176 June 1999) or on Neural Networks (see J.-M.
Valin R. Lefebvre, Bandwidth Extension of Narrowband Speech for Low Bit-Rate Wideband Coding, IEEE Workshop on Speech Coding, Conference Proceedings, pages 130 to 132, September 2000). In these methods, the entries of the codebooks or the weights of the neural networks are generated using training methods requiring large processor and memory resources.
Furthermore, in block 603, a broadband or wideband excitation signal having a spec-trally flat envelope is generated from the narrowband signal. This excitation signal cor-responds to the signal which would be recorded directly behind the vocal cords, i.e. the excitation signal contains information about voicing and pitch, but not about form and structures or the spectral shaping in general. Thus, to retrieve a complete signal, such as a speech signal, the excitation signal has to be weighted with the spectral envelope.
For the generation of excitation signals, non-linear characteristics (see U.
Kornagel, Spectral Widening of the Excitation Signal for Telephone-Band Speech Enhancement, IWAENC 01, Conference Proceedings, pages 215 to 218, September 2001) such as two-ray rectifying or squaring, for example, may be used.
2 For bandwidth extension, the excitation signal xeSC (n) is spectrally colored using the envelope in block 604. After that, the spectral ranges used for the extension are ex-tracted using a band stop filter in block 606 resulting in signal vectors yex1(n) . The band stop filter can be effective, for example, in the range from 200 to 3700 Hz.
The signal vectors x(n) of the received signal are passed through a complementary band pass filter in block 605. Then, the signal components yeS, (n) and y,,(n) are added to obtain a signal vector y(n) with extended bandwidth. In block 607, the differ-ent signal vectors are assembled again and an over-sampling is performed resulting in a signal y(n).
In these prior art systems, the elements and their parameters are implemented once and, then, remain unchanged. Thus, all incoming acoustic signals are treated the same way. In view of this, it is an object underlying the present invention to provide a more flexible method and apparatus for providing an acoustic signal with extended band-width.
This problem is solved by the method according to claim 1 and the apparatus accord-ing to claim 16.
In accordance with the invention, a method for providing an acoustic signal with ex-tended bandwidth is provided, comprising:
(a) automatically determining a current upper and a current lower bandwidth limit of received acoustic signal, (b) automatically determining at least one complementary signal to complement the received acoustic signal between a predefined lower broadband bandwidth limit and the current lower bandwidth limit and/or between the current upper band-width limit and a predefined upper broadband bandwidth limit, wherein the pre-defined lower broadband bandwidth limit is smaller than the current bandwidth
The signal vectors x(n) of the received signal are passed through a complementary band pass filter in block 605. Then, the signal components yeS, (n) and y,,(n) are added to obtain a signal vector y(n) with extended bandwidth. In block 607, the differ-ent signal vectors are assembled again and an over-sampling is performed resulting in a signal y(n).
In these prior art systems, the elements and their parameters are implemented once and, then, remain unchanged. Thus, all incoming acoustic signals are treated the same way. In view of this, it is an object underlying the present invention to provide a more flexible method and apparatus for providing an acoustic signal with extended band-width.
This problem is solved by the method according to claim 1 and the apparatus accord-ing to claim 16.
In accordance with the invention, a method for providing an acoustic signal with ex-tended bandwidth is provided, comprising:
(a) automatically determining a current upper and a current lower bandwidth limit of received acoustic signal, (b) automatically determining at least one complementary signal to complement the received acoustic signal between a predefined lower broadband bandwidth limit and the current lower bandwidth limit and/or between the current upper band-width limit and a predefined upper broadband bandwidth limit, wherein the pre-defined lower broadband bandwidth limit is smaller than the current bandwidth
3 limit and the predefined upper broadband bandwidth limit is larger than the cur-rent upper bandwidth limit, (c) automatically assembling the at least one complementary signal and the re-ceived acoustic signal to obtain an acoustic signal with extended bandwidth.
By determining current upper and lower bandwidth limits of a received acoustic signal and determining a complementary signal between the current bandwidth limits and the respective predefined broadband (or wideband) bandwidth limits, the method according to the invention allows an adaptation of the bandwidth extension to the acoustic signal actually received. For example, when the transmitter uses an ISDN telephone, a broader frequency range is used compared to the case of a mobile phone with a hands-free system. Therefore, the bandwidth of a received acoustic signal will be ex-tended only in those ranges where it is necessary so that the quality of the resulting signal is very high.
In this way, on the one hand, no spectral gaps will occur even if the received signal covers only a very narrow frequency range. On the other hand, when receiving signals covering a relatively broad frequency range, no frequencies are cut-off when determin-ing the complementary signal.
The received acoustic signal may be a digital signal or may be digitized. In the above method, steps (a) to (c) may be preceded by the step of converting the received acous-tic signal to a predetermined sampling rate. Furthermore, steps (a) to (c) may be pre-ceded by the step of extracting a signal vector from the acoustic signal, in particular, the converted acoustic signal. The signal vector may be obtained by sub-sampling the acoustic signal and may comprise a predefined number of entries. Then, subsequent (in time) signal vectors may overlap. The use of signal vectors simplifies further proc-essing of the signals.
Steps (a) to (c) may be preceded by the step of determining a spectral vector of the received acoustic signal. In particular, a window function may be applied to signal vec-tors of the received acoustic signal. For example, a Hann or a Hamming window may be used (see K. D. Kammeyer, K. Kroschel, Digitale Signalverarbeitung, 4th Edition, Teubner, Stuttgart, Germany 1997). Signal vectors, in particular the signal vectors
By determining current upper and lower bandwidth limits of a received acoustic signal and determining a complementary signal between the current bandwidth limits and the respective predefined broadband (or wideband) bandwidth limits, the method according to the invention allows an adaptation of the bandwidth extension to the acoustic signal actually received. For example, when the transmitter uses an ISDN telephone, a broader frequency range is used compared to the case of a mobile phone with a hands-free system. Therefore, the bandwidth of a received acoustic signal will be ex-tended only in those ranges where it is necessary so that the quality of the resulting signal is very high.
In this way, on the one hand, no spectral gaps will occur even if the received signal covers only a very narrow frequency range. On the other hand, when receiving signals covering a relatively broad frequency range, no frequencies are cut-off when determin-ing the complementary signal.
The received acoustic signal may be a digital signal or may be digitized. In the above method, steps (a) to (c) may be preceded by the step of converting the received acous-tic signal to a predetermined sampling rate. Furthermore, steps (a) to (c) may be pre-ceded by the step of extracting a signal vector from the acoustic signal, in particular, the converted acoustic signal. The signal vector may be obtained by sub-sampling the acoustic signal and may comprise a predefined number of entries. Then, subsequent (in time) signal vectors may overlap. The use of signal vectors simplifies further proc-essing of the signals.
Steps (a) to (c) may be preceded by the step of determining a spectral vector of the received acoustic signal. In particular, a window function may be applied to signal vec-tors of the received acoustic signal. For example, a Hann or a Hamming window may be used (see K. D. Kammeyer, K. Kroschel, Digitale Signalverarbeitung, 4th Edition, Teubner, Stuttgart, Germany 1997). Signal vectors, in particular the signal vectors
4 weighted in this way, may be transformed into the Fourier domain using a discrete Fou-rier transform. The resulting vector is a short-term spectral vector. This allows for fur-ther processing in the Fourier domain.
In the above methods, step (b) may comprise determining a broadband spectral enve-lope signal and a broadband excitation signal between the lower and upper broadband bandwidth limits such that the product of spectral envelope signal and excitation signal corresponds to the received acoustic signal according to a predetermined criterion.
Such a decomposition into an envelope signal and an excitation signal simplifies de-termining the current bandwidth limits and increases the accuracy when determining a complementary signal.
Step (a) may comprise comparing a determined broadband spectral envelope signal and a long-term power spectrum of the received acoustic signal. It turned out that the long-term power spectrum is a suitable basis for determining current bandwidth limits of the acoustic signal.
Thus, if current bandwidth limits have been determined in step (a) in this way using a broadband spectral envelope signal of the received acoustic signal, determining a complementary signal in step (b) based on these current bandwidth limits and compris-ing determination of an envelope signal enables to iteratively adapt the current band-width limits by comparing again the (newly) determined envelope signal and a long-term power spectrum. In other words, determining current bandwidth limits in step (a) may use a spectral envelope signal determined according to step (b), particularly in a preceding step or in a preceding iteration of the method.
In particular, if the received acoustic signal has been transformed into the Fourier do-main, determining a long-term power spectrum may comprise performing a first order recursive smoothing of the absolute values squared of the sub-band signals corre-sponding to the acoustic signal. This can be done, in particular, only if a wanted signal, such as a speech signal, has been detected in the received acoustic signal.
In addition, the long-term power spectrum may be normalized, particularly with respect to a long-term power spectrum within predetermined frequency limits.
In the above methods, step (b) may comprise determining a broadband spectral enve-lope signal and a broadband excitation signal between the lower and upper broadband bandwidth limits such that the product of spectral envelope signal and excitation signal corresponds to the received acoustic signal according to a predetermined criterion.
Such a decomposition into an envelope signal and an excitation signal simplifies de-termining the current bandwidth limits and increases the accuracy when determining a complementary signal.
Step (a) may comprise comparing a determined broadband spectral envelope signal and a long-term power spectrum of the received acoustic signal. It turned out that the long-term power spectrum is a suitable basis for determining current bandwidth limits of the acoustic signal.
Thus, if current bandwidth limits have been determined in step (a) in this way using a broadband spectral envelope signal of the received acoustic signal, determining a complementary signal in step (b) based on these current bandwidth limits and compris-ing determination of an envelope signal enables to iteratively adapt the current band-width limits by comparing again the (newly) determined envelope signal and a long-term power spectrum. In other words, determining current bandwidth limits in step (a) may use a spectral envelope signal determined according to step (b), particularly in a preceding step or in a preceding iteration of the method.
In particular, if the received acoustic signal has been transformed into the Fourier do-main, determining a long-term power spectrum may comprise performing a first order recursive smoothing of the absolute values squared of the sub-band signals corre-sponding to the acoustic signal. This can be done, in particular, only if a wanted signal, such as a speech signal, has been detected in the received acoustic signal.
In addition, the long-term power spectrum may be normalized, particularly with respect to a long-term power spectrum within predetermined frequency limits.
5 Alternatively, the long-term power spectrum may be determined in the time domain.
This can be done by determining the auto-correlation and performing an LPC
analysis to obtain corresponding prediction coefficients.
The comparing step may comprise selecting the minimal and maximal frequency for which the long-term power spectrum is larger than or equal to the power spectrum of the determined broadband spectral envelope signal plus a predetermined constant.
This is a particularly simple and reliable way to determine the bandwidth limits. The predetermined constant can be chosen based on empirical or theoretical data.
The predetermined constant may be negative.
In the above methods, determining a broadband spectral envelope signal may com-prise selecting an envelope signal from a codebook according to a predetermined crite-rion.
By using codebooks, the required computing power can be reduced for determining an envelope signal. In principle, different kinds of criteria can be used when selecting an envelope signal from a codebook. In particular, using a predetermined distance crite-rion such as a cepstral distance can be used, particularly if the codebook entries have the form of cepstral vectors.
In particular, selecting an envelope signal may comprise equalizing the received acoustic signal and selecting an envelope signal from the codebook having minimal distance to the equalized acoustic signal according to a predetermined distance crite-rion, in particular, having a minimal cepstral distance.
Equalizing the acoustic signal allows to modify it such that a comparison with envelope signals from the codebook can be simplified. In particular, the received acoustic signal can be equalized in such a way that the resulting signal shows a long-term power spectrum corresponding to the long-term power spectrum of the signal used for training the codebook. Equalizing can be restricted to frequencies between the current upper and lower bandwidth limits of the received acoustic signal; outside these limits, the sig-nal may remain unchanged. In particular, equalizing the received acoustic signal can
This can be done by determining the auto-correlation and performing an LPC
analysis to obtain corresponding prediction coefficients.
The comparing step may comprise selecting the minimal and maximal frequency for which the long-term power spectrum is larger than or equal to the power spectrum of the determined broadband spectral envelope signal plus a predetermined constant.
This is a particularly simple and reliable way to determine the bandwidth limits. The predetermined constant can be chosen based on empirical or theoretical data.
The predetermined constant may be negative.
In the above methods, determining a broadband spectral envelope signal may com-prise selecting an envelope signal from a codebook according to a predetermined crite-rion.
By using codebooks, the required computing power can be reduced for determining an envelope signal. In principle, different kinds of criteria can be used when selecting an envelope signal from a codebook. In particular, using a predetermined distance crite-rion such as a cepstral distance can be used, particularly if the codebook entries have the form of cepstral vectors.
In particular, selecting an envelope signal may comprise equalizing the received acoustic signal and selecting an envelope signal from the codebook having minimal distance to the equalized acoustic signal according to a predetermined distance crite-rion, in particular, having a minimal cepstral distance.
Equalizing the acoustic signal allows to modify it such that a comparison with envelope signals from the codebook can be simplified. In particular, the received acoustic signal can be equalized in such a way that the resulting signal shows a long-term power spectrum corresponding to the long-term power spectrum of the signal used for training the codebook. Equalizing can be restricted to frequencies between the current upper and lower bandwidth limits of the received acoustic signal; outside these limits, the sig-nal may remain unchanged. In particular, equalizing the received acoustic signal can
6 be performed using a normalized long-term power spectrum of the signal used for training the codebooks, particularly using the normalized long-term power spectrum divided by the normalized long-term power spectrum of the received acoustic signal itself.
The codebook may comprise pairs of corresponding envelope signals, each pair com-prising a broadband envelope signal between the lower and upper broadband band-width limits and a corresponding narrowband envelope signal between a lower narrow-band bandwidth limit being larger than the lower broadband bandwidth limit and an upper narrowband bandwidth limit being smaller than the upper broadband bandwidth limit, and selecting an envelope signal may comprise determining a narrowband enve-lope signal having minimal distance to the equalized acoustic signal according to the predetermined distance criterion and selecting the corresponding broadband envelope signal of this pair.
In this way, a simple comparison between the received acoustic signal and the ele-ments of the codebook can be performed as narrowband signals usually match a re-ceived acoustic signal with a narrow bandwidth more closely.
When using a cepstral distance to select an envelope signal, the received acoustic sig-nal, particularly in its equalized form, has to be transformed into the cepstral domain.
Thus, the step of selecting an envelope signal can further comprise the steps of deter-mining the absolute value squared of the sub-band signals of the received acoustic signal, determining an auto-correlation in the time domain, particularly by performing an inverse discrete Fourier transform on the vector of the absolute value squared, de-termining prediction coefficients, particularly using the Levinson-Durbin algorithm, per-forming a recursion to obtain the cepstral coefficients.
In order to determine a spectral envelope from the cepstral vectors, the method may further comprise the steps of recursively transforming a cepstral vector into prediction error coefficients, augmenting the prediction error filter vector by adding a predeter-mined number of zeros and subsequently performing a discrete Fourier transform to obtain an inverse spectrum, determining the reciprocal of each sub-band component to obtain a spectral envelope vector.
The codebook may comprise pairs of corresponding envelope signals, each pair com-prising a broadband envelope signal between the lower and upper broadband band-width limits and a corresponding narrowband envelope signal between a lower narrow-band bandwidth limit being larger than the lower broadband bandwidth limit and an upper narrowband bandwidth limit being smaller than the upper broadband bandwidth limit, and selecting an envelope signal may comprise determining a narrowband enve-lope signal having minimal distance to the equalized acoustic signal according to the predetermined distance criterion and selecting the corresponding broadband envelope signal of this pair.
In this way, a simple comparison between the received acoustic signal and the ele-ments of the codebook can be performed as narrowband signals usually match a re-ceived acoustic signal with a narrow bandwidth more closely.
When using a cepstral distance to select an envelope signal, the received acoustic sig-nal, particularly in its equalized form, has to be transformed into the cepstral domain.
Thus, the step of selecting an envelope signal can further comprise the steps of deter-mining the absolute value squared of the sub-band signals of the received acoustic signal, determining an auto-correlation in the time domain, particularly by performing an inverse discrete Fourier transform on the vector of the absolute value squared, de-termining prediction coefficients, particularly using the Levinson-Durbin algorithm, per-forming a recursion to obtain the cepstral coefficients.
In order to determine a spectral envelope from the cepstral vectors, the method may further comprise the steps of recursively transforming a cepstral vector into prediction error coefficients, augmenting the prediction error filter vector by adding a predeter-mined number of zeros and subsequently performing a discrete Fourier transform to obtain an inverse spectrum, determining the reciprocal of each sub-band component to obtain a spectral envelope vector.
7 In the above methods, the step of selecting an envelope signal may be preceded by providing adapted narrowband codebook envelope signals being adapted to the cur-rent lower and upper bandwidth limits.
Such an adaptation of the codebook entries allows for an improved selection of a cor-responding envelope signal from the codebook. In particular, if the received acoustic signal shows a broader bandwidth than the original narrowband envelope signals of the codebook, the adaptation would result in envelope signals in the codebook having an extended bandwidth. In this way, particularly fricatives can be more reliably detected.
The providing step may comprise processing broadband codebook envelope signals using a long-term power spectrum of the received acoustic signal.
Due to the use of the power spectrum of the received acoustic signal, a suitable adap-tation to the acoustic signal can be obtained. The long-term power spectrum may be normalized; furthermore, the long-term power spectrum of the received acoustic signal may be divided by a normalized long-term power spectrum of a broadband signal used for training of the codebook. The processing of the broadband codebook envelope sig-nals may be performed only for frequencies outside the current bandwidth limits; within the bandwidth limits, the envelope signals may remain unchanged. Processing using the long-term power spectrum may comprise weighting broadband codebook envelope signal vectors using the long-term power spectrum of the received acoustic signal.
In the above methods, determining a broadband excitation signal may be based on prediction error filtering and/or a non-linear characteristic. In this way, suitable excita-tion signals can be generated. Possible non-linear characteristics are disclosed, for example, in U. Kornagel, Spectral Widening of the Excitation Signal for Telephone-Band Speech Enhancement.
In the above methods, the at least one complementary signal may be based on a product of the determined broadband spectral envelope and the determined broadband excitation signal, and step (c) may comprise summing the received acoustic signal be-tween the current lower and upper bandwidth limits and the at least one complemen-tary signal being restricted to the band between the lower broadband bandwidth limit
Such an adaptation of the codebook entries allows for an improved selection of a cor-responding envelope signal from the codebook. In particular, if the received acoustic signal shows a broader bandwidth than the original narrowband envelope signals of the codebook, the adaptation would result in envelope signals in the codebook having an extended bandwidth. In this way, particularly fricatives can be more reliably detected.
The providing step may comprise processing broadband codebook envelope signals using a long-term power spectrum of the received acoustic signal.
Due to the use of the power spectrum of the received acoustic signal, a suitable adap-tation to the acoustic signal can be obtained. The long-term power spectrum may be normalized; furthermore, the long-term power spectrum of the received acoustic signal may be divided by a normalized long-term power spectrum of a broadband signal used for training of the codebook. The processing of the broadband codebook envelope sig-nals may be performed only for frequencies outside the current bandwidth limits; within the bandwidth limits, the envelope signals may remain unchanged. Processing using the long-term power spectrum may comprise weighting broadband codebook envelope signal vectors using the long-term power spectrum of the received acoustic signal.
In the above methods, determining a broadband excitation signal may be based on prediction error filtering and/or a non-linear characteristic. In this way, suitable excita-tion signals can be generated. Possible non-linear characteristics are disclosed, for example, in U. Kornagel, Spectral Widening of the Excitation Signal for Telephone-Band Speech Enhancement.
In the above methods, the at least one complementary signal may be based on a product of the determined broadband spectral envelope and the determined broadband excitation signal, and step (c) may comprise summing the received acoustic signal be-tween the current lower and upper bandwidth limits and the at least one complemen-tary signal being restricted to the band between the lower broadband bandwidth limit
8 and a current lower bandwidth limit and/or to the band between the current upper bandwidth limit and the upper broadband bandwidth limit.
Thus, the complementary signal is based on spectrally coloring the excitation signal using the envelope signal. By adding a complementary signal only outside the current bandwidth limits of the received acoustic signal, artifacts are avoided in the resulting signal with extended bandwidth.
Step (c) may also comprise adapting the power of the complementary signal and/or the received acoustic signal. With this step, the power of the received acoustic signal can be maintained.
In the above-described methods, at least one of the steps may be performed in the cepstral domain. Particularly if the entries of the codebook are cepstral vectors, this allows for performing the method in a simpler way.
Steps (a) to (c) of the above methods may be repeated at predetermined time intervals.
Then, the repeated adaptation to the currently received acoustic signal leads to a per-manent high quality of the resulting broadband signal.
Steps (a) to (c) of the above methods may be repeated only if a wanted signal compo-nent, such as speech activity, is detected in the received acoustic signal.
Particularly in the case of speech signals, an extension of the bandwidth of the received acoustic sig-nal is advantageous. Thus, restricting the method to the case of detected speech activ-ity reduces the required computing power and avoids the presence of artifacts due to mal-adaptation.
The invention also provides a computer program product comprising one or more com-puter-readable media having computer-executable instructions for performing the steps of the above-described methods when run on a computer.
Furthermore, an apparatus for providing an acoustic signal with extended bandwidth is provided, comprising:
Thus, the complementary signal is based on spectrally coloring the excitation signal using the envelope signal. By adding a complementary signal only outside the current bandwidth limits of the received acoustic signal, artifacts are avoided in the resulting signal with extended bandwidth.
Step (c) may also comprise adapting the power of the complementary signal and/or the received acoustic signal. With this step, the power of the received acoustic signal can be maintained.
In the above-described methods, at least one of the steps may be performed in the cepstral domain. Particularly if the entries of the codebook are cepstral vectors, this allows for performing the method in a simpler way.
Steps (a) to (c) of the above methods may be repeated at predetermined time intervals.
Then, the repeated adaptation to the currently received acoustic signal leads to a per-manent high quality of the resulting broadband signal.
Steps (a) to (c) of the above methods may be repeated only if a wanted signal compo-nent, such as speech activity, is detected in the received acoustic signal.
Particularly in the case of speech signals, an extension of the bandwidth of the received acoustic sig-nal is advantageous. Thus, restricting the method to the case of detected speech activ-ity reduces the required computing power and avoids the presence of artifacts due to mal-adaptation.
The invention also provides a computer program product comprising one or more com-puter-readable media having computer-executable instructions for performing the steps of the above-described methods when run on a computer.
Furthermore, an apparatus for providing an acoustic signal with extended bandwidth is provided, comprising:
9 bandwidth determining means for automatically determining a current upper and a current lower bandwidth limit of a received acoustic signal, complementary signal means for automatically determining at least one com-plementary signal to complement the received acoustic signal between a prede-fined lower broadband bandwidth limit and the current lower bandwidth limit and/or between the current upper bandwidth limit and a predefined upper broadband bandwidth limit, wherein the predefined lower broadband bandwidth limit is smaller than the current bandwidth limit and the predefined upper broad-band bandwidth limit is larger than the current upper bandwidth limit, and assembling means for automatically assembling the at least one complementary signal and the received acoustic signal to obtain an acoustic signal with ex-tended bandwidth.
Analogous to the above-described method, such an apparatus provides an advanta-geous way to extend the bandwidth of a received acoustic signal. In particular, due to the determination of current upper and lower bandwidth limits of the received acoustic signal and a corresponding determination of a complementary signal, the quality of the resulting output signal is increased compared to the case of bandwidth extension sys-tems with fixed parameters.
The complementary signal means may comprise a means for determining a broadband spectral envelope signal and a broadband excitation signal between the lower and up-per broadband bandwidth limits such that the product of spectral envelope signal and excitation signal corresponds to the received acoustic signal according to a predeter-mined criterion.
The bandwidth determining means may be configured to compare a determined broadband spectral envelope signal and a long-term power spectrum of the received acoustic signal.
The bandwidth determining means may be configured to select the minimal and maxi-mal frequency for which the long-term power spectrum is larger than or equal to the power spectrum of the determined broadband spectral envelope signal plus a prede-termined constant.
In the above-described apparatus, the means for determining a broadband spectral envelope signal may comprise a means for selecting an envelope signal from a code-book according to a predetermined criterion.
The means for selecting an envelope signal may be configured to equalize the re-ceived acoustic signal and select an envelope signal from the codebook having mini-mal distance to the equalized acoustic signal according to a predetermined distance criterion, in particular, having a minimal cepstral distance.
In the above-described apparatus, the codebook may comprise pairs of corresponding envelope signals, each pair comprising a broadband envelope signal between the lower and upper broadband bandwidth limits and a corresponding narrowband enve-lope signal between a lower narrowband bandwidth limit being larger than the lower broadband bandwidth limit and an upper narrowband bandwidth limit being smaller than the upper broadband bandwidth limit, and the means for selecting an envelope signal may be configured to determine a narrowband envelope signal having minimal distance to the equalized acoustic signal according to the predetermined distance crite-rion and to select the corresponding broadband envelope signal of this pair.
The means for determining a broadband spectral envelope signal may comprise a means for providing adapted narrowband codebook envelope signals being adapted to the current lower and upper bandwidth limits.
The means for providing may be configured to process the broadband codebook enve-lope signal using a long-term power spectrum of the received acoustic signal.
In the above-described apparatus, the means for determining a broadband excitation signal may be configured to determine the broadband excitation signal based on pre-diction error filtering and/or a non-linear characteristic.
The at least one complementary signal may be based on a product of the determined broadband spectral envelope and the determined broadband excitation signal, and the assembling means may be configured to sum the received acoustic signal between the current lower and upper bandwidth limits and the at least one complementary signal being restricted to the band between the lower broadband bandwidth limit and a cur-rent lower bandwidth limit and/or to the band between the current upper bandwidth limit and the upper broadband bandwidth limit.
In the above-described apparatus, at least one of the means may be configured to per-form at least part of its function in the cepstral domain.
The means of the above-described apparatus may be configured to perform their re-spective function repeatedly at predetermined time intervals.
The apparatus may further comprise a wanted signal detector, in particular, a speech detector, and the means may be configured to perform their respective function only if a wanted signal component is detected in the received acoustic signal.
Further features and advantages of the invention will be described in the following with reference to the figures.
Fig. 1 illustrates the structure of an example of an apparatus for providing an acoustic signal with extended bandwidth;
Fig. 2 is a flow diagram of an example of a method for providing an acoustic signal with extended bandwidth;
Fig. 3 illustrates an example of a normalized long-term power spectrum for training a codebook;
Fig 4 illustrates examples of codebook entries;
Fig. 5 illustrates the determination of current bandwidth limits;
Fig. 6 illustrates the structure of a prior art system.
Fig. 1 shows the structure of the signal flow in an apparatus for providing an acoustic signal with extended bandwidth. Fig. 2 is a flow diagram illustrating an example of a method for providing an acoustic signal with extended bandwidth which could be per-formed by the apparatus corresponding to Fig. 1. In view of this, Fig.'s 1 and 2 will be described in the following simultaneously.
According to step 201, an acoustic signal, such as a speech signal, is received via a telephone line. Because of the restricted bandwidth of the telephone line, an extension of the bandwidth is desired to improve the signal quality. Thus, the signal is to be aug-mented so as to obtain a predetermined broader bandwidth. It is to be understood that the method described in the following can be used for bandwidth extension independ-ent of the type of incoming signal and independent of the type of transmission line, i.e., it need not be a telephone line.
The acoustic signal x(n) received by block 101 has already been pre-processed by increasing the sampling rate up to the predetermined broadband or wideband band-width. In this way, however, no additional frequency components are generated.
This can be achieved, for example, by using suitable anti-aliasing or anti-imaging filters.
This kind of bandwidth extension, preferably, is performed only for the "missing" fre-quency ranges; in the case of an analog telephone line, these ranges may be between 0 and 300 Hz and 3400 Hz up to half of the desired sampling rate, for example, up to 3700 Hz.
From the resulting signal x(n), n denoting the time variable, signal vectors x(n) are generated (step 202). This can be achieved by taking every r sampling values up to a certain length. Thus, a signal vector with N. elements has the form:
x(n) = [x(nr), x(nr -1),..., x(nr - NQõo + 1)]T.
It is to be noted that an overlap may exist between neighboring signal vectors. For a desired or final sampling rate of 11.025 kHz, one may take the following values:
r = 64, NanQ = 256.
After that (step 203), a windowing procedure is performed on the signal vector so as to obtain a windowed signal vector xjn) :
x W (n) = Fx(n).
The window matrix F is a diagonal matrix of the form ho 0 0 ... 0 0 h, 0 ... 0 F= 0 0 h2 ... 0 0 0 0 ... h The elements of this matrix can be chosen corresponding to different kinds of windows.
Typical windows are the Hann or Hamming window. The weighted signal vectors are transformed into the Fourier domain using a discrete Fourier transform:
XW(n) = DFT{xw(n)}.
The resulting short-term spectral vector has the form:
X w(n) = f X(e i~ , n), X(e'n', n),..., X(e'n" , n),..., X(e'O"D~-1 , n)~ , ' wherein S2, denotes the frequency variable.
Based on the spectral vectors, a long-term power spectrum of the received acoustic signal is determined in block 102 (step 204). There are different possibilities to esti-mate such a long-term power spectrum. According to one alternative, a first order re-cursive smoothing is performed on the absolute value squared of the sub-band signals X (e'"" , n) :
~3freS~ (S2F,, n -1) + (1- /3fre ) X. (e'n", n)I , during speech activity S. (SZ'õ n) _ S~ (52u , n -1), else.
Preferably, the time constant 8fre is chosen to be close to 1 (0 /3fre < 1) so as to obtain a sufficiently large averaging time.
In principle, the recursive smoothing according to the first line of the above equation may be performed continuously. However, in order to avoid any artefacts, it may be performed only if a wanted signal component is present in the received acoustic signal, for example, if speech activity is detected. For this purpose, a speech detector may be provided as described, for example, in E. Hansler, G. Schmidt, Acoustic Echo and Noise Control - A Practical Approach, Wiley, Hoboken, NJ, USA, 2004.
In order to simplify the further processing, the long-term power spectrum may be nor-malized to the long-term power within a predefined frequency band:
S~ (52P, n) Sxc,norm (Q/t 5 n) _ P.
E(f2,,, n) /l=fl, The band limits S2. and S2Pu denote the lower and upper limits of a predefined fre-quency band. For example, this frequency band may correspond to a telephone band with minimal bandwidth for which the present method is to be used, for example, the limits may be 400 Hz and 3300 Hz. Preferably, the limits correspond to a band which is smaller or at most equal to the frequency band of the narrow frequency band within which the codebook described below has been trained; these limits being denoted by S2, and S2n.
Alternatively, to determine the long-term power spectrum in the frequency domain, an estimation can be performed in the time domain as well. For this purpose, an auto-correlation is estimated for about 10 to 20 sampling cycles of offset.
Afterwards, predic-tion coefficients can be determined using an LPC (linear predictive coding) analysis.
The long-term power spectrum is obtained via a discrete Fourier transform and a divi-sion.
In block 103 (step 205), the acoustic signal is equalized. The equalizing is performed on the spectral vector determined above:
Xeg (n) = Heg (n)X,, (n).
The equalizing matrix Heq (n) is a diagonal matrix of the form He9(en) 0 ... 0 _ 0 Heq(e'"',n) ... 0 He9(n) 0 0 ... He9(e'Q" -1,n) with the entries 1 if ((52p < S2, (n - 1)) or (0, > SZõ (n -1))) He9 (e , n) = SxX,norm (52,, n), else Sxx,norm (o{( I n) and Heq,max , if Heq (e j"v, n) > Hee.m.
He9 (e'~" , n) = Hevmin I lf (He9 (e,fl, I n) < He9,min Heq (e'n" , n), else, I n the equations above, S2, (n -1) and S2u (n -1) denote the current lower and upper bandwidth limits of the received acoustic signal. Thus, for obtaining an updated equal-ized signal, the bandwidth limits at time (n -1) are taken as the current bandwidth lim-its. Furthermore, Sxs,norm (RU In) denotes the normalized long-term power spectrum of the broadband signal which has been used for training the codebook.
Normalizing of such a power spectrum is performed analogously to the case of the long-term power spectrum of the received acoustic signal described above. An example for such a nor-malized long-term power spectrum used for training a codebook is shown in Figure 3.
The equalizing is restricted to minimal and maximum values, for example, to He9,m;,, _ -12 dB, Heg,max =12 dB.
As can be seen from the above, the acoustic signal is equalized only within the current bandwidth limits one time step before. Outside these bandwidth limits, no equalizing takes place.
In the following, determining a broadband spectrum envelope will be described in more detail. An envelope signal corresponding to the received acoustic signal will be deter-mined using a codebook. The used codebook comprises a number of pairs of corre-sponding narrowband and broadband envelope signals. The codebook has been ob-tained by training with a large database on the basis of a starting long-term power spectrum (see Y. Linde, A. Buzo, R. M. Gray, An Algorithm for Vector Quantizer De-sign, IEEE Trans. Comm., vol. COM-28, no. 1, pages 84 - 95, Jan. 1980).
As indicated in Figure 2, the codebook entries are adapted in step 206 (block 104). In particular, the narrowband codebook entries c; .S (n) are adapted.
This is achieved by starting with the broadband entries of the codebook. If the broad-band envelope signals are provided as cepstral vectors c; b(n) , the corresponding spectra C; h(n) are determined. Based on these broadband spectral envelopes, the adapted or optimized narrowband spectra are determined by a multiplication with a weighting matrix:
C;,s (n) = Amoa (n)C;,b (n)=
The weighting matrix is a diagonal matrix of the form:
Hmod (e'" n) 0 . . . 0 0 Hmod (e'o' I n) . . . 0 Hmod (n) -0 0 ... Hmod (e,QNDFT-, e n) with the entries 1, if~SZ,(n-1)<S2IU <S2õ(n-1)~, joa Hmod (e n) - Sxx,norm Ru . n) !~ else.
Sz,norm \~LU , n) Afterwards, cepstral vectors are determined from the resulting spectral narrowband envelopes.
The conversion from spectral vectors to cepstral vectors and vice versa will be de-scribed in the following with respect to step 207 in which broadband spectral envelopes are determined (block 105).
A broadband spectral envelope from the codebook matching the acoustic signal best is determined by comparing the narrowband codebook entries with the spectral envelope of the spectrum of the acoustic signal (after equalizing). The narrowband codebook entry is selected that has the smallness distance to the acoustic signal spectrum. In principle, different distance criteria can be used. The cepstral distance is particularly useful as the codebook entries are provided in the form of cepstral vectors.
When an optimal narrowband codebook entry has been selected, the corresponding broadband codebook entry is determined as the optimal broadband spectral envelope for the received acoustic signal. Due to the adaptation of the narrowband codebook entries as described above, an optimal narrowband envelope can be selected in a very reliable way.
Converting a spectral vector, particularly of the received acoustic signal, to a cepstral vector can be achieved by:
1. Determining the absolute value squared of each sub-band signal Xeq (e'n, , n).
2. Applying an inverse discrete Fourier transform on this vector results in an esti-mation of the auto-correlation in the time domain.
3. Using the Levinson-Durbin algorithm, prediction coefficients (with an order of about 10 to 20) can be determined from the auto-correlation.
4. By performing a recursion with respect to the order, the prediction coefficients are used to determine the cepstral coefficients. Usually, the order corresponds to one and a half of the prediction order.
The optimal cepstral vector of the broadband codebook is designated by Cpt b(n). The resulting broadband spectral envelope has the form:
~ T
Copt,b (n) = lcopt,b(e j", , n), Copt,b (e'~' , n),..., Copt b(eJ vDF7 n)]
Conversion of cepstral vectors into spectral vectors is achieved by:
1. Converting the cepstral vectors using a recursion with respect to the order (as above) to obtain prediction error filter coefficients.
2. By augmenting the prediction error filter vector by a predetermined number of zeros and subsequent performing of a discrete Fourier transform, an inverse spectrum is obtained.
3. By determining the reciprocal of each sub-band component, the vector Copt,b (n) is generated. Divisions by zero have to be treated separately, for example by adding a suitable constant.
Fig. 4 illustrates an example of a codebook with four pairs of entries. In each diagram, a corresponding original narrowband envelope, and a corresponding adapted narrow-band envelope are shown. The original broadband and narrowband codebook entries have been obtained on the basis of a large database for an ISDN telephone connec-tion. As can be seen in this figure, after the adaptation, the resulting optimized entries have a higher upper limit frequency. This allows for an improved detection of fricatives.
In step 208 (block 103), an excitation signal corresponding to the received acoustic signal is generated. This broadband excitation signal shows a spectrally flat envelope.
It corresponds to a signal which would be recorded directly behind the vocal cords.
For determining a broadband excitation signal, first of all, the spectral envelope of the equalized short-term spectrum Xeq (n) is estimated in the form of prediction error filter coefficients. Applying an inverse discrete Fourier transform on this spectral vector al-lows to determine the corresponding time signal. After that, the vector in the time do-main is filtered by a prediction error filter. The corresponding filter coefficients are those that have been determined previously.
Then, a non-linear characteristic, such as a two-way rectification or squaring, is applied to the filtered time domain vector. This generates the missing low frequency and high frequency signal components. A transformation in the Fourier domain provides, then, the spectrum of the extended excitation signal XeSC (n).
Alternatively, determining an excitation signal can be performed in the time sub-band or Fourier domain as well. Examples for this alternative can be found in B.
Iser, G.
Schmidt, Bandwidth Extension of Telephony Speech, Eurasip Newsletter, Volume 16, Number 2, pages 2 to 24, June 2005.
In the following step 209 (block 107), the broadband spectral envelope and the excita-tion signal are used for spectrally coloring the excitation signal. This can be achieved by multiplication in the sub-band or Fourier domain:
Yex: (n) = diag{Copi,b (n)}XeSC (n)=
The diagonal matrix diag{CoP, b(n)} has the form:
CoP, b(ei1, n) 0 ... 0 0 Copt h(e'"' , n) ... 0 diag{Cop, b (n)} _ JQNDFT ' 0 0 ... Cop,b (e ,n) Because of the non-linearity or the prediction error filtering when generating the excita-tion signal, the power of the acoustic signal need not be maintained.
Therefore, a power adaptation may be performed:
Ye,f (n) = K(n)Ye, (n).
The correction factor K can be chosen to be At.
JIXw\e'Q'njz K(n) = u=N, z jna ~YerWe n wherein S2U and S2Pu denote the same bandwidth limits as in the estimation of the long-term power spectrum above.
The current bandwidth limits are adapted in step 210 (block 108). According to one possibility, the bandwidth limits are determined starting with a comparison of the spec-trum of the received acoustic signal and the broadband spectral envelope being re-duced by a predefined constant:
z z l S2, (n) = min S2u E f X (e'~ "~ > CopI b (e'll" ") + K, l S2" (n) = max S2~ E ~IXW (e'~"'" I z > C~,p, b (e'n '" + K, ~ }.
JJ
The parameter Kc can have the value:
Kc = -12 dB.
In Fig. 5, an example for determining the bandwidth limits is illustrated. The above, in-termediate limit values are given by the points of intersection between the lowered broadband spectral envelope and the spectrum of the received acoustic signal.
These intermediate limit values may be recursively smoothed to eliminate temporary mal-estimations. In this case, preferably, smoothing is performed only if speech activity is detected in the current signal frame.
~1(n) = Qbandl ~! (n -1) + (1- ~36aõd, )S2, (n), during speech activity, S2, (n -1), else, Tiu (n) _ j8bandl ?iu (n -1) + (1- j8b,,,,d, )S2v (n), during speech activity, S2õ (n -1), else.
Then, the received acoustic signal is passed through an adaptive band pass filter to retain only components within the current bandwidth limits (block 109) to obtain a spec-tral vector Y,e, (n). Similarly, the spectrally colored excitation signal is passed through a complementary adaptive band stop filter (block 110) so as to obtain a vector Y,,, (n).
An output signal with a standard bandwidth is generated (step 211) by starting with summing these two spectral vectors:
Y(n) = I'lel (n) + Yex, (n).
The components of these vectors are generated as:
Yrel (n) = <T re! (n)Aw (n)f Yex, (n) = G eX, (n)X ex, (n), wherein the weighting matrices G1e, (n) and Gex1(n) are diagonal matrices:
Gtel (e'f2 , n) 0 . . . 0 0 Gter (e'n' , n) . . . 0 G tel (n) ioN
0 0 ... Grer (e p~, _, ~ n) Ge, (e'c , n) 0 . . . 0 G (n) = 0 Gex, (e'K2' , n) ... 0 eX~
0 0 . . . Gex, (e j'N-' , n) The elements of the matrix G1e, (n) are determined as:
jQ' 1, if S2, (n) <- S2AI <- S2õ (n), Giel (e , n) 0, else.
The weights of the complementary weighting matrix are determined so as to yield the unity matrix when summed:
inõ inõ
Ge, (e , n) =1- Gtei (e ~ n).
Alternatively, the transitions at the bandwidth limits can be realized in a smoother way.
The resulting output spectrum Y(n), then, is transformed into the time domain via an inverse Fourier transform:
y(n) = IDFT{Y(n)}, followed by windowing the resulting vector. Particularly when using the above-indicated values for Nana and r and a Hann window, this window function can be used again to obtain windowed time domain vectors:
y w (n) = I+y (n)=
The resulting time domain vectors are, then, assembled using an overlap add method (as described in K. D. Kammeyer, K. Kroschel, Digitale Signalverarbeitung) to obtain the final output signal y(n).
In the above-described steps of the method, more complex filter bank systems may be used instead of the conventional discrete Fourier transform and inverse discrete Fou-rier transform (see, for example, P. P. Vaidyanathan, Multirate Systems and Filter Banks, Prentice Hall, Englewood Cliffs, NJ, USA, 1992).
Further alternatives to the above-described variants are possible as well. For example, the steps performed in the Fourier domain may also be performed in the time domain.
Furthermore, equalizing the acoustic signal may be performed when adapting the nar-rowband codebook entries. Also, the above-described equalizing step may be aug-mented. For example, if an amplification or an attenuation is detected at particular fre-quencies, it may be adjusted within the bandwidth limits as well. In this case, the output vector Yfer (n) is modified with the weighting matrix Hm.d (n).
In addition to the above-described codebook analysis for estimating the broadband spectral envelopes, a so-called linear mapping (see B. Iser, G. Schmidt, Bandwidth Extension of Telephony Speech) may be used additionally.
Further modifications and variations of the present invention will be apparent to those skilled in the art in view of this description. Accordingly, the description is to be con-strued as illustrated only and is for the purpose of teaching those skilled in the art the general manner of carrying out the present invention. It is to be understood that the forms of the invention shown and described herein are to be taken as the presently preferred embodiments.
Analogous to the above-described method, such an apparatus provides an advanta-geous way to extend the bandwidth of a received acoustic signal. In particular, due to the determination of current upper and lower bandwidth limits of the received acoustic signal and a corresponding determination of a complementary signal, the quality of the resulting output signal is increased compared to the case of bandwidth extension sys-tems with fixed parameters.
The complementary signal means may comprise a means for determining a broadband spectral envelope signal and a broadband excitation signal between the lower and up-per broadband bandwidth limits such that the product of spectral envelope signal and excitation signal corresponds to the received acoustic signal according to a predeter-mined criterion.
The bandwidth determining means may be configured to compare a determined broadband spectral envelope signal and a long-term power spectrum of the received acoustic signal.
The bandwidth determining means may be configured to select the minimal and maxi-mal frequency for which the long-term power spectrum is larger than or equal to the power spectrum of the determined broadband spectral envelope signal plus a prede-termined constant.
In the above-described apparatus, the means for determining a broadband spectral envelope signal may comprise a means for selecting an envelope signal from a code-book according to a predetermined criterion.
The means for selecting an envelope signal may be configured to equalize the re-ceived acoustic signal and select an envelope signal from the codebook having mini-mal distance to the equalized acoustic signal according to a predetermined distance criterion, in particular, having a minimal cepstral distance.
In the above-described apparatus, the codebook may comprise pairs of corresponding envelope signals, each pair comprising a broadband envelope signal between the lower and upper broadband bandwidth limits and a corresponding narrowband enve-lope signal between a lower narrowband bandwidth limit being larger than the lower broadband bandwidth limit and an upper narrowband bandwidth limit being smaller than the upper broadband bandwidth limit, and the means for selecting an envelope signal may be configured to determine a narrowband envelope signal having minimal distance to the equalized acoustic signal according to the predetermined distance crite-rion and to select the corresponding broadband envelope signal of this pair.
The means for determining a broadband spectral envelope signal may comprise a means for providing adapted narrowband codebook envelope signals being adapted to the current lower and upper bandwidth limits.
The means for providing may be configured to process the broadband codebook enve-lope signal using a long-term power spectrum of the received acoustic signal.
In the above-described apparatus, the means for determining a broadband excitation signal may be configured to determine the broadband excitation signal based on pre-diction error filtering and/or a non-linear characteristic.
The at least one complementary signal may be based on a product of the determined broadband spectral envelope and the determined broadband excitation signal, and the assembling means may be configured to sum the received acoustic signal between the current lower and upper bandwidth limits and the at least one complementary signal being restricted to the band between the lower broadband bandwidth limit and a cur-rent lower bandwidth limit and/or to the band between the current upper bandwidth limit and the upper broadband bandwidth limit.
In the above-described apparatus, at least one of the means may be configured to per-form at least part of its function in the cepstral domain.
The means of the above-described apparatus may be configured to perform their re-spective function repeatedly at predetermined time intervals.
The apparatus may further comprise a wanted signal detector, in particular, a speech detector, and the means may be configured to perform their respective function only if a wanted signal component is detected in the received acoustic signal.
Further features and advantages of the invention will be described in the following with reference to the figures.
Fig. 1 illustrates the structure of an example of an apparatus for providing an acoustic signal with extended bandwidth;
Fig. 2 is a flow diagram of an example of a method for providing an acoustic signal with extended bandwidth;
Fig. 3 illustrates an example of a normalized long-term power spectrum for training a codebook;
Fig 4 illustrates examples of codebook entries;
Fig. 5 illustrates the determination of current bandwidth limits;
Fig. 6 illustrates the structure of a prior art system.
Fig. 1 shows the structure of the signal flow in an apparatus for providing an acoustic signal with extended bandwidth. Fig. 2 is a flow diagram illustrating an example of a method for providing an acoustic signal with extended bandwidth which could be per-formed by the apparatus corresponding to Fig. 1. In view of this, Fig.'s 1 and 2 will be described in the following simultaneously.
According to step 201, an acoustic signal, such as a speech signal, is received via a telephone line. Because of the restricted bandwidth of the telephone line, an extension of the bandwidth is desired to improve the signal quality. Thus, the signal is to be aug-mented so as to obtain a predetermined broader bandwidth. It is to be understood that the method described in the following can be used for bandwidth extension independ-ent of the type of incoming signal and independent of the type of transmission line, i.e., it need not be a telephone line.
The acoustic signal x(n) received by block 101 has already been pre-processed by increasing the sampling rate up to the predetermined broadband or wideband band-width. In this way, however, no additional frequency components are generated.
This can be achieved, for example, by using suitable anti-aliasing or anti-imaging filters.
This kind of bandwidth extension, preferably, is performed only for the "missing" fre-quency ranges; in the case of an analog telephone line, these ranges may be between 0 and 300 Hz and 3400 Hz up to half of the desired sampling rate, for example, up to 3700 Hz.
From the resulting signal x(n), n denoting the time variable, signal vectors x(n) are generated (step 202). This can be achieved by taking every r sampling values up to a certain length. Thus, a signal vector with N. elements has the form:
x(n) = [x(nr), x(nr -1),..., x(nr - NQõo + 1)]T.
It is to be noted that an overlap may exist between neighboring signal vectors. For a desired or final sampling rate of 11.025 kHz, one may take the following values:
r = 64, NanQ = 256.
After that (step 203), a windowing procedure is performed on the signal vector so as to obtain a windowed signal vector xjn) :
x W (n) = Fx(n).
The window matrix F is a diagonal matrix of the form ho 0 0 ... 0 0 h, 0 ... 0 F= 0 0 h2 ... 0 0 0 0 ... h The elements of this matrix can be chosen corresponding to different kinds of windows.
Typical windows are the Hann or Hamming window. The weighted signal vectors are transformed into the Fourier domain using a discrete Fourier transform:
XW(n) = DFT{xw(n)}.
The resulting short-term spectral vector has the form:
X w(n) = f X(e i~ , n), X(e'n', n),..., X(e'n" , n),..., X(e'O"D~-1 , n)~ , ' wherein S2, denotes the frequency variable.
Based on the spectral vectors, a long-term power spectrum of the received acoustic signal is determined in block 102 (step 204). There are different possibilities to esti-mate such a long-term power spectrum. According to one alternative, a first order re-cursive smoothing is performed on the absolute value squared of the sub-band signals X (e'"" , n) :
~3freS~ (S2F,, n -1) + (1- /3fre ) X. (e'n", n)I , during speech activity S. (SZ'õ n) _ S~ (52u , n -1), else.
Preferably, the time constant 8fre is chosen to be close to 1 (0 /3fre < 1) so as to obtain a sufficiently large averaging time.
In principle, the recursive smoothing according to the first line of the above equation may be performed continuously. However, in order to avoid any artefacts, it may be performed only if a wanted signal component is present in the received acoustic signal, for example, if speech activity is detected. For this purpose, a speech detector may be provided as described, for example, in E. Hansler, G. Schmidt, Acoustic Echo and Noise Control - A Practical Approach, Wiley, Hoboken, NJ, USA, 2004.
In order to simplify the further processing, the long-term power spectrum may be nor-malized to the long-term power within a predefined frequency band:
S~ (52P, n) Sxc,norm (Q/t 5 n) _ P.
E(f2,,, n) /l=fl, The band limits S2. and S2Pu denote the lower and upper limits of a predefined fre-quency band. For example, this frequency band may correspond to a telephone band with minimal bandwidth for which the present method is to be used, for example, the limits may be 400 Hz and 3300 Hz. Preferably, the limits correspond to a band which is smaller or at most equal to the frequency band of the narrow frequency band within which the codebook described below has been trained; these limits being denoted by S2, and S2n.
Alternatively, to determine the long-term power spectrum in the frequency domain, an estimation can be performed in the time domain as well. For this purpose, an auto-correlation is estimated for about 10 to 20 sampling cycles of offset.
Afterwards, predic-tion coefficients can be determined using an LPC (linear predictive coding) analysis.
The long-term power spectrum is obtained via a discrete Fourier transform and a divi-sion.
In block 103 (step 205), the acoustic signal is equalized. The equalizing is performed on the spectral vector determined above:
Xeg (n) = Heg (n)X,, (n).
The equalizing matrix Heq (n) is a diagonal matrix of the form He9(en) 0 ... 0 _ 0 Heq(e'"',n) ... 0 He9(n) 0 0 ... He9(e'Q" -1,n) with the entries 1 if ((52p < S2, (n - 1)) or (0, > SZõ (n -1))) He9 (e , n) = SxX,norm (52,, n), else Sxx,norm (o{( I n) and Heq,max , if Heq (e j"v, n) > Hee.m.
He9 (e'~" , n) = Hevmin I lf (He9 (e,fl, I n) < He9,min Heq (e'n" , n), else, I n the equations above, S2, (n -1) and S2u (n -1) denote the current lower and upper bandwidth limits of the received acoustic signal. Thus, for obtaining an updated equal-ized signal, the bandwidth limits at time (n -1) are taken as the current bandwidth lim-its. Furthermore, Sxs,norm (RU In) denotes the normalized long-term power spectrum of the broadband signal which has been used for training the codebook.
Normalizing of such a power spectrum is performed analogously to the case of the long-term power spectrum of the received acoustic signal described above. An example for such a nor-malized long-term power spectrum used for training a codebook is shown in Figure 3.
The equalizing is restricted to minimal and maximum values, for example, to He9,m;,, _ -12 dB, Heg,max =12 dB.
As can be seen from the above, the acoustic signal is equalized only within the current bandwidth limits one time step before. Outside these bandwidth limits, no equalizing takes place.
In the following, determining a broadband spectrum envelope will be described in more detail. An envelope signal corresponding to the received acoustic signal will be deter-mined using a codebook. The used codebook comprises a number of pairs of corre-sponding narrowband and broadband envelope signals. The codebook has been ob-tained by training with a large database on the basis of a starting long-term power spectrum (see Y. Linde, A. Buzo, R. M. Gray, An Algorithm for Vector Quantizer De-sign, IEEE Trans. Comm., vol. COM-28, no. 1, pages 84 - 95, Jan. 1980).
As indicated in Figure 2, the codebook entries are adapted in step 206 (block 104). In particular, the narrowband codebook entries c; .S (n) are adapted.
This is achieved by starting with the broadband entries of the codebook. If the broad-band envelope signals are provided as cepstral vectors c; b(n) , the corresponding spectra C; h(n) are determined. Based on these broadband spectral envelopes, the adapted or optimized narrowband spectra are determined by a multiplication with a weighting matrix:
C;,s (n) = Amoa (n)C;,b (n)=
The weighting matrix is a diagonal matrix of the form:
Hmod (e'" n) 0 . . . 0 0 Hmod (e'o' I n) . . . 0 Hmod (n) -0 0 ... Hmod (e,QNDFT-, e n) with the entries 1, if~SZ,(n-1)<S2IU <S2õ(n-1)~, joa Hmod (e n) - Sxx,norm Ru . n) !~ else.
Sz,norm \~LU , n) Afterwards, cepstral vectors are determined from the resulting spectral narrowband envelopes.
The conversion from spectral vectors to cepstral vectors and vice versa will be de-scribed in the following with respect to step 207 in which broadband spectral envelopes are determined (block 105).
A broadband spectral envelope from the codebook matching the acoustic signal best is determined by comparing the narrowband codebook entries with the spectral envelope of the spectrum of the acoustic signal (after equalizing). The narrowband codebook entry is selected that has the smallness distance to the acoustic signal spectrum. In principle, different distance criteria can be used. The cepstral distance is particularly useful as the codebook entries are provided in the form of cepstral vectors.
When an optimal narrowband codebook entry has been selected, the corresponding broadband codebook entry is determined as the optimal broadband spectral envelope for the received acoustic signal. Due to the adaptation of the narrowband codebook entries as described above, an optimal narrowband envelope can be selected in a very reliable way.
Converting a spectral vector, particularly of the received acoustic signal, to a cepstral vector can be achieved by:
1. Determining the absolute value squared of each sub-band signal Xeq (e'n, , n).
2. Applying an inverse discrete Fourier transform on this vector results in an esti-mation of the auto-correlation in the time domain.
3. Using the Levinson-Durbin algorithm, prediction coefficients (with an order of about 10 to 20) can be determined from the auto-correlation.
4. By performing a recursion with respect to the order, the prediction coefficients are used to determine the cepstral coefficients. Usually, the order corresponds to one and a half of the prediction order.
The optimal cepstral vector of the broadband codebook is designated by Cpt b(n). The resulting broadband spectral envelope has the form:
~ T
Copt,b (n) = lcopt,b(e j", , n), Copt,b (e'~' , n),..., Copt b(eJ vDF7 n)]
Conversion of cepstral vectors into spectral vectors is achieved by:
1. Converting the cepstral vectors using a recursion with respect to the order (as above) to obtain prediction error filter coefficients.
2. By augmenting the prediction error filter vector by a predetermined number of zeros and subsequent performing of a discrete Fourier transform, an inverse spectrum is obtained.
3. By determining the reciprocal of each sub-band component, the vector Copt,b (n) is generated. Divisions by zero have to be treated separately, for example by adding a suitable constant.
Fig. 4 illustrates an example of a codebook with four pairs of entries. In each diagram, a corresponding original narrowband envelope, and a corresponding adapted narrow-band envelope are shown. The original broadband and narrowband codebook entries have been obtained on the basis of a large database for an ISDN telephone connec-tion. As can be seen in this figure, after the adaptation, the resulting optimized entries have a higher upper limit frequency. This allows for an improved detection of fricatives.
In step 208 (block 103), an excitation signal corresponding to the received acoustic signal is generated. This broadband excitation signal shows a spectrally flat envelope.
It corresponds to a signal which would be recorded directly behind the vocal cords.
For determining a broadband excitation signal, first of all, the spectral envelope of the equalized short-term spectrum Xeq (n) is estimated in the form of prediction error filter coefficients. Applying an inverse discrete Fourier transform on this spectral vector al-lows to determine the corresponding time signal. After that, the vector in the time do-main is filtered by a prediction error filter. The corresponding filter coefficients are those that have been determined previously.
Then, a non-linear characteristic, such as a two-way rectification or squaring, is applied to the filtered time domain vector. This generates the missing low frequency and high frequency signal components. A transformation in the Fourier domain provides, then, the spectrum of the extended excitation signal XeSC (n).
Alternatively, determining an excitation signal can be performed in the time sub-band or Fourier domain as well. Examples for this alternative can be found in B.
Iser, G.
Schmidt, Bandwidth Extension of Telephony Speech, Eurasip Newsletter, Volume 16, Number 2, pages 2 to 24, June 2005.
In the following step 209 (block 107), the broadband spectral envelope and the excita-tion signal are used for spectrally coloring the excitation signal. This can be achieved by multiplication in the sub-band or Fourier domain:
Yex: (n) = diag{Copi,b (n)}XeSC (n)=
The diagonal matrix diag{CoP, b(n)} has the form:
CoP, b(ei1, n) 0 ... 0 0 Copt h(e'"' , n) ... 0 diag{Cop, b (n)} _ JQNDFT ' 0 0 ... Cop,b (e ,n) Because of the non-linearity or the prediction error filtering when generating the excita-tion signal, the power of the acoustic signal need not be maintained.
Therefore, a power adaptation may be performed:
Ye,f (n) = K(n)Ye, (n).
The correction factor K can be chosen to be At.
JIXw\e'Q'njz K(n) = u=N, z jna ~YerWe n wherein S2U and S2Pu denote the same bandwidth limits as in the estimation of the long-term power spectrum above.
The current bandwidth limits are adapted in step 210 (block 108). According to one possibility, the bandwidth limits are determined starting with a comparison of the spec-trum of the received acoustic signal and the broadband spectral envelope being re-duced by a predefined constant:
z z l S2, (n) = min S2u E f X (e'~ "~ > CopI b (e'll" ") + K, l S2" (n) = max S2~ E ~IXW (e'~"'" I z > C~,p, b (e'n '" + K, ~ }.
JJ
The parameter Kc can have the value:
Kc = -12 dB.
In Fig. 5, an example for determining the bandwidth limits is illustrated. The above, in-termediate limit values are given by the points of intersection between the lowered broadband spectral envelope and the spectrum of the received acoustic signal.
These intermediate limit values may be recursively smoothed to eliminate temporary mal-estimations. In this case, preferably, smoothing is performed only if speech activity is detected in the current signal frame.
~1(n) = Qbandl ~! (n -1) + (1- ~36aõd, )S2, (n), during speech activity, S2, (n -1), else, Tiu (n) _ j8bandl ?iu (n -1) + (1- j8b,,,,d, )S2v (n), during speech activity, S2õ (n -1), else.
Then, the received acoustic signal is passed through an adaptive band pass filter to retain only components within the current bandwidth limits (block 109) to obtain a spec-tral vector Y,e, (n). Similarly, the spectrally colored excitation signal is passed through a complementary adaptive band stop filter (block 110) so as to obtain a vector Y,,, (n).
An output signal with a standard bandwidth is generated (step 211) by starting with summing these two spectral vectors:
Y(n) = I'lel (n) + Yex, (n).
The components of these vectors are generated as:
Yrel (n) = <T re! (n)Aw (n)f Yex, (n) = G eX, (n)X ex, (n), wherein the weighting matrices G1e, (n) and Gex1(n) are diagonal matrices:
Gtel (e'f2 , n) 0 . . . 0 0 Gter (e'n' , n) . . . 0 G tel (n) ioN
0 0 ... Grer (e p~, _, ~ n) Ge, (e'c , n) 0 . . . 0 G (n) = 0 Gex, (e'K2' , n) ... 0 eX~
0 0 . . . Gex, (e j'N-' , n) The elements of the matrix G1e, (n) are determined as:
jQ' 1, if S2, (n) <- S2AI <- S2õ (n), Giel (e , n) 0, else.
The weights of the complementary weighting matrix are determined so as to yield the unity matrix when summed:
inõ inõ
Ge, (e , n) =1- Gtei (e ~ n).
Alternatively, the transitions at the bandwidth limits can be realized in a smoother way.
The resulting output spectrum Y(n), then, is transformed into the time domain via an inverse Fourier transform:
y(n) = IDFT{Y(n)}, followed by windowing the resulting vector. Particularly when using the above-indicated values for Nana and r and a Hann window, this window function can be used again to obtain windowed time domain vectors:
y w (n) = I+y (n)=
The resulting time domain vectors are, then, assembled using an overlap add method (as described in K. D. Kammeyer, K. Kroschel, Digitale Signalverarbeitung) to obtain the final output signal y(n).
In the above-described steps of the method, more complex filter bank systems may be used instead of the conventional discrete Fourier transform and inverse discrete Fou-rier transform (see, for example, P. P. Vaidyanathan, Multirate Systems and Filter Banks, Prentice Hall, Englewood Cliffs, NJ, USA, 1992).
Further alternatives to the above-described variants are possible as well. For example, the steps performed in the Fourier domain may also be performed in the time domain.
Furthermore, equalizing the acoustic signal may be performed when adapting the nar-rowband codebook entries. Also, the above-described equalizing step may be aug-mented. For example, if an amplification or an attenuation is detected at particular fre-quencies, it may be adjusted within the bandwidth limits as well. In this case, the output vector Yfer (n) is modified with the weighting matrix Hm.d (n).
In addition to the above-described codebook analysis for estimating the broadband spectral envelopes, a so-called linear mapping (see B. Iser, G. Schmidt, Bandwidth Extension of Telephony Speech) may be used additionally.
Further modifications and variations of the present invention will be apparent to those skilled in the art in view of this description. Accordingly, the description is to be con-strued as illustrated only and is for the purpose of teaching those skilled in the art the general manner of carrying out the present invention. It is to be understood that the forms of the invention shown and described herein are to be taken as the presently preferred embodiments.
Claims (29)
1. Method for providing an acoustic signal with extended bandwidth, comprising-(a) automatically determining a current upper and a current lower bandwidth limit of a received acoustic signal, (b) automatically determining at least one complementary signal to comple-ment the received acoustic signal between a predefined lower broad-band bandwidth limit and the current lower bandwidth limit and/or be-tween the current upper bandwidth limit and a predefined upper broad-band bandwidth limit, wherein the predefined lower broadband band-width limit is smaller than the current bandwidth limit and the predefined upper broadband bandwidth limit is larger than the current upper band-width limit, (c) automatically assembling the at least one complementary signal and the received acoustic signal to obtain an acoustic signal with extended band-width.
2. Method according to claim 1, wherein step (b) comprises determining a broad-band spectral envelope signal and a broadband excitation signal between the lower and upper broadband bandwidth limits such that the product of spectral envelope signal and excitation signal corresponds to the received acoustic sig-nal according to a predetermined criterion.
3. Method according claim 2, wherein step (a) comprises comparing a determined broadband spectral envelope signal and a long-term power spectrum of the re-ceived acoustic signal.
4 Method according to claim 3, wherein the comparing step comprises selecting the minimal and maximal frequency for which the long-term power spectrum is larger than or equal to the power spectrum of the determined broadband spec-tral envelope signal plus a predetermined constant.
5. Method according to any one of claims 2 - 4, wherein determining a broadband spectral envelope signal comprises selecting an envelope signal from a code-book according to a predetermined criterion.
6. Method according to claim 5, wherein selecting an envelope signal comprises equalizing the received acoustic signal and selecting an envelope signal from the codebook having minimal distance to the equalized acoustic signal accord-ing to a predetermined distance criterion, in particular, having a minimal cepstral distance.
7. Method according to claim 6, wherein the codebook comprises pairs of corresponding envelope signals, each pair comprising a broadband envelope signal between the lower and upper broad-band bandwidth limits and a corresponding narrowband envelope signal be-tween a lower narrowband bandwidth limit being larger than the lower broad-band bandwidth limit and an upper narrowband bandwidth limit being smaller than the upper broadband bandwidth limit, and selecting an envelope signal comprises determining a narrowband envelope signal having minimal distance to the equalized acoustic signal according to the predetermined distance criterion and selecting the corresponding broadband envelope signal of this pair.
8. Method according to claim 7, wherein the step of selecting an envelope signal is preceded by providing adapted narrowband codebook envelope signals being adapted to the current lower and upper bandwidth limits.
9. Method according to claim 8, wherein the providing step comprises processing broadband codebook envelope signals using a long-term power spectrum of the received acoustic signal.
10. Method according to any one of claims 2 - 9, wherein determining a broadband excitation signal is based on prediction error filtering and/or a nonlinear charac-teristic.
11. Method according to any one of claims 2 - 10, wherein the at least one complementary signal is based on a product of the determined broadband spectral envelope and the determined broadband excitation signal, and step (c) comprises summing the received acoustic signal between the current lower and upper bandwidth limits and the at least one complementary signal be-ing restricted to the band between the lower broadband bandwidth limit and a current lower bandwidth limit and/or to the band between the current upper bandwidth limit and the upper broadband bandwidth limit.
12. Method according to any one of claims 1- 11, wherein at least one of the steps is performed in the cepstral domain.
13. Method according to any one of claims 1 - 12, wherein steps (a) to (c) are re-peated at predetermined time intervals.
14. Method according to any one of claims 1 - 13, wherein steps (a) to (c) are re-peated only if a wanted signal component, in particular, speech activity, is de-tected in the received acoustic signal.
15. Computer program product comprising one or more computer readable media having computer-executable instructions for performing the steps of the method of any one of claims 1 - 14 when run on a computer.
16. Apparatus for providing an acoustic signal with extended bandwidth, comprising:
bandwidth determining means for automatically determining a current upper and a current lower bandwidth limit of a received acoustic signal, complementary signal means for automatically determining at least one com-plementary signal to complement the received acoustic signal between a prede-fined lower broadband bandwidth limit and the current lower bandwidth limit and/or between the current upper bandwidth limit and a predefined upper broadband bandwidth limit, wherein the predefined lower broadband bandwidth limit is smaller than the current bandwidth limit and the predefined upper broad-band bandwidth limit is larger than the current upper bandwidth limit, and assembling means for automatically assembling the at least one complementary signal and the received acoustic signal to obtain an acoustic signal with ex-tended bandwidth.
bandwidth determining means for automatically determining a current upper and a current lower bandwidth limit of a received acoustic signal, complementary signal means for automatically determining at least one com-plementary signal to complement the received acoustic signal between a prede-fined lower broadband bandwidth limit and the current lower bandwidth limit and/or between the current upper bandwidth limit and a predefined upper broadband bandwidth limit, wherein the predefined lower broadband bandwidth limit is smaller than the current bandwidth limit and the predefined upper broad-band bandwidth limit is larger than the current upper bandwidth limit, and assembling means for automatically assembling the at least one complementary signal and the received acoustic signal to obtain an acoustic signal with ex-tended bandwidth.
17. Apparatus according to claim 16, wherein the complementary signal means comprises a means for determining a broadband spectral envelope signal and a broadband excitation signal between the lower and upper broadband bandwidth limits such that the product of spectral envelope signal and excitation signal cor-responds to the received acoustic signal according to a predetermined criterion.
18. Apparatus according to claim 17, wherein the bandwidth determining means is configured to compare a determined broadband spectral envelope signal and a long-term power spectrum of the received acoustic signal.
19. Apparatus according to claim 18, wherein the bandwidth determining means is configured to select the minimal and maximal frequency for which the long-term power spectrum is larger than or equal to the power spectrum of the determined broadband spectral envelope signal plus a predetermined constant.
20. Apparatus according to any one of claims 17 - 19, wherein the means for de-termining a broadband spectral envelope signal comprises a means for select-ing an envelope signal from a codebook according to a predetermined criterion.
21. Apparatus according to claim 20, wherein the means for selecting an envelope signal is configured to equalize the received acoustic signal and select an enve-lope signal from the codebook having minimal distance to the equalized acous-tic signal according to a predetermined distance criterion, in particular, having a minimal cepstral distance.
22. Apparatus according to claim 21, wherein the codebook comprises pairs of corresponding envelope signals, each pair comprising a broadband envelope signal between the lower and upper broad-band bandwidth limits and a corresponding narrowband envelope signal be-tween a lower narrowband bandwidth limit being larger than the lower broad-band bandwidth limit and an upper narrowband bandwidth limit being smaller than the upper broadband bandwidth limit, and the means for selecting an envelope signal is configured to determine a narrow-band envelope signal having minimal distance to the equalized acoustic signal according to the predetermined distance criterion and to select the correspond-ing broadband envelope signal of this pair.
23. Apparatus according to claim 22, wherein the means for determining a broad-band spectral envelope signal comprises a means for providing adapted nar-rowband codebook envelope signals being adapted to the current lower and up-per bandwidth limits.
24. Apparatus according to claim 23, wherein the means for providing is configured to process the broadband codebook envelope signal using a long-term power spectrum of the received acoustic signal.
25. Apparatus according to any one of claims 17 - 24, wherein the means for de-termining a broadband excitation signal is configured to determine the broad-band excitation signal based on prediction error filtering and/or a nonlinear characteristic.
26. Apparatus according to any one of claims 17 - 25, wherein the at least one complementary signal is based on a product of the determined broadband spectral envelope and the determined broadband excitation signal, and the assembling means is configured to sum the received acoustic signal be-tween the current lower and upper bandwidth limits and the at least one com-plementary signal being restricted to the band between the lower broadband bandwidth limit and a current lower bandwidth limit and/or to the band between the current upper bandwidth limit and the upper broadband bandwidth limit.
27. Apparatus according to any one of claims 16 - 26, wherein at least one of the means is configured to perform at least part of its function in the cepstral do-main.
28. Apparatus according to any one of claims 16 - 27, wherein the means are con-figured to perform their respective function repeatedly at predetermined time in-tervals.
29. Apparatus according to any one of claims 16 - 28, further comprising a wanted signal detector, in particular, a speech detector, and wherein the means are configured to perform their respective function only if a wanted signal compo-nent is detected in the received acoustic signal.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP06017456A EP1892703B1 (en) | 2006-08-22 | 2006-08-22 | Method and system for providing an acoustic signal with extended bandwidth |
| EP06017456.2 | 2006-08-22 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2596411A1 true CA2596411A1 (en) | 2008-02-22 |
Family
ID=37000103
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002596411A Abandoned CA2596411A1 (en) | 2006-08-22 | 2007-08-08 | Method and system for providing an acoustic signal with extended bandwidth |
Country Status (7)
| Country | Link |
|---|---|
| EP (1) | EP1892703B1 (en) |
| JP (1) | JP5150165B2 (en) |
| KR (1) | KR101433833B1 (en) |
| CN (1) | CN101141533B (en) |
| AT (1) | ATE446572T1 (en) |
| CA (1) | CA2596411A1 (en) |
| DE (1) | DE602006009927D1 (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8688441B2 (en) | 2007-11-29 | 2014-04-01 | Motorola Mobility Llc | Method and apparatus to facilitate provision and use of an energy value to determine a spectral envelope shape for out-of-signal bandwidth content |
| US8433582B2 (en) | 2008-02-01 | 2013-04-30 | Motorola Mobility Llc | Method and apparatus for estimating high-band energy in a bandwidth extension system |
| US20090201983A1 (en) | 2008-02-07 | 2009-08-13 | Motorola, Inc. | Method and apparatus for estimating high-band energy in a bandwidth extension system |
| US8463412B2 (en) * | 2008-08-21 | 2013-06-11 | Motorola Mobility Llc | Method and apparatus to facilitate determining signal bounding frequencies |
| JP2010079275A (en) * | 2008-08-29 | 2010-04-08 | Sony Corp | Device and method for expanding frequency band, device and method for encoding, device and method for decoding, and program |
| US8463599B2 (en) | 2009-02-04 | 2013-06-11 | Motorola Mobility Llc | Bandwidth extension method and apparatus for a modified discrete cosine transform audio coder |
| JP5535241B2 (en) * | 2009-12-28 | 2014-07-02 | 三菱電機株式会社 | Audio signal restoration apparatus and audio signal restoration method |
| EP2559026A1 (en) * | 2010-04-12 | 2013-02-20 | Freescale Semiconductor, Inc. | Audio communication device, method for outputting an audio signal, and communication system |
| ES2659001T3 (en) * | 2013-01-29 | 2018-03-13 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Audio encoders, audio decoders, systems, methods and computer programs that use an increased temporal resolution in the temporal proximity of beginnings or endings of fricatives or Africans |
| KR20180056032A (en) | 2016-11-18 | 2018-05-28 | 삼성전자주식회사 | Signal processing processor and controlling method thereof |
| CN107404625B (en) * | 2017-07-18 | 2020-10-16 | 海信视像科技股份有限公司 | Sound effect processing method and device of terminal |
| KR102093819B1 (en) * | 2018-09-10 | 2020-03-26 | 한국과학기술연구원 | Apparatus and method for separating sound sources |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3483958B2 (en) * | 1994-10-28 | 2004-01-06 | 三菱電機株式会社 | Broadband audio restoration apparatus, wideband audio restoration method, audio transmission system, and audio transmission method |
| KR100302370B1 (en) | 1997-04-30 | 2001-09-29 | 닛폰 호소 교카이 | Speech interval detection method and system, and speech speed converting method and system using the speech interval detection method and system |
| US6539355B1 (en) * | 1998-10-15 | 2003-03-25 | Sony Corporation | Signal band expanding method and apparatus and signal synthesis method and apparatus |
| DE1298643T1 (en) | 2000-06-14 | 2003-11-27 | Kabushiki Kaisha Kenwood, Tokio/Tokyo | FREQUENCY INTERPOLATION DEVICE AND FREQUENCY INTERPOLATION METHOD |
| JP3713200B2 (en) * | 2000-11-30 | 2005-11-02 | 株式会社ケンウッド | Signal interpolation device, signal interpolation method and recording medium |
| US6889182B2 (en) | 2001-01-12 | 2005-05-03 | Telefonaktiebolaget L M Ericsson (Publ) | Speech bandwidth extension |
| WO2003019533A1 (en) * | 2001-08-24 | 2003-03-06 | Kabushiki Kaisha Kenwood | Device and method for interpolating frequency components of signal adaptively |
| KR20040035749A (en) * | 2001-08-31 | 2004-04-29 | 코닌클리케 필립스 일렉트로닉스 엔.브이. | Bandwidth extension of a sound signal |
| JP4281349B2 (en) * | 2001-12-25 | 2009-06-17 | パナソニック株式会社 | Telephone equipment |
| CN1887025A (en) * | 2003-12-01 | 2006-12-27 | 皇家飞利浦电子股份有限公司 | Selective audio signal enhancement |
| US20080260048A1 (en) * | 2004-02-16 | 2008-10-23 | Koninklijke Philips Electronics, N.V. | Transcoder and Method of Transcoding Therefore |
| DE602004020765D1 (en) * | 2004-09-17 | 2009-06-04 | Harman Becker Automotive Sys | Bandwidth extension of band-limited tone signals |
-
2006
- 2006-08-22 AT AT06017456T patent/ATE446572T1/en not_active IP Right Cessation
- 2006-08-22 DE DE602006009927T patent/DE602006009927D1/en active Active
- 2006-08-22 EP EP06017456A patent/EP1892703B1/en not_active Not-in-force
-
2007
- 2007-08-08 CA CA002596411A patent/CA2596411A1/en not_active Abandoned
- 2007-08-21 JP JP2007214930A patent/JP5150165B2/en not_active Expired - Fee Related
- 2007-08-22 CN CN2007101466102A patent/CN101141533B/en active Active
- 2007-08-22 KR KR1020070084306A patent/KR101433833B1/en active IP Right Grant
Also Published As
| Publication number | Publication date |
|---|---|
| CN101141533B (en) | 2013-09-04 |
| KR101433833B1 (en) | 2014-08-27 |
| KR20080018132A (en) | 2008-02-27 |
| EP1892703A1 (en) | 2008-02-27 |
| DE602006009927D1 (en) | 2009-12-03 |
| EP1892703B1 (en) | 2009-10-21 |
| CN101141533A (en) | 2008-03-12 |
| JP2008052277A (en) | 2008-03-06 |
| JP5150165B2 (en) | 2013-02-20 |
| ATE446572T1 (en) | 2009-11-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2596411A1 (en) | Method and system for providing an acoustic signal with extended bandwidth | |
| CN1750124B (en) | Bandwidth extension of band limited audio signals | |
| US7035797B2 (en) | Data-driven filtering of cepstral time trajectories for robust speech recognition | |
| KR101461774B1 (en) | A bandwidth extender | |
| CN104520925B (en) | The percentile of noise reduction gain filters | |
| Chen et al. | Speech enhancement using perceptual wavelet packet decomposition and teager energy operator | |
| US8392184B2 (en) | Filtering of beamformed speech signals | |
| EP1970900A1 (en) | Method and apparatus for providing a codebook for bandwidth extension of an acoustic signal | |
| Ren et al. | A Causal U-Net Based Neural Beamforming Network for Real-Time Multi-Channel Speech Enhancement. | |
| EP0929891B1 (en) | Methods and devices for noise conditioning signals representative of audio information in compressed and digitized form | |
| JPWO2011080855A1 (en) | Audio signal restoration apparatus and audio signal restoration method | |
| US5806022A (en) | Method and system for performing speech recognition | |
| CN108198571A (en) | A kind of bandwidth expanding method judged based on adaptive bandwidth and system | |
| Qian et al. | Combining equalization and estimation for bandwidth extension of narrowband speech | |
| Barros et al. | Estimation of speech embedded in a reverberant and noisy environment by independent component analysis and wavelets | |
| You et al. | An invertible frequency eigendomain transformation for masking-based subspace speech enhancement | |
| Rao et al. | Speech enhancement using sub-band cross-correlation compensated Wiener filter combined with harmonic regeneration | |
| Fraile et al. | Mfcc-based remote pathology detection on speech transmitted through the telephone channel-impact of linear distortions: Band limitation, frequency response and noise | |
| Nossier et al. | Two-stage deep learning approach for speech enhancement and reconstruction in the frequency and time domains | |
| TWI749547B (en) | Speech enhancement system based on deep learning | |
| CN114822569A (en) | Audio signal processing method, device, equipment and computer readable storage medium | |
| Heitkaemper et al. | Neural network based carrier frequency offset estimation from speech transmitted over high frequency channels | |
| Sach et al. | EffCRN: An Efficient Convolutional Recurrent Network for High-Performance Speech Enhancement | |
| Lee et al. | Speech Enhancement for Virtual Meetings on Cellular Networks | |
| CN115527550A (en) | Single-microphone subband domain noise reduction method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FZDE | Discontinued |