EP1544848B1 - Amélioration de la qualité d'un signal audio dans le domaine codé - Google Patents
Amélioration de la qualité d'un signal audio dans le domaine codé Download PDFInfo
- Publication number
- EP1544848B1 EP1544848B1 EP20040029839 EP04029839A EP1544848B1 EP 1544848 B1 EP1544848 B1 EP 1544848B1 EP 20040029839 EP20040029839 EP 20040029839 EP 04029839 A EP04029839 A EP 04029839A EP 1544848 B1 EP1544848 B1 EP 1544848B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- parameter
- value
- new
- index
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Not-in-force
Links
- 230000005236 sound signal Effects 0.000 claims description 42
- 238000000034 method Methods 0.000 claims description 38
- 230000002708 enhancing effect Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 5
- 238000013139 quantization Methods 0.000 description 61
- 230000003044 adaptive effect Effects 0.000 description 51
- 239000013598 vector Substances 0.000 description 47
- 238000012937 correction Methods 0.000 description 44
- 239000011295 pitch Substances 0.000 description 43
- IVEKVTHFAJJKGA-BQBZGAKWSA-N (2s)-2-amino-5-[[(2r)-1-ethoxy-1-oxo-3-sulfanylpropan-2-yl]amino]-5-oxopentanoic acid Chemical compound CCOC(=O)[C@H](CS)NC(=O)CC[C@H](N)C(O)=O IVEKVTHFAJJKGA-BQBZGAKWSA-N 0.000 description 21
- 230000006870 function Effects 0.000 description 17
- 230000008859 change Effects 0.000 description 16
- 238000012545 processing Methods 0.000 description 15
- 238000003786 synthesis reaction Methods 0.000 description 14
- 230000003321 amplification Effects 0.000 description 11
- 230000005284 excitation Effects 0.000 description 11
- 238000003199 nucleic acid amplification method Methods 0.000 description 11
- 230000015572 biosynthetic process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 230000005540 biological transmission Effects 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000009499 grossing Methods 0.000 description 6
- 230000004044 response Effects 0.000 description 5
- 230000002194 synthesizing effect Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 230000003595 spectral effect Effects 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 230000001629 suppression Effects 0.000 description 3
- 241000819038 Chichester Species 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000005303 weighing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/173—Transcoding, i.e. converting between two coded representations avoiding cascaded coding-decoding
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
- G10L21/0364—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude for improving intelligibility
Definitions
- the present invention relates to voice enhancement, and in particular to a method and an apparatus for enhancing a coded audio signal.
- TFO is a voice standard to be deployed in the GSM (Global System for Mobile communications) and GSM-evolved 3G (Third Generation) networks. It is intended to avoid the traditional double speech encoding/decoding in mobile-to-mobile call configurations.
- the key inconvenience of a tandem configuration is the speech quality degradation introduced by the double transcoding. According to the ETSI listening tests, this degradation is usually more noticeable when the speech codecs are operating at low rates. Also, higher background noise level increases the degradation.
- the originating and terminating connections are using the same speech codec it is possible to transmit transparently the speech frames received from the originating MS (Mobile Station) to the terminating MS without activating the transcoding functions in the originating and terminating networks.
- Tandem Free Operation is improvement in speech quality by avoiding the double transcoding in the network, possible savings on the inter-PLMN (Public Land Mobile Network) transmission links, which are carrying compressed speech compatible with a 16 kbit/s or 8 kbit/s sub-multiplexing scheme, including packet switched transmission, possible savings in processing power in the network equipment since the transcoding functions in the Transcoder Units are bypassed, and possible reduction in the end-to-end transmission delay.
- PLMN Public Land Mobile Network
- TFO call configuration a transcoder device is physically present in the signal path, but the transcoding functions are bypassed.
- the transcoding device may perform control and protocol conversion functions.
- TrFO Transcoder Free Operation
- no transcoder device is physically present and hence no control or conversion or other functions associated with it are activated.
- the level of speech is an important factor affecting the perceived quality of speech.
- automatic level control algorithms which adjust the speech level to a certain desired target level by increasing the level of faint speech and somewhat decreasing the level of very loud voices.
- the level control is more difficult in the lower modes due to the fact that the fixed codebook gain is no longer scalar quantized but it is vector-quantized together with the adaptive codebook gain.
- this object is achieved by an apparatus and a method of enhancing a coded audio signal comprising indices which represent audio signal parameters which comprise at least a first parameter representing a first characteristic of the audio signal and a second parameter, comprising:
- this object is achieved by an apparatus and a method of enhancing a coded audio signal comprising indices which represent audio signal parameters which comprise at least a first parameter representing a first characteristic of the audio signal and a background noise parameter, comprising:
- the invention may also be embodied as computer program product comprising portions for performing steps when the product is run on a computer.
- a coded audio signal comprising speech and/or noise in a coded domain is enhanced by manipulating coded speech and/or noise parameters of an AMR (Adaptive Multi-Rate) speech codec.
- AMR Adaptive Multi-Rate
- adaptive level control, echo control and noise suppression can be achieved in the network even if speech is not transformed into linear PCM samples, as is the case in TFO, TrFO and future packet networks.
- a method for controlling the level of the AMR coded speech for all the AMR codec modes 12.2 kbit/s, 10.2 kbit/s, 7.95kbit/s, 7.40 kbit/s , 6.70 kbit/s, 5.90 kbit/s, 5.15 kbit/s and 4.75 kbit/s is described.
- the level of the coded speech is adjusted by changing one of the coded speech parameters, namely the quantization index of the fixed codebook gain factor in the modes 12.2 kbit/s and 7.95 kbit/s.
- the fixed codebook gain is jointly vector-quantized with the adaptive codebook gain, and therefore adjusting the level of the coded speech requires changing both the fixed codebook gain factor and the adaptive codebook gain (joint index).
- a new gain index is found such that the error between the desired gain and the realized effective gain becomes minimized.
- the proposed level control does not cause audible artifacts.
- level control is enabled also in lower AMR bit rates (not only 12.2 kbit/s and 7.95 kbit/s).
- the level control in the AMR mode 12.2 kbit/s can be improved by taking into account the required corresponding level control for the comfort noise level.
- Fig. 1 shows a simplified model of speech synthesis in AMR.
- Fig. 2 demonstrates the effect of a DTX operation on a gain manipulation algorithm with noisy child speech samples.
- Fig. 3 shows a diagram illustrating a response of an adaptive codebook to a step-function.
- Fig. 4 shows a non-linear 32-level quantization table of a fixed codebook gain factor in modes 12.2 kbit/s and 7.95 kbit/s.
- Fig. 5 shows a diagram illustrating the difference between adjacent quantization levels in the quantization table of Fig. 4 .
- Fig. 6 shows a vector quantization table for an adaptive codebook gain and a fixed codebook gain in modes 10.2, 7.4 and 6.7 kbit/s.
- Fig. 7 shows a vector quantization table for an adaptive codebook gain and a fixed codebook gain factor in modes 5.90 and 5.15 kbit/s.
- Fig. 8 shows a diagram illustrating a change in the fixed codebook gain when the fixed codebook gain factor is changed one quantization step.
- Figs. 9 and 10 show diagrams illustrating re-quantized levels of the fixed codebook gain factor.
- Fig. 11 illustrates values of terms ⁇ y ⁇ ⁇ z ⁇ and ⁇ y ⁇ g c ⁇ ⁇ z ⁇ with male speech samples.
- Fig. 12 illustrates values of terms ⁇ y ⁇ ⁇ z ⁇ and ⁇ y ⁇ g c ⁇ ⁇ z ⁇ with child speech samples.
- Fig. 13 shows a flow chart illustrating a method of enhancing a coded audio signal according to the invention.
- Fig. 14 shows a schematic block diagram illustrating an apparatus for enhancing a coded audio signal according to the present invention.
- Fig. 15 shows a block diagram illustrating the use of fixed gain.
- Fig. 16 shows a diagram illustrating a high level implementation of the invention in a media gateway.
- an embodiment of the present invention will be described in connection with an AMR coded audio signal comprising speech and/or noise.
- the invention is not limited to AMR coding and can be applied to any audio signal coding technique employing indices corresponding to audio signal parameters.
- audio signal parameters may control a level of synthesized speech.
- the invention can be applied to a audio signal coding technique in which an index indicating a value of an audio signal parameter controlling a first characteristic of the audio signal is transmitted as coded audio signal, in which this index may also indicate a value of an audio signal parameter controlling another audio signal characteristic such as a pitch of the synthesized speech.
- the adaptive multi-rate speech codec (AMR) is presented to the extent necessary for illustrating the preferred embodiments.
- the adaptive multi-rate (AMR) speech codec is based on the code-excited linear predictive (CELP) coding model.
- the AMR encoding process comprises three main steps:
- the fixed codebook gain quantization is performed using moving-average (MA) prediction with fixed coefficients.
- the MA prediction is performed on the innovation energy as follows.
- a predicted fixed codebook gain is computed using the predicted energy as in Eq. (1.2) (by substituting E ( n ) by ⁇ ( n ) and g c by g c ⁇ ).
- the transmitted speech parameters are decoded and speech is synthesized.
- the decoder receives an index to a quantization table that gives the quantified fixed codebook gain correction factor ⁇ gc .
- the index gives both the quantified adaptive codebook gain ⁇ p and the fixed codebook gain correction factor ⁇ gc .
- the fixed codebook gain correction factor gives the fixed codebook gain the same way as described above.
- the fixed codebook gain correction factor ⁇ gc is scalar quantized with 5 bits (32 quantization levels).
- the fixed codebook gain correction factor ⁇ gc and the adaptive codebook gain g p are jointly vector quantized with 7 bits.
- this mode includes smoothing of the fixed codebook gain.
- the fixed codebook gain used for synthesis in the decoder is replaced by a smoothed value of the fixed codebook gains of the previous 5 subframes.
- the smoothing is based on a measure of the stationarity of the short-term spectrum in the LSP (Line Spectral Pair) domain. The smoothing is performed to avoid unnatural fluctuations in the energy contour.
- the fixed codebook gain correction factor ⁇ gc is scalar quantized with 5 bits, as in the mode 12.2 kbit/s.
- This mode includes anti-sparseness processing.
- An adaptive anti-sparseness post-processing procedure is applied to the fixed codebook vector c ( n ) in order to reduce perceptual artifacts arising from the sparseness of the algebraic fixed codebook vectors with only a few non-zero samples per an impulse response.
- the anti-sparseness processing consists of circular convolution of the fixed codebook vector with one of three pre-stored impulse responses. The selection of the impulse response is performed adaptively from the adaptive and fixed codebook gains.
- the fixed codebook gain correction factor ⁇ gc and the adaptive codebook gain g p are jointly vector quantized with 7 bits, as in the mode 10.2 kbit/s.
- the fixed codebook gain correction factor ⁇ gc and the adaptive codebook gain g p are jointly vector quantized with 7 bits, as in the mode 10.2 kbit/s.
- the fixed codebook gain correction factor ⁇ gc and the adaptive codebook gain g p are jointly vector quantized with 6 bits.
- the modes include smoothing of the fixed codebook gain and anti-sparseness processing.
- the fixed codebook gain correction factor ⁇ gc and the adaptive codebook gain g p are jointly vector quantized only every 10 ms by a unique method as described in 3GPP TS 26.090 V4.0.0 (2001-03), "3rd Generation Partnership Project; Technical Specification Group Services and System Aspects; Mandatory Speech Codec speech processing functions; AMR speech codec; Transcoding functions (Release 4)".
- This mode includes smoothing of the fixed codebook gain and anti-sparseness processing.
- DTX discontinuous transmission
- the decoder reconstructs the background noise according to the transmitted noise parameters avoiding thus extremely annoying discontinuities in the background noise in the synthesized speech.
- the comfort noise parameters, information on the level and the spectrum of the background noise are encoded into a special frame called a Silence Descriptor (SID) frame for transmission to the receive side.
- SID Silence Descriptor
- the information on the level of the background noise is of interest. If the gain level were adjusted only during speech frames, the background noise level would change abruptly at the beginning and end of noise only bursts, as illustrated in Fig. 2 .
- the level changes in the background noise are subjectively very annoying see e.g. Kondoz A. M., University of Surrey, UK, "Digital speech coding for low bit rate communications systems," page 336, John Wiley & Sons, Chichester, 1994 . The more annoying the greater the amplification or attenuation is. If the level of speech is adjusted, also the level of the background noise has to be adjusted accordingly to prevent any fluctuations in the background noise level.
- the averaged logarithmic frame energy is quantized by means of a 6-bit algorithmic quantizer. These 6 bits for the energy index are transmitted in the SID frame.
- the fixed codebook gain g c adjusts the level of the synthesized speech in the AMR speech codec, as can be noticed by studying the equation (1.1) and the speech synthesis model shown in Fig. 1 .
- the adaptive codebook gain g p controls the periodicity (pitch) of the synthesized speech, and is limited between [0, 1.2]. As shown in Fig. 1 , an adaptive feedback loop transmits the effect of the fixed codebook gain also to the adaptive codebook branch of the synthesis model thereby adjusting also the voiced part of the synthesized speech.
- the speed at which the change in the fixed codebook gain is transmitted to the adaptive codebook branch depends on the pitch delay T and the pitch gain g p , as illustrated in Fig. 3 .
- the pitch gain and delay vary.
- the simulation with a fixed pitch delay and pitch gain tries to give a rough estimate on the limits to the stabilization time of the adaptive codebook after a change in the fixed codebook gain.
- the pitch delay is limited in AMR between [18, 143] samples, as in the example too, corresponding to high child and low male pitches, respectively.
- the pitch gain may have values between [0,1.2].
- the pitch gain receives values at or above 1 only very short time instants for the adaptive codebook not to go unstable. Therefore, the estimated maximum delay is around few thousand samples, about half a second.
- Fig. 3 shows the response of the adaptive codebook to a step-function (sudden change in g c ) as a function of pitch delay T (integer lag k in Eq. (1. 1)) and pitch gain g p .
- the output of the scaled fixed codebook, g c * c ( n ) changes from 0 to 0.3 at time instant 0 samples.
- the output of the adaptive codebook (and thus also the excitation signal u ( n )) reaches its corresponding level after 108 to 5430 samples, for the pitch delays T and pitch gains g p of the example.
- the fixed codebook gain correction factor ⁇ gc is scalar quantized with 5-bits, giving 32 quantization levels, as shown in Fig. 4 .
- the quantization is nonlinear.
- the quantization steps are shown in Fig. 5 .
- the quantization step is between 1.2 dB to 2.3 dB.
- the lowest mode 4.75 kbit/s uses vector quantization in a unique way.
- the adaptive codebook gains g p and the correction factors ⁇ gc are jointly vector quantized every 10 ms with 6 bits, i.e. two codebook gains of two frames and two correction factors are jointly vector quantized.
- Fig. 5 shows a difference between adjacent quantization levels in the quantization table of the fixed codebook gain factor ⁇ gc in the modes 12.2 kbit/s and 7.95 kbit/s.
- the quantization table is approximately linear between indexes 5 and 28.
- the quantization step in that range is about 1.2 dB.
- Fig. 6 shows the vector quantization table for the adaptive codebook gain and the fixed codebook gain factor in the modes 10.2, 7.4 and 6.7 kbit/s.
- the table is printed so that one index value gives both the fixed codebook gain factor and the corresponding (jointly quantized) adaptive codebook gain.
- Fig. 7 shows the vector quantization table for the adaptive codebook gain and the fixed codebook gain factor in the modes 5.90 and 5.15 kbit/s. Again, the table is printed so that one index value gives both the fixed codebook gain factor and the corresponding (jointly quantized) adaptive codebook gain.
- the speech level control in the parameter domain must take place by adjusting the fixed codebook gain.
- the quantized fixed codebook gain correction factor ⁇ gc is adjusted, which is one of the speech parameters transmitted to the far-end.
- the minimum change for the fixed codebook gain factor (the minimum quantization step) ⁇ 1.2 dB results in ⁇ 3.4 dB change in the fixed codebook gain, and hence in the synthesized speech signal, as shown below.
- 20 ⁇ log 10 ⁇ 2.79 3.4 dB
- Fig. 8 shows a change in the fixed codebook gain (AMR 12.2 kbit/s), when the fixed codebook gain factor is changed one quantization step (in the linear quantization range) first upwards at subframe 6 and then downwards at subframe 16.
- the 1.2 dB amplification (or attenuation) of the fixed codebook gain factor amplifies (or attenuates) the fixed codebook gain gradually 3.4 dB during 5 frames (200 samples).
- the parameter level gain control of coded speech may be made by changing the index value of the fixed codebook gain factor. That is, the index value in the bit stream is replaced by a new value that gives the desired amplification/attenuation.
- the gain values corresponding to the index changes for AMR mode 12.2 kbit/s are listed in the table below. Table I: Parameter level gain values for AMR 12.2 kbit/s.
- Fig. 9 shows the re-quantized levels for cases +3.4, +6.8, +10.2, +13.6 and +17.0 dB signal amplification achieved with the above error minimization procedure.
- Fig. 10 shows also the quantization levels in cases of signal attenuation. Both figures show the quantization levels for the AMR mode 12.2 kbit/s.
- the lowest curve shows the original quantization levels of the fixed codebook gain factor.
- the second lowest curve shows re-quantized levels of the fixed codebook gain factor in the case of +3.4 dB signal level amplification, and the subsequent curves show re-quantized levels of the fixed codebook gain factor in cases +6.8, +10.2, +13.6 and +17 dB signal level amplification, respectively.
- Fig. 10 shows re-quantized levels of the fixed codebook gain factor in cases: -17, -13.6, ..., -3.4, 0, +3.4, ..., +13.6, +17 dB signal level amplification.
- the curve in the middle shows the original quantization levels of the fixed codebook gain factor.
- Equation 2.12 is replaced by: ⁇ ⁇ ⁇ ⁇ gc old - ⁇ ⁇ gc new + weight ⁇ g p_new - g p_old , where the weight is ⁇ 1, and g p_new and g p_o / d are the new and old adaptive codebook gains, respectively.
- the new fixed codebook gain factor index is found as the index which minimizes the error given in Eq. (2.12).
- modes 10.2 kbit/s, 7.40 kbit/s, 6.70 kbit/s, 5.90 kbit/s, 5.15 kbit/s and 4.75 kbit/s the new joint index of the vector quantized fixed codebook gain factor and adaptive gain is found as the index which minimized the error given in Eq. (2.13).
- the rationale behind the Eq. (2.13) is to be able to change the fixed codebook gain factor without introducing audible error to the adaptive codebook gain.
- Fig. 6 shows the vector quantized fixed codebook gain factors and adaptive codebook gains at different index values. From Fig. 6 it can be seen that there is a possibility to change the fixed codebook gain factor without having to change the adaptive codebook gain excessively.
- the adaptive codebook gains g p and the correction factors ⁇ gc are jointly vector quantized every 10 ms with 6 bits, i.e. two codebook gains of two subframes and two correction factors are jointly vector quantized.
- the codebook search is done by minimizing a weighted sum of the error criterion for each of the two subframes.
- the default values of the weighing factors are 1. If the energy of the second subframe is more than two times the energy of the first subframe, the weight of the first subframe is set to 2. If the energy of the first subframe is more than four times the energy of the second subframe, the weight of the second subframe is set to 2.
- the mode 4.75 kbit/s can be processed with the vector quantization schema described above.
- a new gain index (new index value) minimizing the error between the desired ⁇ ⁇ ⁇ ⁇ gc old gain (enhanced first parameter value) and the realized effective gain ⁇ ⁇ gc new (new first parameter value) according to Eq. (2.12) or (2.13) is determined according to the quantization tables for the respective modes.
- the new fixed codebook gain correction factor (and the new adaptive codebook gain in case of modes other than 12.2 kbits/s and 7.95 kbit/s) correspond to the determined new gain index.
- the old gain index (current index value) representing the old fixed codebook gain correction factor ⁇ ⁇ gc old (current first parameter value) (and the old adaptive codebook gain g p _ old (current second parameter value) in case of modes other than 12.2 kbits/s and 7.95 kbit/s) then is replaced by the new gain index.
- the fixed codebook gain is encoded using the fixed codebook gain correction factor ⁇ gc .
- the quantified fixed codebook correction factor has to be multiplied by a correction factor gain ⁇ .
- Realized correction factor gains are denoted with ⁇ ( n - 1), i > 0.
- the new quantized fixed codebook gain becomes: (Note that the prediction g c ⁇ depends on the history of the correction gains, as shown in Equation 2.14)
- g ⁇ c new n ⁇ n ⁇ ⁇ ⁇ gc n ⁇ g c ⁇ new n
- E SQ the fixed codebook quantization error
- g c the target fixed codebook gain.
- the pitch gain g p and the fixed codebook correction factor ⁇ gc are jointly quantized.
- the error criterion is actually a norm of the perceptually weighted error between the target and the synthesized speech.
- E VQ ⁇ g ⁇ p ⁇ y new - ⁇ ⁇ g ⁇ c ⁇ z - g ⁇ p new ⁇ y new - g ⁇ c new ⁇ z ⁇ .
- E SQ ⁇ ⁇ ⁇ ⁇ gc ⁇ g c ⁇ - ⁇ ⁇ gc new ⁇ g c ⁇ new 2
- E SQ ⁇
- This error criterion is simple to evaluate and only the fixed codebook correction factor has to be decoded. Furthermore, four previous realized correction factor gains have to be kept in the memory.
- the error criterion used in the AMR-encoder is more complicated, since the synthesis filters are used. In view of the fact that there is no direct access to the target x , it is approximated by ⁇ p y + ⁇ c z .
- E VQ ⁇ x new - g ⁇ p new ⁇ y new - g ⁇ c new ⁇ z ⁇
- E VQ ⁇ g ⁇ p ⁇ ⁇ ⁇ y - ⁇ ⁇ g ⁇ c ⁇ z - g ⁇ p new ⁇ ⁇ ⁇ y - g ⁇ c new ⁇ z ⁇
- E VQ ⁇ g ⁇ p - g ⁇ p new ⁇ ⁇ ⁇ y + ⁇ ⁇ g ⁇ c - g ⁇ c new ⁇ z ⁇
- E VQ ⁇ g p - g ⁇ p new ⁇ ⁇ ⁇ y + ⁇ ⁇ g ⁇ c - g ⁇ c new ⁇ z ⁇
- E VQ ⁇ g p - g ⁇ p new ⁇ ⁇ ⁇ y + ⁇ ⁇ ⁇ gc
- both codebook vectors have to be decoded and filtered with the LP-synthesis filter. Therefore, LP-synthesis filter parameters have to be decoded. This means that basically all the parameters have to be decoded.
- the codebook vectors are also weighted by a specific weighting filter, but this was not done for this CDALC error criterion.
- Quantization Error Minimization with Memory memory method
- This criterion minimizes quantization error while taking in account the history of the previous correction factors.
- the error criterion is the same as in the first alternative, i.e. the error function to be minimized will be the same as in Equation 3.4. But for the vector quantization the error function becomes little easier to evaluate.
- Equation 3.5 Starting from the error function derived for the first alternative and given in Equation 3.5, minimizing the error of the sum of two components will require decoding the y and z vectors. Practically this means that the whole signal has to be decoded. Instead of minimizing the norm of the error vector, the error can be approximated by the sum of two error components (which would be the case if both vectors y and z are parallel to each other), namely the pitch gain error and the fixed codebook gain error.
- E VQ ⁇ g ⁇ p - g p new 2 ⁇
- This algorithm using fixed pitch gain weight requires decoding (finding a value according to the received quantization index) of both the pitch gain and the correction factor ( ⁇ gc ) and also reconstructing of the fixed codebook gain prediction g c ⁇ . .
- the fixed codebook vector has to be decoded.
- the integer pitch lag is needed for the pitch sharpening of the fixed codebook excitation.
- the energy of the fixed codebook excitation is required for the prediction (see Equation 3.1).
- the energy of the fixed codebook excitation can be estimated, since it is fairly fixed. This allows the creation of a prediction without decoding the fixed codebook vector.
- Fig. 13 shows a flow chart generally illustrating the method of enhancing a coded audio signal comprising coded speech and/or coded noise according to the invention.
- the coded audio signal comprises indices which represent speech parameters and/or noise parameters which comprise at least a first parameter for adjusting a first characteristic of the audio signal, such as the level of synthesized speech and/or noise.
- a current first parameter value is determined from an index corresponding to at least the first parameter, e.g. the fixed codebook gain correction factor ⁇ gc .
- the current first parameter value is adjusted, e.g. multiplied by a, in order to achieve an enhanced first characteristic, thereby obtaining an enhanced first parameter value a ⁇ ⁇ ⁇ gc old .
- a new index value is determined from a table relating index values to at least first parameter values, e.g. a quantization table, such that a new first parameter value corresponding to the new index value substantially matches the enhanced first parameter value.
- a new index value for a ⁇ ⁇ ⁇ gc old is searched such that the equation a ⁇ ⁇ ⁇ gc old - ⁇ ⁇ gc new is minimized, ⁇ ⁇ gc new being the new first parameter value corresponding to the searched new index value.
- a current second parameter value may be determined from the index further corresponding to a second parameter such as the adaptive codebook gain controlling a second characteristic of speech.
- the new index value is determined from the table further relating the index values to second parameter values, e.g. a vector quantization table, such that a new second parameter value corresponding to the new index value substantially matches the current second parameter value.
- a new index value for a ⁇ ⁇ ⁇ gc old and g p_old is searched such that the equation a ⁇ ⁇ ⁇ gc old - ⁇ ⁇ gc new + weight ⁇ g p_new - g p_old is minimized.
- g p _ new is the new second parameter value corresponding to the new index value.
- weight can be ⁇ 1, so that the new index value is determined from the table such that substantially matching the current second parameter value has precedence.
- Fig. 14 shows a schematic block diagram illustrating an apparatus 100 for enhancing a coded audio signal according to the invention.
- the apparatus receives a coded audio signal which comprises indices which represent speech and/or noise parameters which comprise at least a first parameter for adjusting a first characteristic of the audio signal.
- the apparatus comprises a parameter value determination block 11 for determining a current first parameter value from an index corresponding to at least the first parameter, an adjusting block 12 for adjusting the current first parameter value in order to achieve an enhanced first characteristic, thereby obtaining an enhanced first parameter value, and an index value determination block 13 for determining a new index value from a table relating index values to at least first parameter values, such that a new first parameter value corresponding to the new index value substantially matches the enhanced first parameter value.
- the parameter value determination block 11 may further determine a current second parameter value from the index further corresponding to a second parameter, and the index value determination block 13 may then determine the new index value from the table further relating the index values to second parameter values, such that a new second parameter value corresponding to the new index value substantially matches the current second parameter value.
- the index value is optimized simultaneously for both the first and second parameters.
- the index value determination block 13 may determine the new index value from the table such that substantially matching the current second parameter value has precedence.
- the apparatus 100 may further include replacing means for replacing a current value of the index corresponding to the at least first parameter by the determined new index value, and output enhanced coded speech containing the new index value.
- the first parameter value may be the background noise level parameter value which is determined and adjusted and for which a new index value is determined in order to adjust the background noise level.
- the second parameter value may be the background noise level parameter the index value of which is determined in accordance with the adjusted speech level.
- the speech level manipulation requires also manipulating the background noise level parameter during speech pauses in DTX.
- the background noise level parameter the averaged logarithmic frame energy
- the comfort noise level can be adjusted by changing the energy index value.
- the level can be adjusted in 1.5 dB, so finding a suitable comfort noise level corresponding to the change of the speech level is possible.
- the evaluated comfort noise parameters (the average LSF (Line Spectral Frequency) parameter vector f mean and the averaged logarithmic frame energy en log mean ) are encoded into a special frame, called a Silence Descriptor (SID) frame for transmission to the receiver side.
- SID Silence Descriptor
- the parameters give information on the level en log mean and the spectrum ( f mean ) of the background noise. More details can be found in 3GPP TS 26.093 V4.0.0 (2001-03), "3rd Generation Partnership Project; Technical Specification Group Services and System Aspects; Mandatory Speech Codec speech processing functions; AMR speech codec; Source controlled rate operation (Release 6)".
- a parameter value to be adjusted may be the comfort noise parameter value. Accordingly, a new index value index new is determined as mentioned above. In other words, a current background noise parameter index value index may be detected, and a new background noise parameter index value index new may be determined by adding ⁇ 4log 2 ⁇ to the current background noise parameter index value index , wherein ⁇ corresponds to the enhancement of the first characteristic represented by the first speech parameter.
- the level of the synthesized speech signal can be adjusted by manipulating the fixed codebook gain factor index, as shown previously. While being a measure of prediction error, the fixed codebook gain factor index does not discover the level of the speech signal. Therefore, to control the gain manipulation, i.e. to determine whether the level should be changed, the speech signal level must be first estimated.
- the six or seven MSB of the PCM speech samples are transmitted to the far end unchanged, to facilitate a seamless TFO interruption. These six or seven MSB can be used to estimate the speech level.
- the coded speech signal must be at least partially decoded (post-filtering is not necessary) to estimate the speech level.
- Fig. 15 shows a block diagram illustrating a scheme with the possibility of using a constant gain in the gain manipulation described above.
- decoding PCM signals out of the codec signal for using the PCM signals in the gain estimation i.e. speech level estimation
- the speech may be coded with e.g. AMR, AMR-WB (AMR WideBand), GSM FR, GSM EFR, GSM HR speech codecs.
- Fig. 16 shows a high level implementation example of the present invention in an MGW (Media Gateway) of the 3G network architecture.
- MGW Media Gateway
- the present invention may be implemented in a DSP (Digital Signal Processor) of the MGW.
- DSP Digital Signal Processor
- the implementation of the invention is not limited to an MGW.
- coded speech is fed to the MGW.
- the coded speech comprises at least one index corresponding to a value of a speech parameter which adjusts the level of synthesized speech.
- This index may also indicate a value of another speech parameter which is affected by the speech parameter for adjusting the level of synthesized speech. For example, this other speech parameter adjusts the periodicity or pitch of the synthesized speech.
- the index is controlled so as to adjust the level of the speech to a desired level.
- a new index indicating values of the speech parameters affecting the level of the speech such as the fixed codebook gain factor and adaptive codebook gain, is determined by minimizing an error between the desired level and the realized effective level.
- the new index is found which indicates values of the speech parameters realizing the desired level of speech.
- the original index is replaced by the new index and enhanced coded speech is output.
- the partial decoding of speech shown in Fig. 16 relates to controlling means for determining a current level of speech to decide whether the level should be adjusted.
- the above described embodiments of the present invention may not only be utilized in level control itself, but also in noise suppression and echo control (nonlinear processing) in the coded domain.
- Noise suppression can utilize the above technique by e.g. adjusting the comfort noise level during speech pauses.
- Echo control may utilize the above technique e.g. by attenuating the speech signal during echo bursts.
- the present invention is not intended to be limited only to TFO and TrFO voice communication and to voice communication over packet-switched networks, but rather to comprise enhancing coded audio signals in general.
- the invention finds application also in enhancing coded audio signals related e.g. to audio/speech/multimedia streaming applications and to MMS (Multimedia Messaging Service) applications.
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Claims (15)
- Procédé d'amélioration d'un signal audio codé comprenant des indices qui représentent des paramètres de signal audio qui comprennent au moins un premier paramètre représentant une première caractéristique du signal audio et un deuxième paramètre, le procédé comprenant les étapes consistant à :déterminer une valeur de premier paramètre actuelle à partir d'un indice correspondant à un premier paramètre ;ajuster la valeur de premier paramètre actuelle pour atteindre une première caractéristique améliorée, en obtenant de ce fait une valeur de premier paramètre améliorée ;déterminer une valeur de deuxième paramètre actuelle à partir de l'indice correspondant en outre à un deuxième paramètre ; etdéterminer une nouvelle valeur d'indice à partir d'une table liant des valeurs d'indices à des valeurs de premier paramètre et liant des valeurs d'indices à des valeurs de deuxième paramètre, de sorte qu'une nouvelle valeur de premier paramètre correspondant à la nouvelle valeur d'indice et une nouvelle valeur de deuxième paramètre correspondant à la nouvelle valeur d'indice correspondent sensiblement à la valeur de premier paramètre améliorée et à la valeur de deuxième paramètre actuelle.
- Procédé d'amélioration d'un signal audio codé comprenant des indices qui représentent des paramètres de signal audio qui comprennent au moins un premier paramètre représentant une première caractéristique du signal audio et un paramètre de bruit de fond, le procédé comprenant les étapes consistant à :déterminer une valeur de premier paramètre actuelle à partir d'un indice correspondant à au moins un premier paramètre ;ajuster la valeur de premier paramètre actuelle pour atteindre une première caractéristique améliorée, en obtenant de ce fait une valeur de premier paramètre améliorée ;déterminer une nouvelle valeur d'indice à partir d'une table liant des valeurs d'indices à au moins des valeurs de premier paramètre, de sorte qu'une nouvelle valeur de premier paramètre correspondant à la nouvelle valeur d'indice corresponde sensiblement à la valeur de premier paramètre améliorée ;détecter une valeur d'indice de paramètre de bruit de fond actuelle ; etdéterminer une nouvelle valeur d'indice de paramètre de bruit de fond correspondant à la première caractéristique améliorée.
- Procédé selon la revendication 1, comprenant en outre l'étape consistant à :remplacer une valeur actuelle de l'indice correspondant au moins au premier paramètre par la nouvelle valeur d'indice déterminée.
- Procédé selon la revendication 1, comprenant en outre les étapes consistant à :détecter une valeur d'indice de paramètre de bruit de fond actuelle ; etdéterminer une nouvelle valeur d'indice de paramètre de bruit de fond correspondant à la première caractéristique améliorée.
- Procédé selon la revendication 1, comprenant en outre l'étape consistant à déterminer la nouvelle valeur d'indice à partir de la table de sorte qu'une correspondance substantielle de la valeur de deuxième paramètre actuelle ait précédence.
- Procédé selon la revendication 2, comprenant en outre l'étape consistant à :remplacer une valeur actuelle de l'indice correspondant au premier paramètre par la nouvelle valeur d'indice déterminée.
- Appareil destiné à améliorer un signal audio codé comprenant des indices qui représentent des paramètres de signal audio qui comprennent au moins un premier paramètre représentant une première caractéristique du signal audio et un deuxième paramètre, l'appareil comprenant :un moyen de détermination de valeur de paramètre pour déterminer une valeur de premier paramètre actuelle à partir d'un indice correspondant à un premier paramètre et pour déterminer une valeur de deuxième paramètre actuelle à partir de l'indice correspondant en outre à un deuxième paramètre ;un moyen d'ajustement pour ajuster la valeur de premier paramètre actuelle pour atteindre une première caractéristique améliorée, en obtenant de ce fait une valeur de premier paramètre améliorée ; etun moyen de détermination de valeur d'indice pour déterminer une nouvelle valeur d'indice à partir d'une table liant des valeurs d'indices à des valeurs de premier paramètre et liant des valeurs d'indices à des valeurs de deuxième paramètre, dans lequel une nouvelle valeur de premier paramètre correspondant à la nouvelle valeur d'indice et une nouvelle valeur de deuxième paramètre correspondant à la nouvelle valeur d'indice correspondent sensiblement à la valeur de premier paramètre améliorée et à la valeur de deuxième paramètre actuelle.
- Appareil destiné à améliorer un signal audio codé comprenant des indices qui représentent des paramètres de signal audio qui comprennent au moins un premier paramètre représentant une première caractéristique du signal audio et un paramètre de bruit de fond, l'appareil comprenant :un moyen de détermination de valeur de paramètre pour déterminer une valeur de premier paramètre actuelle à partir d'un indice correspondant à au moins un premier paramètre ;un moyen d'ajustement pour ajuster la valeur de premier paramètre actuelle pour atteindre une première caractéristique améliorée, en obtenant de ce fait une valeur de premier paramètre améliorée ;un moyen de détermination de valeur d'indice pour déterminer une nouvelle valeur d'indice à partir d'une table liant des valeurs d'indices à au moins des valeurs de premier paramètre, de sorte qu'une nouvelle valeur de premier paramètre correspondant à la nouvelle valeur d'indice corresponde sensiblement à la valeur de premier paramètre améliorée ;un moyen de détection pour détecter une valeur d'indice de paramètre de bruit de fond actuelle ; etun moyen de détermination pour déterminer une nouvelle valeur d'indice de paramètre de bruit de fond correspondant à la première caractéristique améliorée.
- Appareil selon la revendication 7, comprenant en outre :un moyen de remplacement pour remplacer une valeur actuelle de l'indice correspondant au moins au premier paramètre par la nouvelle valeur d'indice déterminée.
- Appareil selon la revendication 7, comprenant en outre :un moyen de détection pour détecter une valeur d'indice de paramètre de bruit de fond actuelle ; etun moyen de détermination pour déterminer une nouvelle valeur d'indice de paramètre de bruit de fond correspondant à la première caractéristique améliorée.
- Appareil selon la revendication 7, dans lequel le moyen de détermination de valeur d'indice est configuré pour déterminer la nouvelle valeur d'indice à partir de la table de sorte qu'une correspondance substantielle de la valeur de deuxième paramètre actuelle ait précédence.
- Appareil selon la revendication 8, comprenant en outre :un moyen de remplacement pour remplacer une valeur actuelle de l'indice correspondant au premier paramètre par la nouvelle valeur d'indice déterminée.
- Produit de programme informatique, comprenant des parties aptes à effectuer les étapes du procédé selon l'une quelconque des revendications 1 à 6 lorsque le produit est exécuté sur un ordinateur.
- Produit de programme informatique selon la revendication 13, dans lequel ledit produit de programme informatique est directement chargeable dans la mémoire interne de l'ordinateur.
- Support lisible par ordinateur sur lequel est stocké le produit de programme informatique de la revendication 13.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP20040029839 EP1544848B1 (fr) | 2003-12-18 | 2004-12-16 | Amélioration de la qualité d'un signal audio dans le domaine codé |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP03029182 | 2003-12-18 | ||
EP03029182 | 2003-12-18 | ||
US803103 | 2004-03-18 | ||
US10/803,103 US7613607B2 (en) | 2003-12-18 | 2004-03-18 | Audio enhancement in coded domain |
EP20040029839 EP1544848B1 (fr) | 2003-12-18 | 2004-12-16 | Amélioration de la qualité d'un signal audio dans le domaine codé |
Publications (3)
Publication Number | Publication Date |
---|---|
EP1544848A2 EP1544848A2 (fr) | 2005-06-22 |
EP1544848A3 EP1544848A3 (fr) | 2005-09-21 |
EP1544848B1 true EP1544848B1 (fr) | 2010-01-20 |
Family
ID=34527388
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP20040029839 Not-in-force EP1544848B1 (fr) | 2003-12-18 | 2004-12-16 | Amélioration de la qualité d'un signal audio dans le domaine codé |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP1544848B1 (fr) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
RU2682005C1 (ru) * | 2016-10-18 | 2019-03-14 | ФОРД ГЛОУБАЛ ТЕКНОЛОДЖИЗ, ЭлЭлСи | Идентификация контактов с использованием распознавания речи |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8874437B2 (en) | 2005-03-28 | 2014-10-28 | Tellabs Operations, Inc. | Method and apparatus for modifying an encoded signal for voice quality enhancement |
US20060217972A1 (en) * | 2005-03-28 | 2006-09-28 | Tellabs Operations, Inc. | Method and apparatus for modifying an encoded signal |
US20070160154A1 (en) * | 2005-03-28 | 2007-07-12 | Sukkar Rafid A | Method and apparatus for injecting comfort noise in a communications signal |
BRPI0520720A2 (pt) * | 2005-11-30 | 2009-06-13 | Ericsson Telefon Ab L M | método para transcodificação de fala de um primeiro esquema de codificação de fala para um segundo esquema de codificação de fala, transcodificador de fala, e, sistema de telecomunicação |
US8032365B2 (en) | 2007-08-31 | 2011-10-04 | Tellabs Operations, Inc. | Method and apparatus for controlling echo in the coded domain |
EP2518986B1 (fr) * | 2011-07-25 | 2018-03-21 | Huawei Technologies Co., Ltd. | Dispositif et procédé pour le traitement d'écho dans un domaine paramétrique |
CN109286470B (zh) * | 2018-09-28 | 2020-07-10 | 华中科技大学 | 一种主动非线性变换信道加扰传输方法 |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FI116642B (fi) * | 1998-02-09 | 2006-01-13 | Nokia Corp | Puheparametrien käsittelymenetelmä, puhekoodauksen käsittely-yksikkö ja verkkoelementti |
JP4639441B2 (ja) * | 1999-09-01 | 2011-02-23 | ソニー株式会社 | ディジタル信号処理装置および処理方法、並びにディジタル信号記録装置および記録方法 |
JP2005531017A (ja) * | 2002-05-13 | 2005-10-13 | マインドスピード・テクノロジーズ・インコーポレイテッド | パケット網環境における音声のコード変換 |
-
2004
- 2004-12-16 EP EP20040029839 patent/EP1544848B1/fr not_active Not-in-force
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
RU2682005C1 (ru) * | 2016-10-18 | 2019-03-14 | ФОРД ГЛОУБАЛ ТЕКНОЛОДЖИЗ, ЭлЭлСи | Идентификация контактов с использованием распознавания речи |
Also Published As
Publication number | Publication date |
---|---|
EP1544848A2 (fr) | 2005-06-22 |
EP1544848A3 (fr) | 2005-09-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7613607B2 (en) | Audio enhancement in coded domain | |
EP1509903B1 (fr) | Procede et dispositif de masquage efficace d'effacement de trames dans des codec vocaux de type lineaire predictif | |
EP0732686B1 (fr) | Codage CELP à 32 kbit/s à faible retard d'un signal à large bande | |
EP1050040B1 (fr) | Systeme et procede de decodage comportant un postfiltre adaptatif | |
EP1979895B1 (fr) | Procede et dispositif de masquage efficace d'effacement de trames dans des codecs vocaux | |
US6735567B2 (en) | Encoding and decoding speech signals variably based on signal classification | |
US6757649B1 (en) | Codebook tables for multi-rate encoding and decoding with pre-gain and delayed-gain quantization tables | |
JP4222951B2 (ja) | 紛失フレームを取扱うための音声通信システムおよび方法 | |
JP3490685B2 (ja) | 広帯域信号の符号化における適応帯域ピッチ探索のための方法および装置 | |
EP1327241B1 (fr) | Amelioration de signaux acoustiques codes, en termes de perception | |
JP4176349B2 (ja) | マルチモードの音声符号器 | |
JPH08328591A (ja) | 短期知覚重み付けフィルタを使用する合成分析音声コーダに雑音マスキングレベルを適応する方法 | |
JP3955179B2 (ja) | 音声符号化装置、音声復号化装置、およびこれらの方法 | |
US6424942B1 (en) | Methods and arrangements in a telecommunications system | |
Ordentlich et al. | Low-delay code-excited linear-predictive coding of wideband speech at 32 kbps | |
US6205423B1 (en) | Method for coding speech containing noise-like speech periods and/or having background noise | |
CA2378035A1 (fr) | Gestion du bruit du domaine code | |
EP1544848B1 (fr) | Amélioration de la qualité d'un signal audio dans le domaine codé | |
EP3281197B1 (fr) | Codeur audio et procédé de codage d'un signal audio | |
EP0954851A1 (fr) | Vocodeur multi-niveau a codage par transformee des signaux predictifs residuels et quantification sur modeles auditifs | |
US20050102136A1 (en) | Speech codecs | |
CN100369108C (zh) | 编码域中的音频增强的方法和设备 | |
Shoham et al. | pyyy. p. AY CODE-EXCITED LINEAR-PREDICTIVE (ypN (; OF WIDEBAND SPEECH AT 32 KBPS |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20041216 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA HR LV MK YU |
|
PUAL | Search report despatched |
Free format text: ORIGINAL CODE: 0009013 |
|
AK | Designated contracting states |
Kind code of ref document: A3 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA HR LV MK YU |
|
AKX | Designation fees paid |
Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR |
|
17Q | First examination report despatched |
Effective date: 20080808 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REF | Corresponds to: |
Ref document number: 602004025193 Country of ref document: DE Date of ref document: 20100311 Kind code of ref document: P |
|
REG | Reference to a national code |
Ref country code: RO Ref legal event code: EPE |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2337137 Country of ref document: ES Kind code of ref document: T3 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20100120 |
|
LTIE | Lt: invalidation of european patent or patent extension |
Effective date: 20100120 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100520 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100520 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 |
|
REG | Reference to a national code |
Ref country code: HU Ref legal event code: AG4A Ref document number: E007844 Country of ref document: HU |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100421 Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100420 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 |
|
26N | No opposition filed |
Effective date: 20101021 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20101224 Year of fee payment: 7 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: HU Payment date: 20101214 Year of fee payment: 7 Ref country code: RO Payment date: 20101112 Year of fee payment: 7 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20101224 Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20101231 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20110120 Year of fee payment: 7 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20101231 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20101216 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20101231 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20120831 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20101216 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100120 Ref country code: RO Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111216 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111217 Ref country code: IT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111216 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20121213 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20121212 Year of fee payment: 9 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20120102 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FD2A Effective date: 20130703 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111217 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 602004025193 Country of ref document: DE |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20131216 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602004025193 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G10L0019140000 Ipc: G10L0021020000 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140701 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602004025193 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G10L0019140000 Ipc: G10L0021020000 Effective date: 20140925 Ref country code: DE Ref legal event code: R119 Ref document number: 602004025193 Country of ref document: DE Effective date: 20140701 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20131216 |