EP2633520B1 - Parametric encoder for encoding a multi-channel audio signal - Google Patents
Parametric encoder for encoding a multi-channel audio signal Download PDFInfo
- Publication number
- EP2633520B1 EP2633520B1 EP10859153.8A EP10859153A EP2633520B1 EP 2633520 B1 EP2633520 B1 EP 2633520B1 EP 10859153 A EP10859153 A EP 10859153A EP 2633520 B1 EP2633520 B1 EP 2633520B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- audio signal
- parameter
- encoding
- transformed
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Not-in-force
Links
- 230000005236 sound signal Effects 0.000 title claims description 87
- 238000000034 method Methods 0.000 claims description 15
- 230000001131 transforming effect Effects 0.000 claims description 14
- 238000010586 diagram Methods 0.000 description 8
- 230000015572 biosynthetic process Effects 0.000 description 6
- 238000003786 synthesis reaction Methods 0.000 description 6
- 230000001419 dependent effect Effects 0.000 description 3
- 230000002194 synthesizing effect Effects 0.000 description 3
- 210000002370 ICC Anatomy 0.000 description 2
- 238000010988 intraclass correlation coefficient Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
Definitions
- the present invention relates to audio coding.
- Parametric stereo or multi-channel audio coding uses spatial cues to synthesize down-mix - usually mono or stereo - audio signals to signals with more channels.
- the down-mix audio signals result from a superposition of a plurality of audio channel signals of a multi-channel audio signal, e.g. of a stereo audio signal.
- These less channels are waveform coded and side information, i.e. the spatial cues, relating to the original signal channel relations is added as encoding parameters to the coded audio channels.
- the decoder uses this side information to re-generate the original number of audio channels based on the decoded waveform coded audio channels.
- a basic parametric stereo coder may use inter-channel level differences (ILD) as a cue needed for generating the stereo signal from the mono down-mix audio signal. More sophisticated coders may also use the inter-channel coherence (ICC), which may represent a degree of similarity between the audio channel signals, i.e. audio channels. Furthermore, when coding binaural stereo signals e.g. for 3D audio or headphone based surround rendering, also an inter-channel phase difference (IPD) may play a role to reproduce phase/delay differences between the channels.
- IPD inter-channel phase difference
- ICC cues may be relevant for most audio and music contents to re-generate ambience, stereo reverb, source width, and other perceptions related to spatial impression as described in J. Blauert, Spatial Hearing: The Psychophysics of Human Sound Localization, The MIT Press, Cambridge, Massachusetts, USA, 1997 .
- Coherence synthesis may be implemented by using de-correlators in frequency domain as described in E. Schuijers, W. Oomen, B. den Brinker, and J. Breebaart, "Advances in parametric coding for high-quality audio," in Preprint 114th Conv. Aud. Eng. Soc., Mar. 2003 .
- the known synthesis approaches for synthesizing multi-channel audio signals may suffer from an increased complexity.
- the use of ICC parameters e.g. in addition to other parameters, such as inter-channel level differences (ICLDs) and inter-channel phase differences (ICPDs), may increase a bitrate overhead.
- ICLDs inter-channel level differences
- US 2005/180579 A1 discloses a scheme for stereo and multi-channel synthesis of inter-channel correlation (ICC) (normalized cross-correlation) cues for parametric stereo and multi-channel coding.
- ICC inter-channel correlation
- the scheme synthesizes ICC cues such that they approximate those of the original.
- diffuse audio channels are generated and mixed with the transmitted combined (e.g., sum) signal(s).
- the diffuse audio channels are preferably generated using relatively long filters with exponentially decaying Gaussian impulse responses. Such impulse responses generate diffuse sound similar to late reverberation.
- inter-channel level difference ICLD
- inter-channel time difference ICTD
- ICC synthesis are all carried out in the domain of a single short-time Fourier transform (STFT), including the filtering for diffuse sound generation.
- STFT short-time Fourier transform
- WO 2003/219130 A1 discloses a combination device that includes: a detection unit that detects active coded bitstreams that are effective coded bitstreams from a plurality of coded bitstreams within a predetermined time period; a first combining unit that combines, from a plurality of downmix sub-streams included in the coded bitstreams, only downmix sub-streams included in the active coded bitstreams so as to generate a combined downmix sub-stream; and a second combining unit that combines, from a plurality of parameter sub-streams included in the coded bitstreams, only parameter sub-streams included in the active coded bitstreams so as to generate a combined parameter sub-stream.
- US 2003/219130 A1 discloses an auditory scene synthesized from a mono audio signal by modifying, for each critical band, an auditory scene parameter (e.g., an inter-aural level difference (ILD) and/or an inter-aural time difference (ITD)) for each sub-band within the critical band, where the modification is based on an average estimated coherence for the critical band.
- an auditory scene parameter e.g., an inter-aural level difference (ILD) and/or an inter-aural time difference (ITD)
- ILD inter-aural level difference
- ITD inter-aural time difference
- a goal to be achieved by the present invention is to reduce complexity of a parametric coding scheme. This goal is achieved by the features of the independent claims. Further embodiments are apparent from the description, the drawings and from the dependent claims.
- the invention is based on the finding that combining parametric encoding parameters such as ICC parameters may reduce bit rate required for representing the parameters and thus may reduce complexity of the resulting parametric encoding scheme.
- the combined encoding parameters may be applied e.g. only to a certain frequency region in order to improve an audio quality for e.g. speech whereby the complexity and the memory requirements may further be reduced.
- the first and second encoding parameter may be an inter-channel phase difference.
- the first and second encoding parameter may be an inter-channel coherence.
- the first and second encoding parameter may be an inter-channel intensity difference.
- the first and second encoding parameter may be an inter-channel level difference.
- the parameter generator is configured to generate the first encoding parameter and the second encoding parameter upon a basis of a multiplication of values of the first transformed audio signal and of the second transformed audio signal.
- the parameter combiner is configured to determine a weighted average of the first encoding parameter and the second encoding parameter using powers of the a first transformed audio signal and the second transformed signal at the certain frequency as weights to obtain the combined encoding parameter.
- the parameter combiner is configured to determine a weighted average of the first encoding parameter and the second encoding parameter using a frequency-dependent weight to obtain the combined encoding parameter.
- the parameter generator is configured to generate a plurality of encoding parameters from the first transformed audio signal and from the second transformed audio signal at a plurality of frequencies, and wherein the parameter combiner is configured to combine the plurality of the encoding parameters to obtain the combined encoding parameter.
- the parametric encoder further comprises a signal combiner for combining the first transformed audio signal and the second transformed audio signal to obtain a down-mix signal.
- the parametric encoder further comprises an inverse transformer for inversely transforming a combination of the first transformed audio signal and the second transformed audio signal to obtain a down-mix audio signal.
- the invention relates to a method for parametrically encoding a multi-channel audio signal having a first audio signal and a second audio signal, the method having transforming the first audio signal into frequency domain to obtain a first transformed audio signal, and transforming the second audio signal into frequency domain to obtain a second transformed audio signal, generating a first encoding parameter from the first transformed audio signal and from the second transformed audio signal at a first frequency, and generating a second encoding parameter from the first transformed audio signal and from the second transformed audio signal at a second frequency, and combining the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter.
- Fig. 1 shows a diagram of a parametric encoder for encoding a multi-channel audio signal having a first audio signal, x1, and a second audio signal, x2, according to an implementation form.
- the parametric encoder comprises a transformer 101 for transforming the first audio signal into frequency domain to obtain a first transformed audio signal, and for transforming the second audio signal into frequency domain to obtain a second transformed audio signal.
- the transformer 101 may comprise a first transformer 103 for transforming the first audio signal, and a second transformer 105 for transforming the second audio signal.
- the transformer 101 and/or the transformers 103, 105 may be Fourier transformers, by way of example.
- the first and second transformed audio signals are provided to a parameter generator 107 for generating a first encoding parameter from the first transformed signal and from the second transformed audio signal at a first frequency, e.g. at the i-th frequency or in the i-th band.
- the i-th band or "band i" refer to a frequency band i at or in which the parameter generator 107 generates the respective encoding parameter from the first and second transformed signal, and is also referred to as parameter band i.
- the parameter generator 107 is further configured to generate a second encoding parameter from the first and second transformed audio signal at a second frequency or in a second band.
- the first and the second encoding parameters are provided to the parameter combiner 109 which combines the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter according to a principle described herein.
- the parameter combiner 109 may separately obtain encoding parameters for different parameter bands.
- the e.g. stereo input audio channels x1 and x2 are converted to a plurality of sub-bands or parameter bands.
- the corresponding ICC parameters may be estimated.
- One or more ICC parameter combining processes e.g. one of the processes according to the equations (1)-(4), may be applied to the ICC parameters of all or subsets of parameter bands, to compute the combined ICC parameters.
- At least one combined ICC parameter may be put into a bit stream 111 or transmitted to an audio decoder which is not depicted in Fig. 1 .
- the encoding parameter may be any encoding parameter or of any encoding parameter type used for parametric encoding, e.g. inter-channel phase difference or inter-channel intensity difference or an inter-channel level difference or the like, and that the encoder may be adapted to produce combined encoding parameters according one, some or all of the aforementioned encoding parameter types and to include combined encoding parameters of different types in the bitstream 111 as side information.
- the parametric encoder of Fig, 1 may form a parametric stereo encoder which estimates in parameter bands perceptual spatial cue parameters, such as ICLD, ICPD, and/or ICC. If the parameter band index is i, then the estimated parameters in that band are denoted ICLD(i), ICPD(i), and ICC(i). The left and right signal power in a parameter band are denoted P1(i) and P2(i), respectively.
- ICC ⁇ i ⁇ I g i ⁇ P 1 i + P 2 i ⁇ ICC i ⁇ i ⁇ I g i ⁇ P 1 i + P 2 i
- g i is a weight given to frequency (parameter band) i.
- ICC ⁇ i ⁇ I g i ⁇ ICC i ⁇ i ⁇ I g i
- a single full-band ICC performs surprisingly well.
- the combined ICC is computed using all parameter bands, i.e. I contains all parameter band indices.
- the speech quality may be improved by only using ICC in a limited frequency range.
- ICC parameter bands between 500 Hz and 1.5 kHz
- less artifacts occur.
- the combined ICC is computed using only parameter bands between 500 Hz and 1.5 kHz, i.e. I contains only those indices.
- the parametric encoder shown in Fig. 1 may estimate one or more combined ICC parameters by:
- Fig. 2 shows a block diagram of a parametric encoder for decoding a down-mix audio signal according to an implementation form.
- the down-mix signal may be provided by the parametric encoder as shown e.g. in Fig. 1 .
- the parametric decoder comprises a transformer 201 for transforming the down-mix audio signal, as, to obtain a transformed down-mix audio signal having a certain frequency, e.g. an i-th frequency of a plurality of frequencies, or correspondingly a certain band, e.g. an i-th band of a plurality of bands.
- the parametric decoder further comprises a provider 203 for providing a frequency-specific encoding parameter associated with the certain frequency.
- the frequency-specific encoding parameter may be derived from the combined encoding parameter. However, the frequency-specific encoding parameter may correspond to the combined encoding parameter.
- the parametric decoder further comprises an audio synthesizer 205, e.g. a stereo synthesizer, for synthesizing a first audio signal and a second audio signal at the certain frequency or in the certain band from the transformed down-mix audio signal provided by the transformer 201 using the frequency-specific encoding parameter as provided by the provider 203.
- an audio synthesizer 205 e.g. a stereo synthesizer
- the transformer 201 may be a Fourier transformer, wherein the audio synthesizer may synthesize the first and the second audio signal in frequency domain.
- the output signal provided by the synthesizer 205 may correspond to the first and second audio channel signal.
- the parametric encoder may further comprise an inverse transformer 207 for inversely transforming the first and second audio channel signal in time domain in order to obtain a first and second audio channel signal, x1 and x2, in time domain.
- the parametric decoder shown in Fig. 1 uses for all parameter bands, or a subset J thereof, the combined ICC parameter or a modified version thereof.
- One can use the same subset of parameter bands at the decoder as at the encoder, i.e. J I, or a different subset.
- the parametric decoder may receive the down-mix signal s and the stereo parameters, i.e. encoding parameters, amongst which at least one combined ICC parameter may be received.
- the stereo parameters i.e. encoding parameters
- Combined ICC to Band ICC Conversion when one combined ICC is used, combined ICC is applied to all parameter bands. Or, if the combined ICC was estimated only for a subset of bands, J, then the decoder may apply the combined ICC to all bands, to the same subset, or to another subset, e.g. a subset of the same subset.
- the decoder may apply the combined ICCs to parameter bands corresponding to the frequency regions from which the combined ICCs were estimated.
- Fig. 3 shows a diagram of a method for parametrically encoding a multi-channel audio signal having the first and the second audio signal as mentioned above.
- the method comprises transforming 301 the first and second audio signal into frequency domain to obtain a first and second transformed audio signal, generating 303 a first encoding parameter from the first and second transformed audio signal at a first frequency, and a second encoding parameter from the first and second transformed audio signal at a second frequency, and combining 305 the first and second encoding parameter to obtain a combined encoding parameter.
- the method depicted in Fig. 3 may be performed by the parametric encoder as shown in Fig. 1 .
- Fig. 4 shows a block diagram of a method for parametrically decoding a down-mix audio signal upon a basis of a combined encoding parameter.
- the down-mix audio signal may represent a combination, e.g. a superposition, of a first and second audio signal.
- the combined encoding parameter may have features as described above.
- the method comprises transforming 401 the down-mix audio signal to obtain a transformed down-mix audio signal having a certain frequency, providing 403 a frequency-specific encoding parameter associated with the certain frequency upon the basis of the combined encoding parameter, according to the principle described herein, and synthesizing 405 the first and second audio signal at the certain frequency from the transformed down-mix audio signal and from the frequency-specific encoding parameter.
- the method depicted in Fig. 4 may be performed by the parametric decoder as shown in Fig. 2 .
- the parametric decoder shown in Fig. 2 may be a parametric stereo decoder adapted for
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Stereophonic System (AREA)
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2010/078358 WO2012058805A1 (en) | 2010-11-03 | 2010-11-03 | Parametric encoder for encoding a multi-channel audio signal |
Publications (3)
Publication Number | Publication Date |
---|---|
EP2633520A1 EP2633520A1 (en) | 2013-09-04 |
EP2633520A4 EP2633520A4 (en) | 2013-09-04 |
EP2633520B1 true EP2633520B1 (en) | 2015-09-02 |
Family
ID=46023928
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP10859153.8A Not-in-force EP2633520B1 (en) | 2010-11-03 | 2010-11-03 | Parametric encoder for encoding a multi-channel audio signal |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP2633520B1 (es) |
CN (1) | CN102844808B (es) |
ES (1) | ES2553398T3 (es) |
WO (1) | WO2012058805A1 (es) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020193865A1 (en) * | 2019-03-28 | 2020-10-01 | Nokia Technologies Oy | Determination of the significance of spatial audio parameters and associated encoding |
RU2773636C2 (ru) * | 2017-08-10 | 2022-06-06 | Хуавэй Текнолоджиз Ко., Лтд. | Способ кодирования стереопараметров временной области и соответствующий продукт |
US11727943B2 (en) | 2017-08-10 | 2023-08-15 | Huawei Technologies Co., Ltd. | Time-domain stereo parameter encoding method and related product |
Families Citing this family (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102739320B (zh) * | 2012-06-16 | 2014-11-05 | 天地融科技股份有限公司 | 音频数据传输方法、系统、传输装置及电子签名工具 |
WO2014023443A1 (en) | 2012-08-10 | 2014-02-13 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Encoder, decoder, system and method employing a residual concept for parametric audio object coding |
JP6013646B2 (ja) | 2013-04-05 | 2016-10-25 | ドルビー・インターナショナル・アーベー | オーディオ処理システム |
US9838819B2 (en) * | 2014-07-02 | 2017-12-05 | Qualcomm Incorporated | Reducing correlation between higher order ambisonic (HOA) background channels |
RU2763374C2 (ru) * | 2015-09-25 | 2021-12-28 | Войсэйдж Корпорейшн | Способ и система с использованием разности долговременных корреляций между левым и правым каналами для понижающего микширования во временной области стереофонического звукового сигнала в первичный и вторичный каналы |
CN107452387B (zh) * | 2016-05-31 | 2019-11-12 | 华为技术有限公司 | 一种声道间相位差参数的提取方法及装置 |
GB2576769A (en) * | 2018-08-31 | 2020-03-04 | Nokia Technologies Oy | Spatial parameter signalling |
CN110740404B (zh) * | 2019-09-27 | 2020-12-25 | 广州励丰文化科技股份有限公司 | 一种音频相关性的处理方法及音频处理装置 |
CN112731289B (zh) * | 2020-12-10 | 2024-05-07 | 深港产学研基地(北京大学香港科技大学深圳研修院) | 一种基于加权模板匹配的双耳声源定位方法和装置 |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7583805B2 (en) * | 2004-02-12 | 2009-09-01 | Agere Systems Inc. | Late reverberation-based synthesis of auditory scenes |
US7006636B2 (en) * | 2002-05-24 | 2006-02-28 | Agere Systems Inc. | Coherence-based audio coding and synthesis |
EP1671316B1 (en) * | 2003-09-29 | 2007-08-01 | Koninklijke Philips Electronics N.V. | Encoding audio signals |
WO2008132850A1 (ja) * | 2007-04-25 | 2008-11-06 | Panasonic Corporation | ステレオ音声符号化装置、ステレオ音声復号装置、およびこれらの方法 |
CN100571043C (zh) * | 2007-11-06 | 2009-12-16 | 武汉大学 | 一种空间参数立体声编解码方法及其装置 |
US8060042B2 (en) * | 2008-05-23 | 2011-11-15 | Lg Electronics Inc. | Method and an apparatus for processing an audio signal |
RU2495503C2 (ru) * | 2008-07-29 | 2013-10-10 | Панасоник Корпорэйшн | Устройство кодирования звука, устройство декодирования звука, устройство кодирования и декодирования звука и система проведения телеконференций |
CN102016982B (zh) * | 2009-02-04 | 2014-08-27 | 松下电器产业株式会社 | 结合装置、远程通信系统以及结合方法 |
-
2010
- 2010-11-03 WO PCT/CN2010/078358 patent/WO2012058805A1/en active Application Filing
- 2010-11-03 CN CN201080033764.0A patent/CN102844808B/zh active Active
- 2010-11-03 EP EP10859153.8A patent/EP2633520B1/en not_active Not-in-force
- 2010-11-03 ES ES10859153.8T patent/ES2553398T3/es active Active
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
RU2773636C2 (ru) * | 2017-08-10 | 2022-06-06 | Хуавэй Текнолоджиз Ко., Лтд. | Способ кодирования стереопараметров временной области и соответствующий продукт |
US11727943B2 (en) | 2017-08-10 | 2023-08-15 | Huawei Technologies Co., Ltd. | Time-domain stereo parameter encoding method and related product |
WO2020193865A1 (en) * | 2019-03-28 | 2020-10-01 | Nokia Technologies Oy | Determination of the significance of spatial audio parameters and associated encoding |
Also Published As
Publication number | Publication date |
---|---|
EP2633520A1 (en) | 2013-09-04 |
CN102844808B (zh) | 2016-01-13 |
CN102844808A (zh) | 2012-12-26 |
WO2012058805A1 (en) | 2012-05-10 |
EP2633520A4 (en) | 2013-09-04 |
ES2553398T3 (es) | 2015-12-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2633520B1 (en) | Parametric encoder for encoding a multi-channel audio signal | |
US10244319B2 (en) | Audio decoder for audio channel reconstruction | |
EP2834813B1 (en) | Multi-channel audio encoder and method for encoding a multi-channel audio signal | |
EP2702776B1 (en) | Parametric encoder for encoding a multi-channel audio signal | |
US9082396B2 (en) | Audio signal synthesizer | |
JP2015528926A (ja) | マルチチャネルをダウンミックス/アップミックスする場合のため一般化された空間オーディオオブジェクト符号化パラメトリック概念のデコーダおよび方法 | |
WO2013029225A1 (en) | Parametric multichannel encoder and decoder | |
JP2017058696A (ja) | インターチャネル差分推定方法及び空間オーディオ符号化装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20130530 |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 20130724 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20140414 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 19/02 20130101ALN20150218BHEP Ipc: G10L 19/008 20130101ALI20150218BHEP Ipc: G10L 19/00 20130101AFI20150218BHEP Ipc: G10L 13/00 20060101ALI20150218BHEP |
|
INTG | Intention to grant announced |
Effective date: 20150311 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 747064 Country of ref document: AT Kind code of ref document: T Effective date: 20150915 Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 6 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602010027288 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2553398 Country of ref document: ES Kind code of ref document: T3 Effective date: 20151209 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 747064 Country of ref document: AT Kind code of ref document: T Effective date: 20150902 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: FP |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151202 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151203 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160102 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160104 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602010027288 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151103 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20151130 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20151130 |
|
26N | No opposition filed |
Effective date: 20160603 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20151103 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20101103 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 8 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 9 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150902 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NL Payment date: 20201015 Year of fee payment: 11 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20201020 Year of fee payment: 11 Ref country code: GB Payment date: 20201021 Year of fee payment: 11 Ref country code: ES Payment date: 20201208 Year of fee payment: 11 Ref country code: FR Payment date: 20201013 Year of fee payment: 11 Ref country code: IT Payment date: 20201013 Year of fee payment: 11 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 602010027288 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MM Effective date: 20211201 |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20211103 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211103 Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220601 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211103 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FD2A Effective date: 20230214 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211104 |