EP2490459B1 - Procédé de mélange de signaux vocaux - Google Patents

Procédé de mélange de signaux vocaux Download PDF

Info

Publication number
EP2490459B1
EP2490459B1 EP11155021.6A EP11155021A EP2490459B1 EP 2490459 B1 EP2490459 B1 EP 2490459B1 EP 11155021 A EP11155021 A EP 11155021A EP 2490459 B1 EP2490459 B1 EP 2490459B1
Authority
EP
European Patent Office
Prior art keywords
signal
noise
microphone
blending
level
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP11155021.6A
Other languages
German (de)
English (en)
Other versions
EP2490459A1 (fr
Inventor
Bernd Iser
Arthur Wolf
Patrick Hannon
Mohamed Krini
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SVOX AG
Original Assignee
SVOX AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SVOX AG filed Critical SVOX AG
Priority to EP11155021.6A priority Critical patent/EP2490459B1/fr
Publication of EP2490459A1 publication Critical patent/EP2490459A1/fr
Application granted granted Critical
Publication of EP2490459B1 publication Critical patent/EP2490459B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10—General applications
    • H04R2499/11—Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10—General applications
    • H04R2499/13—Acoustic transducers and sound field adaptation in vehicles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/02—Circuits for transducers, loudspeakers or microphones for preventing acoustic reaction, i.e. acoustic oscillatory feedback

Definitions

  • the invention refers to a method and system installed in a car for the communication of people sitting in remote locations. Therefore at least one loudspeaker is installed and at least two microphones, one assigned to each person.
  • the invention relates to method applied to a hands-free telephony system or to an automatic speech recognition system, where conditions and requirements are quite similar. More specifically, the invention relates to a method according to the introductory clause of claim 1 as well as to a software product according to claim 7 and a system according to claim 8.
  • EP1850640B1 shows and describes a typical setup for a basic intercom system with two microphones and one loudspeaker.
  • the mixer in this setup decides depending on a criterion which of the microphone signals shall be switched to the next module or the output.
  • This criterion can depend on the detected occupation of a seat.
  • it has been also proposed in US006549629B2 such decision criterion can depend on the Signal plus Noise to Noise Ratio (SNNR).
  • SNNR Signal plus Noise to Noise Ratio
  • the fundamental disadvantage of such a system is that the feedback is still contained in the signal and therefore SNNR is not a good approach to a Signal to Noise Ratio (SNR).
  • SNR Signal to Noise Ratio
  • a further disadvantage of these systems is a possible change in background-noise level when switching from one speaking person to another (e.g. driver speaks to a passenger in the back while having his window open, and then the co-driver speaks to the passenger in the back having his window closed and driver as well as co-driver are each equipped with a microphone).
  • This change in background-noise level of the signal played back over the loudspeaker might be experienced as unpleasant by a listener.
  • Another disadvantage that occurs is, when the play-back level between two speaking persons differs. This can occur if for example the driver is quite tall and therefore the distance to the microphone is little, which means significant speech level in the microphone signal.
  • US-A-2005/0265560 discloses a system, where a beamformer is used for microphone signal blending. The, thus, blended output signal is then subjected to feedback suppression.
  • This has the disadvantage that the power of the signals to be blended is relative high and, moreover, the output signal of the beamformer is adulterated by the feedback component of the signal.
  • the suppression is merely the suppression of frequencies, which are just developing resonance oscillations, which means that feedback components remain, and only resonance oscillations are prevented.
  • US 2009/0316923 and US 2003/0026437 A1 disclose a feedback suppression or compensation effected before blending the signals of the at least two microphones. If in this connection the term "feedback suppression or compensation" is used, it is in any case a minimization of feedback components of a signal.
  • a feedback compensation is for example described in EP 1 679 874 B1 .
  • Feedback suppression can be applied by filtering, for example with a notch-filter.
  • Another known compensation method uses frequency shifting.
  • the method for voice signal blending is applied in a communication system, such as an indoor communication system, particularly in a vehicle, or in a hands-free telephony system or an automatic speech recognition system, comprising at least two microphones and at least one loudspeaker.
  • the microphone signals are blended with respective weights to be delivered to the at least one loudspeaker.
  • the feedback suppression or compensation is effected before blending the signals of the at least two microphones.
  • US 5 602 962 discloses a solution, where time series of microphone signals are added.
  • the measure according to the characterizing clause of claim 1 For at least a sub-band of the microphone signals the energies of the microphone signals are determined and at least the energies of the noise components are estimated, wherein for blending, a higher weight is given to the microphone signal with the highest ratio of signal energy to noise energy. Sub-dividing the entire frequency band into sub-bands enables it to give individual sub-bands a different weight. Blending will be made for each sub-band with the respective weights.
  • the feedback suppression or compensation is effected before blending the signals of the at least two microphones. This improves the quality of the blended signal and is more robust in relation to feedback components.
  • the feedback components are minimized by estimating the feedback signal or its energy level, preferably its power spectral density, and applying a Wiener filter which eliminates the estimated feedback signal from the signals to be blended.
  • a signal level adjustment of the at least two microphone signals to a predetermined value is carried out particularly immediately before or immediately after blending, wherein the adjustment is depending on the energies of the noise components and minimizes the perceivable difference in the noise level when blending from one microphone to the other.
  • the signal levels of the microphone signals are adjusted to a predetermined value so that there is no perceivable difference in the speech level when blending from one microphone to the other.
  • the noise level of at least two microphones can be unpleasantly different. An adjustment can be achieved, if the characteristics of claim 5 are fulfilled.
  • the microphone signals comprise different components, each with a certain level. If a person close to a microphone is speaking, then the signal of this microphone comprises this voice with a corresponding voice level.
  • the voice levels of the at least two microphone signals are adjusted substantially to the same level by adjusting the microphone signals. Then the difference of the noise levels of the at least two adjusted microphone signals is determined.
  • the noise suppressions in the microphone signals are controlled by adapting the parameters of the noise suppression characteristic in such a way, that the noise suppressed signals have substantially the same level of residual noise and the same level of speech signal in it for each microphone signal.
  • the respective speaking and/or listening, non-speaking party is detected by one of the following measures:
  • Fig. 1 shows a typical setup for a basic intercom system with two microphones 1 and 2 and one loudspeaker 3.
  • a voice mixer 5 which receives the pre-processed signals of the microphones 1, 2 and decides depending on a criterion which of the microphone signals shall be switched to its output and to the next module 6.
  • Fig. 2a one embodiment of the invention is illustrated, where the signals of the microphones 1, 2, after an optional pre-processing stage 4a, are subjected to feedback suppression or compensation in a stage 7.
  • the first component after some optional pre-processing is a feedback suppression or feedback compensation at 7.
  • This module 7 suppresses or compensates for the portion of the loudspeaker signal that is coupling back into the microphone and therefore being an undesired signal component.
  • noise suppression in a noise suppression module 8 can be effected.
  • This module 8 suppresses the background noise components of the microphone signals resulting from the background noise being present in the car cabin and being picked up by the microphones.
  • a practical embodiment with a noise suppression postponed to the voice blender 5a is shown in Fig. 4b .
  • the signal m i of microphone i is made up of the signal from the speaker in the cabin s i , the feedback of the system from the loudspeaker into the microphone f i and the background noise of the cabin recorded by the microphone b i .
  • m i k , ⁇ s i k , ⁇ + f i k , ⁇ + b i k , ⁇
  • i corresponds to the microphone index
  • k to the time interval
  • ⁇ to the frequency band.
  • the noise suppression is located before the voice blender 5a which is not necessarily required (without limitation of generality, see e.g. Fig. 4b ).
  • the blended and noise suppressed signal goes suitably to a NDGC or noise dependent gain control 9.
  • the voice blender module blends the at least two feedback suppressed or compensated and noise suppressed signals according to Fig. 2a together to one output signal following below described criterion. According to Fig. 2b the blended signals are only feedback suppressed or compensated.
  • the blending is made by giving weights to the microphones.
  • the blending or weighting criterion can either be evaluated in a non-frequency selective manner or in a frequency selective manner resulting in a non-frequency selective weighting or in a frequency selective weighting of the microphone signals m i .
  • 4a , 4b , 4c is a function of the ratio of the energy of every microphone signal S m i m i subtracted by the feedback f i (respectively S f i f i for the energy of the feedback component at every microphone) due to the coupling of the loudspeaker signal back into the microphone signal and further subtracted by the noise b i (respectively S b i b i for the energy of the noise component at every microphone) within the microphone signal resulting from the noise present in the car cabin and being picked up by the microphone (for the setup according to Fig. 2b each microphone signal is only subtracted by the feedback) to the estimated noise present in each microphone signal.
  • Double indices are used for the energy of the signal, and for power spectral densities.
  • w i k , ⁇ f S mm k , ⁇ , S ff k , ⁇ + S bb k , ⁇
  • N the number of microphones.
  • ⁇ m is a smoothing constant that has to be chosen between zero and one. This means the current short-term power is weighted with ⁇ m and the estimate of the previous time frame is weighted with (1- ⁇ m ).
  • the feedback component S f i f i ( k , ⁇ ) is estimated in the feedback compensation or feedback suppression module and is therefore given.
  • ⁇ is a small number depending on the sampling rate.
  • the modified SNR after subtracting the power of the feedback signal and of the background noise from the microphone signal power, is set into relation with the sum over all microphones of the modified SNR.
  • the resulting weight is more robust against misinterpretations of feedback power as desired speech power as would be the case using an SNNR as described in US006549629B2 .
  • the voice blender module 5a determines the difference in noise levels between the different signals and causes noise suppression to adapt the parameters of the noise suppression characteristic to result in a noise suppressed signal having the same level of residual noise in it for each speaker.
  • Fig. 4a shows, how a voice blender for carrying out the method according to the invention may be structured. It is clearly visible that feedback suppression or feedback compensation is done for each of the signals stemming from the microphones 1 and 2 in modules 7a and 7b. The output of these modules 7a, 7b is delivered to the voice blender 5a, i.e. to noise estimation modules 10, 10' (see equations 1-4), to power estimation modules 11, 11' (see equation 8) and to multiplicators 12 and 12' for weighting the incoming signals.
  • multiplicators 12, 12' are each controlled by a module Wi which determines the blender weights for each signal of the microphones 1, 2.
  • a summing point 13 At the output of blender 5a is a summing point 13.
  • the output signal v( ⁇ ,k) of the voice blender 5a can be used to influence noise suppression in module 8. This is shown in Fig. 4b , where the weighting signals W i,speech (k, ⁇ ) are supplied to module 8. Noise suppression down to zero gives mostly a bad feeling to the listener, for which reason it is preferred to adjust noise suppression to a predetermined level to a predetermined minimum level. This level typically equals 0.316 meaning a maximum suppression of -10 dB.
  • Fig. 4c a configuration according to Fig. 4c is possible within the scope of the present invention taking equation 8 in mind.
  • the weighting signals W i,speech (k, ⁇ ) are supplied to module 9, the noise dependent gain control, which, to have an information of noise present in the signal, receives also the output signal S bibi (k, ⁇ ) of the noise estimation modules 10 and 10'.
  • the weighting signal W i of the voice blender 5a could be used to control an equalizer 6a which forms part of the post-processor 6 ( Fig. 1 to Fig. 2b ).
  • the decision of the voice blender module is used for the noise dependent gain control module.
  • the noise dependent gain control module is responsible for increasing the level of the output signal by applying a gain g l,NDGC ( S b l b l ( k , ⁇ )), depending on the level of noise perceived by the listening party.
  • This increase of the output signal level depends on a characteristic which maps the level of background noise in the car cabin to a gain applied to the output signal. The characteristic chosen for this mapping depends on which speaker is active. This information is delivered by the voice blender module in the form of the microphone index of the most active microphone l( k ).
  • g l,NDGC,max ( S b l b l ( k, ⁇ )) represents the maximum gain of the NDGC (e.g. 10dB). So all properties of the different NDGC characteristics are described by g l,NDGC,max ( S b l b l ( k, ⁇ )), S bb,l,high ( k, ⁇ ) and S bb,l,low ( k , ⁇ ).
  • S b l b l ( k , ⁇ ) represents the noise component that is present in the microphone signal of the active speaker.
  • equalizer 6a part of post processing 6 of an intercom system is the equalizer 6a.
  • This equalizer 6a can as well be implemented as a multi-channel equalizer individual for each channel.
  • the settings of the equalizer 6a are chosen depending on the voice blender decision of which person is speaking to enhance the perceived quality of the output for the listening party (see Fig.3 and Fig. 5 ).

Landscapes

  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)

Claims (8)

  1. Procédé de mélange de signaux vocaux dans un système de communication, comprenant au moins deux microphones (1, 2) et au moins un haut-parleur (3), dans lequel les au moins deux signaux de microphone sont mélangés (5a) avec des poids respectifs, dans lequel le signal mélangé doit être transmis à l'au moins un haut-parleur, dans lequel une suppression ou compensation de retour (7, 7a, 7b) est effectuée avant de mélanger les signaux des au moins deux microphones,
    caractérisé en ce que
    la bande de fréquences entière des signaux de microphone est subdivisée en sous-bandes, pour au moins une sous-bande de fréquences des signaux de microphone, les énergies des signaux de microphone (11, 11') sont déterminées et les énergies du bruit (10, 10') sont estimées, dans lequel, pour le mélange, des sous-bandes individuelles sont autorisées à avoir un poids différent, un poids supérieur est donné au signal de microphone avec le rapport énergie de signal-énergie du bruit le plus élevé, et le mélange est fait pour chaque sous-bande avec les poids respectifs.
  2. Procédé selon la revendication 1,
    caractérisé en ce qu'un
    ajustement de niveau de signal des aux moins deux signaux de microphone à une valeur prédéterminée est effectué particulièrement immédiatement avant ou immédiatement après le mélange, dans lequel l'ajustement est fonction des énergies des composantes du bruit et minimise la différence dans le niveau de bruit lors du mélange d'un microphone à l'autre.
  3. Procédé selon l'une quelconque des revendications précédentes, caractérisé en ce que les niveaux de signal des signaux de microphone sont ajustés à une valeur prédéterminée de façon à ce qu'il n'y ait pas de différence dans le niveau de signal lors du mélange d'un microphone à l'autre.
  4. Procédé selon la revendication 3,
    caractérisé en ce que
    des niveaux vocaux des au moins deux signaux de microphone sont ajustés essentiellement au même niveau, puis la différence des énergies des composantes du bruit des au moins deux signaux de microphone ajustés est déterminée et la suppression du bruit est réalisée pour générer un signal à bruit supprimé ayant essentiellement le même niveau de bruit résiduel et le même niveau de signal de parole à l'intérieur pour chaque signal de microphone.
  5. Procédé selon l'une quelconque des revendications précédentes, dans lequel le système de communication est un système de communication intérieur dans un véhicule, caractérisé en ce qu'une partie locutrice et/ou une partie auditrice non-locutrice est/sont détectée(e) par l'une des mesures suivantes :
    a) par l'analyse du signal des au moins deux microphones durant le mélange ;
    b) par la détection au moyen de capteurs du véhicule, par ex. pour l'occupation des sièges.
  6. Procédé selon la revendication 5,
    caractérisé en ce qu'au moins
    une des mesures suivantes est prise après la détection :
    a) le poids de signal du signal de microphone où aucune partie locutrice est détectée est réduit, durant l'étape de mélange ;
    b) des caractéristiques d'amplificateur prédéterminées du contrôle du gain, par exemple le contrôle du gain fonction du bruit, pour les auditeurs et/ou les locuteurs, sont fournies et la caractéristique d'amplificateur qui correspond à la position de l'auditeur et/ou du locuteur respectif est choisie.
  7. Produit logiciel adapté pour exécuter le procédé selon l'une quelconque des revendications précédentes.
  8. Système de communication, comprenant au moins un haut-parleur et au moins deux microphones, ainsi qu'un dispositif de traitement du signal, qui est adapté pour exécuter le procédé selon l'une quelconque des revendications 1 à 6.
EP11155021.6A 2011-02-18 2011-02-18 Procédé de mélange de signaux vocaux Active EP2490459B1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP11155021.6A EP2490459B1 (fr) 2011-02-18 2011-02-18 Procédé de mélange de signaux vocaux

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP11155021.6A EP2490459B1 (fr) 2011-02-18 2011-02-18 Procédé de mélange de signaux vocaux

Publications (2)

Publication Number Publication Date
EP2490459A1 EP2490459A1 (fr) 2012-08-22
EP2490459B1 true EP2490459B1 (fr) 2018-04-11

Family

ID=44065560

Family Applications (1)

Application Number Title Priority Date Filing Date
EP11155021.6A Active EP2490459B1 (fr) 2011-02-18 2011-02-18 Procédé de mélange de signaux vocaux

Country Status (1)

Country Link
EP (1) EP2490459B1 (fr)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9516418B2 (en) 2013-01-29 2016-12-06 2236008 Ontario Inc. Sound field spatial stabilizer
US9271100B2 (en) 2013-06-20 2016-02-23 2236008 Ontario Inc. Sound field spatial stabilizer with spectral coherence compensation
US9099973B2 (en) 2013-06-20 2015-08-04 2236008 Ontario Inc. Sound field spatial stabilizer with structured noise compensation
EP2816816B1 (fr) * 2013-06-20 2018-01-17 2236008 Ontario Inc. Stabilisateur spatial du champ sonore avec compensation de bruit structurée
US9106196B2 (en) 2013-06-20 2015-08-11 2236008 Ontario Inc. Sound field spatial stabilizer with echo spectral coherence compensation
WO2015130283A1 (fr) 2014-02-27 2015-09-03 Nuance Communications, Inc. Procédés et appareil pour un contrôle de gain adaptatif dans un système de communication
EP3799444A1 (fr) 2019-09-25 2021-03-31 Oticon A/s Prothèse auditive comportant un système de microphone directionnel

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4330243A1 (de) * 1993-09-07 1995-03-09 Philips Patentverwaltung Sprachverarbeitungseinrichtung
US6549629B2 (en) 2001-02-21 2003-04-15 Digisonix Llc DVE system with normalized selection
JP2004537232A (ja) * 2001-07-20 2004-12-09 コーニンクレッカ　フィリップス　エレクトロニクス　エヌ　ヴィ 多数のマイクロフォンのエコーを抑圧する回路をポストプロセッサとして有する音響補強システム
EP1591995B1 (fr) 2004-04-29 2019-06-19 Harman Becker Automotive Systems GmbH Système de communication d'intérieur pour une cabine de véhicule
ATE396582T1 (de) 2005-01-11 2008-06-15 Harman Becker Automotive Sys Verringerung der rückkopplung von kommunikationssystemen
EP1850640B1 (fr) * 2006-04-25 2009-06-17 Harman/Becker Automotive Systems GmbH Système de communication pour un véhicule
US8385557B2 (en) * 2008-06-19 2013-02-26 Microsoft Corporation Multichannel acoustic echo reduction

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
EP2490459A1 (fr) 2012-08-22

Similar Documents

Publication Publication Date Title
US9711131B2 (en) Sound zone arrangement with zonewise speech suppression
EP2490459B1 (fr) Procédé de mélange de signaux vocaux
US6505057B1 (en) Integrated vehicle voice enhancement system and hands-free cellular telephone system
EP1718103B1 (fr) Compensation de la révérbération et de la rétroaction
EP1885154B1 (fr) Déreverbération des signaux d'un microphone
US7117145B1 (en) Adaptive filter for speech enhancement in a noisy environment
Li et al. Two-stage binaural speech enhancement with Wiener filter for high-quality speech communication
US7039197B1 (en) User interface for communication system
JP5834088B2 (ja) 動的マイクロフォン信号ミキサ
Kamkar-Parsi et al. Instantaneous binaural target PSD estimation for hearing aid noise reduction in complex acoustic environments
US8369901B2 (en) Hands-free telephony and in-vehicle communication
EP1858295B1 (fr) Égaliseur pour le traitement de signaux acoustiques
EP2859772B1 (fr) Détection du bruit caractéristique du vent pour les systèmes de communication embarqués comportant plusieurs zones acoustiques
US20070053524A1 (en) Method and system for communication enhancement in a noisy environment
US9532149B2 (en) Method of signal processing in a hearing aid system and a hearing aid system
US20090034752A1 (en) Constrainted switched adaptive beamforming
Schmidt et al. Signal processing for in-car communication systems
EP2490218B1 (fr) Procédé pour supprimer l'interférence
EP1623600B1 (fr) Procede et systeme d'amelioration des communications dans un environnement bruyant
US20060184361A1 (en) Method and apparatus for reducing an interference noise signal fraction in a microphone signal
EP2416593A1 (fr) Procédé de communication à l'intérieur d'une pièce
Gimm et al. 11 Combination of hands-free and ICC systems
Matheja et al. Dynamic signal combining for distributed microphone systems in car environments
Böhmler et al. Combined echo and noise reduction for distributed microphones
Gimm et al. 10 A Multichannel Spatial Hands-Free Application for In-Car Communication Systems

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20130220

17Q First examination report despatched

Effective date: 20130705

RIC1 Information provided on ipc code assigned before grant

Ipc: H04R 3/00 20060101AFI20170824BHEP

Ipc: H04R 3/02 20060101ALN20170824BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: H04R 3/02 20060101ALN20170913BHEP

Ipc: H04R 3/00 20060101AFI20170913BHEP

INTG Intention to grant announced

Effective date: 20171016

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 989245

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180415

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602011047312

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20180411

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180711

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180711

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180712

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 989245

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180813

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602011047312

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

26N No opposition filed

Effective date: 20190114

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20190218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190218

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20190228

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190228

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190218

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190228

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180811

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20110218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180411

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20231228

Year of fee payment: 14