EP2530673A1 - Audiogerät mit Rauschunterdrückung in einem Sprachsignal unter Verwendung von einem Filter mit fraktionaler Verzögerung - Google Patents

Audiogerät mit Rauschunterdrückung in einem Sprachsignal unter Verwendung von einem Filter mit fraktionaler Verzögerung Download PDF

Info

Publication number
EP2530673A1
EP2530673A1 EP12170407A EP12170407A EP2530673A1 EP 2530673 A1 EP2530673 A1 EP 2530673A1 EP 12170407 A EP12170407 A EP 12170407A EP 12170407 A EP12170407 A EP 12170407A EP 2530673 A1 EP2530673 A1 EP 2530673A1
Authority
EP
European Patent Office
Prior art keywords
speech
signal
filter
equipment
noise
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP12170407A
Other languages
English (en)
French (fr)
Other versions
EP2530673B1 (de
Inventor
Guillaume Vitte
Michael Herve
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Parrot SA
Original Assignee
Parrot SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Parrot SA filed Critical Parrot SA
Publication of EP2530673A1 publication Critical patent/EP2530673A1/de
Application granted granted Critical
Publication of EP2530673B1 publication Critical patent/EP2530673B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02165—Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal

Definitions

  • the invention relates to the treatment of speech in a noisy environment.
  • microphones include one or more microphones (“microphones”) sensitive, capturing not only the voice of the user, but also the surrounding noise, noise that is a disruptive element that can go in some cases to make unintelligible the speaker's words . It is the same if one wants to implement speech recognition techniques, because it is very difficult to perform a form recognition on words embedded in a high noise level.
  • the large distance between the microphone (placed at the dashboard or in an upper corner of the roof of the cockpit) and the speaker (whose distance is constrained by the driving position) causes the capture of a relatively high noise level, which makes it difficult to extract the useful signal embedded in the noise.
  • the highly noisy environment typical of the automotive environment has non-stationary spectral characteristics, that is to say that evolve unpredictably depending on the driving conditions: passage on deformed or paved roads, car radio operating etc.
  • the device is a headset type microphone / headset combined used for communication functions such as "hands-free" telephony functions, in addition to listening to a source audio (music for example) from a device to which the headphones are connected.
  • the headset can be used in a noisy environment (metro, busy street, train, etc.), so that the microphone will not only pick up the word of the wearer of the helmet, but also the noise surrounding.
  • the wearer is certainly protected from this noise by the helmet, especially if it is a model with closed earphones isolating the ear from the outside, and even more if the headset is provided with an "active control of noise”.
  • the distant speaker (the one at the other end of the communication channel) will suffer from the noise picked up by the microphone and being superimposed and interfere with the speech signal of the near speaker (the helmet wearer).
  • certain speech formers essential to the understanding of the voice are often embedded in noise components commonly encountered in the usual environments.
  • the invention relates more particularly to denoising techniques using several microphones, usually two microphones, to judiciously combine the signals picked up simultaneously by these microphones in order to isolate the useful speech components of the noise noise components.
  • a conventional technique consists in placing and orienting one of the microphones so that it mainly captures the voice of the speaker, while the other is arranged to capture a greater noise component than the main microphone.
  • the comparison of the signals captured makes it possible to extract the voice of the ambient noise by spatial coherence analysis of the two signals, with relatively simple software means.
  • the US 2008/0280653 A1 describes such a configuration, where one of the pickups (the one that mainly picks up the voice) is that of a wireless headset carried by the driver of the vehicle, while the other (the one that captures the noise) is that of the telephone device, placed remotely in the passenger compartment of the vehicle, for example hung on the dashboard.
  • this technique has the disadvantage of requiring two remote microphones, the efficiency being even higher than the two microphones are remote. Therefore, this technique is not applicable to a device in which the two microphones are close together, for example two microphones incorporated in the facade of a car radio, or two microphones that would be arranged on one of the shells an earphone.
  • beamforming consists of creating by software means a directivity that improves the signal / noise ratio of the network or "antenna" microphones.
  • the US 2007/0165879 A1 describes such a technique, applied to a pair of non-directional microphones placed back to back.
  • An adaptive filtering of the captured signals makes it possible to derive at the output a signal in which the voice component has been reinforced.
  • the general problem of the invention is, in such a context, to perform an effective noise reduction for delivering to the remote speaker a voice signal representative of the speech transmitted by the close speaker (driver of the vehicle or helmet carrier), by eliminating this signal noise components external noise present in the environment of this speaker nearby.
  • the problem of the invention is also, in such a situation, to be able to implement a set of microphones at a time in reduced numbers (preferably two microphones only) and relatively close (typically a gap of a few centimeters only).
  • Another important aspect of the problem is the need to reproduce a natural and intelligible speech signal, that is to say, undistorted and whose useful frequency spectrum is not amputated by denoising treatments.
  • the invention proposes audio equipment of the general type disclosed by the US 2008/0280653 A1 above, that is to say comprising: a set of two microphonic sensors able to collect the speech of the user of the equipment and to deliver respective noisy speech signals; means for sampling the speech signals delivered by the microphone sensors; and means for denoising a speech signal, receiving as input the samples of the speech signals delivered by the two microphonic sensors, and outputting a speech signal that is not representative of the speech transmitted by the user of the equipment .
  • the denoising means are non-frequency noise reduction means comprising an adaptive filter combiner signals delivered by the two microphonic sensors, operating by iterative search to cancel the noise picked up by one of the microphone sensors on the basis of a noise reference given by the signal delivered by the other microphonic sensor.
  • the adaptive filter is a fractional delay filter, able to model a delay less than the sampling period of the sampling means.
  • the equipment further comprises voice activity detection means capable of delivering a signal representative of the presence or absence of speech by the user of the equipment, and the adaptive filter also receives as input the signal of presence or absence of speech, so as to selectively: i) perform an adaptive search filter parameters in the absence of speech, ii) freeze these parameters of the filter in the presence of speech.
  • the adaptive filter is a LMS mean least squares linear prediction algorithm filter.
  • the equipment comprises a video camera directed towards the user of the equipment and able to capture an image thereof
  • the voice activity detection means comprise suitable video analysis means. analyzing the image produced by the camera and responding in response to said presence or absence of speech signal by said user.
  • the equipment comprises a physiological sensor adapted to come into contact with the head of the user of the equipment to be coupled thereto in order to capture non-acoustic vocal vibrations transmitted by internal bone conduction
  • the voice activity detection means comprise means able to analyze the signal delivered by the physiological sensor and to respond in response to said presence or absence of speech signal by said user, in particular by evaluating the energy of the signal delivered by the physiological sensor and comparison to a threshold.
  • the equipment may in particular be a headset of the combined microphone / headset type, comprising: headphones each comprising a sound reproduction transducer of an audio signal housed in a shell provided with a circumaural pad; said two microphone sensors, arranged on the shell of one of the earphones; and said physiological sensor, incorporated in the pad of one of the earphones and placed in a region thereof able to come into contact with the cheek or temple of the wearer of the helmet.
  • These two microphonic sensors are preferably aligned in a linear array in a main direction directed towards the mouth of the user of the equipment.
  • the Figure 1 schematically illustrates, in block form, the various functions implemented by the invention.
  • the process of the invention is implemented by software means, schematized by a number of functional blocks corresponding to appropriate algorithms executed by a microcontroller or a digital signal processor. Although, for the sake of clarity, the different functions are presented as separate modules, they implement common elements and correspond in practice. to a plurality of functions globally executed by the same software.
  • the signal that is desired to be denoised originates from a network of microphone sensors which, in the minimum configuration illustrated, may simply be an array of two sensors arranged in a predetermined configuration, each sensor consisting of a corresponding respective microphone 10 , 12.
  • the invention may, however, be generalized to a network of more than two microphone sensors, and / or to microphonic sensors, each sensor of which is constituted by a more complex structure than a simple microphone, for example a combination of several microphones and / or or other speech sensors.
  • the microphones 10, 12 are microphones that pick up the signal emitted by the useful signal source (the speech signal from the speaker), and the difference in position between the two microphones induces a set of phase shifts and amplitude variations in the microphone. recording the signals emitted by the useful signal source.
  • the useful signal source the speech signal from the speaker
  • the two microphones 10, 12 are omnidirectional microphones placed a few centimeters apart from each other on the ceiling of a car interior, on the front of a car radio or in an appropriate location on the car. dashboard, or on the shell of one of the headphones headphones, etc.
  • the technique of the invention makes it possible to ensure efficient denoising even for closely spaced microphones, that is to say spaced apart from each other by a distance d such that the maximum phase delay of a signal picked up by one microphone and then the other is less than the sampling period of the signal digitizing converter.
  • This corresponds to a maximum distance of about 4.7 cm for a sampling frequency F e of 8 kHz (and a lesser spacing of half to twice the frequency, etc.).
  • a speech signal emitted by a close speaker will reach one of the microphones before the other, and therefore have a delay, and therefore a phase shift ⁇ , substantially constant.
  • phase shift it can certainly exist also a phase shift between the two microphones 10 and 12.
  • the notion of phase shift being related to the notion of incident wave direction, we can expect this phase difference is different from that of the speech. For example, if a directional noise is directed in the opposite direction to that of the mouth, its phase shift will be - ⁇ if the phase shift for the voice is ⁇ .
  • the noise reduction on the signals picked up by the microphones 10 and 12 is not operated in the frequency domain (as is often the case with conventional denoising techniques) but in the time domain .
  • This noise reduction is effected by means of an algorithm seeking the transfer function between one of the microphones (for example the microphone 10) and the other microphone (the microphone 12) by means of an adaptive combiner 14 setting A predictive filter 16 of LMS ( Least Mean Squares ) type is used.
  • the output of the filter 16 is subtracted at 18 from the signal of the microphone 10 to give a signal S de-noised, applied back to the filter 16 to allow its iterative adaptation as a function of the prediction error. It is thus possible to predict from the signal picked up by the microphone 12 the noise component contained in the signal picked up by the microphone 10 (the transfer function identifying the noise transfer).
  • the adaptive search of the transfer function between the two microphones is performed only during the speech-free phases. For this, the iterative adaptation of the filter 16 is activated only when a voice activity detector VAD ( Voice Activity Detector ) 20 controlled by a sensor 22 indicates that the near speaker is not speaking.
  • VAD Voice Activity Detector
  • the adaptive combiner 14 seeks to optimize the transfer function between the two microphones 10 and 12 so as to reduce the noise component (closed position of the switch 24, as shown in the figure); on the other hand, in the presence of a speech signal confirmed by the voice activity detector 20, the adaptive combiner 14 freezes the parameters of the filter 16 to the value at which they were just before the speech was detected (opening of the switch 24), which avoids any degradation of the speech signal of the nearby speaker.
  • the filtering of the adaptive combiner 14 is a fractional delay filtering, that is to say that it makes it possible to apply a filtering between the signals picked up by the two microphones taking into account the a delay less than the duration of a sample digitizing signals.
  • the interval or offset between two samples corresponds temporally to a duration of Te second.
  • the Figure 3a gives a graphic representation of this function.
  • MicAvant (n) and MicRear (n) being the respective values of the signals from the microphonic sensors 10 and 12.
  • the estimation of ⁇ takes place directly, by the minimization of the error e ( n ) above, without there being need to estimate separately ⁇ and F.
  • L being the length of the filter.
  • the prediction of the filter H gives a fractional delay filter which, ideally and in the absence of speech, cancels the noise of the microphone 10 with reference to the microphone 12 (as indicated above, during speech however, the filter is frozen to prevent any degradation of local speech).
  • the Figure 4 illustrates an example of acoustic response between the two microphones, in the form of a characteristic giving the amplitude A as a function of the coefficients k of the filter F.
  • the different sound reflections that may occur depending on the environment, for example on the windows or other walls of a car interior, create visible peaks on this acoustic response characteristic.
  • the Figure 5 illustrates an example of the result of the convolution G X F of the two filters G (cardinal sinus response) and F (environment of use), in the form of a characteristic giving the amplitude A as a function of the coefficients k of the convoluted filter.
  • the estimate ⁇ can be calculated by an iterative LMS algorithm seeking to minimize the error y (n) - ⁇ X x ( n ) to converge towards the optimal filter.
  • the voice activity detector is here preferably a "perfect” detector, that is to say it delivers a binary signal (absence vs. presence of speech). It is thus distinguished from most voice activity detectors used in known denoising systems, which deliver only a probability of presence of variable speech between 0 and 100% continuously or in successive steps. With such detectors based only on the probability of speech, false detections can be important in noisy environments. To be “perfect”, the voice activity detector can not rely solely on the signal picked up by the microphones; it must have additional information to discriminate the speech and silence phases of the next speaker.
  • a first example of such a detector is illustrated by the Figure 6 , where the voice activity detector 20 operates in response to the signal produced by a camera.
  • This camera is for example a camera 26 installed in the passenger compartment of a motor vehicle, and oriented so that its field of view 28 encompasses in all circumstances the driver's head 30, considered as the close speaker.
  • the signal delivered by the camera 26 is analyzed to determine from the movement of the mouth and lips whether the speaker speaks or not.
  • Video data is added to conventional audio data to improve speech enhancement .
  • This treatment may be used in the context of the present invention to distinguish between the speech phases and the silence phases of the speaker.
  • the advantage of this image analysis technique is to have complementary information completely independent of the acoustic noise environment.
  • a sensor that can be used for the detection of "perfect" voice activity is a physiological sensor capable of detecting certain vocal vibrations of the speaker that are not or only slightly corrupted by the surrounding noise.
  • Such a sensor may consist in particular of an accelerometer or a piezoelectric sensor applied against the cheek or the temple of the speaker.
  • a vibration propagates from the vocal cords to the pharynx and to the bucco-nasal cavity, where it is modulated, amplified and articulated.
  • the mouth, the soft palate, the pharynx, the sinuses and the nasal fossae then serve as a sounding board for this voiced sound and, their wall being elastic, they vibrate in turn and these vibrations are transmitted by internal bone conduction and are perceptible at the cheek and temple.
  • a physiological sensor that collects these noise-free vocal vibrations gives a signal representative of the presence or absence of voiced sounds emitted by the speaker, thus making it possible to discriminate very clearly the speech phases and the speaker's silence phases.
  • Such a physiological sensor may in particular be incorporated into a combined microphone / headset assembly as shown in FIG. Figure 7 .
  • the reference 32 generally designates the helmet according to the invention, which comprises two atria 34 joined by a hoop.
  • Each of the atria is preferably constituted by a closed shell 36, housing a sound reproduction transducer, applied around the ear of the user with the interposition of a pad 38 isolating the ear from the outside.
  • the physiological sensor 40 used for the detection of voice activity is for example an accelerometer integrated in the pad 38 so as to be applied against the cheek or the temple of the user with the closest possible coupling.
  • This physiological sensor 40 may in particular be placed on the inner face of the skin of the pad 38 so that, once the helmet is in place, the sensor is applied against the cheek or the temple of the user under the effect of a slight pressure resulting from the crash cushion material, with only interposition of the outer skin of this pad.
  • the headset also carries the microphones 10, 12 of the circuit for collecting and denoising the speech of the speaker.
  • These two microphones are omnidirectional microphones placed on the shell 36, and they are arranged with the microphone 10 placed in front (closer to the mouth of the wearer of the helmet) and the microphone 12 placed further back.
  • the alignment direction 42 of the two microphones 10, 12 is approximately directed towards the mouth 44 of the helmet wearer.
  • the Figure 8 is a block diagram showing the various functions implemented by the microphone / headset combo of the Figure 7 .
  • This figure shows the two microphones 10 and 12, as well as the voice activity detector 20.
  • the front microphone 10 is the main microphone and the rear microphone 12 serves as input to the adaptive filter 16 of the combiner 14.
  • the signal delivered by the physiological sensor 40 may be used not only as an input signal of a voice activity detector, but also to enrich the signal picked up by the microphones 10 and 12, especially in the low end of the spectrum.
  • the signals delivered by the physiological sensor which correspond to the voiced sounds, are not, strictly speaking, speech since the speech is not only formed of voiced sounds, it contains components that are not born at the level of the voices.
  • vocal cords the frequency content is for example much richer with the sound coming from the throat and emitted by the mouth.
  • the internal bone conduction and the crossing of the skin has the effect of filtering certain vocal components.
  • the signal collected by the physiological sensor is usable only in the low frequencies, mainly in the lower region of the sound spectrum (typically 0-1500 Hz).
  • the signal of a physiological sensor has the considerable advantage of being naturally devoid of parasitic component of noise it will be possible to use this signal in the low end of the spectrum, completing it in the high range (above 1500 Hz) by the (noisy) signals collected by the microphones 10 and 12, after having submitted these signals to a noise reduction operated by the adaptive combiner 14.
  • the complete spectrum is reconstructed by means of the mixing block 46 which receives in parallel: the signal of the physiological sensor 40 for the low end of the spectrum, and the signal of the microphones 10 and 12 after denoising by the adaptive combiner 14 for the top of the spectrum.
  • This reconstruction is performed by summing the signals, which are applied in synchronism with the mixing block 46 so as to avoid any deformation.
  • the resulting signal delivered by the block 46 can be subjected to a final noise reduction by the circuit 48, operated in the frequency domain according to a conventional technique comparable to that described for example in the WO 2007/099222 A1 (Parrot ), to output the final denoised signal S.
  • the system that has just been described makes it possible to obtain excellent overall performance, typically of the order of 30 to 40 dB of noise reduction on the speech signal of the nearby speaker.
  • the adaptive combiner 14 operating on the signals picked up by the microphones 10 and 12 makes it possible in particular, with the fractional-delay filtering, to obtain very good denoising performance in the high frequencies.
EP12170407.6A 2011-06-01 2012-06-01 Audiogerät mit Rauschunterdrückung in einem Sprachsignal unter Verwendung von einem Filter mit fraktionaler Verzögerung Active EP2530673B1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
FR1154825A FR2976111B1 (fr) 2011-06-01 2011-06-01 Equipement audio comprenant des moyens de debruitage d'un signal de parole par filtrage a delai fractionnaire, notamment pour un systeme de telephonie "mains libres"

Publications (2)

Publication Number Publication Date
EP2530673A1 true EP2530673A1 (de) 2012-12-05
EP2530673B1 EP2530673B1 (de) 2013-07-10

Family

ID=44533268

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12170407.6A Active EP2530673B1 (de) 2011-06-01 2012-06-01 Audiogerät mit Rauschunterdrückung in einem Sprachsignal unter Verwendung von einem Filter mit fraktionaler Verzögerung

Country Status (6)

Country Link
US (1) US8682658B2 (de)
EP (1) EP2530673B1 (de)
JP (1) JP6150988B2 (de)
CN (1) CN103002170B (de)
ES (1) ES2430121T3 (de)
FR (1) FR2976111B1 (de)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110447073B (zh) * 2017-03-20 2023-11-03 伯斯有限公司 用于降噪的音频信号处理

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2974655B1 (fr) * 2011-04-26 2013-12-20 Parrot Combine audio micro/casque comprenant des moyens de debruitage d'un signal de parole proche, notamment pour un systeme de telephonie "mains libres".
US20140025374A1 (en) * 2012-07-22 2014-01-23 Xia Lou Speech enhancement to improve speech intelligibility and automatic speech recognition
US9135915B1 (en) * 2012-07-26 2015-09-15 Google Inc. Augmenting speech segmentation and recognition using head-mounted vibration and/or motion sensors
US9685171B1 (en) * 2012-11-20 2017-06-20 Amazon Technologies, Inc. Multiple-stage adaptive filtering of audio signals
CN103871419B (zh) * 2012-12-11 2017-05-24 联想(北京)有限公司 一种信息处理方法及电子设备
FR3002679B1 (fr) * 2013-02-28 2016-07-22 Parrot Procede de debruitage d'un signal audio par un algorithme a gain spectral variable a durete modulable dynamiquement
US9185199B2 (en) 2013-03-12 2015-11-10 Google Technology Holdings LLC Method and apparatus for acoustically characterizing an environment in which an electronic device resides
US20150199950A1 (en) * 2014-01-13 2015-07-16 DSP Group Use of microphones with vsensors for wearable devices
FR3021180B1 (fr) * 2014-05-16 2016-06-03 Parrot Casque audio a controle actif de bruit anc avec prevention des effets d'une saturation du signal microphonique "feedback"
CA2953619A1 (en) 2014-06-05 2015-12-10 Interdev Technologies Inc. Systems and methods of interpreting speech data
US10163453B2 (en) 2014-10-24 2018-12-25 Staton Techiya, Llc Robust voice activity detector system for use with an earphone
CN106157963B (zh) * 2015-04-08 2019-10-15 质音通讯科技（深圳）有限公司 一种音频信号的降噪处理方法和装置及电子设备
JP6501259B2 (ja) * 2015-08-04 2019-04-17 本田技研工業株式会社 音声処理装置及び音声処理方法
EP3147896B1 (de) * 2015-09-25 2023-05-31 Harman Becker Automotive Systems GmbH Aktives strassengeräuschunterdrückungssystem mit übersteuerungserkennung des primären messsignals
JP6732944B2 (ja) * 2016-12-16 2020-07-29 日本電信電話株式会社 目的音強調装置、雑音推定用パラメータ学習装置、目的音強調方法、雑音推定用パラメータ学習方法、プログラム
WO2018119467A1 (en) * 2016-12-23 2018-06-28 Synaptics Incorporated Multiple input multiple output (mimo) audio signal processing for speech de-reverberation
US10366708B2 (en) * 2017-03-20 2019-07-30 Bose Corporation Systems and methods of detecting speech activity of headphone user
JP6821126B2 (ja) * 2017-05-19 2021-01-27 株式会社Ｊｖｃケンウッド ノイズ除去装置、ノイズ除去方法およびノイズ除去プログラム
CN108810692A (zh) * 2018-05-25 2018-11-13 会听声学科技（北京）有限公司 主动降噪系统、主动降噪方法及耳机
US10455319B1 (en) * 2018-07-18 2019-10-22 Motorola Mobility Llc Reducing noise in audio signals
JP2020144204A (ja) * 2019-03-06 2020-09-10 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 信号処理装置及び信号処理方法
CN110049395B (zh) * 2019-04-25 2020-06-05 维沃移动通信有限公司 耳机控制方法及耳机设备
US11871190B2 (en) 2019-07-03 2024-01-09 The Board Of Trustees Of The University Of Illinois Separating space-time signals with moving and asynchronous arrays
US11227587B2 (en) * 2019-12-23 2022-01-18 Peiker Acustic Gmbh Method, apparatus, and computer-readable storage medium for adaptive null-voice cancellation
CN112822592B (zh) * 2020-12-31 2022-07-12 青岛理工大学 一种可定向聆听的有源降噪耳机及控制方法
TWI777729B (zh) * 2021-08-17 2022-09-11 達發科技股份有限公司 適應性主動雜訊消除裝置以及使用其之聲音播放系統
TWI790718B (zh) * 2021-08-19 2023-01-21 宏碁股份有限公司 會議終端及用於會議的回音消除方法
CN115132220B (zh) * 2022-08-25 2023-02-28 深圳市友杰智新科技有限公司 抑制电视噪声的双麦唤醒的方法、装置、设备及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070165879A1 (en) 2006-01-13 2007-07-19 Vimicro Corporation Dual Microphone System and Method for Enhancing Voice Quality
WO2007099222A1 (fr) 2006-03-01 2007-09-07 Parrot Procede de debruitage d'un signal audio
US20080280653A1 (en) 2007-05-09 2008-11-13 Motorola, Inc. Noise reduction on wireless headset input via dual channel calibration within mobile phone

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4672665A (en) * 1984-07-27 1987-06-09 Matsushita Electric Industrial Co. Ltd. Echo canceller
US5574824A (en) * 1994-04-11 1996-11-12 The United States Of America As Represented By The Secretary Of The Air Force Analysis/synthesis-based microphone array speech enhancer with variable signal distortion
US5694474A (en) * 1995-09-18 1997-12-02 Interval Research Corporation Adaptive filter for signal processing and method therefor
US5761318A (en) * 1995-09-26 1998-06-02 Nippon Telegraph And Telephone Corporation Method and apparatus for multi-channel acoustic echo cancellation
US5774562A (en) * 1996-03-25 1998-06-30 Nippon Telegraph And Telephone Corp. Method and apparatus for dereverberation
FI114422B (fi) * 1997-09-04 2004-10-15 Nokia Corp Lähteen puheaktiviteetin tunnistus
US7072831B1 (en) * 1998-06-30 2006-07-04 Lucent Technologies Inc. Estimating the noise components of a signal
US6453285B1 (en) * 1998-08-21 2002-09-17 Polycom, Inc. Speech activity detector for use in noise reduction system, and methods therefor
US6289309B1 (en) * 1998-12-16 2001-09-11 Sarnoff Corporation Noise spectrum tracking for speech enhancement
US7062049B1 (en) * 1999-03-09 2006-06-13 Honda Giken Kogyo Kabushiki Kaisha Active noise control system
JP2000312395A (ja) * 1999-04-28 2000-11-07 Alpine Electronics Inc マイクロホンシステム
US7117145B1 (en) * 2000-10-19 2006-10-03 Lear Corporation Adaptive filter for speech enhancement in a noisy environment
US7206418B2 (en) * 2001-02-12 2007-04-17 Fortemedia, Inc. Noise suppression for a wireless communication device
US7617099B2 (en) * 2001-02-12 2009-11-10 FortMedia Inc. Noise suppression by two-channel tandem spectrum modification for speech signal in an automobile
DE10118653C2 (de) * 2001-04-14 2003-03-27 Daimler Chrysler Ag Verfahren zur Geräuschreduktion
JP3568922B2 (ja) * 2001-09-20 2004-09-22 三菱電機株式会社 エコー処理装置
US6937980B2 (en) * 2001-10-02 2005-08-30 Telefonaktiebolaget Lm Ericsson (Publ) Speech recognition using microphone antenna array
US7167568B2 (en) * 2002-05-02 2007-01-23 Microsoft Corporation Microphone array signal enhancement
KR100602975B1 (ko) * 2002-07-19 2006-07-20 닛본 덴끼 가부시끼가이샤 오디오 복호 장치와 복호 방법 및 프로그램을 기록한 컴퓨터 판독가능 기록매체
US7949522B2 (en) * 2003-02-21 2011-05-24 Qnx Software Systems Co. System for suppressing rain noise
US8073689B2 (en) * 2003-02-21 2011-12-06 Qnx Software Systems Co. Repetitive transient noise removal
CA2473195C (en) * 2003-07-29 2014-02-04 Microsoft Corporation Head mounted multi-sensory audio input system
JP4496379B2 (ja) * 2003-09-17 2010-07-07 財団法人北九州産業学術推進機構 分割スペクトル系列の振幅頻度分布の形状に基づく目的音声の復元方法
JP2005249816A (ja) * 2004-03-01 2005-09-15 Internatl Business Mach Corp <Ibm> 信号強調装置、方法及びプログラム、並びに音声認識装置、方法及びプログラム
JP2006039267A (ja) * 2004-07-28 2006-02-09 Nissan Motor Co Ltd 音声入力装置
US7533017B2 (en) * 2004-08-31 2009-05-12 Kitakyushu Foundation For The Advancement Of Industry, Science And Technology Method for recovering target speech based on speech segment detection under a stationary noise
US7844059B2 (en) * 2005-03-16 2010-11-30 Microsoft Corporation Dereverberation of multi-channel audio streams
FR2908003B1 (fr) * 2006-10-26 2009-04-03 Parrot Sa Procede de reduction de l'echo acoustique residuel apres supression d'echo dans un dispositif"mains libres"
US8175291B2 (en) * 2007-12-19 2012-05-08 Qualcomm Incorporated Systems, methods, and apparatus for multi-microphone based speech enhancement
US8577677B2 (en) * 2008-07-21 2013-11-05 Samsung Electronics Co., Ltd. Sound source separation method and system using beamforming technique

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070165879A1 (en) 2006-01-13 2007-07-19 Vimicro Corporation Dual Microphone System and Method for Enhancing Voice Quality
WO2007099222A1 (fr) 2006-03-01 2007-09-07 Parrot Procede de debruitage d'un signal audio
US20080280653A1 (en) 2007-05-09 2008-11-13 Motorola, Inc. Noise reduction on wireless headset input via dual channel calibration within mobile phone

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
ALAN V: "Signal Processing Series", 1985
B. WIDROW ET AL.: "Adaptative Noise Cancelling: Principles and Applications", PROC. IEEE, vol. 63, no. 12, December 1975 (1975-12-01), pages 1692 - 1716
B. WIDROW: "Adaptative Filters, Aspect of Network and System Theory", 1970, HOLT, RINEHART AND WINSTO, pages: 563 - 587
B. WIDROW; S. STEARNS: "Adaptative Signal Processing", PRENTICE-HALL
DJENDI M ET AL: "Noise Cancellation using Two Closely Spaced Microphones: Experimental Study witha Specific Model and Two Adaptive Algorithms", ACOUSTICS, SPEECH AND SIGNAL PROCESSING, 2006. ICASSP 2006 PROCEEDINGS . 2006 IEEE INTERNATIONAL CONFERENCE ON TOULOUSE, FRANCE 14-19 MAY 2006, PISCATAWAY, NJ, USA,IEEE, PISCATAWAY, NJ, USA, 14 May 2006 (2006-05-14), pages III, XP031386771, ISBN: 978-1-4244-0469-8 *
G. POTAMIANOS ET AL.: "Audio-Visual Automatic Speech Recognition: An Overview, Audio-Visual Speech Processing", 2004, MIT PRESS, pages: 1 - 30

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110447073B (zh) * 2017-03-20 2023-11-03 伯斯有限公司 用于降噪的音频信号处理

Also Published As

Publication number Publication date
ES2430121T3 (es) 2013-11-19
CN103002170B (zh) 2016-01-06
US8682658B2 (en) 2014-03-25
US20120310637A1 (en) 2012-12-06
JP6150988B2 (ja) 2017-06-21
EP2530673B1 (de) 2013-07-10
CN103002170A (zh) 2013-03-27
FR2976111B1 (fr) 2013-07-05
JP2012253771A (ja) 2012-12-20
FR2976111A1 (fr) 2012-12-07

Similar Documents

Publication Publication Date Title
EP2530673B1 (de) Audiogerät mit Rauschunterdrückung in einem Sprachsignal unter Verwendung von einem Filter mit fraktionaler Verzögerung
EP2518724B1 (de) Kombinierte Audioeinheit bestehend aus Mikrofon und Kopfhörer, die Mittel zur Geräuschdämpfung eines nahen Wortsignals umfasst, insbesondere für eine telefonische Freisprechanlage
EP2680262B1 (de) Verfahren zur Geräuschdämpfung eines Audiosignals für eine Multimikrofon-Audiovorrichtung, die in lauten Umgebungen eingesetzt wird
EP2057835B1 (de) Verfahren zur unterdrückung akustischer restechos nach echounterdrückung bei einer freisprecheinrichtung
EP2293594B1 (de) Verfahren zur Filterung von seitlichem nichtstationärem Rauschen für ein Multimikrofon-Audiogerät
EP2309499B1 (de) Verfahren zur optimierten Filterung nicht stationärer Geräusche, die von einem Audiogerät mit mehreren Mikrophonen eingefangen werden, insbesondere eine Freisprechtelefonanlage für Kraftfahrzeuge
EP2538409B1 (de) Verfahren zur Geräuschdämpfung für Audio-Gerät mit mehreren Mikrofonen, insbesondere für eine telefonische Freisprechanlage
EP2772916B1 (de) Verfahren zur Geräuschdämpfung eines Audiosignals mit Hilfe eines Algorithmus mit variabler Spektralverstärkung mit dynamisch modulierbarer Härte
CH702399A2 (fr) Appareil et procédé pour la saisie et le traitement de la voix.
EP0752181A1 (de) Echokompensation mit adaptivem filter im frequenzbereich
EP0998166A1 (de) Anordnung zur Verarbeitung von Audiosignalen, Empfänger und Verfahren zum Filtern und Wiedergabe eines Nutzsignals in Gegenwart von Umgebungsgeräusche
EP0884926B1 (de) Verfahren und Vorrichtung zur optimierten Verarbeitung eines Störsignals während einer Tonaufnahme
FR2906070A1 (fr) Reduction de bruit multi-reference pour des applications vocales en environnement automobile
FR3009121A1 (fr) Procede de suppression de la reverberation tardive d&#39;un signal sonore
WO2017207286A1 (fr) Combine audio micro/casque comprenant des moyens de detection d&#39;activite vocale multiples a classifieur supervise
EP0989544A1 (de) Vorrichtung und Verfahren zur Filterung eines Sprachsignals, Empfänger und Fernsprechsystem
WO2008037925A1 (fr) Reduction de bruit et de distorsion dans une structure de type forward
WO2022207994A1 (fr) Estimation d&#39;un masque optimise pour le traitement de donnees sonores acquises
EP4229636A1 (de) Verfahren und vorrichtung zur echounterdrückung mit variabler grundfrequenz
FR2921747A1 (fr) Procede de traitement d&#39;un signal audio dans un dispositif portable
FR2878399A1 (fr) Dispositif et procede de debruitage a deux voies mettant en oeuvre une fonction de coherence associee a une utilisation de proprietes psychoacoustiques, et programme d&#39;ordinateur correspondant

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20120601

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602012000117

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G10L0021020000

Ipc: G10L0021020800

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 21/0208 20130101AFI20130314BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20130422

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 621363

Country of ref document: AT

Kind code of ref document: T

Effective date: 20130715

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012000117

Country of ref document: DE

Effective date: 20130905

REG Reference to a national code

Ref country code: NL

Ref legal event code: T3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2430121

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20131119

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 621363

Country of ref document: AT

Kind code of ref document: T

Effective date: 20130710

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131111

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130918

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131110

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131010

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131011

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20140411

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012000117

Country of ref document: DE

Effective date: 20140411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140601

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140601

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 4

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20150724

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 602012000117

Country of ref document: DE

Owner name: PARROT AUTOMOTIVE, FR

Free format text: FORMER OWNER: PARROT, PARIS, FR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140602

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20151029 AND 20151104

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

Owner name: PARROT AUTOMOTIVE, FR

Effective date: 20151201

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: NL

Ref legal event code: PD

Owner name: PARROT AUTOMOTIVE; FR

Free format text: DETAILS ASSIGNMENT: VERANDERING VAN EIGENAAR(S), OVERDRACHT; FORMER OWNER NAME: PARROT

Effective date: 20151102

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150630

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150630

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120601

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140630

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130710

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20190619

Year of fee payment: 8

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20200701

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200701

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20230523

Year of fee payment: 12

Ref country code: FR

Payment date: 20230523

Year of fee payment: 12

Ref country code: DE

Payment date: 20230523

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230523

Year of fee payment: 12