EP1830348B1 - Système mains libres utilisé dans les véhicules - Google Patents

Système mains libres utilisé dans les véhicules Download PDF

Info

Publication number
EP1830348B1
EP1830348B1 EP06004158.9A EP06004158A EP1830348B1 EP 1830348 B1 EP1830348 B1 EP 1830348B1 EP 06004158 A EP06004158 A EP 06004158A EP 1830348 B1 EP1830348 B1 EP 1830348B1
Authority
EP
European Patent Office
Prior art keywords
signal
microphone
hands
noise reduction
reduction device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP06004158.9A
Other languages
German (de)
English (en)
Other versions
EP1830348A1 (fr
Inventor
Markus Buck
Tim Haulick
Martin Rössler
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nuance Communications Inc
Original Assignee
Nuance Communications Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nuance Communications Inc filed Critical Nuance Communications Inc
Priority to EP06004158.9A priority Critical patent/EP1830348B1/fr
Priority to JP2007026914A priority patent/JP2007235943A/ja
Publication of EP1830348A1 publication Critical patent/EP1830348A1/fr
Application granted granted Critical
Publication of EP1830348B1 publication Critical patent/EP1830348B1/fr
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
    • H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
    • H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
    • H04R1/406—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers microphones
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02165—Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02166—Microphone arrays; Beamforming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
    • H04R2201/10—Details of earpieces, attachments therefor, earphones or monophonic headphones covered by H04R1/10 but not provided for in any of its subgroups
    • H04R2201/107—Monophonic and stereophonic headphones with microphone for two-way hands free communication
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00—Signal processing covered by H04R, not provided for in its groups
    • H04R2430/20—Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10—General applications
    • H04R2499/13—Acoustic transducers and sound field adaptation in vehicles

Definitions

  • the invention is directed to a hands-free system for speech signal acquisition in a vehicular cabin which shows an improved signal quality.
  • Hands-free systems are used in vehicular cabins, particularly of cars, for different purposes.
  • a hands-free system can be used to enable hands-free telephony.
  • control of devices such as a navigation system or an air conditioning system might rely on a hands-free system.
  • the acquired signals are processed using a speech recognition device to determine control signals as input by a user.
  • At least one microphone is provided to pick up speech signals from a user.
  • These microphones are usually fixedly mounted inside the vehicular cabin. The place for the microphones is chosen based on different criteria. On the one hand, the arrangement and orientation of the microphones with respect to an expected speaker has to be optimized. On the other hand, usually, there are constructional and/or aesthetic or design constraints restricting the number of possible positions.
  • a hands-free system offers a high comfort to a user because no headset or handset is needed.
  • the quality of the signal in particular, the signal-to-noise ratio, acquired by a microphone is rather poor because the acoustic level of the speech signal decreases with the distance from the speaker's mouth to the microphone.
  • One possibility to obtain noise suppression in a signal is to use a microphone array containing several microphones in combination with spatial filtering.
  • usually omni-directional microphones are used in the microphone arrays.
  • a desired directivity of the array can be achieved with the consequence of a noise suppression.
  • the directivity of this system is controlled such that its sensitivity is highest in the direction to an expected speaker. Noise signals coming from other directions, for example, from other speaking persons or from loudspeakers, are suppressed.
  • Possible realizations of such microphone array systems are described in M. Brandstein, D. Ward: Microphone arrays: signal processing techniques and applications, Springer Verlag, Berlin (Germany), 2001 .
  • the microphones of the array are used to both pick up the speech signal and the noise signal.
  • the separation of the noise signal from the speech signal is achieved by processing the entire signals acquired by the microphone array system.
  • the processing of the signals is quite complex, and the resulting signal still contains noise parts.
  • US 5,754,665 discloses a noise canceler comprising two microphones, one being installed at the drivers seat of an automobile and the other being installed at the passenger's seat.
  • An adder outputs a noise-reduced output signal by subtracting an estimated noise signal from the voice signal from the microphone.
  • US 6,738,482 discloses a noise suppression system with a dual microphone echo cancellation.
  • An ear set includes a pair of microphones closely monitored thereon.
  • a hands-free system for speech signal acquisition in a vehicular cabin comprising:
  • Such a hands-free system allows to acquire a signal comprising a speech part and a noise part with the at least one first directional microphone, on the one hand, and to acquire a noise signal with the at least one second directional microphone, on the other hand, as the at least one second directional microphone points in a different direction than the at least one first microphone. Therefore, a signal picked up by the at least one second microphone containing (almost) only noise can be used to remove a noise part from the signal picked up by the at least one first microphone.
  • speech is used to denote any type of wanted signal whereas “noise” denotes an unwanted signal.
  • noise can also comprise signals comprising speech elements, for example, emanating from a loudspeaker or another speaker present in the vehicular cabin; however, these signals are not wanted for a specific purpose.
  • the angle between the pointing directions of the first and the second directional microphones can lie between 20° and 180°, preferably between 90° and 180°, most preferred between 130° and 180°.
  • the noise reduction device can be configured to subtract a signal based on the signal from the at least one second directional microphone from the signal from the at least one first directional microphone.
  • the subtrahend i.e. the signal based on the signal from the at least one second microphone, can be based on or can be equal to the sum or the average of the signals acquired by all second directional microphones.
  • the noise reduction device can be further configured to cancel a noise part in a signal from the at least one second directional microphone using a signal from the at least one first directional microphone.
  • the hands-free system is enabled to acquire enhanced speech signals or wanted signals from two different directions.
  • the at least one first and second microphones can be arranged so that the at least one first microphone points in the direction of driver and the at least one second microphone in the direction of a front seat passenger. Then, if the driver is speaking, the first microphones are responsible for picking up the speech signals whereas the second microphones acquire the noise reference signals. If the front seat passenger is speaking, the roles of the first and second microphones are swept.
  • the noise reduction device is configured symmetrically to cancel a noise part in signals acquired either by the first or the second microphones based on noise reference signals acquired by the other at least one microphone (i.e., the second or the first microphones, respectively).
  • the hands-free system can further comprise a control means for controlling the signal processing of the noise reduction device according to a predetermined criterion in such a way that a noise part in a signal from the at least one first microphone and/or a noise part in a signal from the at least one second microphone is cancelled.
  • Such a control means allows to use the noise reduction system together with the first and second microphones in an advantageous way to acquire a speech signal from one of the two pointing directions of the microphone and suppressing noise parts therein using the microphones pointing in the other direction.
  • this control means For example, always if a speech signal is detected in the signals from the first microphones, the noise reduction device is controlled such that a noise part in the signal from the at least one first microphone is cancelled using the signal from the second microphones as noise reference signal. In this way, a speaker to which the at least one first microphone is pointing is prioritized.
  • control means can be configured such that a noise part in a signal from those microphones is cancelled for which a speech signal is detected first. In this case, each time one speaker stops talking, both the first and the second microphones are ready to acquire a speech signal.
  • the at least one first directional and/or the at least one second directional microphone can be provided as a microphone array with at least two directional microphones, and the noise reduction system can comprise a beamformer for processing the signals from each microphone array.
  • Using a microphone array and a corresponding beamformer, particularly for picking up the wanted signal further increases the signal-to-noise ratio in the signal containing the wanted signal. This leads to a more enhanced resulting signal after cancellation of the noise part. If the at least one second microphone is also used to pick up wanted signals, such an advantageous enhancement of the resulting signal is obtained when using a microphone array and a corresponding beamformer in the case of the second microphones as well.
  • both the first and the second microphones are provided as a microphone array comprising at least two directional microphones
  • a beamformer can be provided for processing the signals from both microphone arrays.
  • a beamformer can be provided for each microphone array.
  • the number of first microphones can equal the number of second microphones.
  • Such a symmetrical case makes it possible to use both the first and the second microphones for picking up a wanted speech signal with similar signal quality.
  • the at least one first and at least one second microphones can be provided pairwise such that one first microphone and one second microphone are arranged on a common support frame, respectively.
  • a dual microphone comprising a first and a second directional microphone, each pointing in a different direction.
  • Dual microphones of this kind can be provided in a very compact form, which has the advantage that not much space is required when mounting the microphones or microphone arrays, in particular, in a vehicular cabin.
  • Each first and second microphone can further be provided in a common housing.
  • each first microphone can have a first output
  • each second microphone can have a second output
  • the noise reduction device can comprise a first input for each first microphone output, a second input for each second microphone output, a cancellation means, a first signal path connecting the at least one first input and the cancellation means, a second signal path connecting the at least one second input and the cancellation means, wherein the cancellation means can be configured to cancel the noise part from the signal received via the first signal path using the signal received via the second signal path
  • each first microphone output is connected to a first input of the noise reduction device and each second microphone output is connected to a second input of the noise reduction device.
  • the cancellation means can be configured to subtract the signal based on the signal received by the second signal path from the signal received by the first signal path.
  • an adaptive filter in particular, based on an LMS, NLMS or RLS algorithm, can be provided on the second signal path between the cancellation means and each second input.
  • adaptive filters result in a further enhancement of the speech signal.
  • the adaptation of the adaptive filter can be based on the output signal of the cancellation means.
  • Each adaptive filter can be configured to be optimized adaptively to estimate the residual noise in the signal received by the cancellation means via the first signal path; thus, the adaptive filters are adapted to maximally cancel out such a residual noise.
  • the at least one first directional microphone can be provided as a microphone array with at least two directional microphones, and a beamformer can be provided on the first signal path, wherein for each first input of the noise reduction device, the first signal path comprises a sub-path connecting the first input and the beamformer.
  • Such an arrangement with a microphone array and a beamformer results in a further enhancement of the signal acquired by the first microphones, particularly with respect to its signal-to-noise ratio.
  • the beamformer can be a fixed beamformer.
  • the beamformer can be a delay-and-sum beamformer in which the signals coming from the different microphones are delayed in such a way that signals entering the microphones from a preferred direction can be added in phase. Then, a summation of the delayed microphone signals is performed.
  • a filter-and-sum beamformer (“superdirective beamformer”) can be used.
  • an adaptive beamformer instead of a fixed beamformer, it is also possible to use an adaptive beamformer.
  • the above hands-free systems can further comprise a third signal path connecting each first input and the cancellation means, wherein a blocking matrix and a subsequent adaptive filter in the direction of signal flow is provided on the third signal path, wherein for each first input of the noise reduction device, the third signal path comprises a sub-path connecting the first input and the blocking matrix and a sub-path connecting the blocking matrix and the adaptive filter.
  • GSC generalized side-lobe canceller
  • the blocking matrix serves to block out parts of the signal so that a noise reference signal is obtained.
  • the blocking matrix may be realized in different ways.
  • the blocking matrix may have a form so that a pairwise subtraction of neighboring input channels is performed. Another possibility is described in the article by L. Griffiths, Ch. Jim: "An alternative approach to linearly constrained adaptive beamforming", IEEE Trans. On Antennas and Propagation, Vol. 30, No. 1, pp. 27 - 34, January 1982 .
  • the previously described hands-free systems may further comprise subtraction means on the second signal path between each second input of the noise reduction device and the cancellation means and a fourth signal path comprising sub-paths connecting the first signal path and each subtraction means, wherein an adaptive filter is provided on each sub-path connecting the first signal path and each subtraction means.
  • the adaptation of the adaptive filters on the sub-paths can be based on the output of the corresponding cancellation means on the second signal path. With these additional adaptive filters, it is possible to compensate for wanted signal portions that leak into the noise reference signals.
  • the above-described hands-free systems can further comprise an adaptation controller with a speech detector for controlling adaptation of an adaptive filter in accordance with the detection of speech.
  • the hands-free systems can comprise an adaptation control for adaptation of all adaptive filters in accordance with the detection of speech.
  • the adaptation controller can be configured to initiate adaptation of an adaptive filter on the second signal path if no speech is detected on the first signal path.
  • the adaptation controller can be configured to initiate adaptation of an adaptive filter on a signal path connecting the first signal path and a subtraction means on the second signal path, if wanted speech is detected in the signal on the first signal path. Such an adaptation control further prevents wanted signal cancellation.
  • each first microphone output can be connected to a second input of the noise reduction device and each second microphone output can be connected to a first input of the noise reduction device
  • the hands-free system can comprise a control means for controlling the signal processing of the noise reduction device according to a predetermined criterion in such a way that a noise part in a signal from the at least one first microphone and/or a noise part in a signal from the at least one second microphone is cancelled by the cancellation means.
  • Such a configuration leads to a symmetrical arrangement. It has the advantage that canceling a noise part in a signal received from the at least one second directional microphone using a signal received from the at least one first directional microphone is enabled.
  • the cancellation means can be further configured to subtract a signal based on the signal received by the first signal path from a signal received by the second signal path.
  • a signal path with an adaptive filter connecting the first input and the cancellation means and/or a signal path with a beamformer connecting the second input and the cancellation means can be provided. These signal paths can be identical to the first and second signal paths mentioned above.
  • the adaptive filter and the beamformer can be configured as those on the first and the second signal path.
  • the control means is configured to control the signal flow in the noise reduction device accordingly.
  • the beamformer and the cancellation means in the noise reduction device are used in the appropriate way depending on which microphones (the first or the second microphones) are used to pick up a desired signal.
  • the noise reduction device of the above-described hands-free systems can be a first noise reduction device to provide a first enhanced output signal
  • the hands-free system can further comprise a second noise reduction device configured to cancel a noise part in a signal from the at least one second directional microphone using a signal from the at least first second directional microphone to provide a second enhanced output signal.
  • the hands-free system can further comprise a signal mixer for combining the first and the second enhanced output signal to provide a combined enhanced output signal.
  • the signal mixer can be configured such that the mixing weights are controlled in a time-varying way depending on a speech activity detected for the at least one first and/or the at least one second directional microphone.
  • the signal mixer can be configured such that one of the mixing weights is set to zero according to a predetermined criterion. As an example, if speech activity is detected for one of the first or the second microphones, the mixing weight for the other of the first or second microphones is set to zero. In this way, the speaker who starts first is prioritized.
  • the second noise reduction device can be configured to process a signal from the at least one first directional microphone and a signal from the at least one second directional microphone at least partly in the same way as the first noise reduction device processes a signal from the at least one second directional microphone and a signal from the at least one first directional microphone, respectively.
  • the second noise reduction device can at least partly comprise the same components as the first noise reduction device as described above.
  • the second noise reduction device can comprise a first input for each second microphone output, a second input for each first microphone output, a cancellation means, a first signal path connecting the at least one first input and the cancellation means, a second signal path connecting the at least one second input and the cancellation means, wherein the cancellation means can be configured to cancel the noise part from the signal received via the first signal path using the signal received via the second signal path, wherein each first microphone output is connected to a second input of the noise reduction device and each second microphone output is connected to a first input of the noise reduction device.
  • the second noise reduction device can comprise a beamformer and/or a cancellation means, similar to the case of the first noise reduction device.
  • the above-described hands-free systems can further comprise an acoustic echo canceller (AEC).
  • AEC acoustic echo canceller
  • Such an acoustic echo canceller can use a loudspeaker excitation signal as a basis to cancel a further noise part in a signal received from the at least one first directional microphone. In this way, an acoustic feedback from loudspeakers into the microphones will also be compensated for.
  • the acoustic echo canceller can comprise one filter for each loudspeaker.
  • the invention further provides the use of a dual microphone comprising a first directional microphone being oriented in a first direction and a second directional microphone being oriented in a second direction different from the first direction, the first and the second microphone being arranged on a common support frame, for forming a microphone array for a hands-free system for speech signal acquisition in a vehicular cabin, in particular, as described above.
  • a vehicular cabin comprising a hands-free system as described above.
  • Fig. 1 schematically illustrates the configuration of a hands-free system for speech signal acquisition and noise suppression.
  • the example shown in this figure comprises two microphone arrays.
  • the first microphone array comprises two first directional microphones 1 being oriented in a first direction as indicated by arrow 2.
  • the second microphone array comprises two second directional microphones 3, being oriented in a second direction different from the first direction.
  • the first microphone array is pointing in the direction of an expected speaker.
  • the output of these first microphones is used to generate a primary signal x P (k).
  • the hands-free system further comprises a noise reduction device 4.
  • a cancellation means 5 forms part of this noise reduction device 4. This cancellation means 5 serves to cancel a noise part from the primary signal x P .
  • the noise reduction device 4 comprises a first input 6 for each first microphone 1.
  • a first signal path 7 connects the first inputs 6 and the cancellation means 5.
  • a beamformer 8 is provided that is a fixed beamformer (FBF) in the present example.
  • the beamformer can be a delay-and-sum beamformer or a filter-and-sum ("superdirective") beamformer.
  • an adaptive beamformer can be used as well.
  • the first signal path 7 comprises sub-paths 7' connecting the first inputs 6 of the noise reduction device 4 and the beamformer 8.
  • the first microphones 1 are used for picking up a wanted speech signal, denoted by x S,1 (k) and x S,2 (k)
  • the second microphones 3 are used for picking up noise signals only (or at least predominantly), denoted by x N,1 (k) and x N,2 (k).
  • the noise reduction device 4 has inputs 9 for receiving the signals from the second microphones 3.
  • a second signal path 10 connects the second inputs 9 and the cancellation means 5.
  • adaptive filters 11 having transfer functions h N,1 (k) and h N,2 (k) are provided between both second inputs 9 and the cancellation means 5.
  • the adaptive filters 11 are used to filter the noise reference signals, wherein these filters are adapted to maximally cancel out residual noise parts of the primary signal x P .
  • adaptation methods to update the adaptive filters 11.
  • LMS, NLMS or RLS algorithms can be used.
  • both microphone arrays comprise two directional microphones.
  • only one directional microphone for each direction can be provided, in which case, no beamformer would be required.
  • each microphone array can comprise more than two directional microphones.
  • the number of microphones in the first and the second microphone array need not be the same.
  • a microphone array with two or more microphones can be provided as the first microphones and less microphones, particularly only one directional microphone, as the at least one second directional microphone.
  • the different directional microphones are provided as independent microphones. Consequently, if such a hands-free system is to be mounted in a vehicular cabin, for example, in the overhead console, each microphone has to be independently mounted.
  • An alternative thereto is shown in Fig. 2 .
  • a dual microphone is schematically depicted.
  • Such a dual microphone comprises a first directional microphone 1 and a second directional microphone 3, both microphones pointing in a different direction.
  • Their pointing direction is indicated by arrows 2.
  • the angle ⁇ between the pointing directions preferably lies between 90° and 180°, in particular, between 130° and 180°.
  • the two directional microphones are mounted on a common substrate (not shown). Furthermore, the illustrated example also comprises a common housing 12. In this way, a very compact arrangement of two directional microphones is obtained.
  • a two of these dual microphones can be arranged so as to form two microphone arrays.
  • the first microphones 1 of both dual microphones form together the first microphone array whereas the second directional microphones 3 of both dual microphones form the second microphone array.
  • Fig. 3 depicts only two dual microphones, it is to be understood that other numbers of dual microphones can be used as well to obtain microphone arrays, each comprising more than two directional microphones.
  • the distance d between two neighboring dual microphones is to be chosen depending on different parameters such as type of beamformer used, distance between the microphone array and the expected position of a speaker's (such as a driver's) mouth, space restrictions for mounting the microphone arrays, etc.
  • the angle ⁇ between the pointing directions can be chosen depending on the intended use of the microphone arrays. For example, if both microphone arrays are used to pick up speech in a car, the directional microphones are to be oriented in the direction of the speaker and the front seat passenger.
  • Fig. 4 illustrates an example of a hands-free system with which the problem of leakage of the wanted speech signal into the noise reference microphones 3 (due to sound reflections or non-ideal directional characteristics of the microphones) is reduced.
  • same reference numerals denote same elements as in Fig. 1
  • a third signal path 13 which connects the first inputs 6 and the cancellation means 5.
  • a blocking matrix (BM) 14 and, in the direction of signal flow, a subsequent adaptive filter 15 with the transfer function h GSC (k) is provided.
  • the blocking matrix 14 is configured to block wanted speech parts in the speech microphone signals x S,1 (k) and x S,2 (k).
  • the resulting signal x B (k) is used as a noise reference signal.
  • a noise reduction device as shown in Fig. 5 can be provided.
  • additional subtraction means 16 are provided on the second signal path 10 between each second input and the cancellation means 5.
  • a fourth signal path 17 connecting the first signal path 7 and the subtraction means 16 are provided.
  • adaptive filters 18 with the transfer function h S,1 (k) and h N,2 (k) are provided on each fourth signal path 17. These additional adaptive filters 18 are used to clean the noise reference signals from speech portions.
  • the resulting signals x R,1 (k) and x R,2 (k) are used as input for the adaptive filters 11.
  • the filters 18 for reducing speech portions should be updated when the desired speaker is active.
  • the noise filters 11 should not be updated when the desired speaker is active in order to prevent further signal cancellation.
  • an adaptation controller (not shown) can be provided comprising a speech detector for providing speech of a desired speaker to freeze or update the adaptive filters, respectively. Then, the speech filters 18 adapted only during speech activity of the desired speaker (“speech adaptive") whereas the noise filters 11 are adapted during noise periods (“noise adaptive").
  • FIG. 6 A further example of a hands-free system is depicted in Fig. 6 .
  • the example of Fig. 6 further comprises an acoustic echo canceller (AEC) 19.
  • AEC acoustic echo canceller
  • This acoustic echo canceller 19 is a further adaptive filter which uses a loudspeaker signal x L (k) as additional noise reference signal. In this way, a further noise part can be subtracted from the primary signal x P in cancellation means 5.
  • the primary signal and/or noise reference signals used in the examples can be weighted depending on the filtering along the respective signal path.
  • an extra filter 19 is preferred for each playback channel in the echo canceller.
  • two filters are advantageous for stereo playback.
  • the structure of the noise reduction device has always been depicted such that the first microphones are used to provide a primary signal, whereas the second microphones are responsible for yielding the noise reference signals.
  • the noise reduction device can also be configured accordingly.
  • the signal processing can be applied for signals either coming from the first or the second microphone array. In the case of a hands-free system mounted in a vehicular cabin, this allows to use the system for enhancing speech signals from both the driver and a front seat passenger, for example'.
  • a control means is to be provided controlling the signal flow from each microphone array to the appropriate elements of the noise reduction device for the specific case. For example, if the driver is speaking, a structure as shown in the figures would be present whereas if the front passenger is speaking, the roles of the microphone arrays and the corresponding connections within the noise reduction device to the filters are to be swept. In other words, in the latter case, the microphones pointing to the driver are used to pick up the noise reference signal, whereas the microphones pointing to the front seat passenger are used to pick up speech.
  • the control of the signal flow of the microphone signals to the respective filters of the noise reduction device may depend on a predetermined criterion. For example, the control may depend on the speech activity of both speakers; the microphone array picking up speech first will be switched to the beamformer, then, this configuration can be maintained until the first speech pause occurs. Alternatively, this signal flow configuration can be maintained permanently (for a predetermined time or until a manual reset is performed).
  • the hands-free systems shown in the figures and described above can comprise a second noise reduction being configured similarly as the first noise reduction device with the difference that the signals from the second directional microphones are fed to the first inputs and the signals from the first directional microphones are fed to the second inputs of the second noise reduction device.
  • the first and second microphone signals are processed in the second noise reduction device in a converse way compared to the case of the first noise reduction device.
  • such a hands-free system comprises also a signal mixer to combine the output signals of the first and the second noise reduction device to obtain a combined enhanced output signal.
  • the mixing weights i.e. the weights with which the signals are multiplied before being summed, can be controlled in a time-varying way depending on the speech activity detected for the first and second microphones.
  • the weights can be controlled such that a speaker who starts speaking first is put through permanently whereas the other speaker is blocked by setting the corresponding mixing weight to zero.

Claims (17)

  1. Système mains libres pour l'acquisition d'un signal vocal dans une cabine d'un véhicule comprenant :
    au moins un premier microphone directionnel (1) orienté dans une première direction (2), chaque premier microphone comportant une première sortie, où l'au moins un premier microphone directionnel est prévu sous forme d'un réseau de microphones avec au moins deux microphones directionnels,
    au moins un second microphone directionnel (3) orienté dans une seconde direction différente de la première direction, chaque second microphone comportant une seconde sortie,
    un dispositif de réduction de bruit (4) configuré pour annuler une partie bruit dans un signal provenant de l'au moins un premier microphone directionnel en utilisant un signal provenant de l'au moins un second microphone directionnel, le dispositif de réduction de bruit comprenant
    une première entrée (6) pour chaque première et/ou seconde sortie microphone,
    une seconde entrée (9) pour chaque seconde et/ou première sortie microphone,
    un moyen d'annulation (5),
    un premier chemin du signal (7) connectant l'au moins une première sortie (6) et le moyen d'annulation (5),
    un second chemin du signal (10) connectant l'au moins une seconde sortie (9) et le moyen d'annulation (5),
    où le moyen d'annulation (5) est configuré pour annuler la partie bruit dans le signal reçu via le premier chemin du signal (7) en utilisant le signal reçu via le second chemin du signal (10), et
    où chaque première sortie de microphone est connectée à une première entrée (6) du dispositif de réduction de bruit (4) et chaque seconde sortie de microphone est connectée à une seconde entrée (9) du dispositif de réduction de bruit,
    un troisième chemin du signal (13) reliant chaque première entrée et le moyen d'annulation, où une matrice de blocage (14) et un filtre adaptatif (15) situé en aval dans la direction du flux de signal est disposée sur le troisième chemin du signal (13), où pour chaque première entrée, le troisième chemin du signal comprend une branche reliant la première entrée et la matrice de blocage et une branche reliant la matrice de blocage et le filtre adaptatif,
    où un générateur de faisceau (8) est disposé sur le premier chemin du signal (7), où pour chaque première entrée du dispositif de réduction de bruit, le premier chemin du signal (7) comprend une branche (7') reliant la première entrée (6) et le générateur de faisceau (8).
  2. Système mains libres selon la revendication 1, où le dispositif de réduction de bruit est configuré pour soustraire le signal basé sur le signal provenant de l'au moins un second microphone directionnel du signal provenant de l'au moins un premier microphone directionnel.
  3. Système mains libres selon la revendication 1 ou 2, où le dispositif de réduction de bruit est configuré en outre pour annuler une partie bruit dans un signal provenant de l'au moins un second microphone directionnel en utilisant un signal provenant de l'au moins un premier microphone directionnel.
  4. Système mains libres selon la revendication 3, comprenant en outre un moyen de commande pour commander le traitement du signal du dispositif de réduction de bruit selon un critère prédéterminé de façon à ce qu'une partie bruit dans un signal provenant de l'au moins un premier microphone et/ou une partie bruit dans un signal provenant de l'au moins un second microphone soit annulé.
  5. Système mains libres selon l'une quelconque des revendications précédentes, où l'au moins un premier microphone directionnel et/ou l'au moins un second microphone directionnel est disposé comme un réseau de microphones avec au moins deux microphones directionnels,
    et
    le système de réduction de bruit comprend un générateur de faisceau (8) pour traiter les signaux en provenance de chaque réseau de microphones.
  6. Système mains libres selon l'une quelconque des revendications précédentes, où le nombre de premiers microphones est égal au nombre de seconds microphones.
  7. Système mains libres selon l'une quelconque des revendications précédentes, où l'au moins un premier et l'au moins un second microphones directionnels sont disposés en paires de telle manière qu'un premier microphone et un second microphone sont respectivement agencés sur un cadre de support commun.
  8. Système mains libres selon l'une quelconque des revendications précédentes, où le moyen d'annulation est configuré pour soustraire le signal basé sur le signal reçu via le second chemin du signal reçu via le premier chemin du signal.
  9. Système mains libres selon l'une quelconque des revendications précédentes, où un filtre adaptatif (11), en particulier basé sur un algorithme LMS, NLMS ou RLS, est disposé sur le second chemin du signal entre le moyen d'annulation et chaque seconde entrée.
  10. Système mains libres selon l'une quelconque des revendications précédentes, où le générateur de faisceau est un générateur de faisceau fixe.
  11. Système mains libres selon l'une quelconque des revendications précédentes, comprenant en outre un moyen de soustraction (16) sur le second chemin du signal entre chaque seconde entrée du dispositif de réduction de bruit et le moyen d'annulation et un quatrième chemin du signal (17) comprenant des branches reliant le premier chemin du signal et chaque moyen de soustraction, où un filtre adaptatif (18) est disposé sur chaque branche reliant le premier chemin du signal et chaque moyen de soustraction.
  12. Système mains libres selon l'une quelconque des revendications précédentes, comprenant en outre une commande d'adaptation avec un détecteur de parole pour commander une adaptation d'un filtre adaptatif conformément à la détection de parole.
  13. Système mains libres selon l'une quelconque des revendications précédentes, où chaque première sortie de microphone est connectée à une seconde entrée (6) du dispositif de réduction de bruit (4) et chaque seconde sortie de microphone est connectée à une première entrée (9) du dispositif de réduction de bruit, et comprenant en outre un moyen de commande pour commander le traitement du signal du dispositif de réduction de bruit selon un critère prédéterminé de telle façon qu'une partie bruit dans un signal en provenance de l'au moins un premier microphone et/ou une partie bruit dans un signal en provenance de l'au moins un second microphone est annulé par le moyen d'annulation.
  14. Système mains libres selon l'une quelconque des revendications précédentes, où le dispositif de réduction de bruit est un premier dispositif de réduction de bruit pour fournir un premier signal de sortie optimisé, et comprenant en outre un second dispositif de réduction de bruit configuré pour annuler une partie bruit dans un signal provenant de l'au moins un second microphone directionnel en utilisant un signal provenant de l'au moins un second microphone directionnel pour fournir un second signal de sortie optimisé.
  15. Système mains libres selon la revendication 14, comprenant en outre un mélangeur de signaux pour combiner le premier et le second signal de sortie optimisé pour fournir un signal de sortie optimisé combiné.
  16. Système mains libres selon la revendication 14 ou 15, où le second dispositif de réduction de bruit est configuré pour traiter un signal provenant de l'au moins un premier microphone directionnel et un signal provenant de l'au moins un second microphone directionnel au moins partiellement de la même manière que le premier dispositif de réduction de bruit traite un signal provenant de l'au moins un second microphone directionnel et un signal provenant de l'au moins un premier microphone directionnel, respectivement.
  17. Système mains libres selon l'une quelconque des revendications précédentes, comprenant en outre un compensateur d'écho acoustique (19).
EP06004158.9A 2006-03-01 2006-03-01 Système mains libres utilisé dans les véhicules Not-in-force EP1830348B1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP06004158.9A EP1830348B1 (fr) 2006-03-01 2006-03-01 Système mains libres utilisé dans les véhicules
JP2007026914A JP2007235943A (ja) 2006-03-01 2007-02-06 音声信号取得のためのハンドフリーシステム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP06004158.9A EP1830348B1 (fr) 2006-03-01 2006-03-01 Système mains libres utilisé dans les véhicules

Publications (2)

Publication Number Publication Date
EP1830348A1 EP1830348A1 (fr) 2007-09-05
EP1830348B1 true EP1830348B1 (fr) 2016-09-28

Family

ID=36615643

Family Applications (1)

Application Number Title Priority Date Filing Date
EP06004158.9A Not-in-force EP1830348B1 (fr) 2006-03-01 2006-03-01 Système mains libres utilisé dans les véhicules

Country Status (2)

Country Link
EP (1) EP1830348B1 (fr)
JP (1) JP2007235943A (fr)

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8218397B2 (en) * 2008-10-24 2012-07-10 Qualcomm Incorporated Audio source proximity estimation using sensor array for noise reduction
KR101791444B1 (ko) * 2010-11-29 2017-10-30 뉘앙스 커뮤니케이션즈, 인코포레이티드 동적 마이크로폰 신호 믹서
KR101248971B1 (ko) * 2011-05-26 2013-04-09 주식회사 마이티웍스 방향성 마이크 어레이를 이용한 신호 분리시스템 및 그 제공방법
US10606546B2 (en) 2012-12-05 2020-03-31 Nokia Technologies Oy Orientation based microphone selection apparatus
EP2760021B1 (fr) * 2013-01-29 2018-01-17 2236008 Ontario Inc. Stabilisateur spatial de champ sonore
US9516418B2 (en) * 2013-01-29 2016-12-06 2236008 Ontario Inc. Sound field spatial stabilizer
US20140270241A1 (en) * 2013-03-15 2014-09-18 CSR Technology, Inc Method, apparatus, and manufacture for two-microphone array speech enhancement for an automotive environment
DE102013005049A1 (de) * 2013-03-22 2014-09-25 Unify Gmbh & Co. Kg Verfahren und Vorrichtung zur Steuerung einer Sprachkommunikation sowie Verwendung derselben
US9106196B2 (en) 2013-06-20 2015-08-11 2236008 Ontario Inc. Sound field spatial stabilizer with echo spectral coherence compensation
US9099973B2 (en) 2013-06-20 2015-08-04 2236008 Ontario Inc. Sound field spatial stabilizer with structured noise compensation
US9271100B2 (en) 2013-06-20 2016-02-23 2236008 Ontario Inc. Sound field spatial stabilizer with spectral coherence compensation
CN105594226B (zh) * 2013-10-04 2019-05-03 日本电气株式会社 信号处理装置、信号处理方法和介质处理装置
US10536773B2 (en) 2013-10-30 2020-01-14 Cerence Operating Company Methods and apparatus for selective microphone signal combining
CN106331959B (zh) * 2016-09-26 2019-10-25 欧仕达听力科技（厦门）有限公司 指向性传声器的降噪方法与装置
CN107331407B (zh) * 2017-06-21 2020-10-16 深圳市泰衡诺科技有限公司 下行通话降噪方法及装置
US20200372926A1 (en) * 2017-12-29 2020-11-26 Harman International Industries, Incorporated Acoustical in-cabin noise cancellation system for far-end telecommunications
WO2019130282A1 (fr) 2017-12-29 2019-07-04 Harman International Industries, Incorporated Système acoustique de réduction du bruit en cabine pour télécommunications à distance
CN111009259B (zh) * 2018-10-08 2022-09-16 杭州海康慧影科技有限公司 一种音频处理方法和装置
DE102019211584A1 (de) * 2019-08-01 2021-02-04 Robert Bosch Gmbh System und Verfahren zur Kommunikation einer mobilen Arbeitsmaschine
JP2022552657A (ja) * 2019-10-10 2022-12-19 シェンツェン・ショックス・カンパニー・リミテッド 音響設備

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2758846B2 (ja) 1995-02-27 1998-05-28 埼玉日本電気株式会社 ノイズキャンセラ装置
WO1997023068A2 (fr) * 1995-12-15 1997-06-26 Philips Electronic N.V. Dispositif d'elimination de bruit adaptatif, systeme de reduction de bruit et emetteur-recepteur
GB9922654D0 (en) 1999-09-27 1999-11-24 Jaber Marwan Noise suppression system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
GRIFFITHS L J ET AL: "AN ALTERNATIVE APPROACH TO LINEARLY CONSTRAINED ADAPTIVE BEAMFORMING", IEEE TRANSACTIONS ON ANTENNAS AND PROPAGATION, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. AP-30, no. 1, 1 January 1982 (1982-01-01), pages 27 - 34, XP000608836, ISSN: 0018-926X, DOI: 10.1109/TAP.1982.1142739 *

Also Published As

Publication number Publication date
EP1830348A1 (fr) 2007-09-05
JP2007235943A (ja) 2007-09-13

Similar Documents

Publication Publication Date Title
EP1830348B1 (fr) Système mains libres utilisé dans les véhicules
US10438605B1 (en) Echo control in binaural adaptive noise cancellation systems in headsets
US8942382B2 (en) Dynamic beamformer processing for acoustic echo cancellation in systems with high acoustic coupling
US9635186B2 (en) Conferencing apparatus that combines a beamforming microphone array with an acoustic echo canceller
JP5307355B2 (ja) ハンズフリーシステムにおけるバッググラウンドノイズリダクション
JP5249207B2 (ja) 適応性のある方向性信号処理を備えた補聴装置
CA2475183C (fr) Methode permettant de reduire le couplage acoustique dans des systemes d'audioconference
US6748086B1 (en) Cabin communication system without acoustic echo cancellation
KR101103794B1 (ko) 멀티 빔 음향시스템
JP2009542057A5 (fr)
WO2004034734A1 (fr) Dispositif reseau et terminal portatif
KR20060128944A (ko) 일반화된 사이드로브 제거를 위한 노이즈 레퍼런스들을생성하는 방법
WO2009034524A1 (fr) Appareil et procede de formation de faisceau audio
US10490208B2 (en) Flexible voice capture front-end for headsets
KR20130078919A (ko) 차량용 핸즈프리 통화 시스템 및 그 제어 방법
KR102579909B1 (ko) 원단 원격통신을 위한 승객실 내 음향 잡음 소거 시스템
US9420115B2 (en) Method using array microphone to cancel echo
US20220394383A1 (en) End-fire array microphone arrangements inside a vehicle
CN113129929A (zh) 用于车辆音频系统的具有空间言语分离的语音闪避
Schmidt Applications of acoustic echo control-an overview
US20220189450A1 (en) Audio processing system and audio processing device
Kellermann Joint design of acoustic echo cancellation and adaptive beamforming for microphone arrays

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK YU

17P Request for examination filed

Effective date: 20080227

AKX Designation fees paid

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

17Q First examination report despatched

Effective date: 20080811

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: NUANCE COMMUNICATIONS, INC.

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602006050381

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G10L0021020000

Ipc: H04R0001400000

RIC1 Information provided on ipc code assigned before grant

Ipc: H04R 1/40 20060101AFI20130419BHEP

Ipc: G10L 21/0216 20130101ALI20130419BHEP

Ipc: G10L 21/0208 20130101ALI20130419BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20160413

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 833649

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161015

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602006050381

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20160928

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 833649

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160928

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161229

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170128

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170130

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602006050381

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20170629

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602006050381

Country of ref document: DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20170301

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20171130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171003

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170331

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170301

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170331

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170331

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170301

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170301

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20060301

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160928

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160928