EP3375204B1 - Audio signal processing in a vehicle - Google Patents

Audio signal processing in a vehicle Download PDF

Info

Publication number
EP3375204B1
EP3375204B1 EP16788664.7A EP16788664A EP3375204B1 EP 3375204 B1 EP3375204 B1 EP 3375204B1 EP 16788664 A EP16788664 A EP 16788664A EP 3375204 B1 EP3375204 B1 EP 3375204B1
Authority
EP
European Patent Office
Prior art keywords
signal
audio signal
audio
limited
vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP16788664.7A
Other languages
German (de)
French (fr)
Other versions
EP3375204A1 (en
Inventor
David Scheler
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Volkswagen AG
Original Assignee
Volkswagen AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Volkswagen AG filed Critical Volkswagen AG
Publication of EP3375204A1 publication Critical patent/EP3375204A1/en
Application granted granted Critical
Publication of EP3375204B1 publication Critical patent/EP3375204B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L21/0232Processing in the frequency domain
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00Circuits for transducers, loudspeakers or microphones
    • H04R3/02Circuits for transducers, loudspeakers or microphones for preventing acoustic reaction, i.e. acoustic oscillatory feedback
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00Stereophonic arrangements
    • H04R5/02Spatial or constructional arrangements of loudspeakers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S7/00Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30Control circuits for electronic adaptation of the sound field
    • H04S7/307Frequency adjustment, e.g. tone control
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L2021/02082Noise filtering the noise being echo, reverberation of the speech
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161Number of inputs available containing the signal or the noise to be suppressed
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02166Microphone arrays; Beamforming
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00Signal processing covered by H04R, not provided for in its groups
    • H04R2430/03Synergistic effects of band splitting and sub-band processing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10General applications
    • H04R2499/13Acoustic transducers and sound field adaptation in vehicles
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00Stereophonic arrangements
    • H04R5/04Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S2400/00Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/05Generation or adaptation of centre channel in multi-channel audio systems

Definitions

  • the present invention relates to a method for audio signal processing in a vehicle and a corresponding audio signal processing apparatus for a vehicle.
  • the present invention relates to audio signal processing with echo cancellation, for example for speech processing.
  • speech dialogue systems are used to assist the driver or occupants.
  • speech dialogue systems serve to control electronic devices without the need for a haptic operator operation.
  • the electronic devices may include, for example, a vehicle computer or a multimedia system of the vehicle. Spoken speech from the driver or occupant is received via a hands-free microphone and fed to speech recognition.
  • An application of microphones in the vehicle interior for e.g. Voice control, telephony or vehicle interior communication can potentially be affected by acoustic coupling of loudspeaker outputs of the vehicle sound system. This can lead to recognition errors in the case of speech recognition, to far-end echo in the case of hands-free telephony, and to feedback in the case of vehicle interior communication. Depending on the application, the consequences are disturbed communication, increased distraction or even disturbing noises and echoes.
  • the audio signals reproduced via the vehicle sound system can comprise, for example, music, traffic news, radio broadcasts, outputs of a navigation system or the (artificial) language of a speech dialogue system.
  • the disturbance of the speech recognition can lead to recognition errors which can make the dialogue inefficient and an increased distraction from the driving task can cause. This can cause dissatisfaction or annoyance to the driver or occupant.
  • a simple solution to the aforementioned problem is to mute the audio playback of, for example, a radio during the voice dialogue or telephone call in the vehicle.
  • the muting of audio playback is often perceived by vehicle users as disturbing and unnecessary.
  • important information can be missed by, for example, a navigation system.
  • the audio playback volume may be temporarily reduced.
  • the extent of the interference by the audio playback is then lower, but generally still so large that it can not be dispensed with a further cleanup of the microphone signal.
  • Couplings are also reduced by constructive-acoustic measures.
  • microphones with a suitable directional characteristic can be used, microphones and loudspeakers inside the vehicle can be suitably arranged relative to one another, or acoustic conditions in the vehicle can be suitably exploited.
  • signal processing components are used to clean up the microphone signals.
  • the signal components coupled into the microphones by the loudspeakers of the vehicle sound system are estimated and removed from the microphone signals.
  • Such methods are referred to as echo cancellation or echo cancellation.
  • transition frequency f c is on the order of 1 kHz.
  • the sum of the left and right channels is high-pass filtered to obtain the frequencies above f c as a monophonic signal.
  • the left (or right) speaker signal is the sum of the low frequencies of the left (or right) channel and high frequencies of both channels.
  • Two echo cancellers are needed: a high frequency mono echo canceller and a low frequency stereo echo canceller.
  • the US 2012/232890 A1 relates to a speech recognition system in which an n-channel system sound is converted by means of a monophonic unit into a single-channel acoustic signal.
  • the single-channel acoustic signal is output to a weighting unit and to an echo canceller.
  • the n-channel system sound is output through multiple speakers in, for example, a vehicle.
  • a microphone is used to receive voice input from a user as well as multichannel system sound output from the speakers and to the echo canceller.
  • a common type of echo cancellation is linear echo cancellation.
  • LEM impulse response Loudspeaker Enclosure Microphone
  • a linear echo cancellation algorithm adaptively estimates the LEM impulse response for each possible microphone-speaker pair. Based on the LEM impulse response, the injected loudspeaker sound components in each microphone signal are then calculated and subtracted therefrom. The rate of adaptation and effective echo cancellation are limited and generally in competition with each other.
  • the disclosed EP 1936939 A1 an echo cancellation in which the microphone signal is divided into subband signals and sub-sampled.
  • a reference audio signal is output through a speaker.
  • the reference audio signal is also sub-sampled and sub-sampled subband signals of the reference audio signal are stored.
  • echoes in the microphone subband signals are estimated and the estimated echoes are subtracted from the microphone subband signals to obtain improved microphone subband signals.
  • the multi-channel audio signal may be, for example, a stereo signal or a surround signal in the vehicle.
  • the estimation problem Due to correlations between the different audio source signals, the estimation problem is mathematically underdetermined. As a consequence, in the event of a sudden appearance of audio source signals, the effectiveness of echo cancellation may be greatly reduced. It may even occur that the LEM estimate diverges, for example when there are changes in the surround sound image. This can occur, for example, when so-called phantom sound sources appear, disappear or shift in the surround panorama.
  • an echo canceller which cooperates with a sound output device with a multi-channel audio unit.
  • the sound output device outputs output sound signals as analog signals of multiple channels through a plurality of speakers.
  • a microphone detects an outside sound and generates an input sound signal as an analog signal.
  • the outside sound echoes the output sound signals.
  • the echo canceller has an echo cancellation function for removing the echo from the input tone signal.
  • the echo canceller receives the output sound signals from the sound output device.
  • the general interfering signals may also include multichannel audio playback. This is for example in the DE 102009051508 A1 considered.
  • a microphone array is used instead of a single microphone. Through the microphone array a multi-channel speech signal is recorded, which is passed instead of a simple speech signal to an echo cancellation unit.
  • the multichannel speech signal recorded by the microphone array is post-processed before being input to the echo canceling unit in a unit downstream of the microphone array for processing the microphone signals by delayed summation of the signals. As a result, the signals of the authorized speakers are separated and all other speaker signals and noise are reduced.
  • the echo canceling unit evaluates the propagation time of the various channels of the multi-channel speech signal and removes all portions of the signal that do not emanate from the authorized speaker's location according to their propagation time.
  • using a microphone array or multiple microphones increases costs, requires more installation space, and requires powerful computational resources.
  • this object is achieved by a method for audio signal processing in a vehicle according to claim 1 and an audio signal processing apparatus for a vehicle according to claim 7.
  • the dependent claims define embodiments of the invention.
  • a method for audio signal processing in a vehicle is provided.
  • a mono audio signal is generated based on a multi-channel audio source signal.
  • the multi-channel audio source signal is, for example, a stereo signal or a surround signal which is to be output in the vehicle via a plurality of loudspeakers of the vehicle.
  • the mono audio signal is limited to a frequency range between a predetermined lower frequency and a predetermined upper frequency.
  • the mono audio signal can be limited, for example, with a bandpass filter to the frequency range between the predetermined lower frequency and the predetermined upper frequency. Limiting the mono audio signal to the frequency domain produces a limited mono audio signal.
  • the limited mono audio signal is output through the plurality of speakers in the vehicle.
  • this voice audio signal contains the limited mono audio signal output via the plurality of loudspeakers.
  • An influence of this limited mono audio signal output via the several loudspeakers on the speech audio signal received via the microphone is compensated by means of the limited mono audio signal. For example, an echo compensation can be performed, which takes into account only the mono audio signal.
  • a complex echo cancellation taking into account a multi-channel audio signal is therefore not required. Instead, only a single-channel echo cancellation is required, which can be realized with relatively low processing power.
  • the echo cancellation taking into account only one echo signal is very reliable, even if the mono audio signal is output through several different speakers, as with a mono audio signal no changes in the multi-channel sound image can occur.
  • the interfering mono audio signal can be largely or completely removed from the voice audio signal.
  • the predetermined lower frequency has a value in the range of 100 Hz to 300 Hz
  • the predetermined upper frequency has a value in the range of 4 kHz to 8 kHz.
  • a speech recognizer which is used, for example, for voice control or voice input in a vehicle, in many cases evaluates audio signals in a limited frequency range of, for example, 100 Hz to 8 kHz in order to recognize the voice input from a user. Therefore, echo cancellation is required only in this limited frequency range. Therefore, the predetermined lower frequency is preferably 100 Hz and the predetermined upper frequency is 8 kHz. As a result, an undisturbed speech signal can be provided to the speech recognizer in the limited frequency range relevant for him.
  • a plurality of limited channel-specific audio signals are generated as a function of the multi-channel audio source signal.
  • a channel-specific audio signal relates, for example, to an audio signal which is determined by the multi-channel audio signal source specifically for a loudspeaker associated with the respective channel. For a stereo source signal, this may include, for example, an audio signal for the right speaker or an audio signal for the left speaker.
  • a respective limited channel-specific audio signal of the plurality of limited channel-specific audio signals is therefore assigned to a respective audio signal of the multi-channel audio source signal.
  • a respective limited channel-specific audio signal is limited to a frequency range which only includes frequencies below the predetermined lower frequency and frequencies above the predetermined upper frequency.
  • a respective limited channel-specific audio signal is formed by a corresponding frequency limitation from the respectively assigned audio signal of the multi-channel audio source signal.
  • the audio signals of the multi-channel audio signal are respectively limited or filtered so that they only include frequencies below the predetermined lower frequency and / or frequencies above the predetermined upper frequency.
  • the plurality of limited channel-specific audio signals are output through the plurality of speakers in the vehicle, so that the effect of multi-channel audio reproduction such as stereo reproduction or surround reproduction can be achieved.
  • an audio reproduction in the vehicle is modified so that in the frequency range between the predetermined lower frequency and the predetermined upper frequency, the multi-channel audio source signal is reproduced in one channel (mono) and in the remaining frequency range multi-channel.
  • the mono audio signal and the plurality of limited channel-specific audio signals may be generated from the multi-channel audio source signal according to the following embodiment.
  • the multi-channel audio source signal is divided into a center signal component which is the same on all channels and a respective side signal component per audio channel of the multi-channel audio source signal.
  • the limited mono audio signal is generated from the center signal component and the plurality of limited channel-specific audio signals are generated from the respective side signal components.
  • the mid-signal component for example, can be used directly as a mono audio signal or suitably scaled used as a mono audio signal.
  • the side signal components may be used directly as the limited channel specific audio signals or in suitably scaled form.
  • the center signal component in the case of a stereo signal, can be formed, for example, from the sum of the right and left audio source signals.
  • the side signal components may be coded together in a difference signal from the difference between the right and left audio source signals and further processed.
  • the center signal component and the side signal components can be generated and processed in a simple manner.
  • the center signal component is formed by averaging respective samples of the audio channels of the multi-channel audio source signal.
  • the respective side signal components are formed by subtracting the center signal component from the respective audio signals of the multi-channel audio source signal. This generation of the center signal component and the side signal components is possible for audio source signals with an arbitrary number of channels.
  • an implementation in, for example, a digital signal processor can be realized in a simple manner.
  • the speech audio signal received via the microphone is limited to a frequency range between the predetermined lower frequency and the predetermined upper frequency.
  • the echo cancellation is applied to the thus limited voice audio signal using the limited mono audio signal.
  • the influence of the limited mono audio signal outputted through the plural speakers to the limited voice audio signal is compensated.
  • the speech recognizer operates generally only in the frequency range between the predetermined lower frequency and the predetermined upper frequency, echo cancellation in a speech audio signal limited thereto is sufficient.
  • spurious signals outside this frequency range are already eliminated before the echo cancellation and have therefore, does not affect echo cancellation and speech recognition, which makes both echo cancellation and speech recognition more reliable.
  • the reproduction of an audio signal is more important to some occupants of the vehicle than to others.
  • audio outputs of a navigation system are more important to the driver than to the remaining occupants
  • audio outputs of a video displayed in the rear of the vehicle are more important for rear passengers than for the driver and front passenger.
  • a plurality of weighting factors associated with the respective loudspeakers may be generated in response to the multi-channel audio source signal.
  • the limited mono audio signal is weighted for each loudspeaker with the weighting factor assigned to the respective loudspeaker. Thereby, a center of gravity of the audio output in the vehicle can be appropriately shifted.
  • the weighted output will not affect the quality of the echo cancellation.
  • the echo cancellation may adjust to the new weight in a relatively short time, for example within a few seconds or minutes.
  • the following weighting may be used.
  • the speaker in the driver's area may output 70% of the mono audio signal, and the remaining three speakers may output only 10% of the mono audio signal, for example.
  • an audio signal processing apparatus for a vehicle.
  • the audio signal processing apparatus is capable of generating a mono audio signal based on a multi-channel audio source signal.
  • the audio signal processing device may have, for example, a summation device.
  • the audio signal processing apparatus is further capable of limiting the mono audio signal to a frequency range between a predetermined lower frequency and a predetermined upper frequency. This can be realized for example with a bandpass filter.
  • the predetermined lower frequency has a value in the range of 100 Hz to 300 Hz
  • the predetermined upper frequency has a value in the range of 4 kHz to 8 kHz.
  • the limited mono audio signal is output through several speakers in the vehicle.
  • the limited mono audio signal is output to a compensation device, for example to a Echo canceller.
  • the compensation device serves to compensate for an influence of the limited mono audio signal output via the plurality of loudspeakers on a voice audio signal received in the vehicle via a microphone by means of the limited mono audio signal.
  • the audio signal processing device is therefore suitable for carrying out the method described above and its embodiments and therefore also comprises the advantages described above.
  • FIG. 1 First, the environment of an audio signal processing device 15 according to the invention in a vehicle 10 will be described. In FIG. 2 Details of the audio signal processing device 15 will be described in conjunction with other components of the vehicle 10. FIG. 3 Finally, Fig. 12 schematically shows the operation of the audio signal processing apparatus 15. Like reference numerals in the figures refer to the same or similar components.
  • FIG. 1 shows a vehicle 10 in a plan view.
  • the vehicle 10 includes a voice recognition system 11.
  • voice commands or instructions from occupants of the vehicle 10 may be captured, processed, and executed.
  • configuration settings of the vehicle 10 or a multimedia system of the vehicle 10 may be changed via appropriate instructions.
  • an audio signal source such as CD or radio
  • a specific radio station can be selected or a title of a CD.
  • a telephone connection to a desired participants are set up or a navigation destination can be set in a navigation system of the vehicle 10.
  • corresponding commands or instructions are received by a driver 12 of the vehicle 10 via a microphone 13.
  • a spoken command of the driver 12 is forwarded by the microphone 13 as a speech audio signal to an audio signal processing device 15.
  • the operation of the audio signal processing device 15 will be described later with reference to FIG FIG. 2 will be described in detail.
  • the processed speech audio signal is supplied to the speech recognition system 11.
  • the speech recognition system 11 evaluates the speech audio signal and detects contained commands and instructions and executes them.
  • the speech recognition system can be coupled with a so-called dialogue system, which can lead a dialogue with the driver via questions and answers.
  • the vehicle 10 further includes an audio signal source 14.
  • the audio signal source 14 may include, for example, a broadcast receiver, a media player such as a CD player or an MP3 player, or a navigation system of the vehicle 10.
  • the audio signal source 14 outputs a multi-channel audio source signal.
  • the multi-channel audio source signal is supplied to and processed by the audio signal processing device 15, as described below with reference to FIG FIG. 2 will be described.
  • the processed multi-channel audio source signal is output from the audio signal processing device 15 to an amplifier 16.
  • the amplifier 16 amplifies the individual signals of the processed multi-channel audio source signal so that they can be reproduced via speakers 17-20 in an interior of the vehicle 10.
  • the vehicle 10 includes four loudspeakers 17-20.
  • the vehicle 10 may include any number of speakers, for example two, three or more than four.
  • the FIG. 1 are the speakers 17-20 assigned to the seats of the vehicle 10.
  • the loudspeaker 17 is associated with a driver's seat of the driver 12, the loudspeaker 18 with a passenger seat, the loudspeaker 19 with a rear right seat and the loudspeaker 20 with a rear left seat.
  • the driver 12 may voice instructions or commands to the speech recognition system 11. This will be in FIG. 1 represented by the dashed arrow between the driver 12 and the microphone 13. While the driver 12 is issuing commands and instructions, multi-channel audio source signals may be supplied from the audio signal source 14 be output through the speakers 17-20. The outputs of the speakers 17-20 also reach the microphone 13, as indicated by the corresponding dashed arrows between the speakers 17-20 and the microphone 13 in the FIG. 1 is shown. However, the outputs from the speakers 17-20 may interfere with speech intelligibility, such that the speech recognition system 11 may not or insufficiently recognize the commands and instructions from the driver 12.
  • FIG. 2 shows details of the audio signal processing device 15 and the speech recognition system 11, which help to reduce or compensate for the influence of the outputs from the loudspeakers 17-20 on the speech signal of the driver 12.
  • the audio signal source 14 in the example of FIG. 2 however, it is clear that the audio signal processing apparatus 15 described below can process any number of channels of a multi-channel audio signal source in the same way.
  • FIG. 2 shown components of the audio signal processing device 15 described.
  • components of the audio signal processing device 15 may not necessarily be designed as concrete components or assemblies, but may be partially or wholly reproduced programmatically and implemented by a suitable controller, such as a microprocessor or a digital signal processor.
  • the audio signal processing device 15 includes inputs via which the multi-channel audio source signal is received by the audio signal source 14.
  • a two-channel stereo audio source signal includes, for example, a left channel L and a right channel R, which are supplied to the audio signal processing device 15.
  • a middle signal component M and for each channel a side signal component S are generated from the two- or multi-channel audio source signal.
  • a common side signal component can be formed as the difference between the left channel L and the right channel R. Since, regardless of the number of side signal components, all side signal components are treated the same in the following FIG. 2 only one path for the side signal components S is shown. This one path can therefore comprise only one side signal component in the case of a stereo signal or several side signal components in the multi-channel case.
  • the middle signal component M may comprise, for example, a sum signal from all the supplied channels.
  • a respective side signal component S may, for example, comprise a difference signal between the respective audio signal of the respective channel of the multi-channel audio source signal and the mid-signal component.
  • the audio signal processing device 15 further comprises a first bandpass filter 23 and a notch filter or notch filter 22.
  • the first bandpass filter 23 has a predetermined lower frequency and a predetermined upper frequency.
  • the first bandpass filter 23 essentially passes only signals having a frequency between the predetermined lower frequency and the predetermined upper frequency. Signals with a frequency below the predetermined lower frequency and signals with a frequency above the predetermined upper frequency are substantially suppressed or at least greatly attenuated.
  • the attenuation may be, for example, 70 dB or more, and in a digital embodiment of the first bandpass filter, the signal above the predetermined upper frequency and below the predetermined lower frequency can be completely suppressed.
  • the notch filter 22 has a frequency response which is substantially inverse to the frequency response of the first bandpass filter 23. That is, the notch filter 22 essentially only passes signals having a frequency below the predetermined lower frequency or above the predetermined upper frequency.
  • the lower predetermined frequency may be 100 Hz
  • the upper predetermined frequency may be 8 kHz, for example.
  • the lower predetermined frequency may be selected in a range of 100 Hz to 300 Hz
  • the upper predetermined frequency may be selected in a range of 4 kHz to 8 kHz. The larger the frequency range between the lower predetermined frequency and the upper predetermined frequency is selected, the more reliable the speech recognition function.
  • reproduction of a multi-channel audio source signal is more affected the greater the frequency range between the lower predetermined frequency and the upper predetermined frequency is selected.
  • a corresponding notch filter 22 having the lower predetermined frequency and the upper predetermined frequency is to be provided for each of these plurality of side signal components.
  • a filtered or frequency-limited mid-signal component Mb is generated.
  • filtered or frequency-limited side signal components Sb are generated.
  • the filtered center signal component Mb and the filtered side signal component Sb are applied to a second signal converter 24, which generates filtered audio signals for the individual channels.
  • the filtered audio signal for a respective single channel may be formed, for example, by summing the filtered center signal component Mb and the corresponding filtered channel-specific side signal component Sb.
  • the filtered audio signals Lb, Rb are output from the audio signal processing device 15 and supplied to the amplifier 16 channel by channel.
  • the audio signal processing device 15 further comprises a second bandpass filter 26.
  • the second bandpass filter 26 has the same filter characteristic as the first bandpass filter 23.
  • the second bandpass filter 26 is coupled on the input side to the microphone 13 and on the output side with an echo canceller 25 of the speech recognition system 11.
  • the echo canceller 25 of the speech recognition system 11, the filtered center signal component Mb is further supplied. Based on the filtered center signal component Mb, the echo canceller 25 performs echo cancellation on the filtered voice signal from the microphone 13.
  • the speech signal processed by the echo canceler 25 is supplied to a speech recognizer 27 of the speech recognition system 11.
  • the audio signal processing device 15 comprises a weighting device 28, which is coupled to the multi-channel audio source signal and / or the audio signal source 14. Based on information from the multi-channel audio source signal or information from the audio signal source 14, the weighting device 28 provides weighting factors with which the filtered audio signals are weighted prior to being output from the second signal converter 24.
  • FIG. 3 shows a method 30 with method steps 31-37, which are carried out by the audio signal processing device 15 in conjunction with the speech recognition system 11.
  • illustrated processing steps can be performed with electronic tools, which include, for example, analog or digital circuits and processing devices.
  • processing devices For example, they may include microprocessors or digital signal processors.
  • the entire functionality of the audio signal processing device 15 can be integrated into, for example, an existing electronic device, for example a digital signal processor of the speech recognition system 11.
  • a multi-channel audio source signal for example a stereo signal or a surround signal
  • a frequency limited mono audio signal and frequency limited channel specific audio signals are generated by the first signal converter 21 and the filters 22 and 23.
  • the frequency-limited center signal component Mb described above may be, for example, the frequency-limited mono audio signal.
  • the frequency-limited side signal components Sb described above may be, for example, the frequency-limited channel-specific audio signals.
  • the frequency-limited mono audio signal and the frequency-limited channel-specific audio signals can also be formed in any other way from the multi-channel audio source signal, for example in a digital signal processor.
  • the limited mono audio signal is output through all the loudspeakers 17-20 and the limited channel specific audio signals are outputted over the loudspeaker associated with the respective channel.
  • the mono audio signal is limited to a relevant frequency range for speech recognition, for example to a frequency range of 100 Hz to 8 kHz.
  • the channel-specific audio signals are limited to a frequency range outside the relevant frequency range for speech recognition, that is, for example, to frequencies below 100 Hz and above 8 kHz. Due to the reduction of the multichannel nature of the audio reproduction within the frequency range relevant to the speech recognizer 27, only the single-channel mono audio signal is present as the disturbing signal for speech recognition. For the vehicle occupant, however, a feeling of spatial perception remains in the sound perception since the multichannel nature is retained for frequencies outside the range relevant for speech recognition.
  • the weighting device 28 may determine an audio centroid for the multi-channel audio source signals or the current signal source based on the information supplied thereto, and the limited mono audio signal corresponding to that audio centroid on the audio source Distribute audio channels. For example, if a voice output of a navigation system represents the multi-channel audio signal source, the limited mono audio signal, for example, for the speaker 17 may be weighted more heavily than for the speakers 18-20, since this information is more relevant to the driver 12 than for the rest of the vehicle occupants.
  • the weighting device 28 may consider further information of the vehicle 10, for example a current seat occupancy in the vehicle.
  • a speech audio signal is received via the microphone 13 in step 35.
  • the received speech audio signal is frequency limited by the second bandpass filter 26.
  • the echo canceller 25 is supplied with the limited mono audio signal and the limited voice audio signal.
  • the echo canceller 25 performs echo cancellation in the voice audio signal using the mono audio signal. Since both the speech audio signal and the mono audio signal are limited to the frequency range relevant to the speech recognition (eg 100Hz-8kHz), the echo cancellation can also be limited to this limited frequency range, whereby less noise occurs and the echo canceller 25 can be made simpler or less Computing power required.
  • single-channel echo cancellation requires only a single audio reference signal, namely the mono audio signal, and only needs to estimate an acoustic impulse response. As a result, system resources are saved in the echo cancellation, which are available for example for the speech recognizer 27.
  • the thus adjusted speech audio signal is supplied to the speech recognizer 27 and processed there to extract corresponding commands and instructions from the spoken speech.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Acoustics & Sound (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Computational Linguistics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Stereophonic System (AREA)

Description

Die vorliegende Erfindung betrifft ein Verfahren für eine Audiosignalverarbeitung in einem Fahrzeug und eine entsprechende Audiosignalverarbeitungsvorrichtung für ein Fahrzeug. Die vorliegende Erfindung betrifft insbesondere eine Audiosignalverarbeitung mit einer Echokompensation, beispielsweise für eine Sprachverarbeitung.The present invention relates to a method for audio signal processing in a vehicle and a corresponding audio signal processing apparatus for a vehicle. In particular, the present invention relates to audio signal processing with echo cancellation, for example for speech processing.

In Fahrzeugen, wie zum Beispiel Personenkraftwagen oder Lastkraftwagen, werden Sprachdialogsysteme zur Unterstützung des Fahrers oder der Insassen verwendet. Sprachdialogsysteme dienen beispielsweise dazu, elektronische Geräte ohne die Notwendigkeit einer haptischen Bedienoperation zu steuern. Die elektronischen Geräte können beispielsweise einen Fahrzeugcomputer oder ein Multimediasystem des Fahrzeugs umfassen. Gesprochene Sprache von dem Fahrer oder Insassen wird über ein Freisprechmikrofon empfangen und einer Spracherkennung zugeführt.In vehicles, such as passenger cars or trucks, speech dialogue systems are used to assist the driver or occupants. For example, speech dialogue systems serve to control electronic devices without the need for a haptic operator operation. The electronic devices may include, for example, a vehicle computer or a multimedia system of the vehicle. Spoken speech from the driver or occupant is received via a hands-free microphone and fed to speech recognition.

Eine Anwendung von Mikrofonen im Fahrzeuginnenraum, für z.B. Sprachbedienung, Telefonie oder Fahrzeuginnenraumkommunikation, kann potenziell durch eine akustische Einkopplung von Lautsprecherausgaben des Fahrzeug-Soundsystems beeinträchtigt werden. Dies kann im Fall einer Spracherkennung zu Erkennungsfehlern, bei Freisprechtelefonie zu Echos am fernen Ende und bei einer Fahrzeuginnenraumkommunikation zu Rückkopplungen führen. Die Konsequenzen sind je nach Anwendung eine gestörte Kommunikation, erhöhte Ablenkung oder sogar störende Geräusche und Echos.An application of microphones in the vehicle interior, for e.g. Voice control, telephony or vehicle interior communication can potentially be affected by acoustic coupling of loudspeaker outputs of the vehicle sound system. This can lead to recognition errors in the case of speech recognition, to far-end echo in the case of hands-free telephony, and to feedback in the case of vehicle interior communication. Depending on the application, the consequences are disturbed communication, increased distraction or even disturbing noises and echoes.

Wenn zum Beispiel während des Sprachdialogs im Fahrzeug gleichzeitig und fortlaufend Audiosignale über das Fahrzeug-Soundsystem wiedergegeben werden, gelangt ein Teil der Audiosignale als akustische Rückkopplung von den Lautsprechern in das Freisprechmikrofon und stört somit die Spracherkennung. Die über das Fahrzeug-Soundsystem wiedergegebenen Audiosignale können beispielsweise Musik, Verkehrsfunknachrichten, Radiosendungen, Ausgaben eines Navigationssystems oder die (künstliche) Sprache eines Sprachdialogsystems umfassen. Die Störung der Spracherkennung kann zu Erkennungsfehlern führen, welche den Dialog ineffizient machen können und eine erhöhte Ablenkung von der Fahraufgabe verursachen können. Dies kann beim Fahrer oder Insassen Unzufriedenheit oder Verärgerung auslösen.If, for example, during the speech dialogue in the vehicle, simultaneous and continuous audio signals are reproduced via the vehicle sound system, a part of the audio signals passes as acoustic feedback from the loudspeakers into the hands-free microphone and thus disturbs the speech recognition. The audio signals reproduced via the vehicle sound system can comprise, for example, music, traffic news, radio broadcasts, outputs of a navigation system or the (artificial) language of a speech dialogue system. The disturbance of the speech recognition can lead to recognition errors which can make the dialogue inefficient and an increased distraction from the driving task can cause. This can cause dissatisfaction or annoyance to the driver or occupant.

Eine einfache Lösung für das vorgenannte Problem besteht darin, während des Sprachdialogs oder Telefonats im Fahrzeug die Audiowiedergabe von beispielsweise einem Radio stumm zu schalten. Die Stummschaltung der Audiowiedergabe wird von Fahrzeugbenutzern jedoch häufig als störend und unnötig empfunden. Darüber hinaus können wichtige Informationen von beispielsweise einem Navigationssystem verpasst werden. Außerdem kann es sein, dass ein Fahrzeugbenutzer sich bei Rückfragen des Sprachdialogsystems und gleichzeitig stumm geschalteter Audiowiedergabe gedrängt fühlt, schnellstmöglich auf die Rückfragen des Sprachdialogsystems zu reagieren.A simple solution to the aforementioned problem is to mute the audio playback of, for example, a radio during the voice dialogue or telephone call in the vehicle. The muting of audio playback is often perceived by vehicle users as disturbing and unnecessary. In addition, important information can be missed by, for example, a navigation system. In addition, it may be that a vehicle user feels in queries of the speech dialogue system and simultaneously muted audio playback urged to respond as quickly as possible to the queries of the speech dialogue system.

Alternativ kann während des Sprachdialogs die Audiowiedergabelautstärke vorübergehend reduziert werden. Für den Spracherkenner ist das Ausmaß der Störung durch die Audiowiedergabe dann zwar geringer, jedoch im Allgemeinen noch so groß, dass auf eine weitere Bereinigung des Mikrofonsignals nicht verzichtet werden kann.Alternatively, during audio dialogue, the audio playback volume may be temporarily reduced. For the speech recognizer, the extent of the interference by the audio playback is then lower, but generally still so large that it can not be dispensed with a further cleanup of the microphone signal.

In begrenztem Maße können o.g. Einkopplungen auch durch konstruktiv-akustische Maßnahmen vermindert werden. Beispielsweise können Mikrofone mit einer geeigneten Richtcharakteristik verwendet werden, Mikrofone und Lautsprecher im Fahrzeuginneren geeignet relativ zueinander angeordnet werden oder akustische Gegebenheiten in dem Fahrzeug geeignet ausgenutzt werden.To a limited extent, o.g. Couplings are also reduced by constructive-acoustic measures. For example, microphones with a suitable directional characteristic can be used, microphones and loudspeakers inside the vehicle can be suitably arranged relative to one another, or acoustic conditions in the vehicle can be suitably exploited.

Da dies jedoch im Allgemeinen nicht ausreicht, werden Signalverarbeitungskomponenten zur Bereinigung der Mikrofonsignale eingesetzt. Dabei werden die von den Lautsprechern des Fahrzeug-Soundsystems in die Mikrofone eingekoppelten Signalanteile geschätzt und aus den Mikrofonsignalen entfernt. Derartige Verfahren werden als Echokompensation oder Echounterdrückung bezeichnet.However, since this is generally not enough, signal processing components are used to clean up the microphone signals. In the process, the signal components coupled into the microphones by the loudspeakers of the vehicle sound system are estimated and removed from the microphone signals. Such methods are referred to as echo cancellation or echo cancellation.

In diesem Zusammenhang offenbart die Veröffentlichung von Jacob Benesty et al: "A Hybrid Mono/Stereo Acoustic Echo Canceler" (APPLICATIONS OF SIGNAL PROCESSING TO AUDIO AND ACOUSTICS, 1997. 1997 IEEE ASSP WORKSHOP ON NEW PALTZ, NY, USA 19-22 OCT. 1997, NEW YORK, NY, USA, IEEE, US, 19 Oktober 1997 (1997-10-19), Seite 4pp, XP010248238, ISBN: 978-0-7803-3908-8 ) einen hybrid Mono/Stereo-Echokompensator, welcher auf einem psychoakustischen Prinzip basiert. Zwei Signale (links und rechts) werden tiefpassgefiltert, um die Frequenzen unterhalb fc zum Realisieren des Stereoeffekts zu erhalten, wobei die Übergangsfrequenz fc in der Größenordnung von 1 kHz liegt. Die Summe des linken und rechten Kanals wird hochpassgefiltert, um die Frequenzen über fc als ein monophones Signal zu erhalten. Das linke (bzw. rechte) Lautsprechersignal ist die Summe aus den niedrigen Frequenzen des linken (bzw. rechten) Kanals und hohen Frequenzen von beiden Kanälen. Zwei Echokompensatoren sind notwendig: ein Monoechokompensator für hohe Frequenzen und ein Stereoechokompensator für niedrige Frequenzen.In this connection, the publication of Jacob Benesty et al: "A Hybrid Mono / Stereo Acoustic Echo Canceler" (APPLICATIONS OF SIGNAL PROCESSING TO AUDIO AND ACOUSTICS, 1997. 1997 IEEE ASSP WORKSHOP ON NEW PALTZ, NY, USA OCTOBER 19-22, 1997, NEW YORK, NY) USA, IEEE, US, October 19, 1997 (1997-10-19), page 4pp, XP010248238, ISBN: 978-0-7803-3908-8 ) a hybrid mono / stereo echo canceller based on a psychoacoustic principle. Two signals (left and right) are low-pass filtered to obtain the frequencies below f c to realize the stereo effect. wherein the transition frequency f c is on the order of 1 kHz. The sum of the left and right channels is high-pass filtered to obtain the frequencies above f c as a monophonic signal. The left (or right) speaker signal is the sum of the low frequencies of the left (or right) channel and high frequencies of both channels. Two echo cancellers are needed: a high frequency mono echo canceller and a low frequency stereo echo canceller.

Die US 2012/232890 A1 betrifft ein Spracherkennungssystem, bei welchem ein n-kanaliger Systemklang mithilfe einer Monofonie-Einheit in ein einkanaliges akustisches Signal gewandelt wird. Das einkanalige akustische Signal wird an eine Gewichtungszuordnungseinheit und an einen Echokompensator ausgegeben. Der n-kanalige Systemklang wird über mehrere Lautsprecher in beispielsweise einem Fahrzeug ausgegeben. Mithilfe eines Mikrofons werden eine Spracheingabe von einem Benutzer sowie der über die Lautsprecher ausgegebene mehrkanalige Systemklang empfangen und dem Echokompensator zugeführt.The US 2012/232890 A1 relates to a speech recognition system in which an n-channel system sound is converted by means of a monophonic unit into a single-channel acoustic signal. The single-channel acoustic signal is output to a weighting unit and to an echo canceller. The n-channel system sound is output through multiple speakers in, for example, a vehicle. A microphone is used to receive voice input from a user as well as multichannel system sound output from the speakers and to the echo canceller.

Eine weit verbreitete Art der Echokompensation ist die lineare Echokompensation.A common type of echo cancellation is linear echo cancellation.

Bei der linearen Echokompensation wird angenommen, dass die Mikrofone, Lautsprecher und ihre jeweiligen Verstärker lineare Übertrager sind und dass sich somit die an einem bestimmten Mikrofon eingekoppelten Lautsprecher-Schallanteile im Mikrofonsignal linear überlagern. Ferner wird angenommen, dass sich diese Lautsprecher-Schallanteile als lineare Faltung des jeweiligen Lautsprecher-Quellsignals mit einer jeweiligen Impulsantwort ergeben. Jede dieser Impulsantworten bezieht sich auf ein bestimmtes Mikrofon-Lautsprecher-Paar und charakterisiert den gesamten elektroakustischen Übertragungsweg vom Lautsprecherquellsignal bis hin zum Mikrofonsignal. Damit werden in einer solchen Impulsantwort unter anderem folgende Größen abgebildet:

  • der Frequenz- und Phasengang des vor den Lautsprecher geschalteten Verstärkers,
  • der Frequenz- und Phasengang des Lautsprechers,
  • die räumliche Abstrahlcharakteristik des Lautsprechers,
  • der akustische Übertragungspfad vom Lautsprecher zum Mikrofon durch den Fahrzeuginnenraum, einschließlich Reflexionen, Beugung, Streuung, Absorption, usw.,
  • die räumliche Empfangscharakteristik des Mikrofons, sowie
  • der Frequenz- und Phasengang des Mikrofons.
In linear echo cancellation, it is assumed that the microphones, loudspeakers and their respective amplifiers are linear transducers and that, thus, the loudspeaker sound components coupled to a specific microphone superimpose linearly in the microphone signal. Furthermore, it is assumed that these loudspeaker sound components result in a linear convolution of the respective loudspeaker source signal with a respective impulse response. Each of these impulse responses refers to a particular pair of microphone-speakers and characterizes the entire electro-acoustic transmission path from the source sound source to the microphone signal. Thus, among other things, the following quantities are mapped in such an impulse response:
  • the frequency and phase response of the amplifier connected in front of the loudspeaker,
  • the frequency and phase response of the loudspeaker,
  • the spatial radiation characteristic of the loudspeaker,
  • the acoustic transmission path from the speaker to the microphone through the vehicle interior, including reflections, diffraction, scattering, absorption, etc.,
  • the spatial reception characteristics of the microphone, as well
  • the frequency and phase response of the microphone.

Diese Impulsantwort wird daher auch als LEM-Impulsantwort (Loudspeaker-Enclosure-Microphone) bezeichnet. Sie ist im Allgemeinen aufgrund von Veränderungen der Fahrzeuginnenraumgeometrie (Insassen und deren Bewegungen, bewegliche Teile, Beladung etc.) sowie der elektroakustischen Eigenschaften von Mikrofonen und Lautsprechern (abhängig von Temperatur, Luftdruck, Feuchtigkeit, Alter, etc.) zeitveränderlich.This impulse response is therefore also referred to as LEM impulse response (Loudspeaker Enclosure Microphone). It is generally due to changes in the Vehicle interior geometry (occupants and their movements, moving parts, loading, etc.) as well as the electro-acoustic properties of microphones and loudspeakers (depending on temperature, air pressure, humidity, age, etc.) are time-variable.

Ein Algorithmus zur linearen Echokompensation schätzt adaptiv die LEM-Impulsantwort für jedes mögliche Mikrofon-Lautsprecher-Paar. Auf der Grundlage der LEM-Impulsantwort werden dann die eingekoppelten Lautsprecher-Schallanteile in jedem Mikrofonsignal berechnet und davon subtrahiert. Die Adaptionsgeschwindigkeit und effektive Echounterdrückung sind begrenzt und stehen im Allgemeinen zueinander in Konkurrenz.A linear echo cancellation algorithm adaptively estimates the LEM impulse response for each possible microphone-speaker pair. Based on the LEM impulse response, the injected loudspeaker sound components in each microphone signal are then calculated and subtracted therefrom. The rate of adaptation and effective echo cancellation are limited and generally in competition with each other.

Verschiedene verbesserte Techniken zur Echokompensation oder Echounterdrückung sind in dem Stand der Technik bekannt, um beispielsweise die Echokompensation zu vereinfachen und somit die benötigte Rechenleistung zu verringern. Dazu offenbart beispielsweise die EP 1936939 A1 eine Echokompensation, bei der das Mikrofonsignal in Subbandsignale aufgeteilt und einer Unterabtastung unterzogen wird. Ein Referenzaudiosignal wird über einen Lautsprecher ausgegeben. Das Referenzaudiosignal wird ebenfalls einer Unterabtastung unterzogen und unterabgetastete Subbandsignale des Referenzaudiosignals gespeichert. Ferner werden Echos in den Mikrofonsubbandsignalen geschätzt und die geschätzten Echos werden von den Mikrofonsubbandsignalen abgezogen, um verbesserte Mikrofonsubbandsignale zu erhalten.Various improved techniques for echo cancellation or echo cancellation are known in the art, for example to simplify echo cancellation and thus reduce the required computational power. For example, the disclosed EP 1936939 A1 an echo cancellation in which the microphone signal is divided into subband signals and sub-sampled. A reference audio signal is output through a speaker. The reference audio signal is also sub-sampled and sub-sampled subband signals of the reference audio signal are stored. Furthermore, echoes in the microphone subband signals are estimated and the estimated echoes are subtracted from the microphone subband signals to obtain improved microphone subband signals.

Bei der Echokompensation ist jedoch eine häufig vorhandene Mehrkanaligkeit des auszugebenden Audiosignals problematisch. Das mehrkanalige Audiosignal kann beispielsweise ein Stereosignal oder ein Surroundsignal in dem Fahrzeug sein.In echo cancellation, however, a frequently present multichannel nature of the audio signal to be output is problematic. The multi-channel audio signal may be, for example, a stereo signal or a surround signal in the vehicle.

Für den Fall mehrerer Audioquellsignale aus mehreren Lautsprechern kommt neben der erhöhten Rechenkomplexität der Algorithmen folgendes Problem hinzu: Aufgrund von Korrelationen zwischen den verschiedenen Audioquellsignalen ist das Schätzproblem mathematisch unterbestimmt. Als Konsequenz kann bei plötzlichem Auftreten von Audioquellsignalen die Wirksamkeit der Echokompensation stark vermindert sein. Es kann sogar auftreten, dass die LEM-Schätzung divergiert, beispielsweise wenn es zu Veränderungen im Surroundklangbild kommt. Dies kann beispielsweise auftreten, wenn sogenannte Phantomschallquellen im Surround-Panorama erscheinen, verschwinden oder verschoben werden.In the case of multiple audio source signals from multiple speakers, in addition to the increased computational complexity of the algorithms, the following problem is added: Due to correlations between the different audio source signals, the estimation problem is mathematically underdetermined. As a consequence, in the event of a sudden appearance of audio source signals, the effectiveness of echo cancellation may be greatly reduced. It may even occur that the LEM estimate diverges, for example when there are changes in the surround sound image. This can occur, for example, when so-called phantom sound sources appear, disappear or shift in the surround panorama.

Es existieren verschiedene Ansätze, hiermit umzugehen, die jedoch entweder zu hörbaren Verzerrungen führen oder sehr rechenintensiv sind (Watermarking, Kalmanfilter-Lösungen).There are several approaches to dealing with this, but they either lead to audible distortions or are very computationally intensive (watermarking, Kalman filter solutions).

Weiterhin ist in diesem Zusammenhang aus der DE 102008027848 A1 beispielsweise ein Echounterdrücker bekannt, welcher mit einer Tonausgabevorrichtung mit einer mehrkanaligen Audioeinheit zusammenarbeitet. Die Tonausgabevorrichtung sendet Ausgabetonsignale als Analogsignale mehrerer Kanäle über mehrere Lautsprecher aus. Ein Mikrofon erfasst einen Außenton und erzeugt ein Eingabetonsignal als ein Analogsignal. Der Außenton umfasst die Ausgabetonsignale als Echo. Der Echounterdrücker besitzt eine Echoauslöschfunktion zum Entfernen des Echos aus dem Eingabetonsignal. Dazu empfängt der Echounterdrücker die Ausgabetonsignale von der Tonausgabevorrichtung. Eine derartige Lösung zur Kompensation mehrkanaliger akustischer Echoquellen ist technisch jedoch sehr komplex und erfordert eine hohe Rechenleistung. Ferner existieren für Kanalzahlen größer als zwei keine expliziten Lösungen.Furthermore, in this context from the DE 102008027848 A1 For example, an echo canceller is known which cooperates with a sound output device with a multi-channel audio unit. The sound output device outputs output sound signals as analog signals of multiple channels through a plurality of speakers. A microphone detects an outside sound and generates an input sound signal as an analog signal. The outside sound echoes the output sound signals. The echo canceller has an echo cancellation function for removing the echo from the input tone signal. For this purpose, the echo canceller receives the output sound signals from the sound output device. Such a solution for the compensation of multi-channel acoustic echo sources is technically very complex and requires a high computing power. Furthermore, for channel numbers greater than two, there are no explicit solutions.

Eine weitere Möglichkeit besteht in einer verbesserten Trennung von Sprachsignalen und allgemeinen Störsignalen. Die allgemeinen Störsignale können auch mehrkanalige Audiowiedergaben umfassen. Dies wird beispielsweise in der DE 102009051508 A1 betrachtet. Zur Reduzierung von Störsignalen bei der Spracherkennung wird statt einem einzelnen Mikrofon ein Mikrofonarray verbaut. Durch das Mikrofonarray wird ein Mehrkanalsprachsignal aufgenommen, welches anstatt eines einfachen Sprachsignals zu einer Echokompensationseinheit geleitet wird. Das durch das Mikrofonarray aufgenommene Mehrkanalsprachsignal wird vor der Eingabe in die Echokompensationseinheit in einer dem Mikrofonarray nachgeschalteten Einheit zur Verarbeitung der Mikrofonsignale durch verzögertes Aufsummieren der Signale nachbearbeitet. Hierdurch werden die Signale der autorisierten Sprecher separiert und alle anderen Sprechersignale und Störsignale reduziert. Außerdem wertet die Echokompensationseinheit die Laufzeit der verschiedenen Kanäle des Mehrkanalsprachsignals aus und entfernt alle Anteile des Signals, die entsprechend ihrer Laufzeit nicht vom Ort des autorisierten Sprechers ausgehen. Die Verwendung eines Mikrofonarrays oder mehrerer Mikrofone erhöht jedoch die Kosten, benötigt mehr Einbauraum und erfordert leistungsstarke Rechenressourcen.Another possibility is an improved separation of speech signals and general interference signals. The general interfering signals may also include multichannel audio playback. This is for example in the DE 102009051508 A1 considered. To reduce interference in speech recognition, a microphone array is used instead of a single microphone. Through the microphone array a multi-channel speech signal is recorded, which is passed instead of a simple speech signal to an echo cancellation unit. The multichannel speech signal recorded by the microphone array is post-processed before being input to the echo canceling unit in a unit downstream of the microphone array for processing the microphone signals by delayed summation of the signals. As a result, the signals of the authorized speakers are separated and all other speaker signals and noise are reduced. In addition, the echo canceling unit evaluates the propagation time of the various channels of the multi-channel speech signal and removes all portions of the signal that do not emanate from the authorized speaker's location according to their propagation time. However, using a microphone array or multiple microphones increases costs, requires more installation space, and requires powerful computational resources.

Eine Aufgabe der vorliegenden Erfindung ist es daher, eine zuverlässige Spracheingabe in einem Fahrzeug bei gleichzeitiger Wiedergabe eines mehrkanaligen Audiosignals zu ermöglichen. Zusätzliche Kosten oder Aufwendungen für beispielsweise zusätzliche Mikrofone oder leistungsstarke Signalverarbeitungseinheiten sollen dabei vermieden werden.It is therefore an object of the present invention to enable reliable voice input in a vehicle while reproducing a multi-channel audio signal. Additional costs or expenses for example, additional microphones or powerful signal processing units should be avoided.

Gemäß der vorliegenden Erfindung wird diese Aufgabe durch ein Verfahren für eine Audiosignalverarbeitung in einem Fahrzeug nach Anspruch 1 und eine Audiosignalverarbeitungsvorrichtung für ein Fahrzeug nach Anspruch 7 gelöst. Die abhängigen Ansprüche definieren Ausführungsformen der Erfindung.According to the present invention, this object is achieved by a method for audio signal processing in a vehicle according to claim 1 and an audio signal processing apparatus for a vehicle according to claim 7. The dependent claims define embodiments of the invention.

Gemäß der vorliegenden Erfindung wird ein Verfahren für eine Audiosignalverarbeitung in einem Fahrzeug bereitgestellt. Bei dem Verfahren wird ein Monoaudiosignal auf der Grundlage eines mehrkanaligen Audioquellsignals erzeugt. Das mehrkanalige Audioquellsignal ist beispielsweise ein Stereosignal oder ein Surroundsignal, welches in dem Fahrzeug über mehrere Lautsprecher des Fahrzeugs auszugeben ist. Das Monoaudiosignal wird auf einen Frequenzbereich zwischen einer vorgegebenen unteren Frequenz und einer vorgegebenen oberen Frequenz begrenzt. Das Monoaudiosignal kann beispielsweise mit einem Bandpassfilter auf den Frequenzbereich zwischen der vorgegebenen unteren Frequenz und der vorgegebenen oberen Frequenz begrenzt werden. Durch das Begrenzen des Monoaudiosignals auf den Frequenzbereich wird ein begrenztes Monoaudiosignal erzeugt.In accordance with the present invention, a method for audio signal processing in a vehicle is provided. In the method, a mono audio signal is generated based on a multi-channel audio source signal. The multi-channel audio source signal is, for example, a stereo signal or a surround signal which is to be output in the vehicle via a plurality of loudspeakers of the vehicle. The mono audio signal is limited to a frequency range between a predetermined lower frequency and a predetermined upper frequency. The mono audio signal can be limited, for example, with a bandpass filter to the frequency range between the predetermined lower frequency and the predetermined upper frequency. Limiting the mono audio signal to the frequency domain produces a limited mono audio signal.

Das begrenzte Monoaudiosignal wird über die mehreren Lautsprecher in dem Fahrzeug ausgegeben. Wenn nun über ein Mikrofon ein Sprachaudiosignal von einem Fahrzeuginsassen oder einem Fahrer des Fahrzeugs empfangen wird, enthält dieses Sprachaudiosignal das über die mehreren Lautsprecher ausgegebene begrenzte Monoaudiosignal. Ein Einfluss dieses über die mehreren Lautsprecher ausgegebenen begrenzten Monoaudiosignals auf das über das Mikrofon empfangene Sprachaudiosignal wird mittels des begrenzten Monoaudiosignals kompensiert. Beispielsweise kann eine Echokompensation durchgeführt werden, welche lediglich das Monoaudiosignal berücksichtigt. Eine komplexe Echokompensation unter Berücksichtigung eines mehrkanaligen Audiosignals ist daher nicht erforderlich. Stattdessen ist lediglich eine einkanalige Echokompensation erforderlich, welche mit verhältnismäßig geringer Rechenleistung realisiert werden kann.The limited mono audio signal is output through the plurality of speakers in the vehicle. Now, when a voice audio signal from a vehicle occupant or a driver of the vehicle is received via a microphone, this voice audio signal contains the limited mono audio signal output via the plurality of loudspeakers. An influence of this limited mono audio signal output via the several loudspeakers on the speech audio signal received via the microphone is compensated by means of the limited mono audio signal. For example, an echo compensation can be performed, which takes into account only the mono audio signal. A complex echo cancellation taking into account a multi-channel audio signal is therefore not required. Instead, only a single-channel echo cancellation is required, which can be realized with relatively low processing power.

Die Echokompensation unter Berücksichtigung von nur einem Echosignal (Monoaudiosignal) ist sehr zuverlässig, auch wenn das Monoaudiosignal über mehrere verschiedene Lautsprecher ausgegeben wird, da mit einem Monoaudiosignal keine Veränderungen im Mehrkanalklangbild auftreten können. Somit kann das störende Monoaudiosignal zu einem großen Teil oder vollständig aus dem Sprachaudiosignal entfernt werden.The echo cancellation taking into account only one echo signal (mono audio signal) is very reliable, even if the mono audio signal is output through several different speakers, as with a mono audio signal no changes in the multi-channel sound image can occur. Thus, the interfering mono audio signal can be largely or completely removed from the voice audio signal.

Die vorgegebene untere Frequenz weist einen Wert im Bereich von 100 Hz bis 300 Hz auf und die vorgegebene obere Frequenz weist einen Wert im Bereich von 4kHz bis 8kHz auf. Ein Spracherkenner, welcher beispielsweise für eine Sprachsteuerung oder eine Spracheingabe in einem Fahrzeug verwendet wird, wertet zur Erkennung der Spracheingaben von einem Benutzer Audiosignale in vielen Fällen in nur einem begrenzten Frequenzbereich von beispielsweise 100 Hz bis 8kHz aus. Daher ist eine Echokompensation nur in diesem begrenzten Frequenzbereich erforderlich. Vorzugsweise beträgt die vorgegebene untere Frequenz daher 100 Hz und die vorgegebene obere Frequenz 8kHz. Dadurch kann dem Spracherkenner in dem für ihn relevanten begrenzten Frequenzbereich ein ungestörtes Sprachsignal bereitgestellt werden.The predetermined lower frequency has a value in the range of 100 Hz to 300 Hz, and the predetermined upper frequency has a value in the range of 4 kHz to 8 kHz. A speech recognizer, which is used, for example, for voice control or voice input in a vehicle, in many cases evaluates audio signals in a limited frequency range of, for example, 100 Hz to 8 kHz in order to recognize the voice input from a user. Therefore, echo cancellation is required only in this limited frequency range. Therefore, the predetermined lower frequency is preferably 100 Hz and the predetermined upper frequency is 8 kHz. As a result, an undisturbed speech signal can be provided to the speech recognizer in the limited frequency range relevant for him.

Um dennoch eine Wirkung einer mehrkanaligen Audiowiedergabe aufrecht zu erhalten, werden bei einer Ausführungsform des Verfahrens zusätzlich mehrere begrenzte kanalspezifische Audiosignale in Abhängigkeit von dem mehrkanaligen Audioquellsignal erzeugt. Ein kanalspezifisches Audiosignal betrifft beispielsweise ein Audiosignal, welches von der mehrkanaligen Audiosignalquelle speziell für einen dem jeweiligen Kanal zugeordneten Lautsprecher bestimmt ist. Bei einem Stereoquellsignal kann dies beispielsweise ein Audiosignal für den rechten Lautsprecher oder ein Audiosignal für den linken Lautsprecher umfassen. Ein jeweiliges begrenztes kanalspezifisches Audiosignal der mehreren begrenzten kanalspezifischen Audiosignale ist daher einem jeweiligen Audiosignal des mehrkanaligen Audioquellsignals zugeordnet. Ein jeweiliges begrenztes kanalspezifisches Audiosignal ist auf einen Frequenzbereich begrenzt, welcher nur noch Frequenzen unterhalb der vorgegebenen unteren Frequenz und Frequenzen oberhalb der vorgegebenen oberen Frequenz umfasst. Ein jeweiliges begrenztes kanalspezifisches Audiosignal wird durch eine entsprechende Frequenzbegrenzung aus dem jeweils zugeordneten Audiosignal des mehrkanaligen Audioquellsignals gebildet. Anders ausgedrückt werden die Audiosignale des mehrkanaligen Audiosignals jeweils derart begrenzt oder gefiltert, dass sie nur noch Frequenzen unter der vorgegebenen unteren Frequenz und/oder Frequenzen über der vorgegebenen oberen Frequenz umfassen. Die mehreren begrenzten kanalspezifischen Audiosignale werden über die mehreren Lautsprecher in dem Fahrzeug ausgegeben, sodass die Wirkung einer mehrkanaligen Audiowiedergabe erzielt werden kann, beispielsweise eine Stereowiedergabe oder eine Surroundwiedergabe. Zusammenfassend wird eine Audiowiedergabe in dem Fahrzeug so modifiziert, dass in dem Frequenzbereich zwischen der vorgegebenen unteren Frequenz und der vorgegebenen oberen Frequenz das mehrkanalige Audioquellsignal einkanalig (Mono) wiedergegeben wird und in dem übrigen Frequenzbereich mehrkanalig.In order nevertheless to maintain an effect of a multi-channel audio reproduction, in one embodiment of the method additionally a plurality of limited channel-specific audio signals are generated as a function of the multi-channel audio source signal. A channel-specific audio signal relates, for example, to an audio signal which is determined by the multi-channel audio signal source specifically for a loudspeaker associated with the respective channel. For a stereo source signal, this may include, for example, an audio signal for the right speaker or an audio signal for the left speaker. A respective limited channel-specific audio signal of the plurality of limited channel-specific audio signals is therefore assigned to a respective audio signal of the multi-channel audio source signal. A respective limited channel-specific audio signal is limited to a frequency range which only includes frequencies below the predetermined lower frequency and frequencies above the predetermined upper frequency. A respective limited channel-specific audio signal is formed by a corresponding frequency limitation from the respectively assigned audio signal of the multi-channel audio source signal. In other words, the audio signals of the multi-channel audio signal are respectively limited or filtered so that they only include frequencies below the predetermined lower frequency and / or frequencies above the predetermined upper frequency. The plurality of limited channel-specific audio signals are output through the plurality of speakers in the vehicle, so that the effect of multi-channel audio reproduction such as stereo reproduction or surround reproduction can be achieved. In summary, an audio reproduction in the vehicle is modified so that in the frequency range between the predetermined lower frequency and the predetermined upper frequency, the multi-channel audio source signal is reproduced in one channel (mono) and in the remaining frequency range multi-channel.

Das Monoaudiosignal und die mehreren begrenzten kanalspezifischen Audiosignale können beispielsweise gemäß nachfolgender Ausführungsform aus dem mehrkanaligen Audioquellsignal erzeugt werden. Bei dieser Ausführungsform wird das mehrkanalige Audioquellsignal in einen auf allen Kanälen gleichen Mittensignalanteil und einen jeweiligen Seitensignalanteil pro Audiokanal des mehrkanaligen Audioquellsignal aufgeteilt. Aus dem Mittensignalanteil wird das begrenzte Monoaudiosignal erzeugt und aus den jeweiligen Seitensignalanteilen werden die mehreren begrenzten kanalspezifischen Audiosignale erzeugt. Der Mittensignalanteil kann beispielsweise direkt als Monoaudiosignal verwendet werden oder geeignet skaliert als Monoaudiosignal verwendet werden. Ebenso können die Seitensignalanteile direkt als die begrenzten kanalspezifischen Audiosignale verwendet werden oder in geeignet skalierter Form. Insbesondere bei einem Stereosignal kann der Mittensignalanteil beispielsweise aus der Summe des rechten und linken Audioquellsignals gebildet werden. Die Seitensignalanteile können gemeinsam in einem Differenzsignal aus der Differenz zwischen dem rechten und linken Audioquellsignals codiert und weiterverarbeitet werden. Dadurch können insbesondere bei der Verarbeitung eines Stereoquellsignals der Mittensignalanteil und die Seitensignalanteile auf einfache Art und Weise erzeugt und verarbeitet werden.For example, the mono audio signal and the plurality of limited channel-specific audio signals may be generated from the multi-channel audio source signal according to the following embodiment. In this embodiment, the multi-channel audio source signal is divided into a center signal component which is the same on all channels and a respective side signal component per audio channel of the multi-channel audio source signal. The limited mono audio signal is generated from the center signal component and the plurality of limited channel-specific audio signals are generated from the respective side signal components. The mid-signal component, for example, can be used directly as a mono audio signal or suitably scaled used as a mono audio signal. Likewise, the side signal components may be used directly as the limited channel specific audio signals or in suitably scaled form. In particular, in the case of a stereo signal, the center signal component can be formed, for example, from the sum of the right and left audio source signals. The side signal components may be coded together in a difference signal from the difference between the right and left audio source signals and further processed. As a result, in particular when processing a stereo source signal, the center signal component and the side signal components can be generated and processed in a simple manner.

Bei einer weiteren Ausführungsform wird der Mittensignalanteil durch eine Mittelwertbildung von jeweiligen Abtastwerten der Audiokanäle des mehrkanaligen Audioquellsignals gebildet. Die jeweiligen Seitensignalanteile werden durch Subtraktion des Mittensignalanteils von den jeweiligen Audiosignalen des mehrkanaligen Audioquellsignals gebildet. Diese Erzeugung des Mittensignalanteils und der Seitensignalanteile ist für Audioquellsignale mit einer beliebigen Anzahl von Kanälen möglich. Darüber hinaus ist eine Implementierung in beispielsweise einem digitalen Signalprozessor auf einfache Art und Weise realisierbar.In another embodiment, the center signal component is formed by averaging respective samples of the audio channels of the multi-channel audio source signal. The respective side signal components are formed by subtracting the center signal component from the respective audio signals of the multi-channel audio source signal. This generation of the center signal component and the side signal components is possible for audio source signals with an arbitrary number of channels. In addition, an implementation in, for example, a digital signal processor can be realized in a simple manner.

Bei einer weiteren Ausführungsform des Verfahrens wird das über das Mikrofon empfangene Sprachaudiosignal auf einen Frequenzbereich zwischen der vorgegebenen unteren Frequenz und der vorgegebenen oberen Frequenz begrenzt. Die Echokompensation wird auf das so begrenzte Sprachaudiosignal unter Verwendung des begrenzten Monoaudiosignals angewendet. Somit wird der Einfluss des über die mehreren Lautsprecher ausgegebenen begrenzten Monoaudiosignals auf das begrenzte Sprachaudiosignal kompensiert. Da der Spracherkenner im Allgemeinen nur in dem Frequenzbereich zwischen der vorgegebenen unteren Frequenz und der vorgegebenen oberen Frequenz arbeitet, ist eine Echokompensation in einem darauf begrenzten Sprachaudiosignal ausreichend. Ferner werden Störsignale außerhalb dieses Frequenzbereichs bereits vor der Echokompensation eliminiert und haben daher keinen Einfluss auf die Echokompensation und die Spracherkennung, wodurch sowohl die Echokompensation als auch die Spracherkennung zuverlässiger arbeiten können.In a further embodiment of the method, the speech audio signal received via the microphone is limited to a frequency range between the predetermined lower frequency and the predetermined upper frequency. The echo cancellation is applied to the thus limited voice audio signal using the limited mono audio signal. Thus, the influence of the limited mono audio signal outputted through the plural speakers to the limited voice audio signal is compensated. Since the speech recognizer operates generally only in the frequency range between the predetermined lower frequency and the predetermined upper frequency, echo cancellation in a speech audio signal limited thereto is sufficient. Furthermore, spurious signals outside this frequency range are already eliminated before the echo cancellation and have therefore, does not affect echo cancellation and speech recognition, which makes both echo cancellation and speech recognition more reliable.

In einigen Fällen ist die Wiedergabe eines Audiosignals für einige Insassen des Fahrzeugs wichtiger als für andere. Beispielsweise sind Audioausgaben eines Navigationssystems für den Fahrer wichtiger als für die übrigen Insassen, wohingegen Audioausgaben eines im Fond des Fahrzeugs wiedergegebenen Videos für Fahrzeuginsassen im Fond wichtiger sind als für den Fahrer und Beifahrer. Gemäß einer Ausführungsform können daher mehrere den jeweiligen Lautsprechern zugeordnete Gewichtungsfaktoren in Abhängigkeit von dem mehrkanaligen Audioquellsignal erzeugt werden. Das begrenzte Monoaudiosignal wird für jeden Lautsprecher mit dem dem jeweiligen Lautsprecher zugeordneten Gewichtungsfaktor gewichtet. Dadurch kann ein Schwerpunkt der Audioausgabe in dem Fahrzeug geeignet verlagert werden.In some cases, the reproduction of an audio signal is more important to some occupants of the vehicle than to others. For example, audio outputs of a navigation system are more important to the driver than to the remaining occupants, whereas audio outputs of a video displayed in the rear of the vehicle are more important for rear passengers than for the driver and front passenger. Thus, according to one embodiment, a plurality of weighting factors associated with the respective loudspeakers may be generated in response to the multi-channel audio source signal. The limited mono audio signal is weighted for each loudspeaker with the weighting factor assigned to the respective loudspeaker. Thereby, a center of gravity of the audio output in the vehicle can be appropriately shifted.

Solange die Gewichtungsfaktoren im Wesentlichen statisch sind, hat die gewichtete Ausgabe keinen Einfluss auf die Qualität der Echokompensation. Wenn die Gewichtung geändert wird, kann sich die Echokompensation in verhältnismäßig kurzer Zeit, beispielsweise innerhalb von wenigen Sekunden oder Minuten, auf die neue Gewichtung einstellen. In dem oben genannten Beispiel der Audioausgaben des Navigationssystems kann bei einem Fahrzeug mit beispielsweise vier Lautsprechern statt einer gleich verteilten Ausgabe des Monoaudiosignals über die vier Lautsprecher folgende Gewichtung verwendet werden. Der Lautsprecher im Bereich des Fahrers kann beispielsweise 70 % des Monoaudiosignals ausgeben und die übrigen drei Lautsprecher können beispielsweise jeweils nur 10 % des Monoaudiosignals ausgeben.As long as the weighting factors are essentially static, the weighted output will not affect the quality of the echo cancellation. When the weight is changed, the echo cancellation may adjust to the new weight in a relatively short time, for example within a few seconds or minutes. In the above example of the audio outputs of the navigation system, in a vehicle having, for example, four speakers instead of an equally distributed output of the mono audio signal via the four speakers, the following weighting may be used. For example, the speaker in the driver's area may output 70% of the mono audio signal, and the remaining three speakers may output only 10% of the mono audio signal, for example.

Gemäß der vorliegenden Erfindung wird weiterhin eine Audiosignalverarbeitungsvorrichtung für ein Fahrzeug bereitgestellt. Die Audiosignalverarbeitungsvorrichtung ist in der Lage, ein Monoaudiosignal auf der Grundlage eines mehrkanaligen Audioquellsignals zu erzeugen. Dazu kann die Audiosignalverarbeitungsvorrichtung beispielsweise eine Summiervorrichtung aufweisen. Die Audiosignalverarbeitungsvorrichtung ist ferner in der Lage, das Monoaudiosignal auf einen Frequenzbereich zwischen einer vorgegebenen unteren Frequenz und einer vorgegebenen oberen Frequenz zu begrenzen. Dies kann beispielsweise mit einem Bandpassfilter realisiert werden. Die vorgegebene untere Frequenz weist einen Wert im Bereich von 100 Hz bis 300 Hz auf und die vorgegebene obere Frequenz weist einen Wert im Bereich von 4kHz bis 8kHz auf. Das begrenzte Monoaudiosignal wird über mehrere Lautsprecher in dem Fahrzeug ausgegeben. Ferner wird das begrenzte Monoaudiosignal zu einer Kompensationsvorrichtung ausgegeben, beispielsweise zu einer Echokompensationsvorrichtung. Die Kompensationsvorrichtung dient dazu, einen Einfluss des über die mehreren Lautsprecher ausgegebenen begrenzten Monoaudiosignals auf ein in dem Fahrzeug über ein Mikrofon empfangenes Sprachaudiosignal mittels des begrenzten Monoaudiosignals zu kompensieren. Die Audiosignalverarbeitungsvorrichtung ist daher zur Durchführung des zuvor beschriebenen Verfahrens und seiner Ausführungsformen geeignet und umfasst daher auch die zuvor beschriebenen Vorteile.According to the present invention, there is further provided an audio signal processing apparatus for a vehicle. The audio signal processing apparatus is capable of generating a mono audio signal based on a multi-channel audio source signal. For this purpose, the audio signal processing device may have, for example, a summation device. The audio signal processing apparatus is further capable of limiting the mono audio signal to a frequency range between a predetermined lower frequency and a predetermined upper frequency. This can be realized for example with a bandpass filter. The predetermined lower frequency has a value in the range of 100 Hz to 300 Hz, and the predetermined upper frequency has a value in the range of 4 kHz to 8 kHz. The limited mono audio signal is output through several speakers in the vehicle. Furthermore, the limited mono audio signal is output to a compensation device, for example to a Echo canceller. The compensation device serves to compensate for an influence of the limited mono audio signal output via the plurality of loudspeakers on a voice audio signal received in the vehicle via a microphone by means of the limited mono audio signal. The audio signal processing device is therefore suitable for carrying out the method described above and its embodiments and therefore also comprises the advantages described above.

Die vorliegende Erfindung wird nachfolgend anhand der beigefügten Figuren im Detail beschrieben werden.

  • Figur 1 zeigt schematisch ein Fahrzeug mit einer Audiosignalverarbeitungsvorrichtung gemäß einer Ausführungsform der vorliegenden Erfindung.
  • Figur 2 zeigt schematisch ein Audiowiedergabesystem und ein Spracherkennungssystem in Verbindung mit einer Audiosignalverarbeitungsvorrichtung gemäß einer Ausführungsform der vorliegenden Erfindung.
  • Figur 3 zeigt schematisch ein Verfahren für eine Audiosignalverarbeitung in einem Fahrzeug gemäß einer Ausführungsform der vorliegenden Erfindung.
The present invention will be described below in detail with reference to the accompanying drawings.
  • FIG. 1 schematically shows a vehicle with an audio signal processing apparatus according to an embodiment of the present invention.
  • FIG. 2 schematically shows an audio playback system and a speech recognition system in connection with an audio signal processing apparatus according to an embodiment of the present invention.
  • FIG. 3 schematically shows a method for audio signal processing in a vehicle according to an embodiment of the present invention.

Die vorliegende Erfindung wird nachfolgend unter Bezugnahme auf die Figuren 1-3 im Detail beschrieben werden. In Figur 1 wird zunächst das Umfeld einer erfindungsgemäßen Audiosignalverarbeitungsvorrichtung 15 in einem Fahrzeug 10 beschrieben. In Figur 2 werden Details der Audiosignalverarbeitungsvorrichtung 15 in Verbindung mit weiteren Komponenten des Fahrzeugs 10 beschrieben werden. Figur 3 zeigt schließlich schematisch die Arbeitsweise der Audiosignalverarbeitungsvorrichtung 15. Gleiche Bezugszeichen in den Figuren betreffen gleiche oder ähnliche Komponenten.The present invention will be described below with reference to FIGS Figures 1-3 will be described in detail. In FIG. 1 First, the environment of an audio signal processing device 15 according to the invention in a vehicle 10 will be described. In FIG. 2 Details of the audio signal processing device 15 will be described in conjunction with other components of the vehicle 10. FIG. 3 Finally, Fig. 12 schematically shows the operation of the audio signal processing apparatus 15. Like reference numerals in the figures refer to the same or similar components.

Figur 1 zeigt ein Fahrzeug 10 in einer Draufsicht. Das Fahrzeug 10 umfasst ein Spracherkennungssystem 11. Mithilfe des Spracherkennungssystems 11 können gesprochene Befehle oder Anweisungen von Insassen des Fahrzeugs 10 erfasst, verarbeitet und ausgeführt werden. Beispielsweise können Konfigurationseinstellungen des Fahrzeugs 10 oder eines Multimediasystems des Fahrzeugs 10 über entsprechende Anweisungen geändert werden. Beispielsweise kann eine Audiosignalquelle, wie zum Beispiel CD oder Radio, gewählt werden. Ferner kann beispielsweise ein bestimmter Radiosender ausgewählt werden oder ein Titel einer CD. Ferner kann mit entsprechenden Anweisungen eine Telefonverbindung zu einem gewünschten Teilnehmer aufgebaut werden oder ein Navigationsziel in einem Navigationssystem des Fahrzeugs 10 eingestellt werden. Dazu werden beispielsweise entsprechende Befehle oder Anweisungen von einem Fahrer 12 des Fahrzeugs 10 über ein Mikrofon 13 empfangen. Ein gesprochener Befehl des Fahrers 12 wird von den Mikrofon 13 als Sprachaudiosignal an eine Audiosignalverarbeitungsvorrichtung 15 weitergeleitet. Die Funktionsweise der Audiosignalverarbeitungsvorrichtung 15 wird später unter Bezugnahme auf Figur 2 im Detail beschrieben werden. Nach der Bearbeitung des Sprachaudiosignals in der Audiosignalverarbeitungsvorrichtung 15 wird das bearbeitete Sprachaudiosignal dem Spracherkennungssystem 11 zugeführt. Das Spracherkennungssystem 11 wertet das Sprachaudiosignal aus und erkennt darin enthaltene Befehle und Anweisungen und führt diese aus. Das Spracherkennungssystem kann mit einem sogenannten Dialogsystem gekoppelt sein, welches über Fragen und Antworten einen Dialog mit dem Fahrer führen kann. FIG. 1 shows a vehicle 10 in a plan view. The vehicle 10 includes a voice recognition system 11. Using the voice recognition system 11, voice commands or instructions from occupants of the vehicle 10 may be captured, processed, and executed. For example, configuration settings of the vehicle 10 or a multimedia system of the vehicle 10 may be changed via appropriate instructions. For example, an audio signal source, such as CD or radio, can be selected. Furthermore, for example, a specific radio station can be selected or a title of a CD. Furthermore, with appropriate instructions a telephone connection to a desired participants are set up or a navigation destination can be set in a navigation system of the vehicle 10. For this purpose, for example, corresponding commands or instructions are received by a driver 12 of the vehicle 10 via a microphone 13. A spoken command of the driver 12 is forwarded by the microphone 13 as a speech audio signal to an audio signal processing device 15. The operation of the audio signal processing device 15 will be described later with reference to FIG FIG. 2 will be described in detail. After the processing of the speech audio signal in the audio signal processing device 15, the processed speech audio signal is supplied to the speech recognition system 11. The speech recognition system 11 evaluates the speech audio signal and detects contained commands and instructions and executes them. The speech recognition system can be coupled with a so-called dialogue system, which can lead a dialogue with the driver via questions and answers.

Das Fahrzeug 10 umfasst ferner eine Audiosignalquelle 14. Die Audiosignalquelle 14 kann beispielsweise ein Rundfunkempfänger, eine Medienwiedergabevorrichtung wie zum Beispiel ein CD Player oder ein MP3-Player, oder ein Navigationssystem des Fahrzeugs 10 umfassen. Die Audiosignalquelle 14 gibt ein mehrkanaliges Audioquellsignal aus. Das mehrkanalige Audioquellsignal wird der Audiosignalverarbeitungsvorrichtung 15 zugeführt und dort bearbeitet, wie es nachfolgend unter Bezugnahme auf Figur 2 beschrieben werden wird. Das bearbeitete mehrkanalige Audioquellsignal wird von der Audiosignalverarbeitungsvorrichtung 15 an einen Verstärker 16 ausgegeben. Der Verstärker 16 verstärkt die einzelnen Signale des bearbeiteten mehrkanaligen Audioquellsignals, sodass diese über Lautsprecher 17-20 in einem Innenraum des Fahrzeugs 10 wiedergegeben werden können.The vehicle 10 further includes an audio signal source 14. The audio signal source 14 may include, for example, a broadcast receiver, a media player such as a CD player or an MP3 player, or a navigation system of the vehicle 10. The audio signal source 14 outputs a multi-channel audio source signal. The multi-channel audio source signal is supplied to and processed by the audio signal processing device 15, as described below with reference to FIG FIG. 2 will be described. The processed multi-channel audio source signal is output from the audio signal processing device 15 to an amplifier 16. The amplifier 16 amplifies the individual signals of the processed multi-channel audio source signal so that they can be reproduced via speakers 17-20 in an interior of the vehicle 10.

In dem in Figur 1 gezeigten Beispiel umfasst das Fahrzeug 10 vier Lautsprecher 17-20. Bei anderen Ausführungsformen der Erfindung kann das Fahrzeug 10 eine beliebige Anzahl von Lautsprechern umfassen, beispielsweise zwei, drei oder mehr als vier. In dem gezeigten Beispiel der Figur 1 sind die Lautsprecher 17-20 den Sitzplätzen des Fahrzeugs 10 zugeordnet. So ist der Lautsprecher 17 einem Fahrersitz des Fahrers 12 zugeordnet, der Lautsprecher 18 einem Beifahrersitz, der Lautsprecher 19 einem hinteren rechten Sitz und der Lautsprecher 20 einem hinteren linken Sitz.In the in FIG. 1 As shown, the vehicle 10 includes four loudspeakers 17-20. In other embodiments of the invention, the vehicle 10 may include any number of speakers, for example two, three or more than four. In the example shown the FIG. 1 are the speakers 17-20 assigned to the seats of the vehicle 10. Thus, the loudspeaker 17 is associated with a driver's seat of the driver 12, the loudspeaker 18 with a passenger seat, the loudspeaker 19 with a rear right seat and the loudspeaker 20 with a rear left seat.

Im Betrieb des Fahrzeugs 10 kann der Fahrer 12 Anweisungen oder Befehle an das Spracherkennungssystem 11 äußern. Dies wird in Figur 1 durch den gestrichelten Pfeil zwischen dem Fahrer 12 und dem Mikrofon 13 dargestellt. Während der Fahrer 12 Befehle und Anweisungen äußert, können mehrkanalige Audioquellsignale von der Audiosignalquelle 14 über die Lautsprecher 17-20 ausgegeben werden. Die Ausgaben der Lautsprecher 17-20 erreichen ebenfalls das Mikrofon 13, wie es durch die entsprechenden gestrichelten Pfeile zwischen den Lautsprechern 17-20 und dem Mikrofon 13 in der Figur 1 dargestellt ist. Die Ausgaben aus den Lautsprechern 17-20 können jedoch die Sprachverständlichkeit stören, sodass das Spracherkennungssystem 11 die Befehle und Anweisungen von dem Fahrer 12 nicht oder nur unzureichend erkennt.In operation of the vehicle 10, the driver 12 may voice instructions or commands to the speech recognition system 11. This will be in FIG. 1 represented by the dashed arrow between the driver 12 and the microphone 13. While the driver 12 is issuing commands and instructions, multi-channel audio source signals may be supplied from the audio signal source 14 be output through the speakers 17-20. The outputs of the speakers 17-20 also reach the microphone 13, as indicated by the corresponding dashed arrows between the speakers 17-20 and the microphone 13 in the FIG. 1 is shown. However, the outputs from the speakers 17-20 may interfere with speech intelligibility, such that the speech recognition system 11 may not or insufficiently recognize the commands and instructions from the driver 12.

Figur 2 zeigt Details der Audiosignalverarbeitungsvorrichtung 15 und des Spracherkennungssystems 11, welche dazu beitragen, den Einfluss der Ausgaben aus den Lautsprechern 17-20 auf das Sprachsignal des Fahrers 12 zu verringern oder zu kompensieren. Zur Vereinfachung der Darstellung ist die Audiosignalquelle 14 in dem Beispiel der Figur 2 nur zweikanalig, also eine Stereoquelle mit einem linken Kanal L und einem rechten Kanal R. Es ist jedoch klar, dass die nachfolgend beschriebene Audiosignalverarbeitungsvorrichtung 15 in gleicher Art und Weise eine beliebige Anzahl von Kanälen einer mehrkanaligen Audiosignalquelle verarbeiten kann. FIG. 2 shows details of the audio signal processing device 15 and the speech recognition system 11, which help to reduce or compensate for the influence of the outputs from the loudspeakers 17-20 on the speech signal of the driver 12. For ease of illustration, the audio signal source 14 in the example of FIG FIG. 2 however, it is clear that the audio signal processing apparatus 15 described below can process any number of channels of a multi-channel audio signal source in the same way.

Bevor die Funktionsweise der Audiosignalverarbeitungsvorrichtung 15 beschrieben werden wird, werden zunächst die in Figur 2 gezeigten Komponenten der Audiosignalverarbeitungsvorrichtung 15 beschrieben. Die in Figur 2 gezeigten Komponenten der Audiosignalverarbeitungsvorrichtung 15 müssen nicht notwendigerweise tatsächlich als konkrete Bauteile oder Baugruppen ausgebildet werden, sondern können teilweise oder insgesamt programmtechnisch nachgebildet werden und durch eine geeignete Steuerung realisiert werden, beispielsweise einen Mikroprozessor oder einen digitalen Signalprozessor.Before the operation of the audio signal processing device 15 will be described, the in FIG. 2 shown components of the audio signal processing device 15 described. In the FIG. 2 shown components of the audio signal processing device 15 may not necessarily be designed as concrete components or assemblies, but may be partially or wholly reproduced programmatically and implemented by a suitable controller, such as a microprocessor or a digital signal processor.

Die Audiosignalverarbeitungsvorrichtung 15 umfasst Eingänge, über welche das mehrkanalige Audioquellsignal von der Audiosignalquelle 14 empfangen wird. Ein zweikanaliges Stereoaudioquellsignal umfasst beispielsweise einen linken Kanal L und einen rechten Kanal R, welche der Audiosignalverarbeitungsvorrichtung 15 zugeführt werden. Mit einem ersten Signalumwandler 21 werden aus dem zwei- oder mehrkanaligen Audioquellsignal ein Mittensignalanteil M und für jeden Kanal ein Seitensignalanteil S erzeugt. Speziell für ein Stereosignal kann statt zweier Seitensignalanteile ein gemeinsamer Seitensignalanteil als Differenz aus dem linken Kanal L und dem rechten Kanal R gebildet werden. Da unabhängig von der Anzahl der Seitensignalanteile alle Seitensignalanteile nachfolgend gleich behandelt werden, ist in Figur 2 nur ein Pfad für die Seitensignalanteile S gezeigt. Dieser eine Pfad kann demzufolge bei einem Stereosignal nur einen Seitensignalanteil oder im mehrkanaligen Fall mehrere Seitensignalanteile umfassen.The audio signal processing device 15 includes inputs via which the multi-channel audio source signal is received by the audio signal source 14. A two-channel stereo audio source signal includes, for example, a left channel L and a right channel R, which are supplied to the audio signal processing device 15. With a first signal converter 21, a middle signal component M and for each channel a side signal component S are generated from the two- or multi-channel audio source signal. Especially for a stereo signal, instead of two side signal components, a common side signal component can be formed as the difference between the left channel L and the right channel R. Since, regardless of the number of side signal components, all side signal components are treated the same in the following FIG. 2 only one path for the side signal components S is shown. This one path can therefore comprise only one side signal component in the case of a stereo signal or several side signal components in the multi-channel case.

Der Mittensignalanteil M kann beispielsweise ein Summensignal aus allen zugeführten Kanälen umfassen. Bei einem Stereosignal kann der Mittensignalanteil M daher das Summensignal aus dem linken Kanal L und dem rechten Kanal R umfassen (M=R+L). Ein jeweiliger Seitensignalanteil S kann beispielsweise ein Differenzsignal zwischen dem jeweiligen Audiosignal des jeweiligen Kanals des mehrkanaligen Audioquellsignals und dem Mittensignalanteil umfassen. Speziell bei einem Stereosignal kann der Seitensignalanteil S auch beispielsweise ein Differenzsignal aus dem rechten Kanal R und dem linken Kanal L umfassen (S=R-L).The middle signal component M may comprise, for example, a sum signal from all the supplied channels. In a stereo signal, therefore, the center signal component M may comprise the sum signal from the left channel L and the right channel R (M = R + L). A respective side signal component S may, for example, comprise a difference signal between the respective audio signal of the respective channel of the multi-channel audio source signal and the mid-signal component. Especially in the case of a stereo signal, the side signal component S may also comprise, for example, a difference signal from the right channel R and the left channel L (S = R-L).

Die Audiosignalverarbeitungsvorrichtung 15 umfasst ferner ein erstes Bandpassfilter 23 und ein Kerbfilter oder Notch-Filter 22. Das erste Bandpassfilter 23 hat eine vorgegebene untere Frequenz und eine vorgegebene obere Frequenz. Das erste Bandpassfilter 23 lässt im Wesentlichen nur Signale mit einer Frequenz zwischen der vorgegebenen unteren Frequenz und der vorgegebenen oberen Frequenz passieren. Signale mit einer Frequenz unterhalb der vorgegebenen unteren Frequenz sowie Signale mit einer Frequenz oberhalb der vorgegebenen oberen Frequenz werden im Wesentlichen unterdrückt oder zumindest stark gedämpft. Bei einer analogen Ausgestaltung des ersten Bandpassfilters 23 kann die Dämpfung beispielsweise 70 dB oder mehr betragen und bei einer digitalen Ausgestaltung des ersten Bandpassfilters kann das Signal oberhalb der vorgegebenen oberen Frequenz und unterhalb der vorgegebenen unteren Frequenz vollständig unterdrückt werden. Das Notch-Filter 22 hat einen Frequenzgang, welcher im Wesentlichen invers zu dem Frequenzgang des ersten Bandpassfilters 23 ist. D.h., das Notch-Filter 22 lässt im Wesentlichen nur Signale mit einer Frequenz unter der vorgegebenen unteren Frequenz oder über der vorgegebenen oberen Frequenz passieren. Die untere vorgegebene Frequenz kann beispielsweise 100 Hz betragen und die obere vorgegebene Frequenz kann beispielsweise 8kHz betragen. Alternativ kann die untere vorgegebene Frequenz in einem Bereich von 100 Hz bis 300 Hz gewählt werden und die obere vorgegebene Frequenz in einem Bereich von 4kHz bis 8kHz gewählt werden. Je größer der Frequenzbereich zwischen der unteren vorgegebenen Frequenz und der oberen vorgegebenen Frequenz gewählt wird, umso zuverlässiger arbeitet die Spracherkennung. Allerdings wird eine Wiedergabe eines mehrkanaligen Audioquellsignals umso stärker beeinträchtigt, je größer der Frequenzbereich zwischen der unteren vorgegebenen Frequenz und der oberen vorgegebenen Frequenz gewählt wird. Für den Fall, dass mehrere Seitensignalanteile erzeugt werden, ist für jeden dieser mehreren Seitensignalanteile ein entsprechendes Notch-Filter 22 mit der unteren vorgegebenen Frequenz und der oberen vorgegebenen Frequenz vorzusehen.The audio signal processing device 15 further comprises a first bandpass filter 23 and a notch filter or notch filter 22. The first bandpass filter 23 has a predetermined lower frequency and a predetermined upper frequency. The first bandpass filter 23 essentially passes only signals having a frequency between the predetermined lower frequency and the predetermined upper frequency. Signals with a frequency below the predetermined lower frequency and signals with a frequency above the predetermined upper frequency are substantially suppressed or at least greatly attenuated. In an analog embodiment of the first bandpass filter 23, the attenuation may be, for example, 70 dB or more, and in a digital embodiment of the first bandpass filter, the signal above the predetermined upper frequency and below the predetermined lower frequency can be completely suppressed. The notch filter 22 has a frequency response which is substantially inverse to the frequency response of the first bandpass filter 23. That is, the notch filter 22 essentially only passes signals having a frequency below the predetermined lower frequency or above the predetermined upper frequency. For example, the lower predetermined frequency may be 100 Hz, and the upper predetermined frequency may be 8 kHz, for example. Alternatively, the lower predetermined frequency may be selected in a range of 100 Hz to 300 Hz, and the upper predetermined frequency may be selected in a range of 4 kHz to 8 kHz. The larger the frequency range between the lower predetermined frequency and the upper predetermined frequency is selected, the more reliable the speech recognition function. However, reproduction of a multi-channel audio source signal is more affected the greater the frequency range between the lower predetermined frequency and the upper predetermined frequency is selected. In the event that several side signal components are generated, a corresponding notch filter 22 having the lower predetermined frequency and the upper predetermined frequency is to be provided for each of these plurality of side signal components.

Durch die Filterung des Mittensignalanteils M mit dem Bandpassfilter 23 wird ein gefilterter oder in der Frequenz begrenzter Mittensignalanteil Mb erzeugt. Durch die Filterung der Seitensignalanteile S mit den Notch-Filtern 22 werden gefilterte oder in der Frequenz begrenzte Seitensignalanteile Sb erzeugt. Der gefiltert Mittensignalanteil Mb und die gefilterten Seitensignalanteil Sb werden einem zweiten Signalumwandler 24 zugeführt, welcher gefilterte Audiosignale für die einzelnen Kanäle erzeugt. Das gefilterte Audiosignal für einen jeweiligen einzelnen Kanal kann beispielsweise durch Summieren des gefilterten Mittensignalanteils Mb und des entsprechenden gefilterten kanalspezifischen Seitensignalanteils Sb gebildet werden. Speziell für den Fall eines Stereoaudioquellsignals gilt beispielsweise Rb=Mb+Sb und Lb=Mb-Sb. Die gefilterten Audiosignale Lb, Rb werden aus der Audiosignalverarbeitungsvorrichtung 15 ausgegeben und kanalweise dem Verstärker 16 zugeführt.By filtering the center signal component M with the bandpass filter 23, a filtered or frequency-limited mid-signal component Mb is generated. By filtering the side signal components S with the notch filters 22, filtered or frequency-limited side signal components Sb are generated. The filtered center signal component Mb and the filtered side signal component Sb are applied to a second signal converter 24, which generates filtered audio signals for the individual channels. The filtered audio signal for a respective single channel may be formed, for example, by summing the filtered center signal component Mb and the corresponding filtered channel-specific side signal component Sb. Specifically, in the case of a stereo audio source signal, Rb = Mb + Sb and Lb = Mb-Sb, for example. The filtered audio signals Lb, Rb are output from the audio signal processing device 15 and supplied to the amplifier 16 channel by channel.

Die Audiosignalverarbeitungsvorrichtung 15 umfasst ferner ein zweites Bandpassfilter 26. Das zweite Bandpassfilter 26 hat die gleiche Filtercharakteristik wie das erste Bandpassfilter 23. Das zweite Bandpassfilter 26 ist eingangsseitig mit dem Mikrofon 13 gekoppelt und ausgangsseitig mit einem Echokompensator 25 des Spracherkennungssystems 11. Dem Echokompensator 25 des Spracherkennungssystems 11 wird ferner der gefilterte Mittensignalanteil Mb zugeführt. Auf der Grundlage des gefilterten Mittensignalanteils Mb führt der Echokompensator 25 eine Echokompensation auf dem gefilterten Sprachsignal von dem Mikrofon 13 durch. Das von dem Echokompensator 25 bearbeitete Sprachsignal wird einem Spracherkenner 27 des Spracherkennungssystems 11 zugeführt.The audio signal processing device 15 further comprises a second bandpass filter 26. The second bandpass filter 26 has the same filter characteristic as the first bandpass filter 23. The second bandpass filter 26 is coupled on the input side to the microphone 13 and on the output side with an echo canceller 25 of the speech recognition system 11. The echo canceller 25 of the speech recognition system 11, the filtered center signal component Mb is further supplied. Based on the filtered center signal component Mb, the echo canceller 25 performs echo cancellation on the filtered voice signal from the microphone 13. The speech signal processed by the echo canceler 25 is supplied to a speech recognizer 27 of the speech recognition system 11.

Außerdem umfasst die Audiosignalverarbeitungsvorrichtung 15 eine Gewichtungsvorrichtung 28, welche mit dem mehrkanaligen Audioquellsignal und/oder der Audiosignalquelle 14 gekoppelt ist. Auf der Grundlage von Informationen des mehrkanaligen Audioquellsignals oder Informationen von der Audiosignalquelle 14 stellt die Gewichtungsvorrichtung 28 Gewichtungsfaktoren bereit, mit welchen die gefilterten Audiosignale vor ihrer Ausgabe aus dem zweiten Signalumwandler 24 gewichtet werden.In addition, the audio signal processing device 15 comprises a weighting device 28, which is coupled to the multi-channel audio source signal and / or the audio signal source 14. Based on information from the multi-channel audio source signal or information from the audio signal source 14, the weighting device 28 provides weighting factors with which the filtered audio signals are weighted prior to being output from the second signal converter 24.

Unter Bezugnahme auf Figur 3 wird nachfolgend die Arbeitsweise der Audiosignalverarbeitungsvorrichtung 15 in dem Fahrzeug 10 beschrieben werden. Figur 3 zeigt ein Verfahren 30 mit Verfahrensschritten 31-37, welche von der Audiosignalverarbeitungsvorrichtung 15 in Verbindung mit dem Spracherkennungssystem 11 ausgeführt werden. Es ist klar, dass die in Figur 3 dargestellten Verarbeitungsschritte mit elektronischen Hilfsmitteln ausgeführt werden können, welche beispielsweise analoge oder digitale Schaltkreise sowie Verarbeitungsvorrichtungen umfassen. Verarbeitungsvorrichtungen können beispielsweise Mikroprozessoren oder digitale Signalprozessoren umfassen. Ferner kann die gesamte Funktionalität der Audiosignalverarbeitungsvorrichtung 15 in beispielsweise eine bestehende elektronische Vorrichtung integriert werden, beispielsweise in einen digitalen Signalprozessor des Spracherkennungssystems 11.With reference to FIG. 3 The operation of the audio signal processing device 15 in the vehicle 10 will be described below. FIG. 3 shows a method 30 with method steps 31-37, which are carried out by the audio signal processing device 15 in conjunction with the speech recognition system 11. It is clear that in FIG. 3 illustrated processing steps can be performed with electronic tools, which include, for example, analog or digital circuits and processing devices. processing devices For example, they may include microprocessors or digital signal processors. Furthermore, the entire functionality of the audio signal processing device 15 can be integrated into, for example, an existing electronic device, for example a digital signal processor of the speech recognition system 11.

Im Schritt 31 wird ein mehrkanaliges Audioquellsignal, beispielsweise ein Stereosignal oder ein Surroundsignal, von der Audiosignalquelle 14 an der Audiosignalverarbeitungsvorrichtung 15 empfangen. In den Schritten 32 und 33 werden mit Hilfe des ersten Signalumwandlers 21 und der Filter 22 und 23 ein in der Frequenz begrenztes Monoaudiosignal und in der Frequenz begrenzte kanalspezifische Audiosignale erzeugt. Der zuvor beschriebene in der Frequenz begrenzte Mittensignalanteil Mb kann beispielsweise das in der Frequenz begrenzte Monoaudiosignal sein. Die zuvor beschriebenen in der Frequenz begrenzten Seitensignalanteile Sb können beispielsweise die in der Frequenz begrenzten kanalspezifischen Audiosignale sein. Das in der Frequenz begrenzte Monoaudiosignal und die in der Frequenz begrenzten kanalspezifischen Audiosignale können jedoch auch auf beliebige andere Art und Weise aus dem mehrkanaligen Audioquellsignal gebildet werden, beispielsweise in einem digitalen Signalprozessor.In step 31, a multi-channel audio source signal, for example a stereo signal or a surround signal, is received by the audio signal source 14 at the audio signal processing device 15. In steps 32 and 33, a frequency limited mono audio signal and frequency limited channel specific audio signals are generated by the first signal converter 21 and the filters 22 and 23. The frequency-limited center signal component Mb described above may be, for example, the frequency-limited mono audio signal. The frequency-limited side signal components Sb described above may be, for example, the frequency-limited channel-specific audio signals. However, the frequency-limited mono audio signal and the frequency-limited channel-specific audio signals can also be formed in any other way from the multi-channel audio source signal, for example in a digital signal processor.

Im Schritt 34 wird das begrenzte Monoaudiosignal über alle Lautsprecher 17-20 ausgegeben und die begrenzten kanalspezifischen Audiosignale werden über dem dem jeweiligen Kanal zugeordneten Lautsprecher ausgegeben. Das Monoaudiosignal ist auf einen für die Spracherkennung relevanten Frequenzbereich begrenzt, beispielsweise auf einen Frequenzbereich von 100 Hz bis 8kHz. Die kanalspezifischen Audiosignale sind auf einen Frequenzbereich außerhalb des für die Spracherkennung relevanten Frequenzbereichs begrenzt, also beispielsweise auf Frequenzen unter 100 Hz und über 8kHz. Durch die Reduktion der Mehrkanaligkeit der Audiowiedergabe innerhalb des für den Spracherkenner 27 relevanten Frequenzbereichs liegt für die Spracherkennung als störendes Signal lediglich das einkanalige Monoaudiosignal vor. Für den oder die Fahrzeuginsassen bleibt jedoch ein Räumlichkeitsgefühl in der Klangwahrnehmung erhalten, da für Frequenzen außerhalb des für die Spracherkennung relevanten Bereichs die Mehrkanaligkeit erhalten bleibt.In step 34, the limited mono audio signal is output through all the loudspeakers 17-20 and the limited channel specific audio signals are outputted over the loudspeaker associated with the respective channel. The mono audio signal is limited to a relevant frequency range for speech recognition, for example to a frequency range of 100 Hz to 8 kHz. The channel-specific audio signals are limited to a frequency range outside the relevant frequency range for speech recognition, that is, for example, to frequencies below 100 Hz and above 8 kHz. Due to the reduction of the multichannel nature of the audio reproduction within the frequency range relevant to the speech recognizer 27, only the single-channel mono audio signal is present as the disturbing signal for speech recognition. For the vehicle occupant, however, a feeling of spatial perception remains in the sound perception since the multichannel nature is retained for frequencies outside the range relevant for speech recognition.

Beim Ausgeben des begrenzten Monoaudiosignals über die Lautsprecher 17-20 kann ein Audioschwerpunkt in dem Fahrzeug geändert werden. Beispielsweise kann die Gewichtungsvorrichtung 28 auf der Grundlage der ihr zur zugeführten Informationen einen Audioschwerpunkt für die mehrkanaligen Audioquellsignale oder die aktuelle Signalquelle bestimmen und das begrenzte Monoaudiosignal gemäß diesem Audioschwerpunkt auf die Audiokanäle verteilen. Wenn beispielsweise eine Sprachausgabe eines Navigationssystems die mehrkanaligen Audiosignalquelle darstellt, kann das begrenzte Monoaudiosignal beispielsweise für den Lautsprecher 17 stärker gewichtet werden als für die Lautsprecher 18-20, da diese Information für den Fahrer 12 relevanter ist als für die übrigen Fahrzeuginsassen. Die Gewichtungsvorrichtung 28. kann weitere Informationen des Fahrzeugs 10 in Betracht ziehen, beispielsweise eine aktuelle Sitzbelegung in dem Fahrzeug.When outputting the limited mono audio signal through the speakers 17-20, an audio center of gravity in the vehicle can be changed. For example, the weighting device 28 may determine an audio centroid for the multi-channel audio source signals or the current signal source based on the information supplied thereto, and the limited mono audio signal corresponding to that audio centroid on the audio source Distribute audio channels. For example, if a voice output of a navigation system represents the multi-channel audio signal source, the limited mono audio signal, for example, for the speaker 17 may be weighted more heavily than for the speakers 18-20, since this information is more relevant to the driver 12 than for the rest of the vehicle occupants. The weighting device 28 may consider further information of the vehicle 10, for example a current seat occupancy in the vehicle.

Für die Spracherkennung wird im Schritt 35 ein Sprachaudiosignal über das Mikrofon 13 empfangen. Im Schritt 36 wird das empfangene Sprachaudiosignal mithilfe des zweiten Bandpassfilters 26 in der Frequenz begrenzt. Dem Echokompensator 25 werden das begrenzte Monoaudiosignal und das begrenzte Sprachaudiosignal zugeführt. Im Schritt 37 führt der Echokompensator 25 eine Echokompensation in dem Sprachaudiosignal unter Verwendung des Monoaudiosignals durch. Da sowohl das Sprachaudiosignal als auch das Monoaudiosignal auf den für die Spracherkennung relevanten Frequenzbereich begrenzt sind (z.B. 100Hz-8kHz), kann die Echokompensation ebenfalls auf diesen begrenzten Frequenzbereich eingeschränkt werden, wodurch weniger Störungen auftreten und der Echokompensator 25 einfacher aufgebaut werden kann bzw. weniger Rechenleistung erfordert. Ferner benötigt die einkanalige Echokompensation nur ein einziges Audioreferenzsignal, nämlich das Monoaudiosignal, und muss nur eine akustische Impulsantwort schätzen. Hierdurch werden Systemressourcen bei der Echokompensation eingespart, welche zum Beispiel für den Spracherkenner 27 zur Verfügung stehen.For speech recognition, a speech audio signal is received via the microphone 13 in step 35. In step 36, the received speech audio signal is frequency limited by the second bandpass filter 26. The echo canceller 25 is supplied with the limited mono audio signal and the limited voice audio signal. In step 37, the echo canceller 25 performs echo cancellation in the voice audio signal using the mono audio signal. Since both the speech audio signal and the mono audio signal are limited to the frequency range relevant to the speech recognition (eg 100Hz-8kHz), the echo cancellation can also be limited to this limited frequency range, whereby less noise occurs and the echo canceller 25 can be made simpler or less Computing power required. Furthermore, single-channel echo cancellation requires only a single audio reference signal, namely the mono audio signal, and only needs to estimate an acoustic impulse response. As a result, system resources are saved in the echo cancellation, which are available for example for the speech recognizer 27.

Das so bereinigte Sprachaudiosignal wird dem Spracherkenner 27 zugeführt und dort verarbeitet, um entsprechende Befehle und Anweisungen aus der gesprochenen Sprache zu extrahieren.The thus adjusted speech audio signal is supplied to the speech recognizer 27 and processed there to extract corresponding commands and instructions from the spoken speech.

BezugszeichenlisteLIST OF REFERENCE NUMBERS

1010
Fahrzeugvehicle
1111
SpracherkennungssystemVoice recognition system
1212
Fahrzeuginsassepassenger
1313
Mikrofonmicrophone
1414
AudiosignalquelleAudio signal source
1515
AudiosignalverarbeitungsvorrichtungAudio signal processing device
1616
Verstärkeramplifier
17-2017-20
Lautsprecherspeaker
2121
erster Signalumwandlerfirst signal converter
2222
Notch-FilterNotch filter
2323
erstes Bandpassfilterfirst bandpass filter
2424
zweiter Signalumwandlersecond signal converter
2525
Echokompensatorecho canceller
2626
zweites Bandpassfiltersecond bandpass filter
2727
Spracherkennerspeech
2828
Gewichtungsvorrichtungweighting device
3030
Verfahrenmethod
31-3731-37
Schrittstep

Claims (8)

  1. A method (30) for an audio signal processing in a vehicle (10), comprising:
    - Generating a mono audio signal of a multichannel audio source signal,
    - limiting the mono audio signal to a frequency range between a prescribed lower frequency and a prescribed upper frequency,
    - outputting the limited mono audio signal via the multiple loudspeakers (17-20) in the vehicle (10), and
    - compensation of an influence of the limited mono audio signal output via the multiple loudspeakers on a voice audio signal received in the vehicle (10) via a microphone (13) by means of the limited mono audio signal,
    characterized in that the prescribed lower frequency has a value in the range of 100 Hz to 300 Hz and the prescribed upper frequency has a value in the range of 4kHz to 8kHz.
  2. A method according to Claim 1, further characterized by:
    - Generating multiple limited channel-specific audio signals depending on the multichannel audio source signal in such a manner, that a respective limited channel-specific audio signal of the multiple limited audio signals is assigned to a respective audio signal of the multichannel audio source signal and is limited to a frequency range below the prescribed lower frequency and/or above the prescribed upper frequency, and
    - outputting the multiple limited channel-specific audio signals via the multiple loudspeakers (17-20) in the vehicle (10).
  3. A method according to Claim 2, characterized in that the multichannel audio source signal is divided into a middle signal component, which is the same on all channels, and a respective side signal component per audio channel of the multichannel audio source signal and the middle signal component is used for generating the limited mono audio signal and the respective side signal components are used for generating the multiple limited channel-specific audio signals.
  4. A method according to Claim 3, characterized in that the middle signal is formed by an averaging of respective sample values of the audio channels of the multichannel audio source signal and the respective side signal components are formed by subtraction of the middle signal from the respective audio signals of the multichannel audio source signal.
  5. A method according to any one of the preceding claims, characterized in that the voice audio signal received via the microphone (13) is limited to a frequency range between the prescribed lower frequency and the prescribed upper frequency and the influence of the limited mono audio signal outputted via the multiple loudspeakers (17-20) on the limited voice audio signal is compensated.
  6. A method according to any one of the preceding claims, further characterized by:
    - The generation of multiple weighting factors assigned to the respective loudspeakers (17-20) in dependence on the multichannel audio source signal, and
    - the outputting of a limited mono audio signal via the respective loudspeaker (17-20), which signal is weighted with the weighting factor assigned to the respective loudspeaker.
  7. An audio signal processing device for a vehicle (10), which is designed
    - to generate a mono audio signal on the basis of a multichannel audio source signal,
    - to limit the mono audio signal to a frequency range between a prescribed lower frequency and a prescribed upper frequency,
    - to output the limited mono audio signal via multiple loudspeakers (17-20) in the vehicle (10), and
    - to output the limited mono audio signal to a compensation device (25), in order to compensate an influence of the limited mono audio signal outputted via the multiple loudspeakers (17-20) on a voice audio signal received in the vehicle (10) via a microphone 13) by means of the limited mono audio signal,
    characterized in that the prescribed lower frequency has a value in the range of 100 Hz to 300 Hz and the prescribed upper frequency has a value in the range of 4kHz to 8kHz.
  8. An audio signal processing device according to Claim 7, characterized in that the audio signal processing device (15) is designed for carrying out the method according to any one of Claims 1-6.
EP16788664.7A 2015-11-10 2016-10-26 Audio signal processing in a vehicle Active EP3375204B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102015222105.9A DE102015222105A1 (en) 2015-11-10 2015-11-10 Audio signal processing in a vehicle
PCT/EP2016/075831 WO2017080830A1 (en) 2015-11-10 2016-10-26 Audio signal processing in a vehicle

Publications (2)

Publication Number Publication Date
EP3375204A1 EP3375204A1 (en) 2018-09-19
EP3375204B1 true EP3375204B1 (en) 2019-06-19

Family

ID=57218876

Family Applications (1)

Application Number Title Priority Date Filing Date
EP16788664.7A Active EP3375204B1 (en) 2015-11-10 2016-10-26 Audio signal processing in a vehicle

Country Status (5)

Country Link
US (1) US10339951B2 (en)
EP (1) EP3375204B1 (en)
CN (1) CN108353229B (en)
DE (1) DE102015222105A1 (en)
WO (1) WO2017080830A1 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102015222105A1 (en) 2015-11-10 2017-05-11 Volkswagen Aktiengesellschaft Audio signal processing in a vehicle
DE102018007433A1 (en) 2018-09-20 2019-02-28 Daimler Ag Audio signal processing system for a vehicle
CN110246517B (en) * 2019-07-08 2021-07-13 广州小鹏汽车科技有限公司 Radio station music identification method, vehicle-mounted system and vehicle
JP7383942B2 (en) * 2019-09-06 2023-11-21 ヤマハ株式会社 In-vehicle sound systems and vehicles
CN111739552A (en) * 2020-08-28 2020-10-02 南京芯驰半导体科技有限公司 Method and system for forming wave beam of microphone array
CN112309416B (en) * 2020-10-20 2022-11-29 中国第一汽车股份有限公司 Vehicle-mounted voice echo eliminating method, system, vehicle and storage medium
CN113096681B (en) * 2021-04-08 2022-06-28 海信视像科技股份有限公司 Display device, multi-channel echo cancellation circuit and multi-channel echo cancellation method

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5828756A (en) * 1994-11-22 1998-10-27 Lucent Technologies Inc. Stereophonic acoustic echo cancellation using non-linear transformations
DE19533541C1 (en) * 1995-09-11 1997-03-27 Daimler Benz Aerospace Ag Method for the automatic control of one or more devices by voice commands or by voice dialog in real time and device for executing the method
JP2003500937A (en) * 1999-05-25 2003-01-07 ブリティッシュ・テレコミュニケーションズ・パブリック・リミテッド・カンパニー Acoustic echo cancellation
US6665645B1 (en) 1999-07-28 2003-12-16 Matsushita Electric Industrial Co., Ltd. Speech recognition apparatus for AV equipment
JP4554044B2 (en) 1999-07-28 2010-09-29 パナソニック株式会社 Voice recognition device for AV equipment
US20050213747A1 (en) * 2003-10-07 2005-09-29 Vtel Products, Inc. Hybrid monaural and multichannel audio for conferencing
NO328256B1 (en) 2004-12-29 2010-01-18 Tandberg Telecom As Audio System
WO2006111370A1 (en) * 2005-04-19 2006-10-26 Epfl (Ecole Polytechnique Federale De Lausanne) A method and device for removing echo in a multi-channel audio signal
JP4896449B2 (en) * 2005-06-29 2012-03-14 株式会社東芝 Acoustic signal processing method, apparatus and program
EP1936939B1 (en) 2006-12-18 2011-08-24 Harman Becker Automotive Systems GmbH Low complexity echo compensation
JP4916394B2 (en) 2007-07-03 2012-04-11 富士通株式会社 Echo suppression device, echo suppression method, and computer program
DE102009051508B4 (en) 2009-10-30 2020-12-03 Continental Automotive Gmbh Device, system and method for voice dialog activation and guidance
US8219394B2 (en) * 2010-01-20 2012-07-10 Microsoft Corporation Adaptive ambient sound suppression and speech tracking
EP2466864B1 (en) 2010-12-14 2019-02-27 Deutsche Telekom AG Transparent decorrelation of the loudspeaker signals of multi-channel echo compensators
JP5649488B2 (en) * 2011-03-11 2015-01-07 株式会社東芝 Voice discrimination device, voice discrimination method, and voice discrimination program
GB2510331A (en) * 2012-12-21 2014-08-06 Microsoft Corp Echo suppression in an audio signal
JP6225920B2 (en) * 2012-12-28 2017-11-08 株式会社ソシオネクスト Device with speech recognition and speech recognition method
AU2014289973A1 (en) * 2013-07-12 2016-03-03 Cirrus Logic International Semiconductor Limited Wind noise reduction
DE102015222105A1 (en) 2015-11-10 2017-05-11 Volkswagen Aktiengesellschaft Audio signal processing in a vehicle

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
US10339951B2 (en) 2019-07-02
CN108353229A (en) 2018-07-31
WO2017080830A1 (en) 2017-05-18
EP3375204A1 (en) 2018-09-19
CN108353229B (en) 2020-10-23
DE102015222105A1 (en) 2017-05-11
US20180358031A1 (en) 2018-12-13

Similar Documents

Publication Publication Date Title
EP3375204B1 (en) Audio signal processing in a vehicle
EP1595427B1 (en) Method and device for the separation of sound signals
DE69738288T2 (en) DEVICE FOR SUPPRESSING A DISTURBING COMPONENT OF AN INPUT SIGNAL
DE10351793B4 (en) Adaptive filter device and method for processing an acoustic input signal
DE69827731T2 (en) Arrangement for active suppression of acoustic echo and noise
DE3925589C2 (en) Method and arrangement for the elimination of interference from speech signals
DE102018127071B3 (en) Audio signal processing with acoustic echo cancellation
DE10018666A1 (en) Dynamic sound optimization in the interior of a motor vehicle or similar noisy environment, a monitoring signal is split into desired-signal and noise-signal components which are used for signal adjustment
DE60004888T2 (en) LANGUAGE DISTRIBUTION SYSTEM
DE102006027673A1 (en) Signal isolator, method for determining output signals based on microphone signals and computer program
EP1251493A2 (en) Method for noise reduction with self-adjusting spurious frequency
EP1771034A2 (en) Microphone calibration in a RGSC-beamformer
DE102014214052A1 (en) Virtual masking methods
DE102010007842A1 (en) Noise suppression device, noise suppression method and recording medium
EP1189419B1 (en) Method and device for eliminating the loudspeaker interference on microphone signals
DE102013011761A1 (en) Motor vehicle has estimating unit and background noise spectrum unit that are designed to open dynamic filter with low background noise and close with strong background noise
DE112016004394T5 (en) Heulunterdrückungsvorrichtung
DE102018117558A1 (en) ADAPTIVE AFTER-FILTERING
DE102014210760B4 (en) operation of a communication system
DE10025655B4 (en) A method of removing an unwanted component of a signal and system for distinguishing between unwanted and desired signal components
EP1718113B1 (en) Hearing-aid device with compensation of acoustic and electromagnetic feedback signal
WO2007036443A1 (en) Method for the active reduction of noise, and device for carrying out said method
DE102014214699A1 (en) Playback of low-frequency audio signals through structure-borne noise
DE69731133T2 (en) Transmission system for correlated signals
DE102016005904A1 (en) Instantaneous noise suppression in a motor vehicle

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20180611

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 502016005192

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04R0003020000

Ipc: G10L0021020800

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
RIC1 Information provided on ipc code assigned before grant

Ipc: H04S 7/00 20060101ALI20190215BHEP

Ipc: G10L 21/0208 20130101AFI20190215BHEP

Ipc: H04R 3/02 20060101ALI20190215BHEP

Ipc: G10L 21/0216 20130101ALN20190215BHEP

Ipc: H04R 5/04 20060101ALN20190215BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20190404

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1146511

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190715

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502016005192

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190919

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190919

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190920

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191021

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191019

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200224

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502016005192

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG2D Information on lapse in contracting state deleted

Ref country code: IS

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191026

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

26N No opposition filed

Effective date: 20200603

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20191031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191026

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20161026

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 1146511

Country of ref document: AT

Kind code of ref document: T

Effective date: 20211026

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211026

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230523

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231024

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20231026

Year of fee payment: 8

Ref country code: DE

Payment date: 20231031

Year of fee payment: 8