RU2801125C1 - Acoustic echo suppression unit - Google Patents

Abstract

FIELD: audio processing.

SUBSTANCE: multi-channel audio signal is received. The multichannel audio signal is processed to obtain the first set of spatial audio components representing the intermediate signal. The first set of spatial audio components is processed to obtain a second set of spatial audio components representing the playback signals of the loudspeakers. Echo suppression is performed by using the first set of spatial audio components or a deviation version of the first set of spatial audio components as a reference signal and by using at least one received microphone signal. The deviation version is obtained by using a spatial component combining unit or by performing processing, in particular linear processing.

EFFECT: increase of efficiency of echo suppression.

17 cl, 4 dwg

Description

Embodiments of the present invention relate to an acoustic echo canceller and a corresponding method. Additional embodiments relate to a computer program. Another embodiment provides for a soundbar or other playback device containing an acoustic echo canceller. Preferred embodiments of the present invention relate to the field of audio signal processing, more specifically to an acoustic echo cancellation (AEC) approach for reproducing multi-channel audio.

Acoustic echo suppression is required for a wide range of applications. For example, it simplifies human-machine interaction with long-range audio capture and call forcing functionality and provides full duplex voice communication. AEC is typically performed using a microphone signal or a plurality of microphone signals and an AEC reference signal or an AEC reference signal containing one, two or more channels. In general, all methods in the literature use loudspeaker drive signals as a reference signal, as shown by FIG. 3.

Fig. 3 shows the audio processing path 10 as well as the acoustic echo cancellation path 20.

The audio processing path 10 includes an audio processor 12 as well as one or more loudspeakers 14. The loudspeakers 14 may be formed by a traditional multi-speaker arrangement (5.1 or 7.2, etc.) or by a soundbar having at least one, preferably two or three or more electroacoustic transducers.

Audio processor 12 receives multi-channel audio (for example, 5.2 or 7.2 signal) and processes it so that surround sound can be reproduced by using speaker 14. For example, audio processor 12 is configured to process the multi-channel audio signal MS to obtain spatial components SC through which the soundbar 14.

The echo cancellation path 20 includes an acoustic echo cancellation unit 22 that is configured to calculate echo cancellation components based on the input signal IS and the reference signal RS. The IS input signal is the microphone signal from one or more microphones (see position 24). Typically, the spatial components SC are used as the reference signal RS. Therefore, the echo cancellation block 22 contains an input for the reference signal RS, which is connected to the audio processor 12, and for one or more microphones 24. The echo cancellation parameters are output to the internal interface 26.

A significant degradation in AEC performance can be expected when a large number of loudspeakers 14 are used and/or there is a high correlation between the loudspeaker drive signals, as is common for multi-channel audio playback with soundbars. Correlation between speaker drive signals can be reduced by applying decorrelation techniques, but this comes at the cost of fidelity and is therefore undesirable for some applications. Therefore, there is a need for an improved approach.

The object of the present invention is to provide an echo cancellation concept having an improved compromise in echo cancellation quality and application. In particular, the challenge is to create an echo cancellation concept applicable to soundbars.

This problem is solved by the objects of independent claims.

Embodiments of the present invention provide an acoustic echo canceller, comprising: an audio processor configured to receive a multi-channel audio signal and comprising: a first stage that is configured to process the multi-channel audio signal to obtain a first set of spatial audio components; and a second stage that is configured to process the first set of spatial audio components to obtain a second set of spatial audio components; an echo cancellation processor configured to perform echo cancellation by using the first set of spatial audio components or a deviation version of the first set of spatial audio components as a reference signal.

According to further embodiments, the echo cancellation processor comprises a spatial component combiner configured to process the first set of spatial audio components to obtain a deviation version of the first set of spatial components to be used as a reference signal.

Embodiments of the present invention are based on the finding that it is preferable to use a two-stage process for audio processing, in which the output/spatial component signal of the first stage is preferred for use as a reference signal for acoustic echo cancellation (compared to the second stage). Here is the first spatial component signal that can be further processed to produce final spatial components, and which is more suitable for acoustic echo cancellation than final spatial components. For example, the intermediate signal may comprise component signals associated with directions in the room to be prepared by rendering as a 3D soundstage by the soundbar. According to a preferred embodiment, the first spatial components are further processed before being used as a reference signal. Therefore, a deviated version of the first set of spatial components can also be used. This deviation version is obtained by using a spatial component combining unit, for example, by performing linear processing or other processing.

In other words, this means that the first spatial components are to be accessed and processed with a spatial component combiner to obtain a reference signal. The resulting reference signal is used for AEC. The use of intermediate signals, i.e. of the first spatial components in combination with the spatial component combining unit allows to obtain a proper AEC reference signal or a proper AEC reference signal, which makes AEC applicable in practice.

According to further embodiments, an acoustic echo canceller is provided, wherein the spatial component combining unit is configured to output at least one reference signal or a reference signal comprising one, two or more channels. According to an embodiment, the spatial component combining unit is configured to linearly combine the first set of spatial components to obtain a reference signal, and/or apply a time-independent downmix matrix to the first set of spatial components to obtain a reference signal. For example, the spatial component combining unit may be configured to perform processing based on the following formula:

,

where R _i is the i-th reference signal, β _ki are weights, C _k is the k-th spatial component signal. Here, each reference signal is obtained from a subset of the first set of spatial components. In addition, each spatial component channel can only be used in a maximum of one reference signaling channel.

According to embodiments, the spatial component combiner is configured to reduce the number of spatial components to obtain at least one reference signal.

Regarding the echo cancellation processor, it should be noted that usually the echo cancellation is based on the microphone signal received through the microphone input. According to further embodiments, an acoustic echo canceller is provided wherein the echo canceller is configured to perform echo cancellation using a single or multi-channel adaptive filter, or a single or multi-channel adaptive filter that is configurable based on a comparison between a reference signal and a microphone signal. .

According to another embodiment, an acoustic echo canceller is provided wherein the first stage is configured to perform non-linear processing, or time-varying processing, or highly time-varying processing, to obtain a first set of spatial audio components. According to another embodiment, the second stage is configured to output the second set of spatial audio components to a playback device or soundbar. It should be noted that according to additional embodiments, acoustic echo cancellation is provided, in which a set of spatial audio components provides the ability to directly drive one or more electro-acoustic transducers of a playback device or control one or more electro-acoustic transducers of a playback device or soundbar by using one or more amplifiers. For example, an acoustic echo canceller is provided in which the second stage is configured to perform linear processing.

Another embodiment provides for a playback device or soundbar containing an acoustic echo canceller.

According to a further embodiment, a method for acoustic echo suppression is provided, comprising:

- reception of a multichannel audio signal;

- processing the multi-channel audio signal to obtain a first set of spatial audio components;

- processing the first set of spatial audio components to obtain a second set of spatial audio components;

- performing echo cancellation by using the first set of spatial audio components or a deviation version of the first set of spatial audio components as a reference signal.

According to additional embodiments, this method may be carried out using a computer.

Further, embodiments of the present invention are explained with reference to the accompanying drawings, in which:

Fig. 1a shows a schematic block diagram illustrating an echo canceller according to the basic embodiment;

Fig. 1b shows a schematic block diagram illustrating an advanced echo cancellation concept according to the basic embodiment;

Fig. 2 shows a schematic block diagram of the sound reduction approach according to the improved approach; And

Fig. 3 shows a schematic block diagram of the prior art sound suppression approach.

Embodiments of the present invention are explained below with reference to the accompanying drawings, in which the same reference numbers are assigned to objects having the same or similar function, so that their description is interchangeable and mutually applicable.

Fig. 1a shows a sound suppressor unit 30 comprising two paths 31 and 41. In path 31, the MS multi-channel audio signal is processed so as to reproduce the MS multi-channel audio signal by using an audio playback device 14, such as a sound bar. Here, the processing, as described in the context of the prior art, is divided into two stages 32 and 34. The first stage 32 processes the multi-channel audio signal MS to obtain an intermediate signal, which is indicated by the position SC_1. This intermediate signal SC_1 represents the first set of spatial component signals. This intermediate signal SC_1 is then further processed by using the second stage 34 to obtain a second set of spatial component signals SC_2 allowing the loudspeaker 14 to be extracted directly. Direct excitation means that no additional spatial processing is used. In other words, the electro-acoustic transducers of the soundbar 14 may be driven by the second set of spatial components (eg, after amplifying the spatial components).

For example, the first stage 32 performs non-linear signal processing and/or (very) time varying processing. The second processing stage 34 may primarily perform linear time-independent processing steps. The prerequisites for the two-stage division are that the processing steps performed by the second stage 34, such as linear time-independent processing steps, may have a negative effect on echo cancellation suitability.

The echo cancellation path 41 performs echo cancellation based on the first spatial component set SC_1. For this, the echo canceller 40 receives the signal SC_1 from the first audio processing stage 32. The echo canceller 40 includes at least an echo canceller 42 performing echo cancellation based on the reference signal RS. According to embodiments, the first set SC_1 of spatial components may be used as a reference signal RS. According to a further (preferred) embodiment, the deviation version of the first spatial component set SC_1 may be used as the RS reference signal. Therefore, the echo canceller 40 may optionally comprise a processor 44 (eg a so-called combiner). This combiner 44 performs processing, such as linear processing, to obtain the reference signal RS based on the first set SC_1 of spatial components.

Just for completeness, it should be noted that typically the echo cancellation performed by object 42 uses, in addition to the RS reference signal, another input signal, such as received through one or more microphones (not shown), as explained below.

Before explaining the improved embodiments, an improved echo cancellation concept is described with respect to its method steps.

Fig. 1b shows the steps of an echo cancellation method 100. In a basic implementation, acoustic echo cancellation 100 comprises three basic steps 132, 142, and 134.

Steps 132 and 134 represent the audio processing performed in the first steps. In the first stage 132, a multi-channel audio signal (see MS in FIG. 1a) that receives a first spatial component set SC_1, which is then further processed by a second stage 134 to obtain a second spatial component set SC_2. Parallel to this processing 132 plus 134, echo cancellation processing is performed in the second stage. This echo cancellation processing uses the SC_1 signal as an input, so that the first echo cancellation step is performed after step 132. Here, the basic echo cancellation step is marked with a reference number 142 performing echo cancellation based on the SC_1 signal used as a reference signal. As explained above, this process step 142 may use additional signals such as a microphone signal as input. If necessary, another processing step 144 may be provided before step 142 . This step 144 allows the processing of the first set of component signals SC_1 to obtain the reference signal RS.

Below with reference to FIG. 2, optional elements of echo cancellation processing are explained.

Fig. 2 shows an echo cancellation unit 30' comprising an audio processor 31 with two stages 32 and 34, as well as an echo cancellation unit 40' comprising two main stages 42 and 44 (see FIG. 1a) as well as one or more microphones 24 and an internal interface processing 26.

With regard to microphones 24 and speakers 14, it should be noted that they can be combined in one common housing or multiple common housings.

Regarding the internal interface processing 26, it should be noted that it can be used as a human-machine interface, for example, by using voice recognition/long range audio capture or for applications such as full duplex communication.

First, the functionality of the entire system 30' is explained.

Due to problems encountered in the prior art methods, it is proposed to use the intermediate processing signals in the soundbar instead of the loudspeaker driving signals as the AEC reference signal. The general approach is illustrated in Fig. 2. The audio processor 31 receives the multi-channel audio signal 115, for example in a format based on the 5.1 surround sound standard or in an immersive 7.1+4H format. The input audio signals 115 are processed by the first processing unit 32 to generate a set SC_1 of first spatial component signals. The resulting set SC_1 of spatial component signals is not suitable for direct playback through the speakers 14 of the soundbar (or other multi-speaker playback systems), but is the basis for further processing by the audio processing stage 39 . The first spatial component signals SC_1 are input to the second audio processing unit 34, in which the second set SC_2 of spatial component signals is generated. Typically, these component signals SC_2 are then reproduced through the speakers 14 of the soundbar, i. e. they represent speaker playback signals.

In order to generate a suitable RS reference signal for the MC-AEC 42, the first set SC_1 of spatial component signals are further processed by the spatial component combiner 44 . Typically, the spatial component combining unit 44 determines the AEC reference signal RS by linearly combining the first set of spatial component signals. In some embodiments, the defined RS AEC reference signal may also correspond to one of the SC_1 spatial component signals without further modification. In exemplary embodiments, the number of RS AEC reference signals is less than the number of SC_1 spatial component signals, i. e. block 44 combining spatial components reduces the number of signals. One advantage is that the configuration and computational complexity of the MC-AEC 42 does not depend directly on the number of loudspeakers 14. The reduction in computational complexity is particularly relevant if the number of loudspeakers 14 included in the soundbar is significantly larger compared to the number of reference speakers. RS AEC signals. Another advantage is that the statistical properties of the speaker drive signals are often not suitable for use directly as an RS AEC reference due to high correlations between different speaker channels, whereas the RS AEC reference derived from the first set of spatial component signals is usually has properties that are better suited for adaptive filtering by MC-AEC 42.

According to embodiments, the echo paths to be modeled by the MC-AEC 42 based on the RS AEC reference signal are preferably (only) slowly time varying and linear. Therefore, it is important to appropriately distribute the various processing steps between the first and second processing units 32 and 34 of the audio processor 31/processing chain 31 in the soundbar. For example, any non-linear or highly time-varying processing steps should be applied in the first processor 32, while the second processing unit should contain primarily linear time-independent processing steps 34.

In some embodiments, the first processor generates the spatial component signals SC_1 associated with the left, right, center, bass, top, and rear of the three-dimensional soundstage rendered by the soundbar 14. A suitable implementation of the spatial component combiner 44 would generate a reference the AEC signal 25 as a linear combination of the first set of spatial components, for example, by applying a time-independent downmix matrix to the spatial components of SC_1, as shown in Equation 1:

,

where R _i is the i-th AEC reference signal, β _ki are weight coefficients, C _k is the k-th spatial component signal. For example, let left, right, center, bass, top, and back be represented by C ₁ , C ₂ , C ₃ , C ₄ , C ₅ and C ₆ , respectively. If a 2-channel AEC reference is required, the spatial component combiner can combine the components by applying the following weighting factors:

In this case, the 2-channel AEC reference signal can be obtained as follows:

R ₁ \u003d C ₁ + C ₂ + C ₄

R ₂ \u003d C ₃ + C _s + C ₆

The processing performed by the echo canceller 30' can be described as follows:

1) Receiving a multi-channel audio signal 115 (at least 2-channel) and at least one microphone signal.

2) Processing the received multi-channel audio signal 115 by the first audio processor 32 to obtain the first set SC_1 of the spatial component signals. The processing may include time-varying and/or non-linear processing steps.

3) Processing the first spatial component signal set SC_1 by the second audio processor 34 to obtain the second spatial component signal set SC_2.

4) Outputting the second set SC_2 of the spatial component signals on the playback device 14 having multiple speakers (for example, a sound bar).

5) Combining the first set SC_1 of spatial component signals to obtain a set of reference signals RS AEC.

This step may be processed using processor 44.

6) Performing echo cancellation, for example, using a multi-channel adaptive filter, based on the RS AEC reference signal and the received microphone signals.

As illustrated, this step is performed by object 42.

According to a further embodiment, an additional step 5a (after step 5 before step 6) may be performed: processing the first set SC_1 of spatial components by the spatial component combiner 44, resulting in a number of AEC reference signals RS less than the number of SC_1 spatial components.

Although some aspects are described in the context of a device, it is obvious that these aspects also represent a description of the corresponding method, with the block or device corresponding to a method step or a feature of a method step. Likewise, the aspects described in the context of a method step also provide a description of the corresponding block or element, or feature of the corresponding device. Some or all of the steps of the method may be performed by (or using) a hardware device such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, some of one or more of the most important steps of the method may be performed by this device.

The encoded audio signal according to the invention may be stored on a digital storage medium or may be transmitted over a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

Depending on certain implementation requirements, embodiments of the invention may be implemented in hardware or in software. The implementation may be performed using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, having electronically readable control signals stored that interact (or are capable of interacting) with programmable computer system in such a way that the corresponding method is carried out. Therefore, the digital storage medium can be machine readable.

Some embodiments of the invention comprise a storage medium having electronically readable control signals that are capable of interacting with a programmable computer system such that one of the methods described herein is implemented.

In general, embodiments of the present invention may be implemented as a computer program product with program code, wherein the program code is configured to perform one of the methods when the computer program product is executed on the computer. The program code may, for example, be stored on a computer-readable medium.

Other embodiments comprise a computer program for carrying out one of the methods described herein, stored on a computer-readable medium.

In other words, an embodiment of the method according to the invention is therefore a computer program having program code for carrying out one of the methods described herein when the computer program is running on a computer.

Therefore, a further embodiment of the methods of the invention is a storage medium (digital storage medium or computer-readable medium) containing a recorded computer program for carrying out one of the methods described herein. The storage medium, digital storage medium or recorded data medium is usually tangible and/or permanent.

Therefore, a further embodiment of the method according to the invention is a data stream or sequence of signals representing a computer program for implementing one of the methods described herein. The data stream or signal sequence, for example, may be configured to be transmitted over a data connection, such as the Internet.

An additional embodiment includes processing means, such as a computer or programmable logic device, configured to perform one of the methods described herein.

An additional embodiment comprises a computer having a computer program installed to implement one of the methods described herein.

A further embodiment according to the invention comprises an apparatus or system capable of transmitting (eg, electronically or optically) a computer program for performing one of the methods described herein to a receiving device. The receiving device may, for example, be a computer, mobile device, storage device, or the like. The device or system, for example, may include a file server for transmitting a computer program to a receiving device.

In some embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a user-programmable gate array may interface with a microprocessor to perform one of the methods described herein. In general, the methods are preferably carried out by any hardware device.

Although in the above embodiments, audio processor 31 is described as having only a first stage 32 and a second stage 34, it should be noted that it may have additional stages, such as an amplifying stage at output 34, an input stage at input 32, and/or a stage between 32 and 34.

The above described embodiments are merely illustrative of the principles of the present invention. It should be understood that modifications and changes to the arrangements and details described herein should be apparent to those skilled in the art. Therefore, it is intended to be limited only by the scope of the following claims, and not by the specific details provided as a description and explanation of the embodiments herein.

Claims (27)

1. Acoustic echo suppression unit, comprising: - an audio processor (31) configured to receive a multi-channel audio signal (MS) and comprising: - the first stage (32), which is configured to process the multi-channel audio signal (MS) to obtain the first set (SC_1) of spatial audio components representing the intermediate signal; And - a second stage (34) which is configured to process the first set (SC_1) of spatial audio components to obtain a second set (SC_2) of spatial audio components representing loudspeaker reproduction signals; - an echo cancellation processor (42) configured to perform echo cancellation by using the first set (SC_1) of spatial audio components or a deviation version of the first set (SC_1) of spatial audio components as a reference signal (RS) and by using at least one received signal microphone wherein the deviation version is obtained by using a spatial component combining unit or by performing processing, especially linear processing. 2. Acoustic echo cancellation unit (30, 30') according to claim 1, wherein the echo cancellation processor (42) comprises a spatial component combining unit (44) configured to process a first set (SC_1) of spatial audio components to obtain a deviation version a first set (SC_1) of spatial components to be used as a reference signal (RS). 3. The acoustic echo cancellation unit (30, 30') of claim 2, wherein the spatial component combining unit (44) is configured to output one or more reference signals (RS) or a reference signal (RS) comprising one, two or more channels. 4. Block (30, 30') acoustic echo suppression according to one of paragraphs. 2 and 3, in which the spatial component combining unit (44) is configured to perform processing so as to output one or more linear combinations of the first set (SC_1) of spatial components as a reference signal (RS), and/or apply a time-independent downmix matrices to a first set (SC_1) of spatial components to obtain a reference signal (RS). 5. Block (30, 30') acoustic echo suppression according to one of paragraphs. 2-4, in which the block (44) for combining spatial components is configured to perform processing based on the following formula: , where R _i is the i-th reference signal (RS), β _ki are weight coefficients, C _k is the k-th spatial component signal. 6. Block (30, 30') acoustic echo cancellation according to one of the previous paragraphs, in which the reference signal (RS) is a subset of the first set (SC_1) of spatial audio components. 7. Block (30, 30') acoustic echo suppression according to one of the previous claims, in which each spatial component of the first set (SC_1) of spatial audio components is included in at most one signal for the reference signal (RS). 8. Block (30, 30') acoustic echo suppression according to one of paragraphs. 2-7, wherein the spatial component combining unit (44) is configured to output one or more signals for a reference signal (RS) having a reduced number compared to the number of spatial audio components of the first set (SC_1) of spatial audio components. 9. Acoustic echo cancellation unit (30, 30') according to one of the preceding claims, wherein the echo cancellation processor (42) performs echo cancellation based on at least the microphone signal received via the microphone input (24). 10. An acoustic echo cancellation unit (30, 30') according to one of the preceding claims, wherein the echo cancellation processor (42) is configured to perform echo cancellation using a single or multi-channel adaptive filter, or a single or multi-channel adaptive filter, which is configurable based on a comparison between the reference signal (RS) and the microphone signal. 11. Block (30, 30') acoustic echo suppression according to one of the previous claims, in which the first stage (32) is configured to perform non-linear processing or time-varying processing, in particular very time-varying processing to obtain the first set (SC_1 ) spatial audio components. 12. Block (30, 30') acoustic echo cancellation according to one of the previous paragraphs, in which the second stage (34) is configured to output the second set (SC_2) of spatial audio components to the playback device (14) or soundbar. 13. Block (30, 30') acoustic echo suppression according to claim 12, in which a set of spatial audio components allows you to directly control one or more electro-acoustic transducers of the playback device (14) or control one or more electro-acoustic transducers of the playback device (14) or sound panel by using one or more amplifiers. 14. Acoustic echo suppression unit (30, 30') according to one of the preceding claims, wherein the second stage (34) is configured to perform linear processing. 15. A playback device (14) comprising an acoustic echo suppression unit (30, 30') according to one of the previous paragraphs. 16. Soundbar containing block (30, 30') acoustic echo suppression according to one of paragraphs. 1-14. 17. A method for acoustic echo suppression, the method comprising the following steps: - receive a multi-channel audio signal (MS); - process (132) multi-channel audio signal (MS) to obtain the first set (SC_1) spatial audio components representing the intermediate signal; processing (134) a first set (SC_1) of spatial audio components to obtain a second set (SC_2) of spatial audio components representing loudspeaker reproduction signals; - performing (142) echo cancellation by using the first set (SC_1) of spatial audio components or a deviation version of the first set (SC_1) of spatial audio components as a reference signal (RS) and by using at least one received microphone signal; wherein the deviation version is obtained by using a spatial component combining unit or by performing processing, especially linear processing.

Images