CN115810362A - Conference terminal and feedback suppression method - Google Patents

Abstract

The invention provides a conference terminal and a feedback suppression method. The transmission sound signal is divided into sub sound signals of a plurality of frequency bands. The sound signal is transmitted for transmission via a network. Different sub-sound signals correspond to different frequency bands. The disturbed frequency bands corresponding to the squeak disturbances are detected from the sub-sound signals of those frequency bands. The power of the sub-sound signals of the disturbed frequency band increases with time. The disturbed frequency bands are influenced by squeak disturbances. The interference bearing is determined from a plurality of input sound signals picked up by a microphone array and passing only through the interfered frequency band. This disturbing azimuthal sound causes squeak noise. Determining a beam pattern of the microphone array from the interference bearing. The gain of the beam pattern in the interference direction is reduced. Thus, squeak noise can be eliminated, and all frequency bands in the input sound signal can be reserved.

Description

Conference terminal and feedback suppression method

Technical Field

The present invention relates to a voice conference, and in particular, to a conference terminal and a feedback suppression method.

Background

Teleconferencing enables people in different locations or spaces to have conversations, and the development of meeting-related devices, protocols, and applications is well established. When a multi-person teleconference is in progress, if more than two mobile devices are operating in the same space, the speakers may be experiencing unpleasant squeak (Howling).

For example, fig. 1 is a schematic diagram of a conventional sound processing architecture. Referring to fig. 1, a sound signal S1 of a mobile device D1 is played through a speaker S. The sound signal S2 received by the microphone R can be processed by the echo cancellation mechanism C to cancel the echo component belonging to the sound signal S1 via the echo path EP, and accordingly generate the sound signal S3. The audio signal S3 is transmitted to the mobile device D2 through the network and then played through the speaker S. However, if the sound played by the mobile device D2 is received by the microphone R of the mobile device D1, a closed loop system (e.g., squeak path HP) is formed, and uncomfortable squeak is also easily generated.

Fig. 2 is a time-frequency diagram illustrating squeak. Referring to fig. 2, squeak is roughly 0.8 kilohertz (kHz) and gets louder over time. It is noted that the current suppression techniques for squeak are all to eliminate sound signals at specific frequencies. However, the signal of a specific frequency in the original main audio signal to be preserved is also eliminated, thereby causing distortion of the audio hearing experience.

Disclosure of Invention

The present invention is directed to a conference terminal and a feedback suppression method that can eliminate squeak and also retain signals of all bands in the main sound signal.

According to the embodiment of the invention, the feedback suppression method is suitable for the conference terminal. The conference terminal includes a Microphone Array (Microphone Array) and a speaker. This feedback suppression method includes (but is not limited to) the following steps: the transmission sound signal is divided into sub sound signals of a plurality of frequency bands. The sound signal is transmitted for transmission via a network. The different sub-sound signals correspond to different frequency bands. The disturbed frequency bands corresponding to the squeak disturbances are detected from the sub-sound signals of those frequency bands. The power of the sub-sound signals of the disturbed one of these frequency bands increases with time. The disturbed frequency bands are influenced by squeak disturbances. The direction of interference is determined from a plurality of input sound signals picked up by the microphone array and passing only through the disturbed frequency band. This disturbing azimuthal sound causes squeak noise. The beam pattern (beam pattern) of the microphone array is determined from the direction of the interference. The gain of the beam pattern in that interference direction is reduced.

According to an embodiment of the present invention, a conference terminal includes (but is not limited to) a microphone array, a speaker, a communication transceiver, and a processor. The microphone array is used for receiving sound. The loudspeaker is used for playing sound. The communication transceiver is used for transmitting or receiving data. The processor is coupled to the microphone array, the speaker, and the communication transceiver. The processor is configured to divide the transmitted sound signal into sub-sound signals of a plurality of frequency bands, detect interfered frequency bands corresponding to squeak interference from the sub-sound signals of those frequency bands, determine an interference azimuth from a plurality of input sound signals picked up by the microphone array and passing only through the interfered frequency bands, and determine a beam pattern of the microphone array according to the interference direction. The sound signal is transmitted for transmission over a network through a communication transceiver. The different sub-sound signals correspond to different frequency bands. The power of the sub-sound signals of the disturbed one of these frequency bands increases with time. The affected frequency band is affected by squeak noise. This directionally disturbing sound causes squeak noise disturbances. The gain of the beam pattern in that interference direction is reduced.

Based on the above, according to the conference terminal and the feedback suppression method of the embodiment of the present invention, the sub sound signals whose power increases with time and the corresponding interfered frequency bands thereof are detected from the plurality of frequency bands, the interference azimuth causing squeak sound interference is determined by the microphone array, and the gain of the interference azimuth is suppressed by beamforming. Thus, all the bands in the main voice signal can be reserved, and squeak can be suppressed.

Drawings

The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.

FIG. 1 is a diagram illustrating a conventional sound processing architecture;

FIG. 2 is a time-frequency diagram illustrating squeak;

FIG. 3 is a schematic diagram of a conference call system according to an embodiment of the present invention;

FIG. 4 is a flow chart of a feedback suppression method according to an embodiment of the invention;

FIG. 5 is a schematic diagram of feedback suppression according to an embodiment of the present invention;

FIG. 6A is a schematic diagram of a beam pattern according to an embodiment of the invention;

fig. 6B is a schematic diagram of a beam pattern according to an embodiment of the invention.

Description of the reference numerals

D1, D2, a mobile device;

S1-S3, sound signals;

s, a loudspeaker;

r is a microphone;

c, echo cancellation;

EP is echo path;

HP is a squeak path;

1: a voice communication system;

10. 20, conference terminals;

50, a network;

11. 21, a radio;

13. 21, a loudspeaker;

15. 25, a communication transceiver;

17. 27 a memory;

19. 29, a processor;

S _Rx receiving voice signals during conversation;

S _Tx transmitting the sound signal;

s410 to S470 and S510 to S570;

S _B1 ～S _BL a secondary sound signal;

B _H an interfered frequency band;

θ _B an interference orientation;

S _M1 ～S _MN 、S _H1 ～S _HN 、S _B inputting a voice signal;

M1-MN are microphones;

PB1, PB2: a beam pattern;

MS, main lobe;

and SS, side valve.

Detailed Description

Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings and the description to refer to the same or like parts.

Fig. 3 is a schematic diagram of the conference call system 1 according to an embodiment of the present invention. Referring to fig. 3, the conference call system 1 includes (but is not limited to) conference terminals 10 and 20 and a cloud server 50.

The conference terminals 10,20 may be wireline phones, mobile phones, web phones, tablet computers, desktop computers, laptops or smart speakers.

The conference terminal 10 includes, but is not limited to, a microphone array 11, a speaker 13, a communication transceiver 15, a memory 17, and a processor 19.

The microphone array 11 includes a plurality of microphones of moving coil (dynamic), capacitor (Condenser), or Electret Condenser (electric Condenser) type. The microphone array 11 may also include other combinations of electronic components, analog-to-digital converters, filters, and audio processors that receive sound waves (e.g., human voice, ambient sound, machine action sound, etc.) and convert them to sound signals. In one embodiment, the microphone array 11 is used for receiving/recording sound to a speaker to obtain an input sound signal. In some embodiments, the input sound signal may include the speaker's voice, the sound emitted by the speakers 13,23, and/or other ambient sounds.

The loudspeaker 13 may be a horn or a loudspeaker. In one embodiment, the speaker 13 is used to play sound.

The communication transceiver 15 is, for example, a transceiver supporting a wired network such as an Ethernet (Ethernet), a fiber optic network, or a cable (which may include (but is not limited to) components such as a connection interface, a signal converter, a communication protocol processing chip), or a wireless network such as a Wi-Fi, a fourth generation (4G), a fifth generation (5G), or a later generation mobile network (which may include (but is not limited to) components such as an antenna, a digital-to-analog/analog-to-digital converter, a communication protocol processing chip). In one embodiment, the communication transceiver 15 is used to transmit or receive data via a network 50 (e.g., the internet, a local area network, or other type of network).

The Memory 17 may be any type of fixed or removable Random Access Memory (RAM), read Only Memory (ROM), flash Memory (flash Memory), hard Disk Drive (HDD), solid-State Drive (SSD), or the like. In one embodiment, the memory 17 is used for storing program codes, software modules, configuration configurations, data (e.g., sound signals, interfered frequency bands, interfering directions or beam patterns) or files.

The processor 19 is coupled to the microphone array 11, the speaker 13, the communication transceiver 15 and the memory 17. The Processor 19 may be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or other Programmable general purpose or special purpose Microprocessor (Microprocessor), digital Signal Processor (DSP), programmable controller, field Programmable Gate Array (FPGA), application-Specific Integrated Circuit (ASIC), or other similar components or combinations thereof. In one embodiment, the processor 19 is configured to execute all or part of the operations of the conference terminal 10, and can load and execute the software modules, files and data stored in the memory 17.

The conference terminal 20 includes, but is not limited to, a microphone array 21, a speaker 23, a communication transceiver 25, a memory 27, and a processor 29. The implementation and functions of the microphone array 21, the speaker 23, the communication transceiver 25, the memory 27 and the processor 29 may refer to the foregoing description of the microphone array 11, the speaker 13, the communication transceiver 15, the memory 17 and the processor 19, which will not be repeated herein. The processor 29 is used for executing all or part of the operations of the conference terminal 20, and can load and execute the software modules, files and data stored in the memory 27.

Hereinafter, the method according to the embodiment of the present invention will be described in conjunction with various devices, components, and modules in the conference communication system 1. The various processes of the method may be adapted according to the implementation, and are not limited thereto.

It should be noted that, for convenience of description, the same components may implement the same or similar operations, and are not repeated herein. For example, the processor 19 of the conference terminal 10 and the processor 19 of the conference terminal 20 may implement the same or similar methods of the embodiments of the present invention.

Fig. 4 is a flow chart of a feedback suppression method according to an embodiment of the invention. Referring to fig. 4, the processor 19 of the conference terminal 10 may transmit the sound signal S _TX Infrasound signal S divided into a plurality of frequency bands B1, B2, \ 8230;, BL _B1 ,S _B2 ,…,S _BL (L is a positive integer) (step S410). Specifically, the sound signal S is transmitted _TX Is for transmitting the sound signal to the conference terminal 20 via the network 50 through the communication transceiver 15. Generally, the processor 19 may record the input sound signal S obtained by the microphone array 11 _M1 ～S _MN Performs sound processing such as filtering, echo cancellation, gain adjustment, etc., and generates a transmission sound signal S based thereon _TX . In one embodiment, the processor 19 may transmit the sound signal S by Fourier transform (Fourier transform), wavelet transform, or impulse response _TX Subsound signal S divided into L (e.g. 64, 128 or 512) bands B1-BL _B1 ～S _BL . Wherein the different infrasound signals S _B1 ～S _BL Corresponding to different frequency bands B1 to BL. E.g. infrasound signal S _B1 Covering only the 600Hz to 700Hz frequency band B1 and the secondary sound signal S _B2 Only the band B2 of 700Hz to 800Hz is covered.

The processor 19 may be adapted to derive the sub-sound signals S from those bands B1-BL _B1 ～S _BL Detecting an interfered band B corresponding to squeak interference _H (step S430). Specifically, squeak noise is characterized by acoustic signals at certain frequencies becoming louder over time. It follows that if one or more of those bandsInterfered frequency band B _H Under influence of squeak noise, this is affected by the noise band B _H The power of the secondary sound signal increases over time.

In one embodiment, processor 19 may determine the interfered frequency band based on the power variation and the single frequency ratio. The power variation is related to the difference or variation of the power of the sub-sound signal of the interfered frequency band at different time points. For example, the power difference of the highest, average or other statistic of the interfered frequency band between time point t-1 and time point t. The greater the power variation, the greater the chance that this band will be subjected to squeak interference. On the other hand, the smaller the power variation, the less likely this band is to be subjected to squeak interference. In addition, the single frequency ratio is that the power of the sub-sound signal of the interfered frequency band occupies all or part of the sub-sound signal S _B1 ～S _BL The ratio of (a) to (b). The greater the proportion of tones, the greater the probability that this band will be subjected to squeak interference. On the other hand, the smaller the proportion of tones, the less likely this band will be subjected to squeak interference.

In one embodiment, processor 19 may determine that the product of the power variation of the interfered frequency band and the single frequency ratio is greater than a threshold value. Suppose a sub-sound signal S for each B1-BL at a time point t _B1 ～S _BL Corresponding power is respectively

The frequency band Bb (B is a value in 1 to L) is the interfered frequency band B _H The possibility is:

left half side

Reflecting the effect of the power change over time (i.e., the aforementioned power change). If the power of the band Bb is increased over time, the value (between-1 and 1) gets closer to 1. On the other hand, the right half

Power-integrated infrasound signal S reflecting frequency band Bb _B1 ～S _BL The ratio of the power of (1) is between 0 and 1 (i.e., the single frequency ratio). Thus, if the infrasound signal S of the band Bb _Bb Louder and louder, then the possibility

The value of (i.e., the product of the power variation and the ratio of single frequencies) will become larger and larger. Furthermore, if possible

Exceeding a defined threshold value T _H (e.g., 0.5, 0.4, or 0.45), there is a high possibility that squeak interference may occur in this band Bb. On the other hand, if there is a possibility

Does not exceed the threshold value T _H The processor 19 can determine that squeak interference has not occurred for this band Bb. The determination equation (2) for the squeak interference detection at the time t may be defined as:

therefore, when the squeak noise interference occurs, the highest possible one, the second highest one or others higher than the threshold value are the interfered frequency band B _H 。

It should be noted that the interfered frequency band B at the same time _H The number of (2) is not limited to one. If the detected condition is met (e.g., the probability or power variation exceeds the corresponding threshold), it is determined as the interfered band B _H 。

The processor 19 may receive sound from the microphone array 11 and only pass through the interfered frequency band B _H A plurality of input sound signals S _M1 ～S _MN Determining a disturbance bearing θ _B (step S450). Specifically, fig. 5 is a schematic diagram of feedback suppression according to an embodiment of the invention. Please refer toReferring to fig. 5, it is assumed that the microphone array 11 includes N microphones M1 to MN (N is a positive integer greater than 1). The sound signals recorded by the microphones M1 to MN are input sound signals S _M1 ～S _MN . It is assumed that the conference terminals 10,20 establish a conference call. For example, a conference is established by video software, voice call software, or telephone dialing, and the speaker can start speaking. After recording/reception by the microphones M1-MN, the processor 19 can obtain the input sound signal S _M1 ～S _MN 。

The processor 19 may compare the input sound signal S _M1 ～S _MN The filtering processes are performed separately (step S510). Specifically, this filtering process is, for example, to allow only the interfered frequency band B _H Passes the signal of (A) and blocks the interfered frequency band B _H Other signals pass through. In one embodiment, the processor 19 may separately convert the input sound signal S by fourier transform, wavelet transform, impulse response, or the like _M1 ～S _MN Divided into L frequency bands B1-BL and only extract the frequency band B belonging to the interfered frequency band _H Input sound signal S _H1 ～S _HN . In another embodiment, the interference may be based on the interfered frequency band B _H Setting a band pass filter, the processor 19 and passing this band pass filter separately for the input sound signal S _M1 ～S _MN Filtering to obtain an input sound signal S _H1 ～S _HN 。

In one embodiment, the processor 19 is based on the input sound signal S _H1 ～S _HN Determines the input sound signal S _H1 ～S _HN The time difference Δ t between them. The correlation referred to here corresponds to the input sound signal S _H1 ～S _HN Phase/time delay between the two. Taking two microphones M1, M2 as an example, the processor 19 is coupled to the input sound signal S _H1 ,S _H2 Cross-correlation (cross-correlation) or other correlation algorithm is used to derive the correlation. Each correlation corresponds to a phase/time delay and the processor 19 can derive the time difference Δ t based on this phase/time delay.

Notably, the direction θ from the disturbance _B The sound of (A) causing squeakAnd (4) interference. For example, the conference terminal 20 is located at the interference orientation θ of the conference terminal 10 _B . And the processor 19 may determine the interference orientation theta based on the time difference deltat and the distance between the plurality of microphones M1-MN in the microphone array 11 _B . That is, the distance traveled by the sound with the time difference Δ t is the disturbance azimuth θ _B Adjacent side in the right triangle, and the distance between two microphones is the interference orientation theta _B Is provided. Taking the two microphones M1, M2 as an example, the distance between the two microphones M1, M2 is d, and the interference orientation θ _B Comprises the following steps:

，v _s is the speed of sound propagation.

In other embodiments, processor 19 may derive interference bearing θ from other sound source localization algorithms _B 。

Referring to FIG. 4, the processor 19 may determine the interference orientation θ _B The beam pattern of the microphone array 11 is determined (step S470). In particular, beamforming (beamforming) techniques adjust parameters (e.g., phase and amplitude) of the elements of a phased array such that signals at certain angles obtain constructive interference and signals at other angles obtain destructive interference. Thus, different parameters will form different beam patterns.

Referring to FIG. 5, if squeak disturbances are detected, the processor 19 determines a disturbance orientation θ corresponding to the squeak disturbances _B Squeak elimination is performed (step S530). The processor 19 may be based on the interference orientation θ _B Determining the parameters of the beam forming (step S550), and reducing the interference direction theta of the beam pattern _B The gain of (c).

In one embodiment, processor 19 aligns null steering (null steering) of the beam pattern to the interference bearing θ _B . In another embodiment, processor 19 aligns the interference bearing θ between the main lobe (main lobe) and the side lobe (sidelobe) in the beam pattern _B 。

In one embodiment, memory 17 records parameters of the beamforming(e.g., amplitudes and phases for different microphones M1-MN) and respective interference orientations θ _B And for use by the processor 19. For example, when the orientation θ is disturbed _B At 30 degrees, the null steering in this beam pattern is toward 30 degrees. In another embodiment, processor 19 may infer the interference orientation θ through a model based machine learning algorithm _B The appropriate beam pattern is used to generate the corresponding parameters.

The processor 19 may generate the input sound signal S according to the parameters of the beamforming _B . At this time, this input sound signal S _B Signals of all frequency bands are still preserved.

In one embodiment, the processor 19 may direct the main beam or main lobe toward the talker of the conference terminal 10 based on beamforming techniques even if squeak interference is not detected.

For example, fig. 6A is a schematic diagram of a beam pattern PB1 according to an embodiment of the invention. Referring to fig. 6A, assuming that squeak interference is not detected, the beam pattern PB1 is directed forward and receives forward audio signals accordingly.

Fig. 6B is a diagram of beam pattern PB2 according to an embodiment of the invention. Referring to fig. 6B, beam pattern PB2 includes main lobe MS and side lobe SS. And the conference terminal 20 is located at the interference orientation theta of the conference terminal 10 _B . Interference orientation theta _B Substantially zero steering between the main lobe MS and the side lobe SS.

On the other hand, referring to fig. 5, the processor 19 can still receive the sound signal S according to the call _Rx (i.e. the sound signal to be played through its loudspeaker 13) to the beamformed input sound signal S _B Echo cancellation is performed (step S570).

In summary, in the conference terminal and the feedback suppression method according to the embodiments of the present invention, the microphone array technology is used to determine the interference orientations corresponding to squeak interferences and eliminate the squeak interferences based on beamforming. Therefore, squeak noise with specific frequency can be eliminated, and all frequency bands of input sound signals can be reserved.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A feedback suppression method applied to a conference terminal including a microphone array and a speaker, the feedback suppression method comprising:

dividing a transmission sound signal into sub sound signals of a plurality of frequency bands, wherein the transmission sound signal is used for transmission via a network, and different sub sound signals correspond to different frequency bands;

detecting an interfered frequency band corresponding to squeak interference according to the sub-sound signals of the frequency band, wherein the power of the sub-sound signals of the interfered frequency band in the frequency band is increased along with time, and the interfered frequency band is influenced by the squeak interference;

determining a jammer orientation from a plurality of input sound signals picked up by the microphone array and passing only through the disturbed frequency band, wherein sounds from the jammer orientation cause the squeak disturbance; and

determining a beam pattern of the microphone array from the interference bearing, wherein a gain of the beam pattern at the interference bearing is reduced.

2. The feedback suppression method according to claim 1, wherein the step of detecting the interfered frequency band corresponding to the squeak interference from the sub-sound signals of the frequency band comprises:

determining the interfered frequency band according to a power variation and a single frequency proportion, wherein the power variation is related to the difference of the power of the secondary sound signal of the interfered frequency band at different time points, and the single frequency proportion is the proportion of the power of the secondary sound signal of the interfered frequency band in the secondary sound signal.

3. The feedback suppression method according to claim 2, wherein the step of determining the interfered frequency band according to the power variation and the single frequency ratio comprises:

determining that a product of the power variation and the single frequency ratio is greater than a critical value.

4. The feedback suppression method according to claim 1, wherein the step of determining the interference bearing from the input sound signal picked up by the microphone array and passing only the interfered frequency band comprises:

determining a time difference between the input sound signals according to a correlation between the input sound signals; and

determining the interference bearing according to a distance between the time difference and a plurality of microphones of the microphone array.

5. The feedback suppression method of claim 1, wherein determining the beam pattern of the microphone array from the interference bearing comprises:

aligning a null steering of the beam pattern to the interference bearing.

6. A conference terminal, comprising:

microphone array for receiving sound;

a speaker for playing sound;

a communication transceiver to transmit or receive data; and

a processor coupled to the microphone array, the speaker, and the communication transceiver, wherein the processor is configured to:

splitting a transmit sound signal into sub sound signals of a plurality of frequency bands, wherein the transmit sound signal is for transmission over a network through the communication transceiver and different ones of the sub sound signals correspond to different ones of the frequency bands;

detecting, from the sub-sound signals of the frequency bands, disturbed frequency bands corresponding to squeak interferences, wherein the power of the sub-sound signals of the disturbed frequency bands in the frequency bands increases over time and the disturbed frequency bands are affected by the squeak interferences;

determining a jammer orientation from a plurality of input sound signals picked up by the microphone array and passing only through the disturbed frequency band, wherein sounds from the jammer orientation cause the squeak disturbance; and

determining a beam pattern of the microphone array from the interference bearing, wherein a gain of the beam pattern at the interference bearing is reduced.

7. The conference terminal of claim 6, wherein said processor is further configured to:

determining the interfered frequency band according to a power variation and a single frequency proportion, wherein the power variation is related to the difference of the power of the secondary sound signal of the interfered frequency band at different time points, and the single frequency proportion is the proportion of the power of the secondary sound signal of the interfered frequency band in the secondary sound signal.

8. The conference terminal of claim 7, wherein the processor is further configured to:

determining that a product of the power variation and the single frequency ratio is greater than a critical value.

9. The conference terminal of claim 6, wherein said processor is further configured to:

determining a time difference between the input sound signals according to a correlation between the input sound signals; and

determining the interference bearing according to a distance between the time difference and a plurality of microphones of the microphone array.

10. The conference terminal of claim 6, wherein the processor is further configured to:

aligning a null steering of the beam pattern to the interference bearing.

Images