CN113870889A - Time delay estimation method and device in echo cancellation and electronic equipment - Google Patents

Time delay estimation method and device in echo cancellation and electronic equipment Download PDF

Info

Publication number
CN113870889A
CN113870889A CN202111070181.1A CN202111070181A CN113870889A CN 113870889 A CN113870889 A CN 113870889A CN 202111070181 A CN202111070181 A CN 202111070181A CN 113870889 A CN113870889 A CN 113870889A
Authority
CN
China
Prior art keywords
signal
power spectrum
frame
time
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111070181.1A
Other languages
Chinese (zh)
Inventor
王青
郑伟军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alipay Hangzhou Information Technology Co Ltd
Original Assignee
Alipay Hangzhou Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alipay Hangzhou Information Technology Co Ltd filed Critical Alipay Hangzhou Information Technology Co Ltd
Priority to CN202111070181.1A priority Critical patent/CN113870889A/en
Publication of CN113870889A publication Critical patent/CN113870889A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L21/0232Processing in the frequency domain
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M9/00Arrangements for interconnection not involving centralised switching
    • H04M9/08Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L2021/02082Noise filtering the noise being echo, reverberation of the speech
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00Reducing energy consumption in communication networks
    • Y02D30/70Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)

Abstract

The embodiment of the specification discloses a time delay estimation method and device in echo cancellation and electronic equipment. Generating an ultrasonic signal, and superposing the ultrasonic signal and a downlink signal to generate a reference signal; collecting a mixed signal; separating a first effective signal from the mixed signal and a second effective signal from the reference signal; determining a first power spectrum of any target frame signal in the first effective signal, and determining a plurality of second power spectrums corresponding to multi-frame signals contained in the second effective signal; and determining a time delay value of the target frame signal relative to the reference signal according to the distance between the first power spectrum and the second power spectrum. Therefore, accurate estimation of a time delay value can be achieved only by performing distance analysis of a frequency spectrum based on the first effective signal and the second effective signal of the frequency band containing the ultrasonic signal, the calculated amount is small, the efficiency is high, and the robustness is high.

Description

Time delay estimation method and device in echo cancellation and electronic equipment
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to a method and an apparatus for estimating a time delay in echo cancellation, and an electronic device.
Background
With the development of the internet, video/voice calls have been widely used. In a two-way voice conversation, a mixed signal picked up by a local microphone includes both a near-end voice signal generated by local personnel and a far-end voice signal played by a playing device, which inevitably generates an echo signal for the far-end personnel, so that echo cancellation is required. In echo cancellation, the most important part is to estimate a delay value to perform subsequent linear and nonlinear echo cancellation, currently, when performing delay estimation, a comparison process is usually performed on a total picked mixed signal and a total downlink signal, a calculation amount is large, near-end speech and noise have a large influence on delay estimation, and the accuracy of delay estimation is poor.
Based on this, there is a need for a more efficient delay estimation scheme in echo cancellation.
Disclosure of Invention
One or more embodiments of the present disclosure provide a method, an apparatus, an electronic device, and a storage medium for estimating a time delay in echo cancellation, so as to solve the following technical problems: there is a need for a more efficient delay estimation scheme in echo cancellation.
To solve the above technical problem, in a first aspect, an embodiment of the present specification provides a method for estimating a time delay in echo cancellation, including:
generating an ultrasonic signal, and superposing the ultrasonic signal and a downlink signal to generate a reference signal;
acquiring a mixed signal, wherein the mixed signal comprises an echo signal generated by playing the reference signal;
separating a first effective signal from the mixed signal and separating a second effective signal from the reference signal, wherein the frequency bands of the first effective signal and the second effective signal comprise the frequency band of the ultrasonic signal;
determining a first power spectrum of any target frame signal in the first effective signal, and determining a plurality of second power spectrums corresponding to multi-frame signals contained in the second effective signal;
and determining a time delay value of the target frame signal relative to the reference signal according to the distance between the first power spectrum and the second power spectrum.
In a second aspect, an embodiment of the present specification provides a delay estimation apparatus in echo cancellation, including:
the signal generation module generates an ultrasonic signal and superposes the ultrasonic signal and the downlink signal to generate a reference signal;
the signal acquisition module is used for acquiring a mixed signal, wherein the mixed signal comprises an echo signal generated by playing the reference signal;
the signal separation module is used for separating a first effective signal from the mixed signal and separating a second effective signal from the reference signal, wherein the frequency bands of the first effective signal and the second effective signal comprise the frequency band of the ultrasonic signal;
a power spectrum determination module, configured to determine, for any target frame signal in the first effective signal, a first power spectrum of the target frame signal, and determine a plurality of second power spectrums corresponding to multiple frame signals included in the second effective signal;
and the time delay estimation module is used for determining a time delay value of the target frame signal relative to the reference signal according to the distance between the first power spectrum and the second power spectrum.
In a third aspect, embodiments of the present specification provide an electronic device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.
In a fourth aspect, embodiments of the present specification provide a non-transitory computer storage medium storing computer-executable instructions that, when read by a computer, cause the one or more processors to perform the method according to the first aspect.
At least one technical scheme adopted by one or more embodiments of the specification can achieve the following beneficial effects: generating an ultrasonic signal, and superposing the ultrasonic signal and a downlink signal to generate a reference signal; collecting a mixed signal; separating a first effective signal from the mixed signal and separating a second effective signal from the reference signal, wherein the frequency bands of the first effective signal and the second effective signal comprise the frequency band of the ultrasonic signal; determining a first power spectrum of any target frame signal in the first effective signal, and determining a plurality of second power spectrums corresponding to multi-frame signals contained in the second effective signal; and determining a time delay value of the target frame signal relative to the reference signal according to the distance between the first power spectrum and the second power spectrum. Therefore, accurate estimation of a time delay value can be achieved only by performing distance analysis of a frequency spectrum based on the first effective signal and the second effective signal of the frequency band containing the ultrasonic signal, the calculated amount is small, the efficiency is high, and the robustness is high.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.
FIG. 1 is a diagram of a system architecture provided by an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of a method for estimating a delay in echo cancellation according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a power spectrum according to embodiments of the present description;
fig. 4 is a schematic diagram of a logical structure of a delay value estimation according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a delay estimation apparatus in echo cancellation according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device provided in an embodiment of the present specification.
Detailed Description
The embodiment of the specification provides a time delay estimation method, a time delay estimation device, time delay estimation equipment and a storage medium in echo cancellation.
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any inventive step based on the embodiments of the present disclosure, shall fall within the scope of protection of the present application.
In a scenario of voice two-way communication (for example, a mobile phone makes a call, a third-party app audio/video call, a conference device multiparty audio/video call, and the like), far-end voice is processed into a downlink signal, and is sent to a playing device (for example, a speaker, a receiver) for playing, a local microphone picks up a local mixed signal (including near-end voice, noise and echo generated when the downlink signal is played), and performs uplink processing (including echo cancellation, noise reduction, volume adjustment, and the like) to form an uplink signal, and sends the uplink signal to a far end. In the process, echo cancellation is a relatively important link, and in echo cancellation, a time delay value of an echo relative to a downlink signal needs to be quickly and accurately estimated.
Based on this, the embodiment of the present application provides a more efficient delay estimation scheme. As shown in fig. 1, fig. 1 is a schematic diagram of a system architecture provided in the embodiment of the present disclosure. In the system architecture, an ultrasonic signal generator is added to generate an ultrasonic signal, the ultrasonic signal is superposed on a downlink signal to generate a reference signal, and part of effective signals containing the ultrasonic signal are extracted from a full signal in a subsequent process to be processed, so that more efficient time delay estimation is realized.
As shown in fig. 2, fig. 2 is a schematic flowchart of a method for estimating a delay in echo cancellation according to an embodiment of the present disclosure, where the method includes:
s201: and generating an ultrasonic signal, and superposing the ultrasonic signal and the downlink signal to generate a reference signal.
When the ultrasonic signal is played through the loudspeaker, high-frequency ultrasonic waves which cannot be directly perceived by human ears are generated. High frequency ultrasound refers to sound waves with frequencies in excess of 20 Hz. In the embodiment of the present specification, for convenience of subsequent processing, the frequency range of the generated ultrasonic signal may be preset to be within a smaller interval, for example, the frequency band (i.e., frequency range) of the generated ultrasonic signal during playing is preset to [20KHz, 22KHz ].
Furthermore, the ultrasonic signal and the downlink signal can be directly superimposed to generate the reference signal. The generated reference signal is played through a loudspeaker on one hand, so that corresponding far-end voice and ultrasonic waves are generated at the near end; at the same time, the generated reference signal will also enter An Echo Cancellation (AEC) device for comparison with the Echo signal, as shown in fig. 1.
Because the ultrasonic wave is not sensed by human ears, the ultrasonic wave is generated while the reference signal is played, but the near-end listening and the actual two-party communication are not influenced. Due to the high frequency of the ultrasonic wave, when the ultrasonic wave in the room with high reverberation meets the wall or other obstacles to be reflected and collected by the microphone, the generated reverberation echo is also small, which is beneficial to the subsequent time delay estimation.
S203, collecting a mixed signal, wherein the mixed signal comprises an echo signal generated by playing the reference signal.
The acquired mixed signal is a signal stream composed of multi-frame signals. The mixed signal includes a speech signal, noise and an echo signal generated when the near end speaks. The echo signal is a signal generated by the reference signal after being played by the near-end playing device and collected by the microphone of the near-end voice. Obviously, the echo signal may generally have a change in energy (attenuated by the environment or increased by the device) compared to the reference signal, but still have a corresponding relationship in frequency, power distribution and phase with respect to the reference signal.
S205, separating a first effective signal from the mixed signal, and separating a second effective signal from the reference signal, wherein the frequency bands of the first effective signal and the second effective signal include the frequency band of the ultrasonic signal.
Because the ultrasonic signal frequency is high, the Fast Fourier Transform (FFT) processing of the whole acquired mixed signal is not required in the processing process, and only the part of the mixed signal containing the frequency band of the ultrasonic signal is required. Thus, it is possible to use a form of separation filtering to directly separate a first effective signal containing a frequency band of the ultrasonic signal from the acquired mixed signal and to separate a second effective signal containing a frequency band of the ultrasonic signal from the reference signal. In general, the frequency bands of the first useful signal and the second useful signal may also be ultrasonic frequency bands that are not audible to the human ear.
For example, if the frequency band of the generated ultrasonic wave is [20KHz, 22KHz ], and the sampling frequency band of the sampler is 0 to 48KHz, then the valid signal of 18KHz to 24KHz can be directly filtered out, and the sampling rate is set to 12 KHz. Thereby separating a first significant signal including multi-frame signals each including 12K samples from the mixed signal, and separating a second significant signal including multi-frame signals from the reference signal.
S207, for any target frame signal in the first effective signal, determining a first power spectrum of the target frame signal, and determining a plurality of second power spectrums corresponding to multi-frame signals included in the second effective signal.
Furthermore, the FFT may be performed on the first effective signal and the second effective signal obtained by the separation, so as to obtain a corresponding first complex spectrum signal and a corresponding second complex spectrum signal.
And calculating the power spectrum based on the first complex spectrum signal and the second complex spectrum signal, thereby obtaining the power spectrum corresponding to each frame. That is, for any target frame signal in the first effective signal, a first power spectrum of the target frame signal is determined, and a plurality of second power spectrums corresponding to multi-frame signals included in the second effective signal are determined.
Meanwhile, in practical application, the current frame signal d (n) may be sequentially regarded as the target frame signal according to the frame sequence in the first effective signal to calculate the first power spectrum, and the plurality of second power spectrums are buffered in the buffer area in advance. Taking the current frame in the first valid signal as the target frame signal d (n), the second power spectrum X (n-m) … X (n) corresponding to the m frames (i.e. the n-m frame to the n frame) before d (n) can be buffered in the buffer.
Since each frame actually contains a plurality of samples, for example, at a sampling rate of 12kHz (i.e., 12000 samples per second), if the frame shift between each frame is set to 10ms, 120 samples are contained between each frame. On the power spectrum, it can be known that each sampling point has its own corresponding power value. The power spectrum of each frame characterizes the distribution of the signal power in the frequency domain at the time corresponding to the frame, specifically, the power spectrum contains some amplitude information in the frequency spectrum, and the phase information is discarded. Fig. 3 is a schematic diagram of a power spectrum according to an embodiment of the present disclosure, as shown in fig. 3. In practical application, a plurality of frequency points can be taken from the frequency band range of the ultrasonic waves contained in the effective signals, for example, if the frequency band of the effective signals is 18kHz to 24kHz and the frequency band range of the ultrasonic waves is 20kHz to 22kHz, 128 frequency points can be taken in the range of 20kHz to 22kHz in a uniformly distributed manner, and then the power spectrum can be represented in a discrete array or vector form. For example, the first power spectrum of the target frame signal may be represented as (power value 1, power value 2 … …, power value 128), where each power value corresponds to a frequency point, and a vector with a dimension of 128 dimensions is obtained. Similarly, vectorization can also be performed in a similar manner for any second power spectrum.
S209, determining a time delay value of the target frame signal relative to the reference signal according to the distance between the first power spectrum and the second power spectrum.
It should be noted that, in the embodiment of the present specification, the first effective signal and the second effective signal obtained by separation are both high-frequency signals, and human voice has been filtered out. Therefore, in a general situation, if the echo attenuation influence is small, for a target frame in the first valid signal and a corresponding frame to be searched in the second valid signal, if there is a correspondence between two frame signals, it is obvious that the corresponding first power spectrum and second power spectrum should be closer. Therefore, the time delay value of the target frame signal relative to the reference signal can be determined according to the distance between the first power spectrum and the second power spectrum.
For example, after determining the current frame as the target frame signal and determining the first power spectrum of the target frame signal, a search may be performed from the buffer, and a second power spectrum whose distance from the first power spectrum meets a preset condition (which may include that the distance is lower than a preset threshold, and/or that the distance is sorted from small to large, etc.) may be determined as the corresponding frame, so that the delay value of the target frame signal relative to the reference signal may be determined according to the time difference between the target frame signal and the corresponding frame.
At least one technical scheme adopted by one or more embodiments of the specification can achieve the following beneficial effects: generating an ultrasonic signal, and superposing the ultrasonic signal and a downlink signal to generate a reference signal; collecting a mixed signal; separating a first effective signal from the mixed signal and separating a second effective signal from the reference signal, wherein the frequency bands of the first effective signal and the second effective signal comprise the frequency band of the ultrasonic signal; determining a first power spectrum of any target frame signal in the first effective signal, and determining a plurality of second power spectrums corresponding to multi-frame signals contained in the second effective signal; and determining a time delay value of the target frame signal relative to the reference signal according to the distance between the first power spectrum and the second power spectrum. Therefore, accurate estimation of a time delay value can be achieved only by performing distance analysis of a frequency spectrum based on the first effective signal and the second effective signal of the frequency band containing the ultrasonic signal, the calculated amount is small, the efficiency is high, and the robustness is high.
In one embodiment, when generating the ultrasonic wave, the ultrasonic wave signal (i.e., the frequency-converted ultrasonic signal) having different frame frequencies in the period may be generated based on a preset window duration as the period. Specifically, the preset window duration may be related to the frame duration setting, for example, the preset window duration may be 100 frames; and randomly extracting the frequency of each frame within a preset range in one period, for example, generating one hundred frequencies from 20kHz, 20.2kHz, 20.4kHz … … to 22kHz at intervals of 0.2kHz within the range from 20kHz to 22kHz, and randomly extracting the frequency of the ultrasonic signal generated by each frame from the one hundred frequencies, ensuring that the absolute value of the difference of the frequencies of any two frames of ultrasonic signals within one period is not lower than 0.2kHz, thereby generating ultrasonic signals with different frame frequencies within the period, and highlighting the frequency difference between each frame of signals within one period. Because the difference between each frame is large, the distance between the first power spectrum and each second power spectrum has a large difference, so that the discrimination of the distance to the second power spectrum is improved, and the accurate calculation of the delay value is facilitated.
In an embodiment, the determination of the delay value may be performed by using a coarse delay search method, that is, m second power spectrums corresponding to m frames of signals before a target signal frame are obtained from a buffer, distances between a first power spectrum and each second power spectrum are respectively calculated, and then the second power spectrum with the minimum distance from the first power spectrum is determined as the target power spectrum; determining a first time point corresponding to the first power spectrum, and determining a second time point corresponding to the target power spectrum; determining a time difference between the first time point and the second time point as the delay value. In this manner, since the first time point and the second time point both correspond to one frame signal, the determined delay value is an integer multiple of the duration of one frame (for example, the delay value is 10 frames). For example, for a sampling rate of 12kHz, if a frame selected during FFT is shifted to 120 samples (i.e., a frame includes 120 samples), it can be known that the duration of a frame is 10ms, and the time delay value obtained by calculation can be accurate to the order of 10 ms.
In an embodiment, a more accurate delay value may be obtained by further adopting the following manner of fine delay search based on the coarse delay search. Specifically, a first time domain characteristic of a plurality of sampling point signals in the target frame signal can be determined; acquiring a continuous multi-frame time delay signal including a frame signal corresponding to a second time point, and determining second time domain characteristics of a plurality of sampling point signals in the time delay signal; cross-correlating the first time domain feature and the second time domain feature to generate a cross-correlation result; and determining the time difference corresponding to the maximum cross-correlation result as the time delay value.
For example, assuming that the current frame is the 100 th frame, when the current frame is taken as the target frame signal, and the delay value is determined to be 10 frames at the same time, the delay signals of consecutive frames (for example, from the 89 th frame to the 91 th frame) including the 90 th frame signal may be selected, and the second time domain characteristics of the multiple sampling point signals in the delay signals are respectively determined, so as to perform cross correlation, and obtain a delay value with finer granularity based on the cross correlation result.
The time domain feature (either the first time domain feature or the second time domain feature) characterizes a change in frequency of a signal over time. The cross-correlation result represents the relationship between two signals after a period of time difference, if the correlation degree of the two signals is high, the cross-correlation result is larger, and if the correlation degree of the two signals is low, the cross-correlation result is smaller. Therefore, a plurality of different time differences (for example, an integral multiple of the time intervals of the sampling points is used as a time difference sequence) may be selected, the first time domain characteristics of the plurality of sampling point signals in the target frame signal and the first time domain characteristics of the plurality of sampling point signals in the delay signal are cross-correlated based on the plurality of different selected time differences, so as to obtain a plurality of different cross-correlation results, when the cross-correlation has a maximum value, it indicates that the two signals are closest in shape at this time, that is, the time difference at this time is the delay value of the target frame signal relative to the reference signal. In other words, in this way delay values at the granularity of the time interval of the sample points can be derived from the result of the cross-correlation.
For example, when 12kHz is used as the sampling frequency, the time interval corresponding to two sampling points is 0.083ms, and the obtained delay value can reach the precision of integral multiple of 0.083ms, which is beneficial to subsequent filtering and nonlinear processing.
As shown in fig. 4, fig. 4 is a schematic diagram of a logic structure of a delay value estimation provided in the embodiment of the present disclosure. In the diagram, Delay0 is the Delay value given by the coarse Delay search, and Delay1 is the more accurate Delay value obtained by performing the fine Delay search based on the coarse Delay search.
In one embodiment, the time-delayed signal may be obtained as follows: acquiring frame signals of continuous appointed frame numbers before and after the second time point to form the time delay signal; or, with the second time point as an end point, acquiring frame signals of specified frame numbers which are continuous before the second time point, and forming the time delay signal. For example, assuming that the number of frames corresponding to the second time point is a 90 th frame, frame signals of consecutive frames before and after the 90 th frame, that is, frames 89 th to 91 th, may be acquired to form a delay signal; or, taking the 90 th frame as an end point, acquiring signals of 2 consecutive frames before the 90 th frame, namely signals of 88 th to 90 th frames, and forming the time delay signal. By the method, the signals corresponding to the current frame always exist in the obtained time delay signals, so that the accuracy of time delay fine search is improved.
In an embodiment, since the ultrasonic wave in the reference signal undergoes echo attenuation after propagation, it appears that the first power spectrum and the second power spectrum are similar in shape, and then the distance between them can be calculated as follows: binarizing the first power spectrum and the second power spectrum based on a preset power threshold and/or a preset frequency threshold to generate a binarized first power spectrum and a binarized second power spectrum; and determining the time delay value of the target frame signal relative to the reference signal according to the distance between the binarized first power spectrum and the binarized second power spectrum.
Namely, the first power spectrum is binarized by using a first power threshold (a power value which is larger than the power threshold is about to be set to 1, and the power threshold which is not larger than the power threshold is about to be 0), the second power spectrum is binarized by using a second power threshold, and the first power threshold and the second power threshold can be set based on experience, so that the values of the first power spectrum and the second power spectrum which have corresponding relations after binarization are the same on corresponding frequency bands, and further, the time delay value of the target frame signal relative to the reference signal can be determined based on the time delay coarse search mode.
For example, for a first power spectrum, assuming that the corresponding vector is (power value 1, power value 2 … …, power value 128), after binarization is performed based on the first power threshold, the possible value is (0, 1 … …, 1), in this way, there are also multiple corresponding vectors corresponding to the second power spectrum in the buffer. Obviously, if the target signal corresponds to a certain frame in the reference signal, the vector corresponding to the second power spectrum in the reference signal will be similar in shape to the first power spectrum, that is, theoretically, the value of the vector corresponding to the second power spectrum is x times (x is greater than 1) that of the first power spectrum, i.e., the theoretical value may be (x power value 1, x power value 2 … …, x power value 128). Based on this, the second power spectrum may be binarized using another larger second power threshold (e.g., the second power threshold x × the first power threshold), resulting in the same similar value (0, 1 … …, 1).
Of course, in practical applications, due to the presence of noise and device-related effects, the two binarized power spectrums may not be exactly the same (i.e., the distance may not be zero), but based on this idea, the distance between each binarized second power spectrum and the binarized first power spectrum in the buffer may be calculated separately, and the delay value of the target frame signal with respect to the reference signal may be determined according to the binarized second power spectrum with the smallest distance, as in the manner of the aforementioned coarse delay search. By the method, the influence of echo attenuation on the power spectrum of the effective signal can be avoided, and the accuracy of time delay value estimation is improved.
In another embodiment, in order to eliminate the interference of the signals of the irrelevant frequency bands on the detection of the effective signal, the system may also perform a test of a plurality of preset frequency bands within the frequency band range of the effective signal at the time of starting. For example, when the frequency band of the effective signal is 20 kHz-22 kHz, the energy of four frequency bands of 20 kHz-20.5 kHz, 20.5 kHz-21 kHz, 21 kHz-21.5 kHz and 21.5 kHz-22 kHz is respectively tested, at this time, whether other equipment interference exists on the site is firstly detected, the frequency band with low noise is selected as the frequency band generated by subsequent ultrasound, and the ultrasound generator periodically generates signals in the frequency band range; when the power spectrum is calculated by subsequent time delay estimation, the calculation is only carried out on the frequency band, and the values of the binarized power spectrum on other frequency bands can be directly set to be 0 so as to reduce interference. For example, if it is found in advance that the frequency band of 20.5kHz to 21kHz has less interference and noise, only the ultrasonic wave of 20.5kHz to 21kHz may be generated, and when the power spectrum of the extracted effective signal is subsequently binarized, the values of the frequency bands other than 20.5kHz to 21kHz (i.e., frequency thresholds) in the first power spectrum and the second power spectrum are both directly set to 0, so as to reduce the influence of the environmental noise on the estimation of the delay value.
It should be noted that, binarization may be performed in only one manner or in both manners simultaneously, based on a preset power threshold or a preset frequency threshold, and the two manners do not conflict with each other. In addition, the system can detect the environment at any time when being started to determine the optimal frequency threshold, namely the frequency threshold changes along with the environment in the frequency band range of the effective signal, and the adaptability of the frequency threshold to the environment is improved.
In one embodiment, after determining the delay value, linear or non-linear echo cancellation may also be performed based on the delay value. For example, assuming no echo attenuation, the hybrid signal and the reference signal may be time-aligned according to the delay value; and sending the aligned mixed signal and the reference signal to a linear filtering and nonlinear processing module for echo cancellation to generate an uplink signal. If echo attenuation exists, the mixed signal can be scaled based on the attenuation coefficient, the scaled mixed signal is aligned with the reference signal, and the aligned mixed signal is sent to the linear filtering and nonlinear processing module for echo cancellation to generate an uplink signal, so that accurate echo cancellation is realized.
Based on the same idea, the embodiments of the present specification further provide an apparatus and an electronic device corresponding to the above method, as shown in fig. 5 and fig. 6.
Fig. 5 is a schematic structural diagram of a delay estimation apparatus in echo cancellation according to an embodiment of the present disclosure, where the apparatus includes:
a signal generating module 501, configured to generate an ultrasonic signal, and superimpose the ultrasonic signal and the downlink signal to generate a reference signal;
a signal collecting module 503, configured to collect a mixed signal, where the mixed signal includes an echo signal generated by playing the reference signal;
a signal separation module 505, configured to separate a first effective signal from the mixed signal and a second effective signal from the reference signal, where frequency bands of the first effective signal and the second effective signal include a frequency band of the ultrasonic signal;
a power spectrum determining module 507, configured to determine, for any target frame signal in the first effective signal, a first power spectrum of the target frame signal, and determine a plurality of second power spectrums corresponding to multiple frame signals included in the second effective signal;
and a delay estimation module 509, configured to determine a delay value of the target frame signal relative to the reference signal according to a distance between the first power spectrum and the second power spectrum.
Further, the signal generating module 501 generates the ultrasonic signals with different frame frequencies in a preset window duration as a period.
Further, the delay estimation module 509 determines a second power spectrum with the minimum distance from the first power spectrum as a target power spectrum; determining a first time point corresponding to the first power spectrum, and determining a second time point corresponding to the target power spectrum; determining a time difference between the first time point and the second time point as the delay value.
Further, when a frame signal includes a plurality of sampling point signals, the delay estimation module 509 determines first time domain characteristics of the plurality of sampling point signals in the target frame signal; acquiring a continuous multi-frame time delay signal including a frame signal corresponding to a second time point, and determining second time domain characteristics of a plurality of sampling point signals in the time delay signal; cross-correlating the first time domain feature and the second time domain feature to generate a cross-correlation result; and determining the time difference corresponding to the maximum cross-correlation result as the time delay value.
Further, the delay estimation module 509 acquires frame signals of consecutive specified frames before and after the second time point to form the delay signal; or, with the second time point as an end point, acquiring frame signals of specified frame numbers which are continuous before the second time point, and forming the time delay signal.
Further, the delay estimation module 509 binarizes the first power spectrum and the second power spectrum based on a preset power threshold to generate a binarized first power spectrum and a binarized second power spectrum; and determining the time delay value of the target frame signal relative to the reference signal according to the distance between the binarized first power spectrum and the binarized second power spectrum.
Further, the apparatus further includes an echo cancellation module 511, which performs time alignment on the mixed signal and the reference signal according to the delay value; and performing echo cancellation according to the aligned mixed signal and the reference signal to generate an uplink signal.
Fig. 6 is a schematic structural diagram of an electronic device provided in an embodiment of the present specification, where the electronic device includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method of fig. 2.
Based on the same idea, the embodiments of the present specification further provide a non-volatile computer storage medium corresponding to the method described above, and store computer-executable instructions, where the instructions cause one or more processors to execute the method described in fig. 2 after the computer reads the computer instructions in the storage medium.
In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually making an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Hardware Description Language), traffic, pl (core universal Programming Language), HDCal (jhdware Description Language), lang, Lola, HDL, laspam, hardward Description Language (vhr Description Language), vhal (Hardware Description Language), and vhigh-Language, which are currently used in most common. It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may thus be considered a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the various elements may be implemented in the same one or more software and/or hardware implementations of the present description.
As will be appreciated by one skilled in the art, the present specification embodiments may be provided as a method, system, or computer program product. Accordingly, embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The description has been presented with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the description. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiments of the apparatus, the device, and the nonvolatile computer storage medium, since they are substantially similar to the embodiments of the method, the description is simple, and for the relevant points, reference may be made to the partial description of the embodiments of the method.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The above description is merely one or more embodiments of the present disclosure and is not intended to limit the present disclosure. Various modifications and alterations to one or more embodiments of the present description will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement or the like made within the spirit and principle of one or more embodiments of the present specification should be included in the scope of the claims of the present specification.

Claims (15)

1. A method of delay estimation in echo cancellation, comprising:
generating an ultrasonic signal, and superposing the ultrasonic signal and a downlink signal to generate a reference signal;
acquiring a mixed signal, wherein the mixed signal comprises an echo signal generated by playing the reference signal;
separating a first effective signal from the mixed signal and separating a second effective signal from the reference signal, wherein the frequency bands of the first effective signal and the second effective signal comprise the frequency band of the ultrasonic signal;
determining a first power spectrum of any target frame signal in the first effective signal, and determining a plurality of second power spectrums corresponding to multi-frame signals contained in the second effective signal;
and determining a time delay value of the target frame signal relative to the reference signal according to the distance between the first power spectrum and the second power spectrum.
2. The method of claim 1, wherein generating an ultrasonic signal comprises:
and generating ultrasonic signals with different frame frequencies in a period by taking a preset window duration as the period.
3. The method of claim 1, determining a delay value of the target frame signal relative to the reference signal based on the distance of the first and second power spectra, comprising:
determining a second power spectrum having a minimum distance from the first power spectrum as a target power spectrum;
determining a first time point corresponding to the first power spectrum, and determining a second time point corresponding to the target power spectrum;
determining a time difference between the first time point and the second time point as the delay value.
4. The method as claimed in claim 3, wherein when a frame signal contains a plurality of sample point signals, the method further comprises:
determining first time domain characteristics of a plurality of sampling point signals in the target frame signal;
acquiring a continuous multi-frame time delay signal including a frame signal corresponding to a second time point, and determining second time domain characteristics of a plurality of sampling point signals in the time delay signal;
cross-correlating the first time domain feature and the second time domain feature to generate a cross-correlation result;
and determining the time difference corresponding to the maximum cross-correlation result as the time delay value.
5. The method of claim 4, wherein obtaining the time delay signals of the consecutive frames including the frame signal corresponding to the second time point comprises:
acquiring frame signals of continuous appointed frame numbers before and after the second time point to form the time delay signal;
or, with the second time point as an end point, acquiring frame signals of specified frame numbers which are continuous before the second time point, and forming the time delay signal.
6. The method of claim 1, wherein determining a delay value of the target frame signal relative to the reference signal as a function of the distance of the first and second power spectra comprises:
binarizing the first power spectrum and the second power spectrum based on a preset power threshold and/or a preset frequency threshold to generate a binarized first power spectrum and a binarized second power spectrum;
and determining the time delay value of the target frame signal relative to the reference signal according to the distance between the binarized first power spectrum and the binarized second power spectrum.
7. The method of claim 1, wherein the method further comprises:
time-aligning the hybrid signal and the reference signal according to the delay value;
and performing echo cancellation according to the aligned mixed signal and the reference signal to generate an uplink signal.
8. A delay estimation apparatus in echo cancellation, comprising:
the signal generation module generates an ultrasonic signal and superposes the ultrasonic signal and the downlink signal to generate a reference signal;
the signal acquisition module is used for acquiring a mixed signal, wherein the mixed signal comprises an echo signal generated by playing the reference signal;
the signal separation module is used for separating a first effective signal from the mixed signal and separating a second effective signal from the reference signal, wherein the frequency bands of the first effective signal and the second effective signal comprise the frequency band of the ultrasonic signal;
a power spectrum determination module, configured to determine, for any target frame signal in the first effective signal, a first power spectrum of the target frame signal, and determine a plurality of second power spectrums corresponding to multiple frame signals included in the second effective signal;
and the time delay estimation module is used for determining a time delay value of the target frame signal relative to the reference signal according to the distance between the first power spectrum and the second power spectrum.
9. The apparatus of claim 8, wherein the signal generating module generates the ultrasonic signals with different frame frequencies in a period of a preset window duration.
10. The apparatus of claim 8, the latency estimation module to determine a second power spectrum having a minimum distance from the first power spectrum as a target power spectrum; determining a first time point corresponding to the first power spectrum, and determining a second time point corresponding to the target power spectrum; determining a time difference between the first time point and the second time point as the delay value.
11. The apparatus of claim 10, wherein when a frame signal comprises a plurality of sampling point signals, the delay estimation module determines a first time domain characteristic of the plurality of sampling point signals in the target frame signal; acquiring a continuous multi-frame time delay signal including a frame signal corresponding to a second time point, and determining second time domain characteristics of a plurality of sampling point signals in the time delay signal; cross-correlating the first time domain feature and the second time domain feature to generate a cross-correlation result; and determining the time difference corresponding to the maximum cross-correlation result as the time delay value.
12. The apparatus of claim 11, wherein the delay estimation module obtains frame signals of a specified number of consecutive frames before and after the second time point to form the delay signal; or, with the second time point as an end point, acquiring frame signals of specified frame numbers which are continuous before the second time point, and forming the time delay signal.
13. The apparatus of claim 8, wherein the delay estimation module generates a first and a second binarized power spectrums by binarizing the first and the second power spectrums based on a preset power threshold; and determining the time delay value of the target frame signal relative to the reference signal according to the distance between the binarized first power spectrum and the binarized second power spectrum.
14. The apparatus of claim 8, further comprising an echo cancellation module to time align the hybrid signal and the reference signal according to the delay value; and performing echo cancellation according to the aligned mixed signal and the reference signal to generate an uplink signal.
15. An electronic device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of claims 1-7.
CN202111070181.1A 2021-09-13 2021-09-13 Time delay estimation method and device in echo cancellation and electronic equipment Pending CN113870889A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111070181.1A CN113870889A (en) 2021-09-13 2021-09-13 Time delay estimation method and device in echo cancellation and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111070181.1A CN113870889A (en) 2021-09-13 2021-09-13 Time delay estimation method and device in echo cancellation and electronic equipment

Publications (1)

Publication Number Publication Date
CN113870889A true CN113870889A (en) 2021-12-31

Family

ID=78995687

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111070181.1A Pending CN113870889A (en) 2021-09-13 2021-09-13 Time delay estimation method and device in echo cancellation and electronic equipment

Country Status (1)

Country Link
CN (1) CN113870889A (en)

Similar Documents

Publication Publication Date Title
CN111418010B (en) Multi-microphone noise reduction method and device and terminal equipment
KR101444100B1 (en) Noise cancelling method and apparatus from the mixed sound
CN101430882B (en) Method and apparatus for restraining wind noise
US9343056B1 (en) Wind noise detection and suppression
JP6703525B2 (en) Method and device for enhancing sound source
CN105118522B (en) Noise detection method and device
CN105578097A (en) Video recording method and terminal
US8423357B2 (en) System and method for biometric acoustic noise reduction
CN104205212A (en) Talker collision in auditory scene
WO2022256577A1 (en) A method of speech enhancement and a mobile computing device implementing the method
CN107452398B (en) Echo acquisition method, electronic device and computer readable storage medium
CN111402910B (en) Method and equipment for eliminating echo
WO2017045512A1 (en) Voice recognition method and apparatus, terminal, and voice recognition device
Yegnanarayana et al. Study of robustness of zero frequency resonator method for extraction of fundamental frequency
CN113870889A (en) Time delay estimation method and device in echo cancellation and electronic equipment
KR20200095370A (en) Detection of fricatives in speech signals
JP2004109712A (en) Speaker's direction detecting device
CN113316075B (en) Howling detection method and device and electronic equipment
CN109378012B (en) Noise reduction method and system for recording audio by single-channel voice equipment
CN111724808A (en) Audio signal processing method, device, terminal and storage medium
Golokolenko et al. Investigation of electric network frequency for synchronization of low cost and wireless sound cards
Gong et al. Noise power spectral density matrix estimation based on modified IMCRA
EP4270983A1 (en) Ear-mounted type device and reproduction method
Romoli et al. A voice activity detection algorithm for multichannel acoustic echo cancellation exploiting fundamental frequency estimation
WO2023125537A1 (en) Sound signal processing method and apparatus, and device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination