CN115835093A - Audio processing method, device, electronic equipment and computer readable storage medium - Google Patents
Audio processing method, device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN115835093A CN115835093A CN202211376944.XA CN202211376944A CN115835093A CN 115835093 A CN115835093 A CN 115835093A CN 202211376944 A CN202211376944 A CN 202211376944A CN 115835093 A CN115835093 A CN 115835093A
- Authority
- CN
- China
- Prior art keywords
- signal
- audio
- modulation
- audio output
- compensation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 23
- 230000005236 sound signal Effects 0.000 claims abstract description 119
- 238000005070 sampling Methods 0.000 claims abstract description 71
- 230000004044 response Effects 0.000 claims description 55
- 238000000034 method Methods 0.000 claims description 46
- 230000015572 biosynthetic process Effects 0.000 claims description 16
- 238000003786 synthesis reaction Methods 0.000 claims description 16
- 238000000354 decomposition reaction Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 5
- 230000002194 synthesizing effect Effects 0.000 claims description 5
- 239000002131 composite material Substances 0.000 claims description 3
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 238000010924 continuous production Methods 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/02—Circuits for transducers, loudspeakers or microphones for preventing acoustic reaction, i.e. acoustic oscillatory feedback
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
The embodiment of the invention provides an audio processing method and device, electronic equipment and a computer readable storage medium, and relates to the field of voice processing. The audio processing method comprises the following steps: acquiring an audio input signal of a current sampling period, performing audio signal processing on the audio input signal to obtain an audio output signal, and playing the audio output signal; the audio signal processing includes: acquiring a front audio output signal, wherein the front audio output signal is an audio output signal of a previous sampling period, acquiring a feedback signal according to the front audio output signal, performing signal compensation on an audio input signal according to the feedback signal to obtain a compensation audio signal, and performing nonlinear modulation on the phase of the compensation audio signal. Compared with the prior art, the audio processing method, the audio processing device, the electronic equipment and the computer-readable storage medium provided by the embodiment of the invention can reduce the distortion of the audio signal played by the loudspeaker while eliminating howling.
Description
Technical Field
The present invention relates to the field of voice communication, and in particular, to an audio processing method, apparatus, electronic device, and computer-readable storage medium.
Background
With the development of the technology, audio equipment such as earphones, loudspeakers, auxiliary listening products and the like are widely applied. However, in order to achieve different functions of noise reduction, sound amplification, auxiliary listening, etc., a microphone and a speaker are installed in the existing audio device at the same time. In the audio equipment provided with the microphone and the loudspeaker, the sound played by the loudspeaker is collected by the microphone, the sound path forms a closed loop, and the signals are continuously superposed and amplified in the sound feedback loop to form positive feedback so as to generate single-frequency howling. In order to fundamentally eliminate howling, the existing echo suppression scheme needs to eliminate a feedback signal fed back from a loudspeaker to a microphone from an audio signal collected by the microphone, but because the microphone and the loudspeaker are closer to each other, a feedback path is shorter, correlation between a microphone input signal and a reference signal played by the loudspeaker is stronger, and the existing echo suppression scheme can cause distortion of a voice signal of a wearer played by the loudspeaker, so that user experience is influenced.
Disclosure of Invention
Objects of the present invention include, for example, providing an audio processing method, apparatus, electronic device and computer readable storage medium capable of reducing distortion of an audio signal played by a speaker while eliminating howling.
Embodiments of the invention may be implemented as follows: in a first aspect, the present invention provides an audio processing method applied to an audio processing apparatus including an audio input device and an audio output device, the method including: acquiring an audio input signal acquired by the audio input device at the current sampling time period, performing audio signal processing on the audio input signal to obtain an audio output signal, and playing the audio output signal through the audio output device; the audio signal processing comprises at least: acquiring a front audio output signal, wherein the front audio output signal is the audio output signal in the previous sampling period, acquiring a feedback signal according to the front audio output signal, performing signal compensation on the audio input signal according to the feedback signal to obtain a compensation audio signal, and performing nonlinear modulation on the phase of the compensation audio signal.
In an alternative embodiment, the audio signal processing further comprises: acquiring a front compensation audio signal, wherein the front compensation audio signal is a compensation audio signal of a previous sampling period; calculating to obtain feedback frequency response data according to the front audio output signal and the front compensation audio signal, and judging whether the feedback frequency response data is larger than preset frequency response data; and if the feedback frequency response data is larger than the preset frequency response data, performing nonlinear modulation on the phase of the compensation audio signal.
In an alternative embodiment, the calculating feedback frequency response data according to the pre-audio output signal and the pre-compensation audio signal includes: acquiring front feedback frequency response data, wherein the front feedback frequency response data is the feedback frequency response data of a previous sampling period; and calculating the feedback frequency response data of the current sampling time period according to the pre-audio output signal, the compensation audio signal and the pre-feedback frequency response data.
In an alternative embodiment, said calculating the feedback frequency response data of the current sampling period according to the pre-audio output signal, the pre-compensation audio signal, and the pre-feedback frequency response data includes: determining the sampling number K according to the length of a preset feedback path; acquiring the audio output signals of K sampling time periods adjacent to the current sampling time period to form an audio output array; and calculating the feedback frequency response data of the current sampling time period according to the audio output array, the compensation audio signal and the front feedback frequency response data.
In an alternative embodiment, the non-linearly modulating the phase of the compensated audio signal includes: performing sub-band decomposition on the compensation audio signal according to the frequency to obtain a plurality of sub-band signals; respectively carrying out nonlinear modulation on the phase of each sub-band signal according to a first modulation method to obtain a plurality of modulated sub-band signals; and performing signal synthesis on the plurality of modulated subband signals.
In an alternative embodiment, the non-linearly modulating the phase of the compensated audio signal includes: performing sub-band decomposition on the compensation audio signal according to the frequency to obtain a plurality of sub-band signals; performing convolution and amplitude limiting processing on each sub-band signal respectively; performing signal synthesis on the plurality of sub-band signals after the convolution and amplitude limiting processing to obtain a synthesized signal; the phase of the composite signal is non-linearly modulated according to a first modulation method.
In an optional embodiment, the first modulation method includes: acquiring a first modulation parameter and a second modulation parameter, and constructing a first modulation formula according to the first modulation parameter and the second modulation parameter, wherein the first modulation parameter is used for adjusting the non-linearity degree of the first modulation formula, and the second modulation parameter is used for adjusting the slope of the first modulation formula; performing nonlinear modulation using the first modulation formula.
In an alternative embodiment, the non-linearly modulating the phase of the compensated audio signal includes: performing sub-band decomposition on the compensation audio signal according to the frequency to obtain a plurality of sub-band signals; acquiring the sub-band signal with the frequency greater than the preset frequency as a target sub-band signal; respectively carrying out nonlinear modulation on the phase of each target subband signal according to a second modulation method to obtain a plurality of modulation subband signals; and synthesizing the sub-band signals except the target sub-band signal and the modulation sub-band signal.
In an alternative embodiment, the non-linearly modulating the phase of the compensated audio signal includes: performing sub-band decomposition on the compensation audio signal according to the frequency to obtain a plurality of sub-band signals; acquiring the sub-band signal with the frequency greater than the preset frequency as a target sub-band signal; performing signal synthesis on the target sub-band signal to obtain a first synthesized signal; carrying out nonlinear modulation on the phase of the first synthetic signal according to a second modulation method to obtain a first modulation synthetic signal; and synthesizing the sub-band signals except the target sub-band signal and the first modulation synthesis signal.
In an optional embodiment, the second modulation method includes: acquiring a first modulation parameter, and constructing a second modulation formula according to the first modulation parameter, wherein the first modulation parameter is used for adjusting the non-linearity degree of the second modulation formula; performing nonlinear modulation using the second modulation formula.
In a second aspect, the present invention provides an audio processing apparatus comprising: the audio input module is used for collecting audio input signals; the processing module is used for acquiring the audio input signal acquired by the audio input module in the current sampling period, and performing audio signal processing on the audio input signal to obtain an audio output signal, wherein the audio signal processing at least comprises: acquiring a front audio output signal, wherein the front audio output signal is the audio output signal in the previous sampling period, acquiring a feedback signal according to the front audio output signal, performing signal compensation on the audio input signal according to the feedback signal to obtain a compensation audio signal, and performing nonlinear modulation on the phase of the compensation audio signal; the audio output module is used for playing the audio output signal; a storage module to store the pre-audio output signal.
In a third aspect, the present invention provides an electronic device comprising: an audio input device, an audio output device, at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the audio processing method of any one of the preceding embodiments.
In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program for execution by a processor to implement the audio processing method of any one of the preceding embodiments.
Compared with the prior art, in the audio processing method, the apparatus, the electronic device and the computer-readable storage medium provided by the embodiments of the present invention, after the audio input device acquires the audio input signal, the feedback signal is obtained according to the previous audio output signal, the previous audio output signal is the audio output signal played by the audio output device in the previous sampling period, the feedback signal obtained according to the previous audio output signal is the estimated signal formed by the portion of the previous audio output signal played by the audio output device and re-acquired by the audio input device, and the audio input signal is subjected to signal compensation according to the feedback signal, so that the portion of the audio input signal, which is played by the audio output device and is re-acquired, can be eliminated, the compensated audio signal thus obtained is the audio signal of the actual environmental audio, and the compensated audio signal is played as the audio output signal, which is not only to effectively eliminate the howling that may be generated; in addition, because the audio processing is a continuous process, the audio output signal obtained in the current sampling period can be used as the previous audio output signal at the next sampling moment to perform audio signal processing on the audio input signal acquired at the next sampling moment, and the phase of the compensation audio signal is subjected to nonlinear modulation, so that the difference between the compensation audio signal and the audio input signal acquired at the next sampling moment can be increased, and the situation that the similarity between the compensation audio signal and the audio content of the audio input signal acquired at the next sampling moment is too high, which leads to excessive audio content being eliminated when the signal compensation is performed on the audio input signal acquired at the next sampling moment is avoided, thereby reducing audio distortion possibly caused in the signal compensation process, and realizing the effect of reducing audio signal distortion while eliminating howling.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a flowchart illustrating an audio processing method according to an embodiment;
fig. 2 is a schematic flowchart of audio signal processing in an audio processing method according to an embodiment;
fig. 3 is a schematic diagram of a frequency-phase response image of a first modulation formula in an audio processing method according to an embodiment;
fig. 4 is a schematic diagram of a frequency-phase response image of a second modulation formula in an audio processing method according to an embodiment;
FIG. 5 is a flowchart illustrating audio signal processing in an audio processing method according to another embodiment;
fig. 6 is a schematic structural diagram of an audio processing apparatus according to a second embodiment;
fig. 7 is a schematic structural diagram of an electronic device according to a third embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. indicate an orientation or a positional relationship based on that shown in the drawings or that the product of the present invention is used as it is, this is only for convenience of description and simplification of the description, and it does not indicate or imply that the device or the element referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and thus should not be construed as limiting the present invention.
Furthermore, the appearances of the terms "first," "second," and the like, if any, are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
It should be noted that the features of the embodiments of the present invention may be combined with each other without conflict.
An embodiment of the present invention provides an audio processing method, which is applied to an audio processing apparatus including an audio input device and an audio output device, and as shown in fig. 1, the audio processing method includes the following steps:
step S101: and acquiring the audio input signal acquired by the audio input equipment in the current sampling period.
In this step, the audio input device continuously collects the external audio, and integrates the external audio data collected in the sampling period at intervals of a preset sampling duration to form an audio input signal.
Step S102: and carrying out audio signal processing on the audio input signal to obtain an audio output signal.
In this step, the audio signal processing process is as shown in fig. 2, and includes the following steps:
step S201: a pre-audio output signal is obtained.
Specifically, the previous audio output signal is the audio output signal of the previous sampling period of the current sampling period.
Step S202: and acquiring a feedback signal according to the pre-audio output signal.
Specifically, in this embodiment, the algorithm filter may be written by using a normalized LMS algorithm, so as to establish a filtering estimation feedback path, and convolve the previous audio output signal with the filtering estimation feedback path to obtain the feedback signal. The feedback signal is an audio signal that may be re-collected by the audio input device after the estimated previous audio output signal is played by the audio output device. It should be understood that the foregoing is only a specific example of obtaining the feedback signal in this embodiment, and is not limited thereto, and in other embodiments of the present invention, other methods may also be used, for example, audio feedback parameters between the audio input device and the audio output device are measured in advance by a measuring device, and a feedback signal is obtained according to the measured audio feedback parameters when used, and other methods may be used flexibly according to actual needs.
Step S203: and performing signal compensation on the audio input signal according to the feedback signal to obtain a compensated audio signal.
Specifically, in this step, the feedback signal is removed from the audio input signal at the current sampling time, so as to obtain the compensated audio signal after echo cancellation.
Step S204: the phase of the compensated audio signal is non-linearly modulated.
Specifically, in this step, the compensation audio signal is decomposed in advance according to the frequency, a plurality of sub-band signals with different frequencies are obtained after decomposition, and then the phase of each sub-band signal is nonlinearly modulated according to a first modulation method, so as to obtain a plurality of modulated sub-band signals; and carrying out signal synthesis on the plurality of modulation subband signals to obtain the audio output signal of the current sampling period. The first modulation method is to use a first modulation formula to perform nonlinear modulation on the phase of each subband signal, wherein the first modulation formula comprises a first modulation parameter and a second modulation parameter, the first modulation parameter is used for adjusting the nonlinear degree of the first modulation formula, and the second modulation parameter is used for adjusting the slope of the first modulation formula.
Further, in an embodiment of the present invention, the first modulation formula is:
wherein α is a first modulation parameter, the magnitude of α affects the non-linearity of the first modulation formula, β is a second modulation parameter, the magnitude of β affects the slope of the first modulation formula, H (Z) is the magnitude of the phase to be adjusted corresponding to different frequencies, Z is the frequency, and N is a preset constant.
Specifically, fig. 3 is a schematic diagram of a frequency-phase response image of the first modulation formula when α is set to 0.4 and β is set to 0.6. As shown in FIG. 3, the first modulation formula is in the high frequency part (greater than 1.0 × 10) 4 Part of Hz) phase adjustmentLinearity degree is less than 0.5 x 10 compared with low frequency part 4 Hz portion) is more non-linear, since the human auditory system is generally sensitive to phase non-linear changes at low frequencies and insensitive to phase non-linear changes at high frequencies. Therefore, the phase adjustment amplitude of the first modulation formula is larger in the high frequency part than in the low frequency part, and a better effect of weakening the correlation between the audio input signal and the audio output signal can be achieved without changing the hearing sense.
In another embodiment of the present invention, the compensation audio signal may also be sub-band decomposed according to the frequency to obtain a plurality of sub-band signals; performing convolution and amplitude limiting processing on each sub-band signal respectively; performing signal synthesis on the plurality of sub-band signals after convolution and amplitude limiting processing to obtain a synthesized signal; the phase of the composite signal is non-linearly modulated according to a first modulation method. And synthesizing the signals of the plurality of sub-band signals after convolution and amplitude limiting processing to obtain a synthesized signal, and then performing nonlinear modulation on the phase of the synthesized signal by using a first modulation method, so that the calculated amount in the phase modulation process can be reduced, and the overall audio processing efficiency is improved.
In another embodiment of the present invention, the compensation audio signal may also be sub-band decomposed according to the frequency to obtain a plurality of sub-band signals; acquiring a sub-band signal with frequency greater than preset frequency as a target sub-band signal; respectively carrying out nonlinear modulation on the phase of each target sub-band signal according to a second modulation method to obtain a plurality of modulation sub-band signals; and synthesizing the sub-band signals except the target sub-band signal and the modulation sub-band signal. The second modulation method is to use a second modulation formula to perform nonlinear modulation on the phase of each target subband signal, wherein the second modulation formula comprises a first modulation parameter, and the first modulation parameter is used for adjusting the nonlinear degree of the first modulation formula.
Further, in an embodiment of the present invention, the second modulation formula is:
wherein α is a first modulation parameter, the magnitude of α affects the non-linearity of the first modulation formula, H (Z) is the magnitude of a phase to be adjusted corresponding to different frequencies, Z is a frequency, and N is a preset constant.
Specifically, fig. 4 is a schematic diagram of a frequency-phase response image of the second modulation formula when α is set to 0.4. The target sub-band signal with the frequency greater than the preset frequency is subjected to phase modulation by using the second modulation method, so that the influence on the partial sub-band signal with the frequency less than the preset frequency is avoided, meanwhile, the calculated amount in the phase modulation process can be reduced, the audio processing efficiency is improved, and the fidelity of the played audio is improved.
In another embodiment of the present invention, the compensation audio signal may also be sub-band decomposed according to the frequency to obtain a plurality of sub-band signals; performing signal synthesis on the sub-band signal with the frequency greater than the preset frequency to obtain a first synthesized signal; carrying out nonlinear modulation on the phase of the first synthetic signal according to a second modulation method to obtain a first modulation synthetic signal; and performing signal synthesis on the subband signals except the target subband signal and the first modulation synthesis signal. Performing signal synthesis on the sub-band signal with the frequency greater than the preset frequency to obtain a first synthesized signal; and carrying out nonlinear modulation on the phase of the first synthetic signal according to a second modulation method, so that the calculated amount in the phase modulation process can be reduced, and the overall audio processing efficiency is improved.
Further, the foregoing steps S201 to S204 are merely illustrative of the audio signal processing process in one embodiment of the present invention, and in another embodiment of the present invention, the audio signal processing process may also be as shown in fig. 5, which includes the following steps.
Step S301: a pre-audio output signal is acquired.
Step S302: and acquiring a feedback signal according to the pre-audio output signal.
Step S303: and performing signal compensation on the audio input signal according to the feedback signal to obtain a compensated audio signal.
It is to be understood that steps S301 to S303 in the present embodiment are substantially the same as steps S201 to S203 in the previous embodiment, and specific reference may be made to the specific description of the previous embodiment.
Step S304: a pre-compensation audio signal is acquired.
The pre-compensated audio signal is a compensated audio signal of a previous sampling period.
Step S305: and calculating to obtain feedback frequency response data according to the front audio output signal and the front compensation audio signal.
Specifically, in an embodiment of the present invention, pre-feedback frequency response data may be obtained, where the pre-feedback frequency response data is feedback frequency response data of a previous sampling period; and calculating to obtain feedback frequency response data of the current sampling time period according to the front audio output signal, the front compensation audio signal and the front feedback frequency response data.
Further, it can be according to the formula
And calculating to obtain feedback frequency response data.
Where μ is the iteration step coefficient, W (n) = [ W 0 (n),w 1 (n),w 2 (n),...,w L-1 (n)] T W (n) is the estimated feedback path, n is the sampling period, W (n) is the feedback frequency response data of the previous sampling period, W (n + 1) is the feedback frequency response data of the current sampling period, R (n) = [ R (n), R (n-1),. So, R (n-L + 1)] T R (n) is the compensated audio signal of the previous sampling period, r (n-L + 1) is the compensated audio signal of the previous L +2 sampling period, and L is the preset feedback path length of the audio output device for feeding back the audio to the audio input device. That is, determining the sampling number K according to the preset feedback path length L, where K = L +2, obtaining the audio output signals of K sampling periods adjacent to the current sampling period to form an audio output array, where R (n) is an inverse array of the audio output array, and d (n) = s (n) -z (n), and z (n) = W T (n) R (n), where s (n) is the audio input signal for sampling period n, z (n) is the feedback signal for sampling period n, and d (n) is the compensated audio signal for sampling period n. By using the formula, the audio output array and the pre-compensation audio signal can be output according to the audioAnd calculating the signal and the front feedback frequency response data to obtain the feedback frequency response data of the current sampling time period.
Step S306: and judging whether the feedback frequency response data is larger than the preset frequency response data, if so, executing step S307, and if not, executing step S308.
In this step, the preset frequency response data is threshold data for generating a howling risk, and the feedback frequency response data is greater than the preset frequency response data, which indicates that the howling risk may be generated, step S306 is executed, otherwise, the feedback frequency response data is not greater than the preset frequency response data, which indicates that the howling risk may not be generated, and step S307 is executed.
Step S307: the phase of the compensated audio signal is non-linearly modulated.
Step S308: the compensated audio signal is taken as the audio output signal.
The phase of the compensation audio signal is subjected to nonlinear modulation, feedback frequency response data is calculated before the phase of the compensation audio signal is subjected to nonlinear modulation, when the feedback frequency response data is larger than preset frequency response data, the phase of the compensation audio signal is subjected to nonlinear modulation, otherwise, the phase of the compensation audio signal is not subjected to nonlinear modulation, and therefore on the premise that howling is not generated, the frequency of modulating the phase of the compensation audio signal is reduced, the calculated amount of phase modulation is reduced, and the efficiency of audio signal processing is improved.
Step S103: and playing the audio output signal through the audio output device.
Compared with the prior art, in the audio processing method provided by the first embodiment of the present invention, after the audio input device acquires the audio input signal, the feedback signal is obtained according to the previous audio output signal, where the previous audio output signal is the audio output signal played by the audio output device at the previous sampling period, the feedback signal obtained according to the previous audio output signal is the estimated signal formed by the portion of the previous audio output signal that is re-acquired by the audio input device after being played by the audio output device, and the audio input signal is subjected to signal compensation according to the feedback signal, so that the portion of the audio output device that is re-acquired by the audio that is played by the audio input device, which is included in the audio input signal, can be eliminated, the compensated audio signal thus obtained is the audio signal of the actual environmental audio, and the compensated audio signal is played as the audio output signal, which can effectively eliminate possible howling; in addition, because the audio processing is a continuous process, the audio output signal obtained in the current sampling period can be used as the previous audio output signal at the next sampling moment to perform audio signal processing on the audio input signal acquired at the next sampling moment, and the phase of the compensation audio signal is subjected to nonlinear modulation, so that the difference between the compensation audio signal and the audio input signal acquired at the next sampling moment can be increased, and the situation that the similarity between the compensation audio signal and the audio content of the audio input signal acquired at the next sampling moment is too high, which leads to excessive audio content being eliminated when the signal compensation is performed on the audio input signal acquired at the next sampling moment is avoided, thereby reducing audio distortion possibly caused in the signal compensation process, and realizing the effect of reducing audio signal distortion while eliminating howling.
An embodiment of the present invention provides an audio processing apparatus, as shown in fig. 6, including: the audio input module 601, the audio input module 601 is used for collecting audio input signals; the processing module 602, the processing module 602 is configured to obtain an audio input signal acquired by the audio input module 601 in a current sampling period, and perform audio signal processing on the audio input signal to obtain an audio output signal, where the audio signal processing at least includes: acquiring a front audio output signal, wherein the front audio output signal is an audio output signal in a previous sampling period, acquiring a feedback signal according to the front audio output signal, performing signal compensation on an audio input signal according to the feedback signal to obtain a compensation audio signal, and performing nonlinear modulation on the phase of the compensation audio signal; the audio output module 603, the audio output module 603 is configured to play the audio output signal; a storage module 604, wherein the storage module 604 is configured to store the pre-audio output signal.
Compared with the prior art, in the audio processing apparatus provided in the second embodiment of the present invention, after the audio input module 601 acquires the audio input signal, the processing module 602 acquires a feedback signal according to the previous audio output signal stored in the storage module 604, where the previous audio output signal is an audio output signal that is stored in the storage module 604 and played by the audio output module 603 in the previous sampling period, and a feedback signal acquired according to the previous audio output signal is a signal formed by a portion, which is re-acquired by the audio input module 601 after the previous audio output signal is played by the audio output module 603, and performs signal compensation on the audio input signal according to the feedback signal, so that a portion, which is re-acquired by the audio played by the audio output module 603 and included in the audio input signal, can be eliminated, the compensated audio signal thus obtained is an audio signal of an actual environmental audio, and the processing module 602 plays the compensated audio signal as the audio output signal, which can effectively eliminate howling that may be generated; in addition, because the audio processing is a continuous process, the audio output signal obtained in the current sampling period can be used as the previous audio output signal at the next sampling moment to perform audio signal processing on the audio input signal acquired at the next sampling moment, and the phase of the compensation audio signal is subjected to nonlinear modulation, so that the difference between the compensation audio signal and the audio input signal acquired at the next sampling moment can be increased, and the situation that the similarity between the compensation audio signal and the audio content of the audio input signal acquired at the next sampling moment is too high, which leads to excessive audio content being eliminated when the signal compensation is performed on the audio input signal acquired at the next sampling moment is avoided, thereby reducing audio distortion possibly caused in the signal compensation process, and realizing the effect of reducing audio signal distortion while eliminating howling.
An embodiment of the present invention relates to an electronic device, as shown in fig. 7, including: an audio input device 701, an audio output device 702, at least one processor 703; and a memory 704 communicatively coupled to the at least one processor 703; the memory 704 stores instructions executable by the at least one processor 703, and the instructions are executed by the at least one processor 703 to enable the at least one processor 703 to execute the audio processing method in the above embodiments.
Where the memory and processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting together one or more of the various circuits of the processor and the memory. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And the memory may be used to store data used by the processor in performing operations.
The fourth embodiment of the present invention relates to a computer-readable storage medium in which a computer program is stored. The computer program realizes the above-described method embodiments when executed by a processor.
That is, as can be understood by those skilled in the art, all or part of the steps in the method according to the above embodiments may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps in the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (13)
1. An audio processing method applied to an audio processing apparatus including an audio input device and an audio output device, the method comprising:
acquiring an audio input signal acquired by the audio input device at the current sampling time period, performing audio signal processing on the audio input signal to obtain an audio output signal, and playing the audio output signal through the audio output device;
the audio signal processing includes: acquiring a front audio output signal, wherein the front audio output signal is the audio output signal in the previous sampling period, acquiring a feedback signal according to the front audio output signal, performing signal compensation on the audio input signal according to the feedback signal to obtain a compensation audio signal, and performing nonlinear modulation on the phase of the compensation audio signal.
2. The method of claim 1, wherein the audio signal processing further comprises:
acquiring a front compensation audio signal, wherein the front compensation audio signal is a compensation audio signal of a previous sampling period;
calculating to obtain feedback frequency response data according to the front audio output signal and the front compensation audio signal, and judging whether the feedback frequency response data is larger than preset frequency response data;
if the feedback frequency response data is larger than the preset frequency response data, executing the following steps: the phase of the compensated audio signal is non-linearly modulated.
3. The method of claim 2, wherein calculating feedback frequency response data from the pre-audio output signal and the pre-compensated audio signal comprises:
acquiring front feedback frequency response data, wherein the front feedback frequency response data is the feedback frequency response data of a previous sampling period;
and calculating the feedback frequency response data of the current sampling time period according to the front audio output signal, the front compensation audio signal and the front feedback frequency response data.
4. The method of claim 3, wherein the calculating the feedback frequency response data for a current sampling period based on the pre-audio output signal, the pre-compensated audio signal, and the pre-feedback frequency response data comprises:
determining the sampling number K according to the length of a preset feedback path;
acquiring the audio output signals of K sampling time periods adjacent to the current sampling time period to form an audio output array;
and calculating the feedback frequency response data of the current sampling time period according to the audio output array, the compensation audio signal and the pre-feedback frequency response data.
5. The method of claim 1, wherein the non-linearly modulating the phase of the compensated audio signal comprises:
performing sub-band decomposition on the compensation audio signal according to the frequency to obtain a plurality of sub-band signals;
respectively carrying out nonlinear modulation on the phase of each sub-band signal according to a first modulation method to obtain a plurality of modulated sub-band signals;
and performing signal synthesis on the plurality of modulated subband signals.
6. The method of claim 1, wherein the non-linearly modulating the phase of the compensated audio signal comprises:
performing sub-band decomposition on the compensation audio signal according to the frequency to obtain a plurality of sub-band signals;
performing convolution and amplitude limiting processing on each sub-band signal respectively;
performing signal synthesis on the plurality of sub-band signals after the convolution and amplitude limiting processing to obtain a synthesized signal;
the phase of the composite signal is non-linearly modulated according to a first modulation method.
7. The method according to claim 5 or 6, wherein the first modulation method comprises:
performing nonlinear modulation by using a first modulation formula;
the first modulation formula comprises a first modulation parameter and a second modulation parameter, the first modulation parameter is used for adjusting the non-linearity degree of the first modulation formula, and the second modulation parameter is used for adjusting the slope of the first modulation formula.
8. The method of claim 1, wherein the non-linearly modulating the phase of the compensated audio signal comprises:
performing sub-band decomposition on the compensation audio signal according to the frequency to obtain a plurality of sub-band signals;
acquiring the sub-band signal with the frequency greater than the preset frequency as a target sub-band signal;
respectively carrying out nonlinear modulation on the phase of each target subband signal according to a second modulation method to obtain a plurality of modulation subband signals;
and performing signal synthesis on the subband signals except the target subband signal and the modulation subband signal.
9. The method of claim 1, wherein the non-linearly modulating the phase of the compensated audio signal comprises:
performing sub-band decomposition on the compensation audio signal according to the frequency to obtain a plurality of sub-band signals;
acquiring the sub-band signal with the frequency greater than the preset frequency as a target sub-band signal;
performing signal synthesis on the target sub-band signal to obtain a first synthesis signal;
carrying out nonlinear modulation on the phase of the first synthetic signal according to a second modulation method to obtain a first modulation synthetic signal;
and synthesizing the sub-band signals except the target sub-band signal and the first modulation synthesis signal.
10. The method according to claim 8 or 9, wherein the second modulation method comprises:
performing nonlinear modulation by using a second modulation formula;
the second modulation equation includes a first modulation parameter that is used to adjust a degree of non-linearity of the second modulation equation.
11. An audio processing apparatus, comprising:
the audio input module is used for collecting audio input signals;
the processing module is used for acquiring the audio input signal acquired by the audio input module in the current sampling period, and performing audio signal processing on the audio input signal to obtain an audio output signal, wherein the audio signal processing at least comprises: acquiring a front audio output signal, wherein the front audio output signal is the audio output signal in a previous sampling period, acquiring a feedback signal according to the front audio output signal, performing signal compensation on the audio input signal according to the feedback signal to obtain a compensation audio signal, and performing nonlinear modulation on the phase of the compensation audio signal;
the audio output module is used for playing the audio output signal;
a storage module to store the pre-audio output signal.
12. An electronic device, comprising:
an audio input device, an audio output device, at least one processor; and a memory communicatively coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the audio processing method of any one of claims 1 to 10.
13. A computer-readable storage medium storing a computer program, characterized in that the computer program is executed by a processor to implement the audio processing method of any one of claims 1 to 10.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211376944.XA CN115835093A (en) | 2022-11-04 | 2022-11-04 | Audio processing method, device, electronic equipment and computer readable storage medium |
| PCT/CN2023/103752 WO2024093307A1 (en) | 2022-11-04 | 2023-06-29 | Audio processing method and apparatus, electronic device, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211376944.XA CN115835093A (en) | 2022-11-04 | 2022-11-04 | Audio processing method, device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115835093A true CN115835093A (en) | 2023-03-21 |
Family
ID=85526639
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211376944.XA Pending CN115835093A (en) | 2022-11-04 | 2022-11-04 | Audio processing method, device, electronic equipment and computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115835093A (en) |
| WO (1) | WO2024093307A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117640868A (en) * | 2024-01-23 | 2024-03-01 | 宁波菊风系统软件有限公司 | Intelligent double-recording system and method |
| WO2024093307A1 (en) * | 2022-11-04 | 2024-05-10 | 恒玄科技(上海)股份有限公司 | Audio processing method and apparatus, electronic device, and computer-readable storage medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016059784A1 (en) * | 2014-10-17 | 2016-04-21 | パナソニックＩｐマネジメント株式会社 | Apparatus for removing howling and method for removing howling |
| JP2018125831A (en) * | 2017-02-03 | 2018-08-09 | ミミー電子株式会社 | Hearing aid circuit for eliminating howling by varying signal frequency by variable delay amount |
| CN109862463A (en) * | 2018-12-26 | 2019-06-07 | 广东思派康电子科技有限公司 | Earphone audio playback method, earphone and its computer readable storage medium |
| CN111583949A (en) * | 2020-04-10 | 2020-08-25 | 南京拓灵智能科技有限公司 | Howling suppression method, device and equipment |
| CN111586512B (en) * | 2020-04-30 | 2022-01-04 | 歌尔科技有限公司 | Howling prevention method, electronic device and computer readable storage medium |
| CN115065921A (en) * | 2022-05-16 | 2022-09-16 | 武汉左点科技有限公司 | Method and device for preventing hearing aid from howling |
| CN115835093A (en) * | 2022-11-04 | 2023-03-21 | 恒玄科技(上海)股份有限公司 | Audio processing method, device, electronic equipment and computer readable storage medium |
-
2022
- 2022-11-04 CN CN202211376944.XA patent/CN115835093A/en active Pending
-
2023
- 2023-06-29 WO PCT/CN2023/103752 patent/WO2024093307A1/en unknown
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024093307A1 (en) * | 2022-11-04 | 2024-05-10 | 恒玄科技(上海)股份有限公司 | Audio processing method and apparatus, electronic device, and computer-readable storage medium |
| CN117640868A (en) * | 2024-01-23 | 2024-03-01 | 宁波菊风系统软件有限公司 | Intelligent double-recording system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024093307A1 (en) | 2024-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115835093A (en) | Audio processing method, device, electronic equipment and computer readable storage medium | |
| US9870783B2 (en) | Audio signal processing | |
| CN100525101C (en) | Method and apparatus to record a signal using a beam forming algorithm | |
| US8085959B2 (en) | Hearing compensation system incorporating signal processing techniques | |
| US8611552B1 (en) | Direction-aware active noise cancellation system | |
| Kates et al. | Speech intelligibility enhancement | |
| US8170248B2 (en) | Feedback compensation in a sound processing device | |
| US10819305B2 (en) | Method and device for adjusting sound quality | |
| US8538052B2 (en) | Generation of probe noise in a feedback cancellation system | |
| US9343073B1 (en) | Robust noise suppression system in adverse echo conditions | |
| US20070172080A1 (en) | Acoustic feedback suppression | |
| JP2016195439A (en) | Frequency band compression using dynamic threshold | |
| CN101635873B (en) | Adaptive long-term prediction filter for adaptive whitening | |
| US20220312129A1 (en) | Feedback elimination in a hearing aid | |
| EP3236677B1 (en) | Tonality-driven feedback canceler adaptation | |
| US8295508B2 (en) | Processing an audio signal | |
| Akhtar et al. | Acoustic feedback cancellation in hearing aids using dual adaptive filtering and gain-controlled probe signal | |
| CN111629313A (en) | Hearing device comprising a loop gain limiter | |
| CN113593518A (en) | Howling suppression method and device, in-ear earphone and storage medium | |
| US12010486B2 (en) | Detection of feedback path change | |
| CN116405822A (en) | Bass enhancement system and method applied to open Bluetooth headset | |
| EP4021008B1 (en) | Voice signal processing method and device | |
| AU2011226820B2 (en) | Method for frequency compression with harmonic correction and device | |
| CN112584275B (en) | Sound field expansion method, computer equipment and computer readable storage medium | |
| CN114501211A (en) | Active noise reduction circuit with liyin permeability, method, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |