WO2023226193A1 - Audio processing method and apparatus, and non-transitory computer-readable storage medium - Google Patents

Abstract

An audio processing method, an audio processing apparatus, and a non-transitory computer-readable storage medium. The audio processing method comprises: generating a control instruction on the basis of a first audio signal; generating a second audio signal on the basis of the control instruction; and outputting the second audio signal to suppress a third audio signal, wherein the sum of the phase of the second audio signal and the phase of the third audio signal is less than a phase threshold, and the time at which the first audio signal appears is earlier than the time at which the third audio signal appears.

Description

Audio processing method and device, non-transitory computer-readable storage medium

This application requires U.S. Provisional Patent Application No. 63/344,642 submitted on May 23, 2022, U.S. Provisional Patent Application No. 63/351,439 submitted on June 13, 2022, and U.S. Provisional Patent Application No. 63/351,439 submitted on June 14, 2022 Priority is granted to U.S. Provisional Patent Application No. 63/352,213, the contents of which are incorporated herein by reference in their entirety.

Technical field

Embodiments of the present disclosure relate to an audio processing method, an audio processing device, and a non-transitory computer-readable storage medium.

Background technique

At present, noise reduction methods mainly include active noise reduction and passive noise reduction. Active noise reduction uses the noise reduction system to generate an inverse signal that is equal to the external noise to neutralize the noise, thereby achieving the noise reduction effect. Passive noise reduction mainly achieves the noise reduction effect by forming a closed space around the object or using sound insulation materials to block external noise.

Active noise reduction usually uses lagging inverted audio to destructively superimpose the originally received audio (for example, noise) to achieve the effect of audio suppression. An active noise reduction process is as follows: First, the audio Vn generated by the sound source is received through the microphone, and the received audio Vn is sent to the processor. Then, the processor performs inversion processing on the audio Vn to generate inverted audio. Vn' and output the inverted audio Vn' to the speaker, and the speaker emits the inverted audio Vn'. The human ear can receive the inverted audio Vn’ and the audio Vn, and the inverted audio Vn’ and the audio Vn can be destructively superimposed to achieve the effect of suppressing the audio. In this active noise reduction, due to the time required for signal processing and signal transmission, the time of the inverted audio Vn' output by the speaker must lag behind the time of the audio Vn originally received by the microphone. Therefore, the human ear receives The time to the inverted audio Vn' must also lag behind the time when the human ear receives the audio Vn, and the silencing effect is poor, and may even be impossible to achieve. There must be a delay from the input end (i.e. microphone) to the output end (i.e. speaker). The lower the delay from the input end to the output end, the smaller the time difference between the human ear receiving the inverted audio Vn' and the received audio Vn, the smaller the noise reduction. The better. Therefore, active noise reduction has extremely strict requirements on end-to-end delay, so the architecture of the active noise reduction system must use high-speed analog-to-digital converters and high-speed computing hardware to achieve low latency and achieve better audio suppression effects. , resulting in high development costs and less elastic architecture. Therefore, how to avoid the impact of end-to-end delay on active noise reduction and how to achieve better audio suppression effects have become problems that need to be solved.

Contents of the invention

To address the above problems, at least one embodiment of the present disclosure provides an audio processing method, which includes: generating a control instruction based on a first audio signal; generating a second audio signal based on the control instruction; and outputting the second audio signal to Suppressing a third audio signal, wherein the sum of the phases of the second audio signal and the third audio signal is less than a phase threshold, and the first audio signal appears earlier than the third audio signal. time.

For example, in the audio processing method provided by at least one embodiment of the present disclosure, the outputting the second audio signal to suppress the third audio signal includes: based on the control instruction, determining the output of the second audio signal. The first moment; outputting the second audio signal at the first moment, wherein the third audio signal starts to appear from the second moment, and the absolute time difference between the first moment and the second moment is The value is less than the time threshold.

For example, in the audio processing method provided by at least one embodiment of the present disclosure, the time difference between the first moment and the second moment is 0.

For example, in the audio processing method provided by at least one embodiment of the present disclosure, generating a control instruction based on a first audio signal includes: acquiring the first audio signal; processing the first audio signal to predict a fourth audio signal; based on the fourth audio signal, the control instruction is generated.

For example, in the audio processing method provided by at least one embodiment of the present disclosure, the second audio signal and/or the third audio signal and/or the fourth audio signal are periodic or intermittent time domain Signal.

For example, in the audio processing method provided by at least one embodiment of the present disclosure, processing the first audio signal to predict a fourth audio signal includes: generating a first audio feature code based on the first audio signal ; Query the lookup table based on the first audio feature coding to obtain the second audio feature coding; predict and obtain the fourth audio signal based on the second audio feature coding.

For example, in the audio processing method provided by at least one embodiment of the present disclosure, the lookup table includes at least one first encoding field.

For example, in the audio processing method provided by at least one embodiment of the present disclosure, the lookup table further includes at least one second encoding field, and multiple first encoding fields constitute one second encoding field.

For example, in the audio processing method provided by at least one embodiment of the present disclosure, the second audio feature encoding includes at least one of the first encoding field and/or at least one of the second encoding field.

For example, in the audio processing method provided by at least one embodiment of the present disclosure, obtaining the first audio signal includes: collecting an initial audio signal; performing downsampling processing on the initial audio signal to obtain the first audio signal. Signal.

For example, in the audio processing method provided by at least one embodiment of the present disclosure, obtaining the first audio signal includes: collecting an initial audio signal; filtering the initial audio signal to obtain the first audio signal .

For example, in the audio processing method provided by at least one embodiment of the present disclosure, the phase of the second audio signal is opposite to the phase of the third audio signal.

At least one embodiment of the present disclosure also provides an audio processing device, including: an instruction generation module configured to generate a control instruction based on a first audio signal; and an audio generation module configured to generate a second audio based on the control instruction. signal; an output module configured to output the second audio signal to suppress a third audio signal; wherein the sum of the phases of the second audio signal and the phase of the third audio signal is less than a phase threshold, the The first audio signal appears earlier than the third audio signal.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the output module includes a time determination sub-module and an output sub-module, and the time determination sub-module is configured to determine to output the first time based on the control instruction. The first moment of the second audio signal; the output sub-module is configured to output the second audio signal at the first moment, wherein the third audio signal begins to appear from the second moment, and the first moment The absolute value of the time difference between the second moment and the second moment is less than the time threshold.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the time difference between the first time and the second time is 0.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the instruction generation module includes an audio acquisition sub-module, a prediction sub-module and a generation sub-module, and the audio acquisition sub-module is configured to acquire the first audio signal; the prediction sub-module is configured to process the first audio signal to predict a fourth audio signal; the generation sub-module is configured to generate the control instruction based on the fourth audio signal.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the second audio signal and/or the third audio signal and/or the fourth audio signal are periodic or intermittent time domain Signal.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the prediction sub-module includes a query unit and a prediction unit, the query unit is configured to generate a first audio feature encoding based on the first audio signal and a prediction unit based on the first audio signal. The first audio feature coding queries a lookup table to obtain a second audio feature coding; the prediction unit is configured to predict the fourth audio signal based on the second audio feature coding.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the lookup table includes at least one first encoding field.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the lookup table further includes at least one second encoding field, and multiple first encoding fields constitute one second encoding field.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the second audio feature encoding includes at least one of the first encoding field and/or at least one of the second encoding field.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the audio acquisition sub-module includes a collection unit and a down-sampling processing unit, the collection unit is configured to collect an initial audio signal; the down-sampling processing unit is Configured to perform downsampling processing on the initial audio signal to obtain the first audio signal.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the audio acquisition sub-module includes an acquisition unit and a filtering unit, the acquisition unit is configured to acquire an initial audio signal; the filtering unit is configured to The initial audio signal is filtered to obtain the first audio signal.

For example, in the audio processing device provided by at least one embodiment of the present disclosure, the phase of the second audio signal is opposite to the phase of the third audio signal.

At least one embodiment of the present disclosure also provides an audio processing device, including: one or more memories non-transiently storing computer-executable instructions; one or more processors configured to run the computer-executable instructions, Wherein, the computer-executable instructions implement the audio processing method according to any embodiment of the present disclosure when run by the one or more processors.

At least one embodiment of the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are implemented when executed by a processor. An audio processing method according to any embodiment of the present disclosure.

According to the audio processing method, audio processing device and non-transitory computer-readable storage medium provided by any embodiment of the present disclosure, a future inverted audio signal is generated by learning the characteristics of the current audio signal (ie, the first audio signal) (i.e., the second audio signal) to suppress the future audio signal (i.e., the third audio signal) to avoid the problem of out-of-synchronization of the inverted audio signal and the audio signal that needs to be suppressed due to the delay between the input end and the output end, Improving the noise canceling effect can significantly reduce or even eliminate the impact of input-to-output delay on noise canceling, and the audio suppression effect is better than the backward active noise canceling system commonly used in the industry.

Description of the drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure and do not limit the present disclosure. .

Figure 1 is a schematic block diagram of an audio processing system provided by at least one embodiment of the present disclosure;

Figure 2A is a schematic flow chart of an audio processing method provided by at least one embodiment of the present disclosure;

Figure 2B is a schematic flow chart of step S10 shown in Figure 2A;

Figure 2C is a schematic flow chart of step S102 shown in Figure 2B;

Figure 3 is a schematic diagram of a first audio signal and a third audio signal provided by at least one embodiment of the present disclosure;

Figure 4 is a schematic diagram of a third audio signal and a fourth audio signal provided by at least one embodiment of the present disclosure;

Figure 5A is a schematic diagram of an audio signal provided by some embodiments of the present disclosure;

Figure 5B is an enlarged schematic diagram of the audio signal in the dotted rectangular frame P1 in Figure 5A;

Figure 6 is a schematic block diagram of an audio processing device provided by at least one embodiment of the present disclosure;

Figure 7 is a schematic block diagram of another audio processing device provided by at least one embodiment of the present disclosure; and

FIG. 8 is a schematic diagram of a non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present disclosure.

Unless otherwise defined, technical terms or scientific terms used in this disclosure shall have the usual meaning understood by a person with ordinary skill in the art to which this disclosure belongs. "First", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprising" mean that the elements or things appearing before the word include the elements or things listed after the word and their equivalents, without excluding other elements or things. Words such as "connected" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some well-known functions and well-known components.

At least one embodiment of the present disclosure provides an audio processing method. The audio processing method includes: generating a control instruction based on the first audio signal; generating a second audio signal based on the control instruction; and outputting the second audio signal to suppress the third audio signal. The sum of the phases of the second audio signal and the phase of the third audio signal is less than the phase threshold, and the first audio signal appears earlier than the third audio signal.

In the audio processing method provided by embodiments of the present disclosure, by learning the characteristics of the current audio signal (ie, the first audio signal), a future inverted audio signal (ie, the second audio signal) is generated to suppress the future audio signal ( That is, the third audio signal), avoids the problem of out-of-synchronization between the inverted audio signal and the audio signal that needs to be suppressed due to the delay between the input end and the output end, improves the noise canceling effect, and can significantly reduce or even eliminate the impact of the input end on the output The effect of terminal delay on noise reduction is better than that of the backward active noise reduction system commonly used in the industry.

Embodiments of the present disclosure also provide an audio processing device and a non-transitory computer-readable storage medium. The audio processing method can be applied to the audio processing device provided by the embodiment of the present disclosure, and the audio processing device can be configured on an electronic device. The electronic device may be a personal computer, a mobile terminal, a car headrest, etc. The mobile terminal may be a mobile phone, a headset, a tablet computer or other hardware devices.

The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

Figure 1 is a schematic block diagram of an audio processing system provided by at least one embodiment of the present disclosure. Figure 2A is a schematic flow chart of an audio processing method provided by at least one embodiment of the present disclosure. Figure 2B is shown in Figure 2A Figure 2C is a schematic flow chart of step S10 shown in Figure 2B. Figure 3 is a schematic diagram of a first audio signal and a third audio signal provided by at least one embodiment of the present disclosure.

The audio processing system shown in Figure 1 can be used to implement the audio processing method provided by any embodiment of the present disclosure, for example, the audio processing method shown in Figure 2A. As shown in Figure 1, the audio processing system may include an audio receiving part, an audio processing part and an audio output part. The audio receiving part can receive the audio signal Sn1 emitted by the sound source at time tt1, and then transmit the audio signal Sn1 to the audio processing part. The audio processing part processes the audio signal Sn1 to predict the future inverted audio signal Sn2; then the The future inverted audio signal Sn2 is output through the audio output section. The future inverted audio signal Sn2 may be used to suppress the future audio signal Sn3 generated by the sound source at time tt2 later than time tt1. For example, the target object (e.g., human ear, etc.) can receive the inverted audio signal Sn2 and the future audio signal Sn3 at the same time, so that the future inverted audio signal Sn2 and the future audio signal Sn3 can be destructively superimposed, thereby achieving noise elimination. .

For example, the audio receiving part may include a microphone, an amplifier (for example, a microphone amplifier), an analog to digital converter (ADC), a downsampler, etc., and the audio processing part may include an AI engine and/or a digital signal Processor (Digital Signal Processing, DSP), etc., the audio output part can include an upsampler, a digital to analog converter (digital to analog converter, DAC), an amplifier (for example, a speaker amplifier), a speaker, etc.

As shown in Figure 2A, an audio processing method provided by one embodiment of the present disclosure includes steps S10 to S12. In step S10, a control instruction is generated based on the first audio signal; in step S11, a second audio signal is generated based on the control instruction; in step S12, the second audio signal is output to suppress the third audio signal.

For example, the first audio signal may be the audio signal Sn1 shown in FIG. 1 , the second audio signal may be the inverted audio signal Sn2 shown in FIG. 1 , and the third audio signal may be the future audio signal Sn3 shown in FIG. 1 .

For example, the audio receiving part can receive a first audio signal; the audio processing part can process the first audio signal to generate a control instruction, and generate a second audio signal based on the control instruction; the audio output part can output the second audio signal, thereby achieving Suppress third audio signal.

For example, the first audio signal appears earlier than the third audio signal. As shown in Figure 3, the time when the first audio signal starts to appear is t11, and the time when the third audio signal starts to appear is t21. On the time axis t, time t11 is earlier than time t21. For example, the time period during which the first audio signal exists may be the time period between time t11 and time t12, and the time period during which the third audio signal exists may be the time period between time t21 and time t22. Taking into account factors such as the time of the signal processing process, time t12 and time t21 may not be the same time, and time t12 is earlier than time t21.

It should be noted that, in the embodiment of the present disclosure, "the time period in which the audio signal exists or the time in which it appears" means the time period in which the audio corresponding to the audio signal exists or the time in which it appears.

For example, the sum of the phases of the second audio signal and the phase of the third audio signal is less than the phase threshold. The phase threshold can be set according to the actual situation, and this disclosure does not specifically limit this. For example, in some embodiments, the phase of the second audio signal is opposite to the phase of the third audio signal, so that complete silence can be achieved, that is, the third audio signal is completely suppressed. At this time, when the second audio signal and the third audio signal When received by an audio collection device (for example, a microphone, etc.), the error energy of the audio signal received by the audio collection device is 0; if the second audio signal and the third audio signal are received by the human ear, it is equivalent to the person not hearing the sound. .

For example, in some embodiments, the first audio signal may be the time-domain audio signal with the maximum volume (maximum amplitude) between time t11 and time t12, and the first audio signal is not an audio signal of a specific frequency, so the implementation of the present disclosure The audio processing method provided in the example does not need to extract spectral features from the audio signal to generate a spectrogram, which can simplify the audio signal processing process and save processing time.

For example, the first audio signal and the third audio signal may be audio signals generated by the external environment, machines, etc., the sound of machine operation, the sound of electric drills and electric saws during decoration, etc. For example, machines may include household appliances (air conditioners, range hoods, washing machines, etc.) and the like.

For example, in some embodiments, as shown in Figure 2B, step S10 may include steps S101 to 103. In step S101, a first audio signal is obtained; in step S102, the first audio signal is processed to predict Fourth audio signal; in step S103, a control instruction is generated based on the fourth audio signal. In the audio processing method provided by embodiments of the present disclosure, an audio signal that has not yet been generated (ie, the fourth audio signal) is predicted by learning the characteristics of the current audio signal (ie, the first audio signal).

For example, the fourth audio signal is a predicted future audio signal. For example, on the time axis, the time period in which the fourth audio signal exists is later than the time period in which the first audio signal exists, for example, the time period in which the fourth audio signal exists. The segment is the same as the time period in which the third audio signal exists, so the time period in which the fourth audio signal exists may also be the time period between time t21 and time t22 shown in FIG. 3 .

Figure 4 is a schematic diagram of a third audio signal and a fourth audio signal provided by at least one embodiment of the present disclosure. In the example shown in Figure 4, the horizontal axis represents time (Time), the vertical axis represents amplitude (Amplitude), and the amplitude can be expressed as a voltage value. As shown in Figure 4, in one embodiment, the predicted fourth audio signal is substantially the same as the third audio signal.

For example, in one embodiment, the third audio signal and the fourth audio signal may be exactly the same. In this case, the phase of the second audio signal finally generated based on the fourth audio signal is opposite to the phase of the third audio signal, thereby achieving complete Silencing.

For example, in step S102, processing the first audio signal to predict the fourth audio signal may include processing the first audio signal through a neural network to predict the fourth audio signal.

For example, neural networks may include recurrent neural networks, long short-term memory networks, or generative adversarial networks. In embodiments of the present disclosure, the characteristics of the audio signal can be learned based on artificial intelligence, thereby predicting the audio signal of a certain future time period that has not yet occurred, and thereby generating an inverted audio signal of the future time period to suppress the time period audio signal.

For example, in some embodiments, as shown in Figure 2C, step S102 may include steps S1021 to 1023. In step S1021, a first audio feature code is generated based on the first audio signal; in step S1022, based on the first audio signal, The feature coding queries the lookup table to obtain the second audio feature coding; in step S1023, based on the second audio feature coding, a fourth audio signal is predicted.

For example, the first audio signal may be an analog signal, and the first audio signal may be processed through an analog-to-digital converter to obtain a processed first audio signal. The processed first audio signal may be a digital signal. Based on the processed The first audio signal may generate a first audio feature code.

For another example, the first audio signal may be a digital signal, such as a PDM (Pulse-density-modulation, pulse density modulation) signal. In this case, the first audio feature code may be generated directly based on the first audio signal. PDM signals can be represented by binary numbers 0 and 1.

For example, any suitable encoding method may be used to implement the first audio feature encoding. For example, in some embodiments, when representing an audio signal, the changing state of the audio signal can be used to describe the audio signal, and multi-bits can be used to represent the changing state of the audio signal. For example, two bits (2bits) can be used to represent the changing state of the audio signal. In some examples, as shown in Table 1 below, 00 means that the audio signal becomes larger, 01 means that the audio signal becomes smaller, 10 means that there is no audio signal, and 11 means that there is no audio signal. The audio signal remains unchanged.

Bits	The changing state of the audio signal
00	Audio signal becomes louder
01	Audio signal becomes smaller
10	no audio signal
11	Audio signal remains unchanged

Table 1

"The audio signal becomes larger" means that the amplitude of the audio signal in the unit time period (each time step) becomes larger with time, and "the audio signal becomes smaller" means that the amplitude of the audio signal in the unit time period increases with time. The time becomes smaller, "the audio signal remains unchanged" means that the amplitude of the audio signal in the unit time period does not change with time, and "no audio signal" means that there is no audio signal in the unit time period, that is, the amplitude of the audio signal is 0.

Figure 5A is a schematic diagram of an audio signal provided by some embodiments of the present disclosure. Figure 5B is an enlarged schematic diagram of the audio signal in the dotted rectangular box P1 in Figure 5A.

In Figure 5A, the abscissa is time (ms, milliseconds), and the ordinate is the amplitude of the audio signal (volts, volts). As shown in FIG. 5A , the audio signal V is a periodically changing signal, and the periodic pattern of the audio signal V is the pattern shown by the dotted rectangular frame P2.

As shown in Figure 5B, the amplitude of the audio signal represented by the waveform segment 30 does not change with time t, and the time corresponding to the waveform segment 30 is a unit time period, then the waveform segment 30 can be expressed as audio feature coding (11); similarly Ground, the amplitude of the audio signal represented by waveform segment 31 gradually increases with time t, and the time corresponding to waveform segment 31 is four unit time segments, then waveform segment 31 can be expressed as audio feature encoding (00,00,00, 00); the amplitude of the audio signal represented by waveform segment 32 remains unchanged with time t, the time corresponding to waveform segment 32 is a unit time period, and waveform segment 32 can be represented as audio feature encoding (11); represented by waveform segment 33 The amplitude of the audio signal gradually becomes smaller with time t, and the time corresponding to the waveform segment 33 is six unit time periods, then the waveform segment 33 can be expressed as the audio feature code (01,01,01,01,01,01); The amplitude of the audio signal represented by waveform segment 34 does not change with time t, and the time corresponding to waveform segment 34 is a unit time period, then waveform segment 34 can be expressed as audio feature encoding (11); the audio signal represented by waveform segment 35 The amplitude of the signal gradually increases with time t, and the time corresponding to the waveform segment 35 is eight unit time segments, then the waveform segment 35 can be expressed as audio feature encoding (00,00,00,00,00,00,00,00 ); By analogy, waveform segment 36 can be expressed as audio feature coding (01,01,01,01,01,01,01,01,01,01,01,01), and waveform segment 37 can be expressed as audio feature coding (11), the waveform segment 38 can be expressed as audio feature encoding (00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00). Therefore, the audio feature encoding corresponding to the audio signal shown in Figure 5B can be expressed as {11,00,00,00,00,11,01,01,01,01,01,01,11,00,00,00, 00,00,00,00,00,01,01,01,01,01,01,01,01,01,01,01,01,11,00,00,00,00,00,00,00, 00,00,00,00,00,00,00,00,00,…}.

For example, in some embodiments, a lookup table (codebook) includes at least one first code field. For example, in other embodiments, the lookup table further includes at least one second encoding field, and multiple first encoding fields constitute a second encoding field, so that dimensionally reduced high-order features can be formed from combinations of low-level features. For example, the coding method of the coding field (codeword, for example, the codeword may include a first coding field and a second coding field) in the lookup table may be the same as the coding method of the above-mentioned first audio feature coding.

For example, in some embodiments, when two bits are used to represent the changing state of the audio signal to implement feature encoding, the first encoding field may be one of 00, 01, 10, and 11. 00, 01, 10 and 11 can be combined to form the second encoding field. For example, a second encoding field may be represented as {00,00,00,01,01,01,11,11,01,…}, which is composed of a combination of 00, 01 and 11.

For example, when the lookup table includes a plurality of second encoding fields, the number of first encoding fields included in each of the plurality of second encoding fields may be different.

It should be noted that when more bits (for example, 3 bits, 4 bits, etc.) are used to represent the changing state of the audio signal to implement feature encoding, the types of the first coding field can be more, for example, when 3 bits are used to represent When the audio signal changes state, there can be up to eight types of first coding fields. At this time, the first coding fields can be part or all of 000, 001, 010, 011, 100, 101, 110 and 111.

For example, one or more second encoding fields can also be combined to obtain a third encoding field, or one or more second encoding fields and one or more first encoding fields can be combined to obtain a third encoding field, similarly Alternatively, one or more third coding fields may be combined or one or more third coding fields may be combined with the first coding field and/or the second coding field to obtain a higher order coding field. In embodiments of the present disclosure, low-order feature codes can be combined to obtain high-order feature codes, thereby achieving more efficient and longer predictions.

For example, the second audio feature encoding includes at least one first encoding field and/or at least one second encoding field. For example, in some embodiments, the second audio feature encoding may include one or more complete second encoding fields, or the second audio feature encoding may include part of the first encoding field in one second encoding field.

It should be noted that when the lookup table includes a third encoding field, the second audio feature encoding may include at least one first encoding field and/or at least one second encoding field and/or at least one third encoding field.

For example, in one embodiment, the lookup table includes the second encoding field W1, the second encoding field W2, and the second encoding field W3, and W1={11,00,00,00,00,11,01,01,01 ,01,01,01,11,00,00,00,00,00,00,00,00,01,01,01,01,01,01,01,01,01,01,01,01,11 ,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,….}, W2＝{11,01,00,00,01, 01,01,01,01,01,01,….}, W3＝{11,00,01,00,00,01,01,01,11,00,00,00,01,01,01,01 ,01,01,01,01,01,….}.

In one embodiment, as shown in Figure 5B, starting from time t31, the audio collection device continues to collect the first audio signal. When the first feature encoding field corresponding to the first audio signal collected by the audio collection device is expressed as {11 }, corresponding to waveform segment 30, a query is performed based on the lookup table to determine whether there is a certain coding field (including the first coding field and the second coding field) in the lookup table, including {11}. In the above example, the query The second encoding field W1, the second encoding field W2, and the second encoding field W3 in the lookup table all include {11}. At this time, the second encoding field W1, the second encoding field W2, and the second encoding field W3 are all used as to-be-coded fields. The encoding fields to be output in the output encoding field list.

Then, as shown in Figure 5B, when the second feature encoding field corresponding to the first audio signal collected by the audio collection device is represented as {00}, corresponding to the first unit time period in the waveform segment 31, continue the search. Query the table (at this time, you can only query the coding field to be output in the coding field column to be output, which can save query time. However, you can also query the entire lookup table) to determine whether a certain encoding exists in the lookup table. The field includes {11,00}. In the above example, it is found that the second encoding field W1 and the second encoding field W3 in the lookup table both include {11,00}, because the second encoding field W2 includes {11,01}. , and does not include {11,00}, thus not meeting the characteristics of the first audio signal collected by the audio collection device. Therefore, the second encoding field W2 can be deleted from the list of encoding fields to be output. At this time, the second encoding field W2 Field W1 and the second encoding field W3 serve as the encoding fields to be output in the encoding field list to be output.

Then, when the third feature encoding field corresponding to the first audio signal collected by the audio collection device is represented as {00}, corresponding to the second unit time period in the waveform segment 31, continue to query the lookup table to determine Check whether there is a certain encoding field in the lookup table that includes {11,00,00}. In the above example, the second encoding field W1 in the lookup table is queried and includes {11,00,00}. Then, it can be predicted that the next audio signal should be the pattern of the second encoding field W1. For the first three coding fields {11,00,00} in the second coding field W1, since their corresponding audio signals have passed in time, the fourth field in the second coding field W1 can be output (ie {00}) is used as the predicted second audio coding feature. At this time, the second audio feature coding is expressed as {00,00,11,01,01,01,01,01,01 ,11,00,00,00,00,00,00,00,00,01,01,01,01,01,01,01,01,01,01,01,01,11,00,00,00 ,00,00,00,00,00,00,00,00,00,00,00,00,00,…….}.

It should be noted that in actual applications, how many feature coding fields are matched before determining the second audio feature coding can be adjusted according to actual application scenarios, design requirements and other factors. For example, in the above example, when 3 matching fields (in actual In the application, if 10, 20, 50, etc.) feature coding fields can be matched, the second audio feature coding can be determined.

For example, in the above example, the first audio feature code corresponding to the first audio signal includes 3 feature code fields and is represented as {11,00,00}. As shown in Figure 5B, the time period corresponding to the first audio signal It is from time t31 to time t32. When considering factors such as the system's signal processing time, the system actually needs to output the second audio signal at time t33, which is later than time t32. At this time, the first two feature coding fields in the second audio feature coding {00 The time period corresponding to ,00} (that is, the time period between time t32 and time t33) has passed, so the audio feature encoding corresponding to the predicted fourth audio signal is actually expressed as {11,01,01,01,01 ,01,01,11,00,00,00,00,00,00,00,00,01,01,01,01,01,01,01,01,01,01,01,01,11,00 ,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,…}.

For example, if the third audio signal and the fourth audio signal are exactly the same, the audio feature code corresponding to the third audio signal is also expressed as {11,01,01,01,01,01,01,11,00,00,00 ,00,00,00,00,00,01,01,01,01,01,01,01,01,01,01,01,01,11,00,00,00,00,00,00,00 ,00,00,00,00,00,00,00,00,00,…}.

For example, the second audio signal is a signal obtained by inverting the fourth audio signal, that is, the second audio signal can be {11,01,01,01,01,01,01,11,00,00,00, 00,00,00,00,00,01,01,01,01,01,01,01,01,01,01,01,01,11,00,00,00,00,00,00,00, 00,00,00,00,00,00,00,00,00,….} The inverted audio signal of this pattern.

For example, in some embodiments, the duration of the second audio signal, the duration of the third audio signal, and the duration of the fourth audio signal are substantially the same, eg, identical.

For example, in some embodiments, the leading feature coding field may be set for at least part of the first coding field and/or the second coding field in the lookup table. For example, the leading feature coding field may be set for the second coding field W1 {11,00 ,00}, when the leading feature coding field is detected, the second coding field W1 is output as the second audio feature coding. In this case, when it is detected that the first audio feature code corresponding to the first audio signal is {11,00,00}, the first audio feature code corresponding to the first audio signal and the preamble feature code field {11,00, 00} matching, so that the second encoding field W1 can be output as the second audio feature encoding.

For another example, the leading feature coding field {11,00,00,01,01} can be set for the second coding field W1. When some fields in the leading feature coding field are detected, the second coding field W1 and the leading feature coding field are The remaining fields in the feature encoding field are output as the second audio feature encoding. In this case, when it is detected that the first audio feature encoding corresponding to the first audio signal is {11,00,00}, the first audio signal corresponding The first audio feature encoding matches the first three fields {11,00,00} in the leading feature encoding field, so that the remaining fields {01,01} and the second encoding field W1 in the leading feature encoding field can be output as the third 2. Audio feature encoding. At this time, the time corresponding to the first two feature coding fields {01,01} in the second audio feature coding (that is, the remaining fields in the leading feature coding field) can be the time for the system to process the signal, so that the predicted first The audio feature encoding corresponding to the four audio signals may be the complete second encoding field W1.

It should be noted that the length of the leading feature encoding field can be adjusted according to actual conditions, and this disclosure does not limit this.

It is worth noting that for look-up tables, when the memory used to store the look-up table is large enough and the content stored in the look-up table is rich enough (that is, there are enough combinations of encoding fields in the look-up table), the user's desire to eliminate all types of audio signals. For neural networks, when the samples used to train the neural network are rich enough and the types of samples are rich enough, any type of audio signal that the user wants to eliminate can be predicted based on the neural network.

For example, the lookup table may be stored in the memory in the form of a table, etc. The embodiments of the present disclosure do not limit the specific form of the lookup table.

For example, predictions in neural networks can be achieved by looking up tables.

For example, the second audio signal and/or the third audio signal and/or the fourth audio signal are periodic or intermittent time domain signals, and the second audio signal and/or the third audio signal and/or the fourth audio signal The signal characteristics are periodic or intermittent time domain amplitude changes, that is, the second audio signal and/or the third audio signal and/or the fourth audio signal have the characteristics of continuous repetition or intermittence repetition, and have a fixed pattern. For intermittent audio signals, since there is no audio signal during the pause period of the intermittent audio signal, there is no spectral feature to be extracted during the pause period, but the pause period can become the time domain feature of the intermittent audio signal. one.

For example, in some embodiments, step S101 may include: collecting an initial audio signal; performing downsampling on the initial audio signal to obtain a first audio signal.

Since the sampling rate of the initial audio signal collected by the audio acquisition device is high, it is not conducive to the back-end audio signal processing device (for example, artificial intelligence engine (AI (Artificial Intelligence) Engine), digital signal processor (Digital Signal) Processing (DSP for short), etc.), therefore, the initial audio signal can be down-sampled to achieve frequency reduction, which is convenient for processing by the audio signal processing device. For example, the frequency can be reduced to 48K Hz or even lower.

For example, in other embodiments, step S101 may include: collecting an initial audio signal; and filtering the initial audio signal to obtain a first audio signal.

In some application scenarios, being too quiet is not safe. Therefore, filtering can also be performed through a bandwidth controller (Bandwidth controller) to suppress audio signals within a specific frequency range. For continuous and intermittent audio signals (for example, knocking or dripping noise, etc.), the effective bandwidth of the first audio signal is set to the frequency range corresponding to the audio signal that needs to be suppressed, for example, 1K ~ 6K Hz , thereby ensuring that users can still hear more important sounds. For example, when used in the automotive field, it must be ensured that the driver can hear the horn, etc. to improve driving safety.

For example, in some embodiments, filtering processing and downsampling processing can also be used in combination, and the present disclosure does not limit the processing order of filtering processing and downsampling processing. For example, in some embodiments, obtaining the first audio signal may include: collecting an initial audio signal; filtering the initial audio signal to obtain an audio signal within a predetermined frequency range; and downsampling the audio signal within the predetermined frequency range. Processing to obtain the first audio signal; alternatively, obtaining the first audio signal may include: collecting an initial audio signal; performing downsampling processing on the initial audio signal; and performing filtering processing on the downsampled audio signal to obtain the first audio signal.

For example, the control instruction may include the time at which the second audio signal is output, the fourth audio signal, a control signal instructing to invert the fourth audio signal, and the like.

For example, in some embodiments, step S11 may include: based on the control instruction, determining a fourth audio signal and a control signal indicating inverting the fourth audio signal; based on the control signal, inverting the fourth audio signal Processed to generate a second audio signal.

For example, in some embodiments, step S12 may include: determining a first moment to output the second audio signal based on the control instruction; and outputting the second audio signal at the first moment.

For example, the third audio signal starts to appear from the second moment, and the absolute value of the time difference between the first moment and the second moment is less than the time threshold. It should be noted that the time threshold can be specifically set according to the actual situation, and this disclosure does not limit this. The smaller the time threshold, the better the silencing effect.

For example, in some embodiments, the time difference between the first moment and the second moment is 0, that is, the moment when the second audio signal starts to be output and the moment when the third audio signal starts to appear are the same. In the example shown in Figure 3 , the time when the second audio signal starts to be output and the time when the third audio signal starts to appear are both time t21.

For example, the time difference between the first moment and the second moment can be set according to the actual situation. For example, the first moment and the second moment can be set to ensure that the second audio signal and the third audio signal are transmitted to the target object at the same time, thereby avoiding The transmission of audio signals causes the second audio signal and the third audio signal to be out of sync, further improving the noise canceling effect. For example, the target object can be a human ear, a microphone, etc.

For example, the second audio signal can be output through a device such as a speaker that can convert an electrical signal into a sound signal for output.

It should be noted that when the audio collection device does not collect the audio signal, the audio processing method provided by the present disclosure may not be executed until the audio collection device collects the audio signal, thereby saving power consumption.

In embodiments of the present disclosure, the audio processing method can reduce or eliminate periodic audio signals (for example, noise) in environmental audio signals. For example, in application scenarios such as libraries, the sound of construction at a nearby construction site can be eliminated. wait. This type of scenario does not require special knowledge of the audio signals that you want to keep. It simply reduces the target sounds to be silenced in the environment that need to be eliminated. These target sounds to be silenced usually have the characteristics of continuous repetition or intermittence repetition, so they can be predicted through prediction. Predicted. It should be noted that the "target sound to be silenced" can be determined according to the actual situation. For example, for an application scenario such as a library, when there is a construction site around the library, the external environment audio signal can include two audio signals. The first The audio signal can be the sound of drilling at the construction site, and the second audio signal can be the sound of discussions by people around you. Usually, the sound of construction site drilling has periodic characteristics and usually has a fixed pattern. However, the discussion sound most likely does not have a fixed pattern and does not have periodic characteristics. At this time, the target sound to be silenced is the construction site drilling sound. Through the audio processing method provided by the embodiments of the present disclosure, it is possible to predict the drilling sound at the construction site, thereby eliminating or reducing the drilling sound at the construction site.

The audio processing method provided by embodiments of the present disclosure can be applied to automobile driving headrests to create a silent zone near the driver's ears to avoid unnecessary external audio signals (such as engine noise, road noise, wind noise, and tire noise). Noise signals while the car is driving) interfere with the driver. For another example, this audio processing method can also be applied to hair dryers, range hoods, vacuum cleaners, non-inverter air conditioners and other equipment to reduce the operating sound emitted by these equipment, allowing users to stay in noisy environments without being affected by the surrounding environment. The impact of environmental noise. This audio processing method can also be applied to headphones, etc., to reduce or eliminate external sounds, so that users can better receive the sounds from the headphones (music or phone calls, etc.).

At least one embodiment of the present disclosure also provides an audio processing device. Figure 6 is a schematic block diagram of an audio processing device provided by at least one embodiment of the present disclosure.

As shown in FIG. 6 , the audio processing device 600 includes an instruction generation module 601 , an audio generation module 602 and an output module 603 . The components and structures of the audio processing device 600 shown in FIG. 6 are only exemplary and not restrictive. The audio processing device 600 may also include other components and structures as needed.

The instruction generation module 601 is configured to generate a control instruction based on the first audio signal. The instruction generation module 601 is used to execute step S10 shown in Figure 2A.

The audio generation module 602 is configured to generate a second audio signal based on the control instruction. The audio generation module 602 is used to perform step S11 shown in Figure 2A.

The output module 603 is configured to output the second audio signal to suppress the third audio signal. The output module 603 is used to perform step S12 shown in Figure 2A.

For a specific description of the functions implemented by the instruction generation module 601, please refer to the relevant description of step S10 shown in FIG. 2A in the embodiment of the above audio processing method. For a specific description of the functions implemented by the audio generation module 602, please refer to the above audio For the relevant description of step S11 shown in FIG. 2A in the embodiment of the processing method, for a specific description of the functions implemented by the output module 603, please refer to the relevant description of step S12 shown in FIG. 2A in the embodiment of the audio processing method. . The audio processing device can achieve similar or identical technical effects to the foregoing audio processing method, which will not be described again here.

For example, the first audio signal appears earlier than the third audio signal.

For example, the sum of the phases of the second audio signal and the third audio signal is less than the phase threshold. In some embodiments, the phase of the second audio signal is opposite to the phase of the third audio signal, so that the third audio signal can be completely suppressed. .

For example, in some embodiments, the instruction generation module 601 may include an audio acquisition sub-module, a prediction sub-module and a generation sub-module. The audio acquisition sub-module is configured to acquire the first audio signal; the prediction sub-module is configured to process the first audio signal to predict a fourth audio signal; the generation sub-module is configured to generate a control instruction based on the fourth audio signal.

For example, the second audio signal and/or the third audio signal and/or the fourth audio signal are periodic or intermittent time domain signals.

For example, the third audio signal and the fourth audio signal may be exactly the same.

For example, in some embodiments, the prediction sub-module may process the first audio signal based on a neural network to predict the fourth audio signal. For example, the prediction sub-module may include the AI engine and/or digital signal processor in the audio processing part shown in Figure 1. The AI engine may include a neural network. For example, the AI engine may include a recurrent neural network, a long short-term memory network, or At least one neural network among generative adversarial networks and the like.

For example, in some implementations, the prediction sub-module includes a query unit and a prediction unit. The query unit is configured to generate a first audio feature code based on the first audio signal and query the lookup table based on the first audio feature code to obtain a second audio feature code. The prediction unit is configured to predict the fourth audio signal based on the second audio feature encoding.

For example, the lookup unit may include memory for storing lookup tables.

For example, in some embodiments, the lookup table may include at least one first encoding field. For example, in other embodiments, the lookup table further includes at least one second encoding field, and multiple first encoding fields constitute one second encoding field. Regarding the specific content of the lookup table, reference may be made to the relevant descriptions in the embodiments of the audio processing method described above, and repeated details will not be described again.

For example, the second audio feature encoding includes at least one first encoding field and/or at least one second encoding field.

For example, in some embodiments, the audio acquisition sub-module includes an acquisition unit and a downsampling processing unit. The acquisition unit is configured to collect the initial audio signal; the down-sampling processing unit is configured to perform down-sampling processing on the initial audio signal to obtain the first audio signal.

For example, in some embodiments, the audio acquisition sub-module includes an acquisition unit and a filtering unit. The acquisition unit is configured to acquire an initial audio signal; and the filtering unit is configured to filter the initial audio signal to obtain a first audio signal.

For example, the audio acquisition sub-module can be implemented as the audio receiving part shown in Figure 1. For example, the collection unit may include an audio collection device, such as a microphone in the audio receiving part shown in FIG. 1 , or the like. For example, the acquisition unit may also include an amplifier, an analog-to-digital converter, etc.

For example, in some embodiments, the output module 603 may include a moment determination sub-module and an output sub-module. The time determination sub-module is configured to determine a first time to output the second audio signal based on the control instruction; the output sub-module is configured to output the second audio signal at the first time.

For example, the output module 603 may be implemented as the audio output part shown in FIG. 1 .

For example, the third audio signal starts to appear from the second moment, and the absolute value of the time difference between the first moment and the second moment is less than the time threshold.

For example, the time difference between the first time and the second time may be zero.

For example, the output sub-module may include audio output devices such as speakers. For example, the output sub-module may also include a digital-to-analog converter, etc.

For example, the instruction generation module 601, the audio generation module 602, and/or the output module 603 may be hardware, software, firmware, or any feasible combination thereof. For example, the instruction generation module 601, the audio generation module 602 and/or the output module 603 can be a dedicated or general-purpose circuit, chip or device, or a combination of a processor and a memory. The embodiments of the present disclosure do not limit the specific implementation forms of each of the above modules, sub-modules and units.

At least one embodiment of the present disclosure also provides an audio processing device. FIG. 7 is a schematic block diagram of another audio processing device provided by at least one embodiment of the present disclosure.

For example, as shown in FIG. 7 , the audio processing device 700 includes one or more memories 701 and one or more processors 702 . One or more memories 701 are configured to store non-transitory computer-executable instructions; one or more processors 702 are configured to execute the computer-executable instructions. The computer-executable instructions, when executed by one or more processors 702, implement the audio processing method according to any of the above embodiments. For the specific implementation and related explanations of each step of the audio processing method, please refer to the description of the above embodiments of the audio processing method, and will not be described again here.

For example, in some embodiments, the audio processing device 700 may further include a communication interface and a communication bus. The memory 701, the processor 702 and the communication interface can communicate with each other through the communication bus, and the memory 701, the processor 6702 and the communication interface and other components can also communicate through a network connection. This disclosure does not limit the type and function of the network.

For example, the communication bus may be a Peripheral Component Interconnect Standard (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus can be divided into address bus, data bus, control bus, etc.

For example, the communication interface is used to implement communication between the audio processing device 700 and other devices. The communication interface may be a Universal Serial Bus (USB) interface, etc.

For example, the processor 702 and the memory 701 can be provided on the server side (or cloud).

For example, processor 702 may control other components in audio processing device 700 to perform desired functions. The processor 702 may be a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable Logic devices, discrete gate or transistor logic devices, discrete hardware components. The central processing unit (CPU) can be X86 or ARM architecture, etc.

For example, memory 701 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, and the like. One or more computer-executable instructions may be stored on the computer-readable storage medium, and the processor 702 may execute the computer-executable instructions to implement various functions of the audio processing device 700 . Various applications and various data can also be stored in the storage medium.

For example, for detailed description of the process of audio processing performed by the audio processing device 700, reference may be made to the relevant descriptions in the embodiments of the audio processing method, and repeated details will not be described again.

For example, in some embodiments, the audio processing device 700 may be embodied in the form of a chip, a small device/device, or the like.

FIG. 8 is a schematic diagram of a non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure. For example, as shown in Figure 8, one or more computer-executable instructions 1001 may be non-transitory stored on a non-transitory computer-readable storage medium 1000. For example, one or more steps in the audio processing method described above may be performed when the computer-executable instructions 1001 are executed by a processor.

For example, the non-transitory computer-readable storage medium 1000 can be applied in the above-mentioned audio processing device 700, and for example, it can include the memory 701 in the audio processing device 700.

For description of the non-transitory computer-readable storage medium 1000, reference may be made to the description of the memory 701 in the embodiment of the audio processing device 600 shown in FIG. 7, and repeated descriptions will not be repeated.

At least one embodiment of the present disclosure provides an audio processing method, an audio processing device and a non-transitory computer-readable storage medium. By learning the characteristics of the current audio signal, an audio signal that has not yet been generated (ie, the fourth audio signal) is predicted, The audio signal predicted based on this generates a future inverted audio signal to suppress the future audio signal, avoiding the problem of the inverted audio signal being out of sync with the audio signal that needs to be suppressed due to the delay between the input end and the output end, and improving noise reduction. The effect can significantly reduce or even eliminate the impact of the input-to-output delay on noise reduction, and the audio suppression effect is better than that of the backward active noise reduction system commonly used in the industry; because the first audio signal is a time domain signal , the first audio signal is not an audio signal of a specific frequency, so the audio processing method provided by the embodiment of the present disclosure does not need to extract spectral features from the audio signal to generate a spectrogram, thereby simplifying the audio signal processing process and saving processing time. ; In the lookup table, low-order feature codes can be combined to obtain high-order feature codes, thereby achieving more efficient and longer predictions; and in this audio processing method, filtering processing can also be performed through the bandwidth controller , thereby achieving suppression of audio signals within a specific frequency range to ensure that users can still hear more important sounds. For example, when used in the automotive field, it must be ensured that the driver can hear the horn, etc. to improve driving safety. property; in addition, when no audio signal is collected, the audio processing method provided by the present disclosure may not be executed until the audio signal is collected, thereby saving power consumption.

Regarding this disclosure, there are still several points that need to be explained:

(1) The drawings of the embodiments of this disclosure only refer to the structures related to the embodiments of this disclosure, and other structures can refer to common designs.

(2) Without conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

The above are only specific implementation modes of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims (26)

1. An audio processing method including:

   Based on the first audio signal, generate a control instruction;

   Based on the control instruction, generate a second audio signal;

   outputting the second audio signal to suppress the third audio signal,

   Wherein, the sum of the phases of the second audio signal and the third audio signal is less than a phase threshold, and the first audio signal appears earlier than the third audio signal.
2. The audio processing method according to claim 1, wherein the outputting the second audio signal to suppress the third audio signal includes:

   Based on the control instruction, determine a first moment to output the second audio signal;

   Output the second audio signal at the first moment,

   Wherein, the third audio signal starts to appear from the second time, and the absolute value of the time difference between the first time and the second time is less than a time threshold.
3. The audio processing method according to claim 2, wherein the time difference between the first time and the second time is 0.
4. The audio processing method according to any one of claims 1 to 3, wherein generating a control instruction based on the first audio signal includes:

   Obtain the first audio signal;

   Process the first audio signal to predict a fourth audio signal;

   The control instruction is generated based on the fourth audio signal.
5. The audio processing method according to claim 4, wherein the second audio signal and/or the third audio signal and/or the fourth audio signal are periodic or intermittent time domain signals.
6. The audio processing method according to claim 4 or 5, wherein said processing the first audio signal to predict a fourth audio signal includes:

   Generate a first audio feature code based on the first audio signal;

   Query a lookup table based on the first audio feature coding to obtain a second audio feature coding;

   Based on the second audio feature encoding, the fourth audio signal is predicted.
7. The audio processing method of claim 6, wherein the lookup table includes at least one first encoding field.
8. The audio processing method according to claim 7, wherein the lookup table further includes at least one second encoding field, and a plurality of the first encoding fields constitute one second encoding field.
9. The audio processing method according to claim 8, wherein the second audio feature encoding includes at least one of the first encoding field and/or at least one of the second encoding field.
10. The audio processing method according to any one of claims 4 to 9, wherein said obtaining the first audio signal includes:

    Collect initial audio signal;

    Perform downsampling processing on the initial audio signal to obtain the first audio signal.
11. The audio processing method according to any one of claims 4 to 9, wherein said obtaining the first audio signal includes:

    Collect initial audio signal;

    The initial audio signal is filtered to obtain the first audio signal.
12. The audio processing method according to any one of claims 1 to 11, wherein the phase of the second audio signal is opposite to the phase of the third audio signal.
13. An audio processing device, including:

    an instruction generation module configured to generate a control instruction based on the first audio signal;

    an audio generation module configured to generate a second audio signal based on the control instruction;

    an output module configured to output the second audio signal to suppress the third audio signal;

    Wherein, the sum of the phases of the second audio signal and the third audio signal is less than a phase threshold, and the first audio signal appears earlier than the third audio signal.
14. The audio processing device according to claim 13, wherein the output module includes a time determination sub-module and an output sub-module,

    The time determination submodule is configured to determine the first time to output the second audio signal based on the control instruction;

    The output sub-module is configured to output the second audio signal at the first moment,

    Wherein, the third audio signal starts to appear from the second time, and the absolute value of the time difference between the first time and the second time is less than a time threshold.
15. The audio processing device according to claim 14, wherein the time difference between the first time and the second time is 0.
16. The audio processing device according to any one of claims 13 to 15, wherein the instruction generation module includes an audio acquisition sub-module, a prediction sub-module and a generation sub-module,

    The audio acquisition sub-module is configured to acquire the first audio signal;

    The prediction sub-module is configured to process the first audio signal to predict a fourth audio signal;

    The generating sub-module is configured to generate the control instruction based on the fourth audio signal.
17. The audio processing device according to claim 16, wherein the second audio signal and/or the third audio signal and/or the fourth audio signal are periodic or intermittent time domain signals.
18. The audio processing device according to claim 16 or 17, wherein the prediction sub-module includes a query unit and a prediction unit,

    The query unit is configured to generate a first audio feature code based on the first audio signal and query a lookup table based on the first audio feature code to obtain a second audio feature code;

    The prediction unit is configured to predict the fourth audio signal based on the second audio feature encoding.
19. The audio processing device of claim 18, wherein the lookup table includes at least one first encoding field.
20. The audio processing device according to claim 19, wherein the lookup table further includes at least one second encoding field, and a plurality of the first encoding fields constitute one second encoding field.
21. The audio processing device according to claim 20, wherein the second audio feature encoding includes at least one of the first encoding field and/or at least one of the second encoding field.
22. The audio processing device according to any one of claims 16 to 21, wherein the audio acquisition sub-module includes a collection unit and a downsampling processing unit,

    The collection unit is configured to collect an initial audio signal;

    The down-sampling processing unit is configured to perform down-sampling processing on the initial audio signal to obtain the first audio signal.
23. The audio processing device according to any one of claims 16 to 21, wherein the audio acquisition sub-module includes a collection unit and a filtering unit,

    The collection unit is configured to collect an initial audio signal;

    The filtering unit is configured to perform filtering processing on the initial audio signal to obtain the first audio signal.
24. The audio processing device according to any one of claims 13 to 23, wherein the phase of the second audio signal is opposite to the phase of the third audio signal.
25. An audio processing device, including:

    One or more memories that non-transitoryly store computer-executable instructions;

    one or more processors configured to execute the computer-executable instructions,

    Wherein, when the computer-executable instructions are run by the one or more processors, the audio processing method according to any one of claims 1 to 12 is implemented.
26. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when executed by a processor, the computer-executable instructions implement any one of claims 1 to 12 The audio processing method described in the item.

Images