CN114598970A

CN114598970A - Audio processing method and device, electronic equipment and storage medium

Info

Publication number: CN114598970A
Application number: CN202210229393.8A
Authority: CN
Inventors: 周岭松
Original assignee: Beijing Xiaomi Mobile Software Co Ltd; Beijing Xiaomi Pinecone Electronic Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd; Beijing Xiaomi Pinecone Electronic Co Ltd
Priority date: 2022-03-10
Filing date: 2022-03-10
Publication date: 2022-06-07

Abstract

The present disclosure relates to an audio processing method, apparatus, electronic device, and storage medium, the method comprising: the method comprises the steps of obtaining a first audio signal received by wearable equipment in an environment where the wearable equipment is located, obtaining a second audio signal, wherein the second audio signal is obtained by the wearable equipment after the wearable equipment conducts transparent filtering processing on the first audio signal based on a transparent filtering coefficient, adjusting the transparent filtering coefficient under the condition that the first audio signal is not matched with the second audio signal, conducting the transparent filtering processing on the first audio signal through the adjusted transparent filtering coefficient, generating a third audio signal for playing, and matching the third audio signal with the first audio signal. Therefore, the transparent filtering coefficient can be adjusted according to different environmental sounds, and the environmental sounds heard by the user when the user uses the wearable device are close to the environmental sounds in the no-ear state.

Description

Audio processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of signal processing, and in particular, to an audio processing method and apparatus, an electronic device, and a storage medium.

Background

In the related art, when a user wears a TWS (True Wireless Stereo) headset, the user cannot hear the external sound due to the blockage of the headset. Through adopting sound to pass through the technique, the environmental sound that will gather, through the penetrating filter filtering in the earphone after, broadcast to the user's duct with loudspeaker on the earphone in, the environmental sound that reveals to the user's duct in the stack, sound when realizing wearing the earphone is the same with the sound under the empty ear state. However, for different environments and wearing modes, the transparent filter cannot be completely matched with the actual environment, so that transparent transmission of environment sound cannot be accurately realized, and a user may hear greater or smaller environmental noise, thereby bringing poor user experience to the user.

Disclosure of Invention

To overcome the problems in the related art, the present disclosure provides an audio processing method, apparatus, electronic device, and storage medium.

According to a first aspect of the embodiments of the present disclosure, there is provided an audio processing method, including:

acquiring a first audio signal received by the wearable device in an environment;

acquiring a second audio signal, wherein the second audio signal is obtained by the wearable device after the wearable device performs through filtering processing on the first audio signal based on a through filtering coefficient;

adjusting the pass-through filter coefficients if the first audio signal does not match the second audio signal;

and performing through filtering processing on the first audio signal through the adjusted through filtering coefficient to generate a third audio signal for playing, wherein the third audio signal is matched with the first audio signal.

Optionally, the adjusting the pass-through filter coefficient includes:

determining a target filter coefficient from a plurality of candidate filter coefficients according to the first audio signal and the second audio signal;

updating the pass-through filter coefficient of the wearable device with the target filter coefficient.

Optionally, the determining, by the first audio signal and the second audio signal, a target filter coefficient among a plurality of candidate filter coefficients, where the plurality of candidate filter coefficients correspond to a plurality of amplitude ranges in a one-to-one manner, includes:

acquiring a difference value between the average amplitude of the first audio signal in a preset frequency range and the average amplitude of the second audio signal in the preset frequency range;

and determining a filter coefficient corresponding to a target amplitude range in the multiple candidate filter coefficients as the target filter coefficient, wherein the target amplitude range is an amplitude range in which the average amplitude difference value is located.

Optionally, the audio processing method further includes:

acquiring a difference value between the average amplitude of the first audio signal and the average amplitude of the second audio signal;

determining that the first audio signal does not match the second audio signal if the average amplitude difference is greater than a preset average amplitude difference.

Optionally, the obtaining an average amplitude difference value of the first audio signal and the second audio signal includes:

generating a first frequency response curve of the first audio signal and a second frequency response curve of the second audio signal;

determining a first average amplitude value of the first frequency response curve in the preset frequency range and a second average amplitude value of the second frequency response curve in the preset frequency range;

and subtracting the first average amplitude value from the second average amplitude value to generate a difference value of the average amplitude values.

Optionally, the audio processing method further includes:

identifying audio properties of the first audio signal, the audio properties including an environmental noise floor property and/or a vocal property of the environment;

and under the condition that the voice attribute is not included in the audio attributes, judging whether the first audio signal is matched with the second audio signal.

Optionally, the plurality of pass-through filter coefficients of the wearable device are determined by:

acquiring a test audio signal of a test audio received by the wearable device in an unworn state;

acquiring a noise reduction audio signal of the test audio subjected to noise reduction by the wearable device in a state that the wearable device is worn;

generating a test frequency response curve of the test audio signal and a passive noise reduction frequency response curve of the noise reduction audio signal;

generating a compensation frequency response curve according to the test frequency response curve and the passive noise reduction frequency response curve;

determining a reference frequency response curve of the compensation frequency response curve by setting a recursion performance parameter of a recursion filter;

adjusting a recursive performance parameter of the recursive filter based on the reference frequency response curve to determine a plurality of pass-through filter coefficients.

Optionally, the determining a reference frequency response curve of the compensated frequency response curve by setting a recursive performance parameter of a recursive filter includes:

performing a first recursive performance parameter update procedure, the first recursive performance parameter update step comprising: inputting initial recursive performance parameters in the recursive filter to generate an initial analog frequency response curve; under the condition that the difference value between the average amplitude of the initial analog frequency response curve in the preset frequency range and the average amplitude of the compensation frequency response curve in the preset frequency range is larger than the preset average amplitude difference value, updating the recursive performance parameters of the recursive filter by taking the initial recursive performance parameters as a reference to obtain updated recursive performance parameters;

performing a second recursive performance parameter update procedure, the second recursive performance parameter update procedure comprising: inputting the updated recursive performance parameters into the recursive filter to generate a corresponding analog frequency response curve; under the condition that the difference value between the average amplitude value of the corresponding analog frequency response curve in the preset frequency range and the average amplitude value of the compensation frequency response curve in the preset frequency range is greater than the preset average amplitude value difference value and is smaller than the average amplitude value difference value in the last recursive performance parameter updating process, updating the recursive performance parameters of the recursive filter by taking the updated recursive performance parameters as a reference, wherein the last recursive performance parameter updating process is the first recursive performance parameter updating process or the second recursive performance parameter updating process executed last time;

and repeatedly executing the second recursive performance parameter updating process until the average amplitude difference value in the second recursive performance parameter updating process reaches the preset average amplitude difference value, and taking the analog frequency response curve generated in the second recursive performance parameter updating process as a reference frequency response curve.

Optionally, the adjusting the recursive performance parameter of the recursive filter based on the reference frequency response curve to determine a plurality of pass-through filter coefficients includes:

determining the recursive performance parameter of the recursive filter corresponding to the reference frequency response curve as an initial pass-through filter coefficient;

adjusting the recursive performance parameters of the recursive filter to enable the reference frequency response curve to translate for multiple times in the direction of the amplitude coordinate axis according to a preset rule, and generating a plurality of corresponding detection frequency response curves;

determining a difference value between the average amplitude of each detection frequency response curve and the average amplitude of the reference frequency response curve, and a pass-through filter coefficient corresponding to each detection frequency response curve, wherein the pass-through filter coefficient corresponding to each detection frequency response curve and the initial pass-through filter coefficient are used as the plurality of pass-through filter coefficients;

and constructing a corresponding relation between the average amplitude difference value corresponding to each detection frequency response curve and the multiple through filter coefficients based on each detection frequency response curve.

According to a second aspect of an embodiment of the present disclosure, there is provided an audio processing apparatus including:

a first acquisition module configured to acquire a first audio signal received by the wearable device in an environment in which the wearable device is located;

a second obtaining module configured to obtain a second audio signal, where the second audio signal is obtained by the wearable device after performing through filtering processing on the first audio signal based on a through filtering coefficient;

an adjustment module configured to adjust the pass-through filter coefficients if the first audio signal does not match the second audio signal;

a generating module configured to perform pass-through filtering processing on the first audio signal through the adjusted pass-through filtering coefficient, and generate a third audio signal for playing, where the third audio signal is matched with the first audio signal.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

adjusting the pass filter coefficient if the first audio signal does not match the second audio signal;

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the audio processing method provided by the first aspect of the present disclosure.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects: .

In the above scheme, a second audio signal is obtained by obtaining a first audio signal received by the wearable device in an environment where the wearable device is located, the second audio signal is obtained by the wearable device after performing transparent filtering processing on the first audio signal based on a transparent filtering coefficient, the transparent filtering coefficient is adjusted under the condition that the first audio signal is not matched with the second audio signal, the first audio signal is subjected to the transparent filtering processing through the adjusted transparent filtering coefficient, a third audio signal used for playing is generated, and the third audio signal is matched with the first audio signal. Like this, through judging whether the first audio signal of wearable equipment in the environment of locating matches with the second audio signal after through the penetrating filtering processing to adjust penetrating filter coefficient, make the third audio signal that obtains after the adjustment and first audio signal phase-match, realize adjusting penetrating filter coefficient according to the environment sound of difference, the environment sound that makes the user hear when using wearable equipment more approaches the environment sound under the empty ear state.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flow diagram illustrating an audio processing method according to an example embodiment.

Fig. 2 is a diagram illustrating a method for adjusting pass-through filter coefficients according to an exemplary embodiment.

Fig. 3 is a schematic diagram illustrating a method for determining a plurality of pass-through filter coefficients of a wearable device according to an exemplary embodiment.

Fig. 4 is a schematic diagram illustrating a frequency response curve according to an example embodiment.

Fig. 5 is a schematic diagram illustrating a method of determining a reference frequency response curve according to an example embodiment.

Fig. 6 is a block diagram illustrating an audio processing device according to an example embodiment.

FIG. 7 is a block diagram illustrating an electronic device in accordance with an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

Fig. 1 is a flowchart illustrating an audio processing method, which is used in a terminal as shown in fig. 1, according to an exemplary embodiment, and includes the following steps.

In step S11, a first audio signal received by the wearable device in the environment is acquired.

It should be noted that the wearable device in this embodiment may be a Wireless bluetooth headset, for example, a TWS (True Wireless Stereo) headset, a Wireless smart helmet, or other wearable devices that can be used to transmit audio signals into the ear of the user. When the wearable device is used by a user, in order to enable audio signals to be transmitted into the ear canal of the user through the wearable device, the wearable device is required to cover at least the ear hole of the user. The wearable device may include a feedforward microphone configured to collect ambient sounds in an environment in which the wearable device is located, and convert the collected ambient sounds into a first audio signal according to the wearable device.

In step S12, a second audio signal is obtained, where the second audio signal is obtained by the wearable device through-filtering the first audio signal based on a through-filtering coefficient.

It can be understood that, in this embodiment, the wearable device has a pass-through filter, and in the pass-through mode, the wearable device can perform pass-through filtering processing on the first audio signal acquired from the environment by using a preset pass-through filtering coefficient, so as to obtain a corresponding second audio signal. For example, the preset pass-through filter coefficient of the wearable device may be a pass-through filter coefficient pre-stored in the wearable device, and after the wearable device is turned on, the pass-through filter coefficient pre-stored in the wearable device is read, and the first audio signal is subjected to pass-through filtering processing to generate a second audio signal. The wearable device may further include a pass-through filter, which obtains a pass-through filter coefficient after adjusting based on the audio signal in the previous environment after moving from one audio environment to another audio environment when being used, and performs pass-through filtering processing on the first audio signal in the new environment according to the pass-through filter coefficient obtained by the previous adjustment, so as to obtain a second audio signal, where the pass-through filter coefficient is suitable for the audio signal in the previous environment, but when the environmental audio changes or the wearable device is moved to another environment when being used, the pass-through filter coefficient at this time may not be suitable for the new audio environment, and therefore analysis and matching are required, and the pass-through filter coefficient is adjusted.

Optionally, after the step S12, the audio processing method further includes:

a difference between the average amplitude of the first audio signal and the average amplitude of the second audio signal is obtained.

In the case where the difference is greater than the preset amplitude difference, it is determined that the first audio signal does not match the second audio signal.

It can be understood that, comparing the acquired first audio signal with the second audio signal, optionally, determining an audio curve of the first audio signal and the second audio signal by an audio detection device, and determining whether the first audio signal and the second audio signal are matched with each other by comparing similarity of the audio curves. Alternatively, whether the first frequency response curve and the second frequency response curve are matched with each other may be determined by calculating an average amplitude difference value of the first audio signal and the second audio signal, and comparing whether the average amplitude difference value exceeds a preset average amplitude difference value.

Under first audio signal and the second audio signal assorted condition, then confirm that penetrating filter coefficient this moment can be with the penetrating audio processing of environment audio frequency for close audio signal, transmit to user's auditory canal through wearable equipment and enable the user still can hear the audio signal that is close to with the environment audio frequency when using wearable equipment to carry out filtering processing to the audio signal in the environment of locating based on penetrating filter coefficient. For example, in the present embodiment, whether the first audio signal and the second audio signal match is determined by calculating an average amplitude difference value of the first audio signal and the second audio signal, and when the average amplitude difference value of the first audio signal and the second audio signal is smaller than or equal to a preset average amplitude difference value, it indicates that the first audio signal and the second audio signal match; when the average amplitude difference value of the first audio signal and the second audio signal is greater than the preset average amplitude difference value, determining that the first audio signal is not matched with the second audio signal and the transparent filter coefficient needs to be correspondingly adjusted, optionally, the preset average amplitude difference value is used for comparing the matching degree of the first audio signal and the second audio signal, under the environment with higher precision requirement, a smaller preset average amplitude difference value can be set, and under the condition that the matching degree of the first audio signal and the second audio signal is higher, the average amplitude difference value can be smaller than or equal to the preset average amplitude difference value; in an environment with lower accuracy requirements, a smaller preset average amplitude difference value can be set.

Optionally, the step of obtaining the difference between the average amplitude of the first audio signal and the average amplitude of the second audio signal may include:

a first frequency response curve of the first audio signal and a second frequency response curve of the second audio signal are generated.

And determining a first average amplitude of the first frequency response curve in a preset frequency range and a second average amplitude of the second frequency response curve in the preset frequency range.

And the first average amplitude value and the second average amplitude value are subjected to difference to generate a difference value of the average amplitude values.

It should be noted that, the environmental audio is analyzed, amplitude signals at different frequencies can be obtained, and a frequency response curve corresponding to the audio signal is obtained by detecting the amplitudes of the audio signal at different frequencies. In this embodiment, the acquired first audio signal and the acquired second audio signal are converted into a first frequency response curve and a second frequency response curve, respectively. It can be understood that the frequency response curve obtained after the conversion is a frequency response curve of the environmental audio in the full frequency band, and for the human ear corresponding to the user, the audio frequency range that can be heard by the human ear is 20Hz to 20000Hz, so that when the audio signal comparison is performed, only the audio signal in the frequency range that can be heard by the human ear needs to be compared, in this embodiment, to make the matching result more accurate, the average amplitude value of the 1kHz to 5kHz frequency band is intercepted to calculate, a first average amplitude value difference value of the first frequency response curve in the 1kHz to 5kHz frequency band is determined, and a second average amplitude value difference value of the second frequency band curve in the 1kHz to 5kHz frequency band is determined, and a difference value between the first average amplitude value and the second average amplitude value is calculated and used as the average amplitude value difference value, it can be understood that the average amplitude value difference value at this time represents a numerical value difference between the first average amplitude value and the second average amplitude value by a numerical value, and expressing the magnitude relation of the first average amplitude and the second average amplitude by positive and negative relations.

Optionally, after the step S12, the audio processing method further includes:

audio properties of the first audio signal are identified, the audio properties including an ambient noise floor property and/or a vocal property of an environment in which the first audio signal is located.

And under the condition that the voice attribute is not included in the audio attribute, judging whether the first audio signal is matched with the second audio signal.

It can be understood that the background noise in the environment represents a series of relatively regular environmental noises, and the human voice in the environment has the characteristics of relatively large frequency fluctuation, relatively large individual difference, instability and the like relative to the background noise frequency in the environment, so that the human voice is more difficult to detect and analyze compared with the background noise frequency in the environment. Therefore, in this embodiment, whether the transparent filter coefficient of the wearable device needs to be adjusted is determined mainly by analyzing the bottom noise frequency in the environment, and whether a human Voice exists in the first audio signal is determined by performing Voice Activity Detection ((VAD) on the collected first audio signal in the environment, so as to identify that the audio attribute of the first audio signal is the environment bottom noise attribute or the human Voice attribute.

In step S13, in the case that the first audio signal does not match the second audio signal, the pass-through filter coefficients are adjusted.

For example, when the first frequency response signal does not match the second frequency response signal, it indicates that the pass-through filter coefficient at this time is not suitable for the current environment, and therefore, the pass-through filter coefficient of the wearable device needs to be adjusted.

Fig. 2 is a schematic diagram illustrating a method for adjusting pass filter coefficients according to an exemplary embodiment, where the step S13 may include:

in step S131, a target filter coefficient is determined among the plurality of candidate filter coefficients according to the first audio signal and the second audio signal.

In step S132, the target filter coefficient is used to update the pass-through filter coefficient of the wearable device.

For example, in this embodiment, a plurality of candidate filter coefficients are preset in the wearable device, and a corresponding target filter coefficient is selected from the plurality of candidate filter coefficients by analyzing the first audio signal and the second audio signal. Optionally, a pass-through filtering test may be performed in a silent environment, a correspondence between an average amplitude difference between the ambient audio and the audio transmitted in the ear canal and a pass-through filtering coefficient is established, so as to establish a one-to-one correspondence between a plurality of candidate filtering coefficients and a plurality of average amplitude differences, and a target filtering coefficient is determined by comparing the average amplitude difference between the first audio signal and the second audio signal and according to the correspondence between the plurality of candidate filtering coefficients and the average amplitude difference. And updating a transparent filter coefficient of the wearable device by adopting the target filter coefficient to perform transparent filtering processing on the first audio signal in the environment.

Optionally, the step S131 includes:

and acquiring the difference value of the average amplitude of the first audio signal in the preset frequency range and the average amplitude of the second audio signal in the preset frequency range.

And determining a filter coefficient corresponding to the target amplitude range in the plurality of candidate filter coefficients as a target filter coefficient, wherein the target amplitude range is an amplitude range in which the average amplitude difference value is located.

For example, a first frequency response curve corresponding to the first audio signal and a second frequency response curve corresponding to the second audio signal may be determined in the above manner, a first average amplitude of the first frequency response curve in a preset frequency range and a second average amplitude of the second frequency response curve in the preset frequency range may be determined through calculation, and an average amplitude difference value may be determined according to the first average amplitude and the second average amplitude.

It can be understood that a plurality of alternative filter coefficients are preset in the wearable device, and one of the alternative filter coefficients corresponds to a section of amplitude range, and the amplitude ranges corresponding to the alternative filter coefficients are different. And determining the amplitude range corresponding to the average amplitude difference value as a target amplitude range by identifying the amplitude range, and taking the filter coefficient corresponding to the target amplitude range as a target filter coefficient.

In step S14, the first audio signal is subjected to pass-through filtering processing by the adjusted pass-through filtering coefficient, so as to generate a third audio signal for playing, where the third audio signal matches with the first audio signal.

For example, in step S14, after the pass-through filter coefficient is adjusted, the wearable device performs pass-through filtering on the acquired first audio signal by using the adjusted pass-through filter coefficient, so as to obtain a third audio signal, where the third audio signal matches with the first audio signal, so that the user can clearly hear the ambient sound based on the wearable device.

In the scheme, by acquiring a first audio signal received by the wearable device in the environment, the wearable device at least covers the earhole when being worn; the method comprises the steps of obtaining a second audio signal, wherein the second audio signal is obtained after the wearable device conducts through filtering processing on the first audio signal based on a through filtering coefficient, under the condition that the first audio signal is not matched with the second audio signal, the through filtering coefficient is adjusted, the first audio signal is conducted through filtering processing through the adjusted through filtering coefficient, a third audio signal used for playing is generated, and the third audio signal is matched with the first audio signal. Like this, through judging whether the first audio signal of wearable equipment in the environment of locating matches with the second audio signal after through the penetrating filtering processing to adjust penetrating filter coefficient, make the third audio signal that obtains after the adjustment and first audio signal phase-match, realize adjusting penetrating filter coefficient according to the environment sound of difference, the environment sound that makes the user hear when using wearable equipment more approaches the environment sound under the empty ear state.

Fig. 3 is a schematic diagram illustrating a method for determining a plurality of pass-through filter coefficients of a wearable device according to an exemplary embodiment, and referring to fig. 3, the plurality of pass-through filter coefficients may be determined by:

in step S21, a test audio signal of a test audio received in a state where the wearable device is not worn is acquired.

It is understood that the wearable device for audio transmission needs to be subjected to acoustic characteristic detection in a sound damping chamber before shipping, and the acoustic characteristic of the wearable device is determined by controlling variables in the sound damping chamber. Other environmental noise is not present in the anechoic chamber, and the background noise in the environment is simulated by playing test audio with known frequency and corresponding amplitude in the anechoic chamber. Test audio is collected by a feed-forward microphone of the wearable device and converted into a test audio signal.

In step S22, a noise reduction audio signal obtained by reducing the noise of the test audio by the wearable device is acquired in a state where the wearable device is worn.

It can be understood that the noise reduction audio signal is a part of the test audio signal that is propagated into the ear canal from the gap between the wearable device and the ear when the wearable device is in use, because the ear of the user is shielded but not completely shielded, and because the propagation path is blocked and interfered, the frequency and the corresponding amplitude of the part of the test audio signal are different from each other. At the moment, the wearable device is in a passive noise reduction state, feedback sound in the ear canal of the user is collected through a feedback microphone arranged in the wearable device facing the ear canal, and a noise reduction audio signal is determined based on the feedback sound.

In step S23, a test frequency response curve of the audio signal and a passive noise reduction frequency response curve of the noise reduction audio signal are generated.

Fig. 4 is a schematic diagram illustrating a frequency response curve according to an exemplary embodiment, and referring to fig. 4, it should be noted that when determining the compensation value, the audio signal needs to be converted into the frequency response curve as shown in fig. 4, and the test audio signal and the noise reduction audio signal are compared by determining the amplitude difference value at each frequency in the frequency response curve. Based on a signal conversion device in the wearable equipment, a test frequency response curve is generated by the received test audio signal, and a passive noise reduction frequency response curve is generated by the noise reduction audio signal.

In step S24, a compensation frequency response curve is generated according to the test frequency response curve and the passive noise reduction frequency response curve.

For example, in this embodiment, the wearable device needs to process the test audio signal through the pass-through filter, so that the audio signal obtained after the processing is superimposed with the noise reduction audio signal, and the audio signal matched with the test audio signal can be heard in the ear canal of the user. Therefore, the test audio signal needs to be compensated by the pass-through filter, and the compensation frequency response curve that needs to be compensated by the pass-through filter is generated by calculating the amplitude difference value of the test frequency response curve and the noise reduction frequency response curve at each frequency.

In step S25, a reference frequency response curve of the compensation frequency response curve is determined by setting a recursive performance parameter of the recursive filter.

For example, in step S25, after the compensation frequency Response curve that needs to be compensated by the transparent filter is determined through the above steps, a corresponding relationship between the compensation frequency Response curve and the filter coefficient corresponding to the transparent filter needs to be established.

Fig. 5 is a schematic diagram illustrating a method for determining a reference frequency response curve according to an exemplary embodiment, and referring to fig. 5, the step S25 may include:

performing a first recursive performance parameter update procedure, the first recursive performance parameter update step comprising: inputting initial recursive performance parameters in a recursive filter to generate an initial analog frequency response curve; and under the condition that the difference value between the average amplitude of the initial simulation frequency response curve in the preset frequency range and the average amplitude of the compensation frequency response curve in the preset frequency range is greater than the difference value of the preset average amplitudes, updating the recursion performance parameters of the recursion filter by taking the initial recursion performance parameters as the reference to obtain the updated recursion performance parameters.

Performing a second recursive performance parameter update procedure comprising: inputting the updated recursive performance parameters into a recursive filter to generate a corresponding analog frequency response curve; and under the condition that the difference value between the average amplitude of the corresponding analog frequency response curve in the preset frequency range and the average amplitude of the corresponding compensation frequency response curve in the preset frequency range is greater than the preset average amplitude difference value and is smaller than the difference value of the average amplitude in the last recursive performance parameter updating process, updating the recursive performance parameters of the recursive filter by taking the updated recursive performance parameters as the reference, wherein the last recursive performance parameter updating process is a first recursive performance parameter updating process or a second recursive performance parameter updating process executed last time.

The second recursive performance parameter updating process is repeatedly executed until the average amplitude difference value in the second recursive performance parameter updating process of this time reaches the preset average amplitude difference value, and the analog frequency response curve generated in the second recursive performance parameter updating process of this time is used as the reference frequency response curve, so that the reference frequency response curve matched with the compensation frequency response is generated as shown in fig. 5.

The executing the first recursive performance parameter updating process may be to randomly initialize the recursive performance parameter of the recursive filter to obtain a first recursive performance parameter, and obtain a first recursive signal corresponding to the first recursive performance parameter.

And calculating first difference values of the first recursive signal and the sound pressure level amplitude values of the same frequency of the compensation audio signal corresponding to the compensation frequency response curve, and calculating the sum of the first difference values to obtain a first sum value.

And determining whether the first recursive signal meets a preset recursive condition according to the first sum, wherein the preset recursive condition can be whether the first sum is smaller than a preset threshold value.

And if the first recursive signal meets the preset recursive condition, namely the first sum is smaller than a preset threshold value, generating a reference frequency response curve corresponding to the compensation frequency response curve according to the first recursive signal.

If the first recursive signal does not meet the preset recursive condition, namely the first sum is greater than or equal to a preset threshold value, on the basis of the first recursive performance parameter, randomly updating the recursive performance parameter of the recursive filter to obtain a second recursive performance parameter, and inputting the first recursive signal into the recursive filter to obtain a second recursive signal corresponding to the second recursive performance parameter.

And calculating second difference values of the sound pressure level amplitudes of the second recursive signal and the compensation audio signal at the same frequency, and calculating the sum of the second difference values to obtain a second sum value.

And determining whether the second recursive signal meets the preset recursive condition or not according to the second sum value. And if the second recursive signal meets the preset recursive condition, generating a reference frequency response curve corresponding to the compensation frequency response curve according to the second recursive signal.

And if the second recursion signal does not meet the preset recursion condition, determining a reference recursion performance parameter from the first recursion performance parameter and the second recursion performance parameter according to the magnitude relation between the second sum and the first sum. For example, in the case where the second sum value is larger than the first sum value, the first recursive performance parameter is determined as the reference recursive performance parameter; in the case where the second sum value is smaller than the first sum value, the second recursive performance parameter is determined as the reference recursive performance parameter.

On the basis of the reference recursive performance parameter, randomly updating the recursive performance parameter of the recursive filter to obtain a third recursive performance parameter, and inputting a recursive signal corresponding to the reference recursive performance parameter into the recursive filter to obtain a third recursive signal corresponding to the third recursive performance parameter. For example, in the case where the second recursive performance parameter is the reference recursive performance parameter, the recursive performance parameter of the recursive filter is randomly updated on the basis of the second recursive performance parameter to obtain a third recursive performance parameter.

And calculating a third difference value of the sound pressure level amplitude values of the same frequency of the third recursive signal and the compensation audio signal, and calculating the sum of all the third difference values to obtain a third sum value.

Determining whether the third recursive signal meets the preset recursive condition according to the third sum, and if the third recursive signal meets the preset recursive condition, generating a reference frequency response curve corresponding to the compensation frequency response curve according to the third recursive signal;

if the third recursive signal does not meet the preset recursive condition, determining a reference recursive performance parameter for randomly updating the recursive performance parameter of the recursive filter next time from the reference recursive performance parameter and the third recursive performance parameter according to the magnitude relation between the third sum and the difference value between the recursive signal corresponding to the reference recursive performance parameter and the sound pressure level amplitude value of the compensation audio signal at the same frequency; and the number of the first and second electrodes,

and executing the steps from randomly updating the recursion performance parameter of the recursion filter on the basis of the reference recursion performance parameter to determining the reference recursion performance parameter of the recursion filter which is randomly updated next time until the recursion signal corresponding to the reference recursion performance parameter meets the preset recursion condition, and generating a reference frequency response curve corresponding to the compensation frequency response curve according to the recursion signal corresponding to the reference recursion performance parameter.

In step S26, the recursive performance parameters of the recursive filter are adjusted based on the reference frequency response curve to determine a plurality of pass-through filter coefficients.

It can be understood that the through filter coefficient corresponding to the reference frequency response curve is obtained in a single test audio environment, and in practical applications, the ambient audio signal processed by the through filter coefficient is affected by factors such as the nature of the ambient noise floor and the way in which the user wears the wearable device, and thus has a deviation from an ideal audio signal. Therefore, in this embodiment, a plurality of through filter coefficients need to be determined based on the reference frequency response curve, for example, by simulating the determination process of the reference frequency response curve, a plurality of simulated frequency response curves having the same shape as the reference frequency response curve but different amplitudes are simulated in the recursive filter based on the reference frequency response curve, and the plurality of through filter coefficients are determined by reading the recursive performance parameters corresponding to the plurality of simulated frequency response curves.

Optionally, the step S26 may include:

and determining the recursive performance parameter of the recursive filter corresponding to the reference frequency response curve as an initial pass-through filter coefficient.

And adjusting the recursive performance parameters of the recursive filter to enable the reference frequency response curve to translate for multiple times in the direction of the amplitude coordinate axis according to a preset rule, and generating a plurality of corresponding detection frequency response curves.

And determining the difference value of the average amplitude of each detection frequency response curve and the average amplitude of the reference frequency response curve, the through filter coefficient corresponding to each detection frequency response curve, and the through filter coefficient corresponding to each detection frequency response curve and the initial through filter coefficient as a plurality of through filter coefficients.

And constructing a corresponding relation between the difference value of the average amplitude values corresponding to the detection frequency response curves and the multiple through filter coefficients based on the detection frequency response curves.

In an example, in this embodiment, the initial pass-through filter coefficient corresponding to the reference frequency response curve is determined by using the recursive performance parameter corresponding to the reference frequency response curve. By the approximation method of the recursive filter, a curve segment of the basic frequency response curve in the preset frequency range is intercepted, and the curve segment is translated for a plurality of times in the direction of the amplitude coordinate axis, so as to generate a plurality of detection frequency response curves which are identical to the reference frequency response curve and different in amplitude at each frequency, for example, the reference frequency response curve can be translated upwards by 5 amplitudes and translated downwards by 5 amplitudes, so as to obtain D1, D2, D3, D4 and D5 detection frequency response curves which are respectively 1dB, 2dB, 3dB, 4dB and 5dB different in average amplitude from the reference frequency response curve in the 5 forward directions, and D6, D7, D8, D9 and D10 detection frequency response curves which are respectively-1 dB, -2dB, -3dB, -4dB and-5 dB in average amplitude difference from the 5 reverse directions. And determining the transparent filter coefficient corresponding to each detection frequency response curve by reading the recursion performance parameters of each detection frequency response curve in the recursion filter, and establishing the corresponding relation between the average amplitude difference value and the transparent filter coefficient. After the wearable device is subjected to through filtering, the average amplitude difference value of the second frequency response curve obtained after through filtering and the reference frequency response curve in a preset frequency range is compared, and a target filtering coefficient of the wearable device is determined based on the corresponding relation.

Fig. 6 is a block diagram illustrating an audio processing device according to an example embodiment. Referring to fig. 6, the audio processing apparatus 100 includes a first obtaining module 110, a second obtaining module 120, an adjusting module 130, and a generating module 140.

The first obtaining module 110 is configured to obtain a first audio signal received by the wearable device in an environment in which the wearable device is located.

The second obtaining module 120 is configured to obtain a second audio signal, where the second audio signal is obtained by the wearable device through-filtering the first audio signal based on the through-filtering coefficient.

The adjusting module 130 is configured to adjust the pass-through filter coefficients if the first audio signal does not match the second audio signal.

The generating module 140 is configured to perform a pass-through filtering process on the first audio signal through the adjusted pass-through filtering coefficient, and generate a third audio signal for playing, where the third audio signal matches the first audio signal.

Optionally, the adjusting module may include:

a determination sub-module configured to determine a target filter coefficient among the plurality of candidate filter coefficients from the first audio signal and the second audio signal.

An update sub-module configured to update the pass-through filter coefficients of the wearable device with the target filter coefficients.

Optionally, the determining sub-module may be configured to:

and acquiring the difference value between the average amplitude of the first audio signal in the preset frequency range and the average amplitude of the second audio signal in the preset frequency range.

And determining a filter coefficient corresponding to the target amplitude range in the multiple candidate filter coefficients as a target filter coefficient, wherein the target amplitude range is the amplitude range in which the difference value is located.

Optionally, the audio processing apparatus 100 may further include:

a third obtaining module configured to obtain a difference value between the average amplitude of the first audio signal and the average amplitude of the second audio signal.

A first determination module configured to determine that the first audio signal does not match the second audio signal if the difference is greater than a preset average magnitude difference.

Optionally, the third obtaining module may be configured to:

Optionally, the audio processing apparatus 100 may further include:

an identification module configured to identify audio properties of the first audio signal, the audio properties comprising an ambient noise floor property and/or a vocal property of an environment in which the first audio signal is located.

The judging module is configured to judge whether the first audio signal is matched with the second audio signal or not under the condition that the voice attribute is not included in the audio attributes.

Optionally, the audio processing apparatus 100 may further include:

a fourth obtaining module configured to obtain a test audio signal of the test audio received in a state that the wearable device is not worn.

The fifth acquisition module is configured to acquire a noise reduction audio signal of the test audio subjected to noise reduction by the wearable device in a state that the wearable device is worn.

The first execution module is configured to generate a test frequency response curve of the test audio signal and a passive noise reduction frequency response curve of the noise reduction audio signal.

And the second execution module is configured to generate a compensation frequency response curve according to the test frequency response curve and the passive noise reduction frequency response curve.

A second determination module configured to determine a reference frequency response curve of the compensated frequency response curve by setting a recursive performance parameter of the recursive filter.

A third determination module configured to adjust a recursive performance parameter of the recursive filter based on the reference frequency response curve to determine a plurality of pass-through filter coefficients.

Optionally, the second determining module may be configured to:

performing a first recursive performance parameter update procedure, the first recursive performance parameter update step comprising: inputting initial recursive performance parameters in a recursive filter to generate an initial analog frequency response curve; and under the condition that the difference value between the average amplitude of the initial simulation frequency response curve in the preset frequency range and the average amplitude of the compensation frequency response curve in the preset frequency range is greater than the difference value of the preset average amplitudes, updating the recursive performance parameters of the recursive filter by taking the initial recursive performance parameters as the reference to obtain the updated recursive performance parameters.

Performing a second recursive performance parameter update procedure, the second recursive performance parameter update procedure comprising: inputting the updated recursive performance parameters into a recursive filter to generate a corresponding analog frequency response curve; and under the condition that the difference value between the average amplitude of the corresponding analog frequency response curve in the preset frequency range and the average amplitude of the corresponding compensation frequency response curve in the preset frequency range is greater than the preset average amplitude difference value and is smaller than the difference value of the average amplitude in the last recursive performance parameter updating process, updating the recursive performance parameters of the recursive filter by taking the updated recursive performance parameters as the reference, wherein the last recursive performance parameter updating process is a first recursive performance parameter updating process or a second recursive performance parameter updating process executed last time.

And repeatedly executing the second recursive performance parameter updating process until the average amplitude difference value in the second recursive performance parameter updating process reaches the preset average amplitude difference value, and taking the simulated frequency response curve generated in the second recursive performance parameter updating process as a reference frequency response curve.

Optionally, the third determining module may be configured to:

And determining the difference value between the average amplitude of each detection frequency response curve and the average amplitude of the reference frequency response curve, and the through filter coefficients corresponding to each detection frequency response curve, wherein the through filter coefficients corresponding to each detection frequency response curve and the initial through filter coefficients are used as a plurality of through filter coefficients.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

The present disclosure also provides a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the audio processing method provided by the present disclosure.

Fig. 7 is a block diagram illustrating an electronic device 700 in accordance with an example embodiment. The electronic device 700 may be configured as a wearable device, for example, the electronic device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a medical device, an exercise device, an audio device, a personal digital assistant, and the like.

Referring to fig. 7, the electronic device 700 may include one or more of the following components: a processing component 702, a memory 704, a power component 706, a multimedia component 708, an audio component 710, an input/output (I/O) interface 712, a sensor component 714, and a communication component 716.

The processing component 702 generally controls overall operation of the electronic device 700, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to perform all or a portion of the steps of the method of audio processing described above. Further, the processing component 702 may include one or more modules that facilitate interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate interaction between the multimedia component 708 and the processing component 702.

The memory 704 is configured to store various types of data to support operations at the electronic device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 704 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power component 706 provides power to the various components of the electronic device 700. The power components 706 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 700.

The multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 700 is in an operation mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 710 is configured to output and/or input audio signals. For example, the audio component 710 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 700 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in the memory 704 or transmitted via the communication component 716. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.

The I/O interface 712 provides an interface between the processing component 702 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the electronic device 700. For example, the sensor assembly 714 may detect an open/closed state of the electronic device 700, the relative positioning of components, such as a display and keypad of the electronic device 700, the sensor assembly 714 may also detect a change in the position of the electronic device 700 or a component of the electronic device 700, the presence or absence of user contact with the electronic device 700, orientation or acceleration/deceleration of the electronic device 700, and a change in the temperature of the electronic device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 700 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described audio processing methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 704 comprising instructions, executable by the processor 720 of the electronic device 700 to perform the audio processing method described above is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In another exemplary embodiment, a computer program product is also provided, which comprises a computer program executable by a programmable apparatus, the computer program having code portions for performing the audio processing method described above when executed by the programmable apparatus.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice in the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An audio processing method applied to a wearable device includes:

2. The audio processing method of claim 1, wherein the adjusting the pass-through filter coefficients comprises:

3. The audio processing method according to claim 2, wherein the candidate filter coefficients correspond to amplitude ranges in a one-to-one manner, and the determining a target filter coefficient among the candidate filter coefficients according to the first audio signal and the second audio signal comprises:

and determining a filter coefficient corresponding to a target amplitude range in the multiple candidate filter coefficients as the target filter coefficient, wherein the target amplitude range is an amplitude range in which the difference value is located.

4. The audio processing method of claim 1, wherein the method further comprises:

determining that the first audio signal does not match the second audio signal if the difference is greater than a preset amplitude difference.

5. The audio processing method of claim 4, wherein obtaining the difference between the average amplitude of the first audio signal and the average amplitude of the second audio signal comprises:

6. The audio processing method of claim 1, wherein the method further comprises:

7. The audio processing method of any of claims 2-6, wherein the plurality of pass-through filter coefficients of the wearable device are determined by:

8. The audio processing method of claim 7, wherein the determining the reference frequency response curve of the compensated frequency response curve by setting recursive performance parameters of a recursive filter comprises:

9. The audio processing method of claim 7, wherein said adjusting recursive performance parameters of the recursive filter based on the reference frequency response curve to determine a plurality of pass-through filter coefficients comprises:

10. An audio processing apparatus, comprising:

a first acquisition module configured to acquire a first audio signal received by the wearable device in an environment where the wearable device is located;

the generating module is configured to perform through-filtering processing on the first audio signal through the adjusted through-filtering coefficient, and generate a third audio signal for playing, where the third audio signal is matched with the first audio signal.

11. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

12. A computer-readable storage medium, on which computer program instructions are stored, which program instructions, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 9.