CN116782084A - Audio signal processing method and device, earphone and storage medium - Google Patents

Audio signal processing method and device, earphone and storage medium Download PDF

Info

Publication number
CN116782084A
CN116782084A CN202210216457.0A CN202210216457A CN116782084A CN 116782084 A CN116782084 A CN 116782084A CN 202210216457 A CN202210216457 A CN 202210216457A CN 116782084 A CN116782084 A CN 116782084A
Authority
CN
China
Prior art keywords
audio signal
equalization
audio
test
earphone
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210216457.0A
Other languages
Chinese (zh)
Inventor
练添富
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN202210216457.0A priority Critical patent/CN116782084A/en
Publication of CN116782084A publication Critical patent/CN116782084A/en
Pending legal-status Critical Current

Links

Abstract

An audio signal processing method and device, a headset and a storage medium, wherein the method is applied to the headset and comprises the following steps: outputting a first test audio signal and collecting a received audio signal corresponding to the first test audio signal; determining audio equalization parameters according to the received audio signals and the first test audio signals so as to equalize the second test audio to be output and output the equalized second test audio signals; and acquiring hearing detection information fed back for the second test audio signal, and determining an audio compensation parameter according to the hearing detection information, wherein the audio compensation parameter is used for carrying out adaptive equalization matched with the ear shape and/or the wearing state of the earphone of the user on the target audio signal to be output, and the audio compensation parameter is used for compensating the target audio signal after carrying out the adaptive equalization. By implementing the embodiment of the application, the earphone can provide personalized audio equalization and compensation processing for different users, and the effectiveness of the earphone in optimizing and adjusting the audio signals is improved.

Description

Audio signal processing method and device, earphone and storage medium
Technical Field
The present application relates to the field of electronic devices, and in particular, to an audio signal processing method and apparatus, an earphone, and a storage medium.
Background
Currently, when an earphone user wears the earphone, because actual scenes such as physiological conditions of the auditory canal of the user and the wearing mode of the earphone are often greatly different, the user is often easy to cause listening feeling which is not in line with expectations, and the experience of using the earphone by the user is poor. However, in practice, it is found that the conventional audio processing method (such as volume adjustment and noise reduction) is difficult to effectively cope with the above situations, so that the earphone cannot perform appropriate processing on the audio signal to be output according to the actual requirement of the user, thereby reducing the effectiveness of the earphone in performing optimal adjustment on the audio signal.
Disclosure of Invention
The embodiment of the application discloses an audio signal processing method and device, an earphone and a storage medium, which can enable the earphone to provide personalized audio equalization and compensation processing for different users, and is beneficial to improving the effect of the earphone on adaptively processing audio signals, so that the effectiveness of the earphone on optimizing and adjusting the audio signals can be improved.
An embodiment of the present application in a first aspect discloses an audio signal processing method applied to an earphone, where the earphone includes a speaker and a feedback microphone, the method includes:
Outputting a first test audio signal through the speaker;
collecting a received audio signal corresponding to the first test audio signal through the feedback microphone;
determining an audio equalization parameter according to the received audio signal and the first test audio signal;
according to the audio equalization parameters, performing equalization processing on second test audio to be output, and outputting an equalized second test audio signal through the loudspeaker;
acquiring hearing test information fed back for the second test audio signal;
and determining an audio compensation parameter according to the hearing detection information, wherein the audio compensation parameter is used for performing adaptive equalization processing matched with the ear shape and/or the wearing state of the earphone of a user on a target audio signal to be output, and the audio compensation parameter is used for performing hearing compensation on the target audio signal subjected to the adaptive equalization processing by using the audio compensation parameter.
A second aspect of an embodiment of the present application discloses an audio signal processing apparatus, which is applied to an earphone, the earphone includes a speaker and a feedback microphone, and the audio signal processing apparatus includes:
an audio output unit for outputting a first test audio signal through the speaker;
An audio receiving unit, configured to collect, by using the feedback microphone, a received audio signal corresponding to the first test audio signal;
a first determining unit, configured to determine an audio equalization parameter according to the received audio signal and the first test audio signal;
the audio equalization unit is used for performing equalization processing on second test audio to be output according to the audio equalization parameters and outputting an equalized second test audio signal through the loudspeaker;
the information acquisition unit is used for acquiring hearing detection information fed back for the second test audio signal;
and the second determining unit is used for determining audio compensation parameters according to the hearing detection information, wherein the audio compensation parameters are used for performing adaptive equalization processing matched with the ear shape and/or the wearing state of the earphone of the user on the target audio signal to be output, and the audio compensation parameters are used for performing hearing compensation on the target audio signal subjected to the adaptive equalization processing by using the audio compensation parameters.
A third aspect of the embodiment of the present application discloses an earphone, including a memory and a processor, where the memory stores a computer program, where the computer program when executed by the processor causes the processor to implement all or part of the steps in any one of the audio signal processing methods disclosed in the first aspect of the embodiment of the present application.
A fourth aspect of the embodiments of the present application discloses a computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements all or part of the steps of any one of the audio signal processing methods as disclosed in the first aspect of the embodiments of the present application.
Compared with the related art, the embodiment of the application has the following beneficial effects:
in the embodiment of the application, the earphone applying the audio signal processing method may include a speaker and a feedback microphone, and output a first test audio signal through the speaker, and then collect a received audio signal corresponding to the first test audio signal through the feedback microphone. On the basis, the earphone can determine the audio equalization parameters according to the received audio signals and the first test audio signals, and perform equalization processing on the second test audio to be output according to the audio equalization parameters so as to output the equalized second test audio signals through the loudspeaker. Further, the earphone may acquire hearing test information fed back for the second test audio signal, and determine the audio compensation parameter according to the hearing test information. The audio equalization parameters may be used to perform adaptive equalization processing matching with the ear shape and/or the wearing state of the earphone of the user on the target audio signal to be output, and the audio compensation parameters may be used to perform hearing compensation on the target audio signal after performing adaptive equalization processing by using the audio equalization parameters. Therefore, by implementing the embodiment of the application, the audio signal adjustment parameters in multiple dimensions can be obtained by continuously carrying out different types of tests according to the difference of the actual scenes such as the ear shape, the hearing characteristics, the wearing state of the earphone and the like of different users wearing the earphone, and the target audio signal to be output subsequently is correspondingly balanced and compensated, so that the comprehensive adjustment of the target audio signal is realized. Through realizing above-mentioned audio signal processing, can make the earphone provide individualized audio equalization and compensation processing to different users, not only be favorable to improving the effect of earphone self-adaptation processing audio signal, still be convenient for carry out more accurate noise reduction to the target audio signal and handle to can promote the effectiveness that the earphone carries out the optimization adjustment to audio signal.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly explain the drawings needed in the embodiments, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1A is a schematic view of an application scenario of an audio signal processing method according to an embodiment of the present application;
fig. 1B is a schematic diagram of another application scenario of an audio signal processing method according to an embodiment of the present application;
fig. 2 is a schematic flow chart of an audio signal processing method according to an embodiment of the present application;
FIG. 3 is a flow chart of mixing a base audio signal with a low frequency signal according to an embodiment of the present application;
FIG. 4 is a flow chart of another audio signal processing method according to an embodiment of the present application;
FIG. 5 is a schematic diagram of amplitude-frequency response corresponding to a test ear transfer function and a target ear transfer function according to an embodiment of the present application;
FIG. 6 is a graph of an amplitude-frequency response of an equalizer configured with first equalization parameters determined from the ear transfer function shown in FIG. 5;
FIG. 7 is a schematic diagram of a system amplitude-frequency response of an earphone with different leakage levels according to an embodiment of the present application;
FIG. 8 is a graph of the amplitude-frequency response of an equalizer configured with a second equalization parameter determined from the different leakage levels shown in FIG. 7;
FIG. 9 is a schematic diagram of the system amplitude-frequency response after equalization by the equalizer of FIG. 8;
fig. 10 is a flowchart of yet another audio signal processing method according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of an earphone according to an embodiment of the present application;
fig. 12 is a schematic diagram of an audio signal processing apparatus according to an embodiment of the present application;
fig. 13 is a schematic diagram of a headset according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It should be noted that the terms "comprises" and "comprising," along with any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiment of the application discloses an audio signal processing method and device, an earphone and a storage medium, which can enable the earphone to provide personalized audio equalization and compensation processing for different users, and is beneficial to improving the effect of the earphone on adaptively processing audio signals, so that the effectiveness of the earphone on optimizing and adjusting the audio signals can be improved.
The following detailed description will be given with reference to the accompanying drawings.
Referring to fig. 1A and fig. 1B together, fig. 1A is a schematic view of an application scenario of an audio signal processing method according to an embodiment of the present application, and fig. 1B is a schematic view of another application scenario of an audio signal processing method according to an embodiment of the present application. As shown in fig. 1A, the application scenario may include a user 10 and an earphone 20, where the user 10 may detect, through the earphone 20, influences of personalized factors such as an ear shape (or an ear canal physiological condition) of the user 10, a wearing state of the earphone 20, various hearing characteristics (such as different degrees of hearing impairment, different style preference, etc.) of the user 10 on a self-listening effect, and further determine corresponding audio equalization or compensation parameters for the personalized factors, so as to configure a suitable filter to perform corresponding audio signal processing on a target audio signal to be output by the earphone 20, so as to improve a listening effect of the user 10 using the earphone 20.
Illustratively, the headphones 20 may utilize different types of test audio signals to detect the impact of the various personalization factors described above on the listening effect of the user 10. In some embodiments, the test audio signal may comprise a first test audio signal, which may comprise a base audio signal and/or a low frequency signal. The basic audio signal may refer to an audio signal with a sound frequency in a main frequency band (typically 20-20000 Hz) within a human ear listening range, that is, a frequency band more common in daily life (e.g., 500-4000 Hz, etc.), and may be used to detect an influence of an ear shape of the user 10 on listening to the first test audio signal; the low-frequency signal may refer to an audio signal having a low sound frequency, for example, lower than the listening range of the human ear, that is, having a sound frequency of 20Hz or lower, and may be used to detect the influence of the wearing state of the earphone 20 worn by the user 10 on the listening of the first test audio signal, especially, the situation of low-frequency leakage in the first test audio signal (which is represented by irregular attenuation of the test audio signal in the middle-low frequency band) caused by improper wearing mode of the earphone, improper size of the earplug, and the like.
Alternatively, the first test audio signal may also be obtained by mixing the above-mentioned basic audio signal with a low frequency signal. By mixing the basic audio signal and the low-frequency signal, the earphone 20 can simultaneously analyze the influence of a plurality of different individuation factors by using the obtained first test audio signal, and further can set corresponding audio equalization parameters according to the different individuation factors, so that ear-shaped adaptive equalization, wearing leakage adaptive equalization and the like can be simultaneously realized, and the equalization effect of the earphone 20 on the audio signal is facilitated to be improved.
In some embodiments, the test audio signal may further comprise a second test audio signal, which may comprise a pure tone signal at a certain frequency point (e.g. 500Hz, 1000Hz, etc.), i.e. an audio signal consisting of only the audio signal component corresponding to the frequency point and not comprising audio signal components of other frequencies. The pure tone signal is adopted as the second test audio signal, the hearing sensitivity degree of the user 10 on the frequency point can be accurately judged through the corresponding hearing detection process, and then corresponding audio compensation parameters can be set for the hearing detection information, so that compensation corresponding to the hearing characteristics of the user is further realized, and the audio compensation effect can be further improved on the basis that the earphone 20 performs audio equalization on the audio signal.
In an embodiment of the present application, the headset 20 may include a speaker and a feedback microphone that may be located between the speaker and the user when the headset 20 is worn by the user 10. Specifically, the earphone 20 may output the first test audio signal through its speaker, where the first test audio signal may be generated in real time when the earphone 20 is actively triggered and detected by the user 10, or may be generated and stored in the earphone 20 in advance. On this basis, the earphone 20 may collect the received audio signal corresponding to the first test audio signal through its feedback microphone, and determine the corresponding audio equalization parameter according to the received audio signal and the first test audio signal. Further, the earphone 20 may perform equalization processing on the second test audio to be output according to the audio equalization parameter, and output the second test audio signal after the equalization processing through the speaker thereof again. The earphone 20 may acquire hearing test information fed back by the user 10 for the second test audio signal, and may then determine corresponding audio compensation parameters according to the hearing test information. The above-mentioned audio equalization parameters may be used to perform adaptive equalization processing matching with the ear shape and/or the wearing state of the earphone of the user 10 on the target audio signal to be output by the earphone 20, and the audio compensation parameters may be used to perform further hearing compensation on the target audio signal after the adaptive equalization processing using the above-mentioned audio equalization parameters.
Therefore, the method can acquire the audio signal adjustment parameters in multiple dimensions by continuously performing different types of tests according to the difference of the actual scenes such as the ear shape, the hearing characteristics, the wearing state of the earphone and the like of different users 10 wearing the earphone 20, and accordingly perform corresponding equalization and compensation on the target audio signal to be output subsequently, so as to realize comprehensive adjustment on the target audio signal. By implementing the above audio signal processing, the headphones 20 can provide personalized audio equalization and compensation processing for different users 10, which is beneficial to improving the effect of adaptively processing audio signals by the headphones 20, so that the target audio signal received by the users 10 can restore the actual initial sound quality (i.e. restore the tone, pitch, etc. when the target audio signal is recorded or generated) as much as possible; on the other hand, the earphone 20 is further conducive to more accurate noise reduction processing for the target audio signal, so that the problem that the target audio signal which is not subjected to equalization and compensation is deformed, attenuated and the like in the propagation process is avoided, and the earphone 20 misjudges the initial tone quality of the target audio signal and performs the noise reduction operation which is not matched with the actual requirement of the user 10, thereby improving the effectiveness of the earphone 20 in optimizing and adjusting the audio signal.
Optionally, as shown in fig. 1B, the earphone 20 may also establish a communication connection with the terminal device 30, so that when the listening characteristic of the user 10 needs to be detected, the user 10 may interact with the terminal device 30 to trigger, by the terminal device 30, the earphone 20 to output the above-mentioned second test audio signal. After the earphone 20 outputs the second test audio signal, the hearing test information fed back by the user 10 interacting with the terminal device 30 may be further obtained, so as to determine the corresponding audio compensation parameter. The terminal device 30 may include various devices or systems with wireless communication functions, such as a mobile phone, an intelligent wearable device, a vehicle-mounted terminal, a tablet computer, a PC (Personal Computer, a personal computer), a PDA (Personal Digital Assistant, a personal digital assistant), and the like, which are not particularly limited in the embodiment of the present application.
Referring to fig. 2, fig. 2 is a schematic flow chart of an audio signal processing method according to an embodiment of the application, and the method can be applied to the above-mentioned earphone, where the earphone may include a speaker and a feedback microphone. As shown in fig. 2, the audio signal processing method may include the steps of:
202. the first test audio signal is output through a speaker.
In the embodiment of the application, in order to determine the influence of personalized factors such as the ear shape (or physiological condition of the auditory canal) of the earphone user, the wearing state of the earphone wearing and the like on the self-hearing effect of the user, the basic audio signal and/or the low-frequency signal can be adopted as the first test audio signal.
The basic audio signal may include a white noise signal, an audio data signal (such as a music file, a recording file, an audio data signal corresponding to audio data with actual information such as chat voice, etc.), etc., which can cover a relatively large frequency range, especially a main frequency band included in a human ear listening range, so as to be used for detecting an influence of an audio system in which the earphone is located (i.e., a channel in which an audio signal output by the earphone is transmitted between the earphone and a user) on the first test audio signal in the frequency range. Specifically, since the ear-shaped personalized differences of different users mainly affect the audio signal transmission process in a higher frequency range (such as a frequency band of 1000Hz or more), by detecting the above-mentioned basic audio signal, analysis can be performed in a subsequent step for the higher frequency range, so as to realize corresponding ear-shaped adaptive equalization based on the ear-shaped personalized differences of different users.
The low-frequency signal may include a pure-tone signal with a sound frequency lower than a wearing leakage threshold, where the wearing leakage threshold may represent a lowest frequency (e.g., 20Hz, 30Hz, 50Hz, etc.) at which the audio signal in the ear canal leaks or leaks significantly due to different wearing states of the earphone, etc., in the case that the user wears the earphone. To test the effect of actual low frequency leakage, low frequency signals with sound frequencies below the wear leakage threshold described above, such as frequencies of 10Hz, 12Hz, 15Hz, etc., may be used. Taking the wearing leakage threshold of 20Hz as an example, the low-frequency signal may only be used in the ultralow frequency range of the infrasound frequency, so as to detect the influence of the audio system in which the earphone is located on the first test audio signal in the ultralow frequency range. On the basis, the low-frequency leakage possibly caused by different earphone wearing states of users mainly affects the audio signal transmission process of a lower frequency range (such as a frequency band below 1000 Hz), and the low-frequency signal can be analyzed in a subsequent step by detecting the low-frequency signal so as to realize corresponding wearing leakage self-adaptive equalization based on personalized differences of the earphone wearing states of different users.
Alternatively, the first test audio signal may be obtained by mixing a base audio signal with a low frequency signal. For example, the above-mentioned base audio signal may be filtered through a high-pass filter, and then the filtered base audio signal may be mixed with a low-frequency signal. The cut-off frequency of the high-pass filter can be higher than the frequency corresponding to the low-frequency signal, so that the interference of a basic audio signal to the low-frequency signal can be effectively reduced, and the ear-shaped self-adaptive equalization and the wearing leakage self-adaptive equalization can be independently realized conveniently.
Referring to fig. 3, fig. 3 is a schematic flow chart of mixing a basic audio signal and a low frequency signal according to an embodiment of the application. As shown in fig. 3, when the first test audio signal needs to be generated, the base audio signal may be input to the high-pass filter, the base audio signal is filtered by the high-pass filter, and then the filtered base audio signal and the low-frequency signal are input to the mixer together for mixing. The mixer may be used to directly add the filtered basic audio signal to the low frequency signal, or may be used to adjust gain of the filtered basic audio signal to the low frequency signal, add the low frequency signal, adjust delay, and add the low frequency signal to obtain the mixed first test audio signal.
204. A received audio signal corresponding to the first test audio signal is collected by a feedback microphone.
In the embodiment of the application, after the earphone outputs the first test audio signal, the earphone can immediately collect the received audio signal corresponding to the first test audio signal through the built-in feedback microphone. The first test audio signal is transmitted in the audio system where the earphone is located, and after being received by the feedback microphone, the first test audio signal can be used for evaluating the influence of the audio signal in the transmission process of the audio system so as to determine corresponding audio equalization parameters in the subsequent steps. It will be appreciated that the above-described audio system may also be approximately replaced by a path for the transmission of audio signals between the speaker and the feedback microphone, since the feedback microphone is located between the speaker and the user.
For example, the feedback microphone of the earphone may continuously collect the audio signal, so that the received audio signal collected by the feedback microphone at a time near the time stamp (e.g., delayed by 0.01 ms, delayed by 0.1 ms, etc.) may be obtained according to the time stamp of the output of the first test audio signal by the speaker. In some embodiments, the feedback microphone of the earphone may not be continuously turned on, but after the speaker outputs the first test audio signal, the speaker triggers the on, and the audio signal collected after the feedback microphone is turned on is used as the received audio signal corresponding to the first test audio signal. Optionally, for the received audio signal collected by the feedback microphone, the earphone may further utilize a signal processing module disposed in the earphone to compare waveforms of the first test audio signal output by the speaker with those of the received audio signal, and when the comparison result indicates that the waveform similarity between the test audio signal and the received audio signal meets a similarity threshold (such as 50%, 80%, etc.), the received audio signal may be identified as a received audio signal corresponding to the first test audio signal.
206. An audio equalization parameter is determined from the received audio signal and the first test audio signal.
In the embodiment of the application, after the earphone collects the received audio signal, the difference between the received audio signal and the first test audio signal can be analyzed according to the received audio signal so as to evaluate the influence of the audio signal in the transmission process of the audio system where the earphone is located, and further, the audio equalization parameters required for equalizing the influence can be determined.
In some embodiments, the audio equalization parameters may include a first equalization parameter and a second equalization parameter. The first equalization parameter may correspond to the basic audio signal, and is used for implementing ear-shaped adaptive equalization, so as to equalize the audio signal in a higher frequency range (such as a frequency band of 1000Hz or more) according to ear-shaped personalized differences of different users; the second equalization parameter may correspond to the low-frequency signal, and be used for implementing adaptive equalization of wearing leakage, so as to equalize the audio signal in the lower frequency range (such as the frequency band below 1000 Hz) according to personalized differences of the earphone wearing states of different users, especially different situations of low-frequency leakage in the audio signal caused by improper earphone wearing modes, improper earplug sizes, and the like.
In other embodiments, the audio equalization parameters may not be differentiated, i.e., the earphone may directly determine the audio equalization parameters for comprehensively equalizing the full-band audio signal according to the received audio signal and the first test audio signal. It should be noted that, no matter what type of audio equalization parameter is adopted, the ear-shaped adaptive equalization and the wearing leakage adaptive equalization can be respectively realized, so that the target audio signal to be output by the earphone can be comprehensively compensated, the influence possibly suffered by the transmission of the target audio signal in the audio system where the earphone is positioned can be equalized, and the target audio signal heard by the user can restore the actual initial sound quality as much as possible.
208. And carrying out equalization processing on the second test audio to be output according to the audio equalization parameters, and outputting the second test audio signal after the equalization processing through a loudspeaker.
In the embodiment of the application, in order to further perform corresponding audio signal compensation according to the hearing characteristics of the user so as to improve the listening effect when the user uses the earphone, the hearing detection information corresponding to the user needs to be acquired. Therefore, the earphone can evaluate the hearing characteristics of the user by outputting the second test audio signal and collecting the feedback condition of the user aiming at the second test audio signal so as to obtain corresponding hearing detection information. The second test audio signal may include pure tone signals at each frequency point to be detected (e.g., 500Hz, 1000Hz, etc.).
After the audio equalization parameters are determined, the earphone may configure a corresponding target equalization filter based on the audio equalization parameters, and perform equalization processing on the second test audio signal to be output through the target equalization filter, so as to offset the problems of deformation, attenuation and the like possibly occurring in the transmission process of the second test audio signal as much as possible, thereby improving accuracy and reliability of hearing detection. On this basis, the earphone may output the equalized second test audio signal through its speaker in a manner similar to the output of the first test audio signal described above.
210. And acquiring hearing test information fed back for the second test audio signal.
In the embodiment of the application, when the earphone acquires the hearing detection information fed back for the second test audio signal, the hearing detection result corresponding to the second test audio signal is determined by interaction with the user, namely based on whether the user hears the feedback of the second test audio signal. The hearing test information may include subjective judgment information about whether the user hears the second test audio signal, or may include a critical sound intensity (i.e., the sound intensity of the second test audio signal when the user just hears the second test audio signal) further determined according to the subjective judgment information, a audible sound intensity range, and the like.
In one embodiment, when the user acquires the above-mentioned fed-back hearing test information only through the earphone, this may be achieved by detecting a user operation for the earphone. For example, the user operation for the earphone may include a touch operation, a voice operation, a mobile operation, and the like.
For example, when the user receives the second test audio signal, the designated touch point on the earphone may be touched, so that when the earphone detects a touch operation for the designated touch point, the user may determine that the user receives the hearing state of the second test audio signal, and further obtain corresponding hearing detection information.
For another example, when the user hears the second test audio signal, the user may directly issue an "hear" voice command; when the user does not hear the second test audio signal, the user can directly send out a voice command of 'not hearing', so that the earphone can analyze the voice command detected by the earphone to determine whether the user hears the second test audio signal.
For another example, the user may also perform head movements in different directions according to whether the second test audio signal is heard, so that the headset may detect its own movement state through the sensor to determine whether the corresponding user is hearing the hearing state of the second test audio signal. For example, when the user hears the second test audio signal, the head may be tilted left so that the headphones detect a tendency to move left; when the user does not hear the second test audio signal, the head can be tilted to the right so that the earphone detects a trend of moving to the right, and then the earphone can determine the hearing detection information fed back by the user for the second test audio signal according to the detected moving trend.
In another embodiment, when the user further obtains the above-mentioned fed-back hearing test information through the terminal device communicatively connected to the earphone, this may also be achieved by obtaining a user operation for the terminal device. For example, the user operation for the terminal device may include a touch operation, a button click operation, or the like. When the terminal device detects the user operation, whether the user hears the hearing state of the second test audio signal or not can be determined according to the user operation, and the hearing state is sent to the earphone. On the basis, the earphone can further acquire the hearing detection information fed back for the second test audio signal according to the received hearing state.
212. And determining an audio compensation parameter according to the hearing detection information, wherein the audio compensation parameter is used for performing adaptive equalization processing matched with the ear shape and/or the wearing state of the earphone of the user on the target audio signal to be output, and the audio compensation parameter is used for performing hearing compensation on the target audio signal subjected to the adaptive equalization processing by using the audio equalization parameter.
In the embodiment of the application, the earphone can call the hearing test information through the built-in processor, and analyze the hearing characteristics of the user (such as the existence of different degrees of hearing impairment, different style favorites and the like) according to the hearing test information so as to determine the hearing sensitivity degree of the user on different frequency components of the audio signal. For example, if it is determined according to the hearing detection information that the hearing sensitivity of the user at a certain frequency point is low, that is, the user cannot easily hear the audio signal of the frequency component, the frequency component of the audio signal may be enhanced subsequently; if it is determined according to the hearing test information that the hearing sensitivity of the user at a certain frequency point is too high, that is, the user is easily stimulated by the audio signal of the frequency component, the frequency component of the audio signal can be reserved or weakened later. According to the user hearing characteristics obtained through the analysis, the earphone can further calculate corresponding audio compensation parameters, and the audio compensation parameters can be used for compensating target audio signals to be output by the earphone, namely, compensating corresponding to the hearing characteristics of the user respectively aiming at different frequency components of the target audio signals.
Illustratively, the audio equalization parameters may include equalization filter parameters, and the compensation parameters may include compensation filter parameters (each may include tap coefficients for configuring a filter, etc.). The earphone can configure a corresponding target equalization filter through the equalization filter parameters, and configure a corresponding target compensation filter through the compensation filter parameters. On the basis, the configured target equalization filter can be used for carrying out adaptive equalization on a target audio signal to be output by the earphone, and the target compensation filter can be used for further compensating the target audio signal subjected to adaptive equalization by using the target equalization filter.
In some embodiments, either the target equalization filter or the target compensation filter described above may be comprised of one or more filters. Specifically, when an audio signal in a specific frequency range needs to be equalized or compensated, a band-pass filter or a band-stop filter of a corresponding frequency band can be configured for filtering; when more complex equalization or compensation is required for the audio signals of multiple frequency bands, the corresponding filtering can also be performed by configuring a cascaded FIR (Finite Impulse Response, finite length unit impulse response) filter or IIR (Infinite Impulse Response, infinite length unit impulse response) filter.
Therefore, by implementing the audio signal processing method described in the above embodiment, different types of tests can be continuously performed to obtain audio signal adjustment parameters in multiple dimensions according to the difference of actual scenes such as ear shapes, hearing characteristics, wearing states of the headphones and the like of different users wearing the headphones, and accordingly, corresponding equalization and compensation are performed on the target audio signal to be output, so that comprehensive adjustment of the target audio signal is achieved. Through realizing above-mentioned audio signal processing, can make the earphone provide individualized audio equalization and compensation processing to different users, not only be favorable to improving the effect of earphone self-adaptation processing audio signal, still be convenient for carry out more accurate noise reduction to the target audio signal and handle to can promote the effectiveness that the earphone carries out the optimization adjustment to audio signal.
Referring to fig. 4, fig. 4 is a flowchart of another audio signal processing method according to an embodiment of the present application, and the method may be applied to the above-mentioned earphone, where the earphone may include a speaker and a feedback microphone. As shown in fig. 4, the audio signal processing method may include the steps of:
402. the first test audio signal is output through a speaker.
404. A received audio signal corresponding to the first test audio signal is collected by a feedback microphone.
Step 402 and step 404 are similar to step 202 and step 204 described above, and are not repeated here.
406. And determining a test ear transfer function corresponding to the received audio signal according to the received audio signal and the first test audio signal.
In the embodiment of the application, after the earphone collects the received audio signal, the influence of the corresponding first test audio signal in the transmission process of the audio system where the earphone is located can be estimated according to the received audio signal, so as to deduce a test ear transfer function determined by the individual difference of the user ear, and the test ear transfer function can be used for calculating the first equalization parameter for realizing ear adaptive equalization in the subsequent step.
The test ear transfer function may be calculated based on frequency domain forms obtained by fourier transforming the received audio signal and the first test audio signal, respectively. In some embodiments, the above received audio signal may be first subjected to frame-division windowing by a signal processing module (e.g., DSP module) built in the earphone, that is, the macroscopically unstable audio signal is divided into a plurality of audio signal frames with short-time stationarity (e.g., audio signal frames with frame length of 10-30 ms), and then FFT (Fast Fourier Transform ) operation is performed on the received audio signal based on each frame after the windowing, to obtain a frequency domain form corresponding to the received audio signal. Similarly, the first test audio signal may also be subjected to a corresponding framing and windowing process and FFT operation, and then may be further calculated to obtain a desired test ear transfer function in combination with the received audio signal.
408. And calculating a first equalization parameter corresponding to the user ear according to the test ear transfer function and the target ear transfer function.
In the embodiment of the application, after the earphone determines the test ear transfer function, the earphone can also acquire a corresponding target ear transfer function. The target ear transfer function may include an ear transfer function measured when the earphone is in a standard ear jig (such as IEC711, etc.), that is, the earphone is placed in the standard ear jig with good air tightness in the anechoic room environment, and the ear transfer function at this time is detected; the method can also comprise the steps of counting the obtained ear transfer function, for example, obtaining transfer functions when a large number of users wear the earphone normally in the environment of the anechoic room, and calculating the average value of the transfer functions to obtain corresponding target ear transfer functions; for another example, if the transfer function of the plurality of users when wearing headphones normally is expressed in the form of a function curve, the average value curve may be obtained for the function curve, and the function corresponding to the average value curve may be determined as the target ear function.
On the basis, the earphone can calculate a first equalization parameter corresponding to the ear shape of the user according to the test ear shape transfer function and the target ear shape transfer function A number. Wherein the first equalization parameters may comprise tap coefficients, gain coefficients, etc. for configuring filters comprised in the respective equalizer in order to facilitate ear-shaped adaptive equalization of the target audio signal to be output by the headphones in a subsequent step. Illustratively, the first equalization parameter configures a transfer function H of the equalizer e (k) This can be calculated by the following equation 1:
equation 1:
wherein H is p (k) Corresponding to the test ear transfer function, H t (k) Corresponding to the target ear transfer function, alpha is a correction factor. It should be noted that k in the above formula 1 may represent the divided frequency band sub-bands, that is, the earphone may calculate the first equalization parameters corresponding to each frequency band sub-band according to the test ear transfer function and the target ear transfer function corresponding to each frequency band sub-band. The above-described respective frequency domain sub-bands may be respective sub-frequency ranges divided within the frequency range of the first test audio signal, for example.
For example, referring to fig. 5 and fig. 6 together, fig. 5 is a schematic diagram of an amplitude-frequency response of an equalizer (denoted by a "first equalizer") configured with a first equalization parameter determined according to the ear-shaped transfer function shown in fig. 5 according to the embodiment of the present application, where the test ear-shaped transfer function and the target ear-shaped transfer function correspond to each other. As shown in fig. 5, the dashed line represents the frequency response of the above-described test ear transfer function, while the solid line represents the frequency response of the target ear transfer function, which are clearly distinguished in the higher frequency range of the frequency band of 1000Hz or more. By employing the first equalizer as shown in fig. 6, the audio signal can be equalized in the above-described higher frequency range so that the equalized ear transfer function (dotted line) is as close as possible to the target ear transfer function (solid line), thereby realizing ear-shaped adaptive equalization.
410. And carrying out signal cancellation processing on the received audio signal based on the first test audio signal to obtain an error audio signal.
In the embodiment of the application, the received audio signal collected by the earphone device through the feedback microphone of the earphone device can generate low-frequency leakage with different degrees according to different wearing states of the earphone of a user, but the main audio component of the received audio signal is still the transmitted first test audio signal. In order to highlight the influence of the low-frequency leakage, the earphone may remove the transmitted first test audio signal from the received audio signal to obtain a corresponding error audio signal, and then calculate a second equalization parameter corresponding to the wearing state of the earphone based on the error audio signal in a subsequent step.
Optionally, in order to determine the appropriate second equalization parameter, the earphone may further configure an output equalizer with the initial second equalization parameter, and filter the first test audio signal through the output equalizer to obtain an output audio signal corresponding to the first test audio signal. On the basis, the earphone can remove the transmitted output audio signal from the received audio signal (namely, replace the first test audio signal) to obtain a corresponding error audio signal, and then the second equalization parameter is updated based on the error audio signal in the subsequent step so as to improve the accuracy of wearing leakage self-adaptive equalization of the earphone.
For the first test audio signal S (n), it may be defined as follows, for example:
equation 2:
S(n)＝[s(n),s(n-1),...,s(n-N+1)] T
where N is the number of coefficients of the output equalizer to be determined, i.e. the output equalizer may be determined by N coefficients (initial value 0). After filtering the first test audio signal S (n) by the output equalizer Ha (n), the resulting output audio signal Sout (n) can be represented by the following formula 3:
equation 3:
Sout(n)＝Ha T (n)S(n)
on this basis, the earphone may filter the output audio signal Sout (n) through a transfer function filter Hsm' (n) to simulate the transmission influence of the audio transmission system in which the earphone is located on the output audio signal, and obtain a transfer audio signal Sh (n) corresponding to the output audio signal Sout (n), as shown in the following formula 4:
equation 4:
Sh(n)＝Hsm'(n) T Sout'(n)
further, the earphone may calculate an error between the transmitted audio signal Sh (n) and the received audio signal Sm (n) collected by the feedback microphone thereof, to obtain an error audio signal Se (n):
equation 5:
Se(n)＝Sm(n)-Sh(n)
412. and calculating a second equalization parameter corresponding to the wearing state of the earphone according to the error audio signal.
In the embodiment of the present application, the earphone may calculate the mean square error J between the reception audio signal Sm (n) and the transmission audio signal Sh (n) based on the error audio signal. Specifically, the above mean square error J may be calculated as shown in the following equation 6:
Equation 6:
J＝E[Se 2 (n)]＝E[Sm 2 (n)]-2E[Sm(n)Ha T (n)S(n)]+E[Ha T (n)S(n)S T (n)Ha(n)]
where E may represent a mathematical expectation. Further, according to the above-mentioned mean square error J, the earphone may update the output equalizer Ha (n), as shown in the following equation 7:
equation 7:
Ha(n+1)＝Ha(n)+2uSe(n)S(n)
wherein u is a step factor used for updating. After updating the output equalizer Ha (n) to obtain Ha (n+1), the earphone may re-execute the signal cancellation process, filter the first test audio signal S (n) by using the new output equalizer Ha (n+1), that is, repeat the calculation steps shown in formulas 3 to 7 until the update stop condition is satisfied, and determine the updated second equalization filter according to the output equalizer Ha (n) obtained when the update stop condition is satisfied. The update stopping condition may include an iteration number condition (for example, the number of updates reaches an upper limit of the number of times) and/or an iteration parameter condition (for example, the output equalizer Ha (n) or the step factor u satisfies a certain numerical condition), which is not specifically limited in the embodiment of the present application.
In the embodiment of the application, as different wearing states of the earphone of the user can generate different degrees of low-frequency leakage, the earphone can divide different leakage degrees according to different signal energy ranges in advance. Illustratively, the headphones may divide the respective leakage levels by a uniform step size (e.g., 0.2 units of normalized signal energy, 0.4 units of normalized signal energy, etc.), or may divide the different leakage levels by other distribution manners. Referring to fig. 7, fig. 7 is a schematic diagram showing a system amplitude-frequency response at different leakage levels according to an embodiment of the present application. As shown in fig. 7, if curve a is the frequency response corresponding to when the user wears the earphone normally (i.e. when no low frequency leakage occurs), curve B, C, D, E may represent the frequency response at different leakage levels, and may be ranked B, C, D, E according to the leakage severity level from low to high. It should be noted that each different leakage degree can be matched with the corresponding leakage frequency response curve, and the leakage frequency response curve matched with each leakage degree can be obtained by detecting (i.e. detecting frequency response) the earphone when the wearing state of the earphone respectively meets each leakage degree. On the basis, the earphone can determine corresponding second equalization parameters according to different leakage degrees, and then can configure a corresponding equalizer (expressed by a second equalizer) according to the second equalization parameters so as to wear leakage self-adaptive equalization on a target audio signal to be output by the earphone.
Referring to fig. 8 and 9 together, fig. 8 is a schematic diagram of the amplitude-frequency response of the second equalizer configured by the second equalization parameters determined by the different leakage levels shown in fig. 7, and fig. 9 is a schematic diagram of the amplitude-frequency response of the system equalized by the second equalizer shown in fig. 8. It will be appreciated that the more severe the leakage, the higher the level of equalization compensation of its corresponding second equalizer for audio signals in the lower frequency range below the 1000Hz band. Illustratively, the second equalizer corresponding to the frequency curve B in fig. 8 may be used for equalizing and compensating the target audio signal to be output by the earphone under the leakage degree corresponding to the frequency curve B in fig. 7; the second equalizer corresponding to the frequency curve E in fig. 8 may be used to perform equalization compensation on the target audio signal to be output by the earphone at the leakage level corresponding to the frequency curve E in fig. 7. By performing the equalization compensation, as shown in fig. 9, the frequency response of the earphone at various leakage levels tends to be uniform, as close to the non-leakage state as possible.
In some embodiments, the earphone may first perform the steps 406 and 408, and calculate a first equalization parameter corresponding to the ear shape of the user; the above steps 410 and 412 are performed again, and a second equalization parameter corresponding to the wearing state of the earphone is calculated. In other implementations, the situation may be reversed, that is, the headset may first perform the steps 410 and 412 to calculate the second equalization parameter corresponding to the wearing state of the headset; the above steps 406 and 408 are performed again, and a first equalization parameter corresponding to the user's ear shape is calculated. Optionally, the earphone may further calculate the first equalization parameter and the second equalization parameter at the same time, so as to improve efficiency of adaptive equalization of the audio signal by the earphone.
414. And carrying out equalization processing on the second test audio to be output according to the first equalization parameter and the second equalization parameter, and outputting the second test audio signal subjected to equalization processing through a loudspeaker.
Step 414 is similar to step 208 described above. It should be noted that, when corresponding adaptive equalization needs to be performed on an audio signal to be output by the earphone (whether the second test audio signal or a target audio signal to be output by the earphone subsequently), the earphone may implement ear-shaped adaptive equalization according to the first equalization parameter first and then implement wearing leakage adaptive equalization according to the second equalization parameter; or the wearing leakage self-adaptive equalization can be realized according to the second equalization parameters, and then the ear-shaped self-adaptive equalization can be realized according to the first equalization parameters.
416. And acquiring hearing test information fed back for the second test audio signal.
418. And determining an audio compensation parameter according to the hearing detection information, wherein the audio compensation parameter is used for performing adaptive equalization processing matched with the ear shape and/or the wearing state of the earphone of the user on the target audio signal to be output, and the audio compensation parameter is used for performing hearing compensation on the target audio signal subjected to the adaptive equalization processing by using the audio equalization parameter.
Step 416 and step 418 are similar to step 210 and step 212 described above, and are not repeated here.
Therefore, by implementing the audio signal processing method described in the above embodiment, different types of tests can be continuously performed to obtain audio signal adjustment parameters in multiple dimensions according to the difference of actual scenes such as ear shapes, hearing characteristics, wearing states of the headphones and the like of different users wearing the headphones, and accordingly, corresponding equalization and compensation are performed on the target audio signal to be output, so that comprehensive adjustment of the target audio signal is achieved. In addition, through dividing the ear-shaped self-adaptive equalization and wearing leakage self-adaptive equalization, the method is beneficial to comprehensively compensating the target audio signal to be output by the earphone so as to equalize the influence possibly suffered by the transmission of the target audio signal in the audio system where the earphone is positioned, so that the target audio signal heard by the user is restored to the actual initial sound quality as much as possible. Through realizing the audio signal processing, the earphone can provide personalized audio equalization and compensation processing for different users, and the effect of the adaptive processing of the audio signal of the earphone is improved, so that the effectiveness of the earphone in optimizing and adjusting the audio signal can be improved.
Referring to fig. 10, fig. 10 is a flowchart of another audio signal processing method according to an embodiment of the present application, and the method can be applied to the above-mentioned earphone, where the earphone may specifically include a speaker, a feedforward microphone, and a feedback microphone. As shown in fig. 10, the audio signal processing method may include the steps of:
1002. the first test audio signal is output through a speaker.
1004. A received audio signal corresponding to the first test audio signal is collected by a feedback microphone.
Step 1002 and step 1004 are similar to step 202 and step 204 described above, and are not repeated here.
1006. And determining a test ear transfer function corresponding to the received audio signal according to the received audio signal and the first test audio signal.
1008. And calculating a first equalization parameter corresponding to the user ear according to the test ear transfer function and the target ear transfer function.
Step 1006 and step 1008 are similar to step 406 and step 408, and are not described herein.
1010. And carrying out signal cancellation processing on the received audio signal based on the first test audio signal to obtain an error audio signal.
1012. And calculating a second equalization parameter corresponding to the wearing state of the earphone according to the error audio signal.
Step 1010 and step 1012 are similar to step 410 and step 412 described above, and are not repeated here.
1014. And carrying out audio loudness correction on the second test audio signal to be output on the target test frequency point to obtain a corrected second test audio signal.
In the embodiment of the application, when the earphone needs to output the second test audio signal to carry out hearing test on the user, so that corresponding audio signal compensation is realized for the hearing characteristic of the user in the subsequent step, the earphone can carry out audio loudness correction on the second test audio signal to be output according to part of specific frequency points, so that the amplitude of the second test audio signal output by the earphone on each test frequency point is the same when the second test audio signal reaches the eardrum in the ear of the user, thereby reducing errors possibly generated in the hearing test process as much as possible and improving the accuracy and reliability of hearing test.
Illustratively, the earphone may perform audio loudness correction on the second test audio signal to be output at a target test frequency point (e.g., 500Hz frequency point, 1000Hz frequency point, 2000Hz frequency point, etc.), including increasing audio loudness or decreasing audio loudness, so that a corrected second test audio signal may be obtained. On the basis, the earphone can balance the corrected second test audio signal according to the audio balance parameters, and the balanced second test audio signal is output through the loudspeaker.
1016. And carrying out equalization processing on the corrected second test audio signal according to the first equalization parameter and the second equalization parameter, and outputting the equalized second test audio signal through a loudspeaker.
Step 1016 is similar to step 414 described above and will not be described again.
1018. And acquiring hearing test information fed back for the second test audio signal.
1020. And determining an audio compensation parameter according to the hearing detection information, wherein the audio compensation parameter is used for performing adaptive equalization processing matched with the ear shape and/or the wearing state of the earphone of the user on the target audio signal to be output, and the audio compensation parameter is used for performing hearing compensation on the target audio signal subjected to the adaptive equalization processing by using the audio equalization parameter.
Step 1018 and step 1020 are similar to step 210 and step 212, and are not repeated here.
1022. Ambient sound signals are collected by a feedforward microphone.
Referring to fig. 11, fig. 11 is a schematic structural diagram of an earphone according to an embodiment of the application. As shown in fig. 11, the earphone may include a feedforward microphone 113 in addition to the speaker 111 and the feedback microphone 112 disposed in front of the speaker 111, and the feedforward microphone 113 may be disposed behind the speaker 111 (i.e., when the earphone is worn by a user, the feedforward microphone 113 is located between the speaker 111 and the external environment) to collect an external environmental sound signal through the feedforward microphone 113.
1024. And calculating according to the environmental sound signal to obtain the environmental sound parameter.
By way of example, the ambient sound parameters may include various parameters for characterizing the intensity of ambient noise, such as sound intensity, sound power, spectral energy, etc. In the embodiment of the application, after the earphone collects the environmental sound through the feedforward microphone, the environmental sound can be analyzed to calculate the corresponding environmental sound parameter. In some embodiments, when the environmental sound parameter meets the noise equalization condition, the earphone may perform noise equalization processing on the target audio signal compensated by using the audio compensation parameter, so as to reduce noise interference and improve the tone quality experience of the user.
The spectral energy dependent ambient sound parameters will be described below as an example. In some embodiments, before the earphone device specifically calculates the environmental sound parameter, the environmental sound signal may be windowed and divided according to a unit window length to obtain at least one frame of environmental sound sub-signal, fourier transform is performed on each frame of environmental sound sub-signal, and according to the transformed environmental sound sub-signals of each frame, the spectral energy corresponding to the environmental sound signal is calculated.
Specifically, the earphone may perform framing and windowing processing on the above-mentioned ambient sound signal through a signal processing module (for example, DSP module) built in the earphone. Further, a frame of ambient sound signal obtained after the frame division is windowed may be subjected to short-time fourier transform by an algorithm such as FFT, as shown in the following formula 8 (the form is not shown):
equation 8:
wherein x (n) is an ambient sound signal, and may represent an nth frame in the ambient sound signal; m may represent the time sequence in the corresponding fourier transform X (k, m), and k may represent the frequency domain subband sequence. On this basis, the process of calculating the spectrum energy by the earphone device according to the transformed ambient sound sub-signals of each frame may be as follows in equation 9:
equation 9:
PS(k,m)＝(1-α)*PS(k,m-1)+α*|X(k,m)| 2
where PS (k, m) may represent a power spectral density of the mth frame ambient sound sub-signal corresponding to the kth frequency domain sub-band, and α may represent an iteration factor, i.e., a weighting factor of a modulus of the current frame sub-band spectrum signal. It can be seen that the earphone may calculate, according to the transformed ambient sound sub-signals of each frame, a power spectral density corresponding to each frequency domain sub-band of the ambient sound signal, where each frequency domain sub-band is a frequency domain component of the ambient sound signal in each corresponding frequency range.
It can be understood that if m is equal to 1, the earphone device may actually calculate, according to the transformed mth frame ambient sound signal (i.e. the 1 st frame), a power spectrum density corresponding to the mth frame ambient sound signal; if M is greater than 1 and less than or equal to M (M is the total frame number and M is a positive integer), the earphone device may calculate a power spectral density corresponding to the mth frame ambient sound signal according to the transformed mth frame ambient sound signal and a power spectral density corresponding to the M-1 th frame ambient sound signal.
On this basis, the earphone can compare the above spectral energy (expressed by the power spectral density PS (k, m)) with the standard energy PS (k), and take the resulting ratio as an ambient sound parameter, namely:
equation 10:
1026. and under the condition that the environmental sound parameters meet the noise balance condition, performing noise balance processing on the target audio signal subjected to hearing compensation processing by utilizing the audio compensation parameters.
In some embodiments, if the environmental sound parameter Filter (k, m) > 1 indicates that the noise of the environment where the earphone is located is too large, the earphone may perform noise equalization processing on the target audio signal compensated by using the audio compensation parameter; if the environmental sound parameter Filter (k, m) is less than or equal to 1, the noise of the environment where the earphone is positioned is smaller, and the earphone can not perform the noise equalization processing.
It should be noted that, the above steps 1022 and 1024 may not be performed immediately after the step 1020, but may be performed at any time during the use of the earphone, and the obtained ambient sound parameters may be stored. In some embodiments, after the earphone performs the above step 1020, step 1026 may be performed, where the earphone may call the environmental sound parameters stored at this time and determine whether the environmental sound parameters meet the noise equalization condition, so as to determine whether to perform noise equalization processing on the target audio signal after the compensation using the audio compensation parameters.
Therefore, by implementing the audio signal processing method described in the above embodiment, different types of tests can be continuously performed to obtain audio signal adjustment parameters in multiple dimensions according to the difference of actual scenes such as ear shapes, hearing characteristics, wearing states of the headphones and the like of different users wearing the headphones, and accordingly, corresponding equalization and compensation are performed on the target audio signal to be output, so that comprehensive adjustment of the target audio signal is achieved. Through realizing above-mentioned audio signal processing, can make the earphone provide individualized audio equalization and compensation processing to different users, not only be favorable to improving the effect of earphone self-adaptation processing audio signal, still be convenient for carry out more accurate noise reduction to the target audio signal and handle to can promote the effectiveness that the earphone carries out the optimization adjustment to audio signal. In addition, further noise equalization processing is realized outside the audio equalization and compensation, noise interference is reduced, and tone quality experience of a user is further improved.
Referring to fig. 12, fig. 12 is a schematic diagram of an audio signal processing apparatus according to an embodiment of the present application, which can be applied to the above-mentioned earphone, and the earphone can include a speaker, a feedback microphone and a feedforward microphone. As shown in fig. 12, the audio signal processing apparatus may include an audio output unit 1201, an audio receiving unit 1202, a first determining unit 1203, an information acquiring unit 1204, and a second determining unit 1205, wherein:
an audio output unit 1201 for outputting a first test audio signal through a speaker;
an audio receiving unit 1202 for collecting a received audio signal corresponding to the first test audio signal through a feedback microphone;
a first determining unit 1203 configured to determine an audio equalization parameter according to the received audio signal and the first test audio signal;
the audio output unit 1201 is further configured to perform equalization processing on the second test audio to be output according to the audio equalization parameter, and output the second test audio signal after the equalization processing through the speaker;
an information obtaining unit 1204, configured to obtain hearing test information fed back for the second test audio signal;
A second determining unit 1205 is configured to determine an audio compensation parameter according to the hearing test information, where the audio compensation parameter is used for performing adaptive equalization processing matching with the ear shape and/or the wearing state of the earphone of the user on the target audio signal to be output, and the audio compensation parameter is used for performing hearing compensation on the target audio signal after performing adaptive equalization processing by using the audio compensation parameter.
Therefore, by adopting the audio signal processing device described in the above embodiment, the audio signal adjustment parameters in multiple dimensions can be obtained by continuously performing different types of tests according to the difference of the actual scenes such as the ear shape, the hearing characteristics, the wearing state of the earphone and the like of different users wearing the earphone, and accordingly, the target audio signal to be output is correspondingly balanced and compensated, so that the comprehensive adjustment of the target audio signal is realized. Through realizing above-mentioned audio signal processing, can make the earphone provide individualized audio equalization and compensation processing to different users, not only be favorable to improving the effect of earphone self-adaptation processing audio signal, still be convenient for carry out more accurate noise reduction to the target audio signal and handle to can promote the effectiveness that the earphone carries out the optimization adjustment to audio signal.
In an embodiment, the first test audio signal may include a base audio signal, and the first determining unit 1203 may specifically be configured to:
determining a first equalization parameter according to the received audio signal and the basic audio signal, wherein the first equalization parameter is used for performing ear-shaped adaptive equalization matched with the ear shape of a user on the audio signal to be output; and/or the number of the groups of groups,
the first test audio signal may include a low frequency signal, where the frequency of the low frequency signal is lower than the wearing leakage threshold, and the first determining unit 1203 may specifically be configured to:
and determining a second equalization parameter according to the received audio signal and the low-frequency signal, wherein the second equalization parameter is used for carrying out wearing leakage adaptive equalization matched with the wearing state of the earphone on the audio signal to be output.
In an embodiment, the first test audio signal may be obtained by mixing a base audio signal filtered by a high-pass filter with a low-frequency signal, where the frequency of the low-frequency signal is lower than a wearing leakage threshold value, and the cut-off frequency of the high-pass filter is higher than a frequency corresponding to the low-frequency signal.
In an embodiment, the above audio equalization parameters may include a first equalization parameter corresponding to an ear shape of the user, and the first determining unit 1203 may specifically be configured to:
Determining a test ear transfer function corresponding to the received audio signal according to the received audio signal and the first test audio signal;
and calculating the first equalization parameter according to the test ear transfer function and the target ear transfer function.
The first determining unit 1203 may specifically include:
and calculating and obtaining a first equalization parameter corresponding to each frequency domain sub-band according to the test ear transfer function and the target ear transfer function corresponding to each frequency domain sub-band, wherein each frequency domain sub-band is divided into sub-frequency ranges in the frequency range of the first test audio signal.
In an embodiment, the above audio equalization parameter may include a second equalization parameter corresponding to a wearing state of the headset, and the first determining unit 1203 may specifically be configured to:
performing signal cancellation processing on the received audio signal based on the first test audio signal to obtain an error audio signal;
the second equalization parameter is calculated from the error audio signal.
The first determining unit 1203 may specifically include:
filtering the first test audio signal through a transfer function filter to obtain a transfer audio signal corresponding to the first test audio signal, wherein the transfer function filter is used for representing the transmission influence of an audio transmission system where the earphone is positioned on the first test audio signal;
And calculating the error between the received audio signal and the transmitted audio signal to obtain an error audio signal.
On this basis, the first determining unit 1203 may specifically include:
calculating a mean square error between the received audio signal and the transmitted audio signal based on the error audio signal;
and updating the transfer function filter according to the mean square error, and re-executing the step of filtering the first test audio signal through the transfer function filter to obtain a transfer audio signal corresponding to the first test audio signal until the update stop condition is met, and determining the second equalization parameter based on the transfer function filter obtained when the update stop condition is met.
In one embodiment, the audio output unit 1201 may be further specifically configured to, when outputting the second test audio signal:
performing audio loudness correction on a second test audio signal to be output on the target test frequency point to obtain a corrected second test audio signal;
and according to the audio equalization parameters, equalizing the corrected second test audio signal, and outputting the equalized second test audio signal through a loudspeaker.
In an embodiment, the audio equalization parameters may include equalization filter parameters, the audio compensation parameters may include compensation filter parameters, and the audio signal processing apparatus may further include a configuration unit, not shown, which may be configured to:
configuring a target equalization filter through the equalization filter parameters, wherein the target equalization filter is used for carrying out self-adaptive equalization on a target audio signal to be output;
and configuring a target compensation filter through the compensation filter parameters, wherein the target compensation filter is used for compensating the target audio signal subjected to adaptive equalization by using the target equalization filter.
In one embodiment, the audio signal processing apparatus may further include an environmental sound collection unit, a third determination unit, and a noise equalization processing unit, which are not illustrated, wherein:
the environmental sound collection unit is used for collecting environmental sound signals through the feedforward microphone;
the third determining unit is used for calculating and obtaining the environmental sound parameters according to the environmental sound signals;
a noise equalization processing unit configured to perform noise equalization processing on a target audio signal subjected to hearing compensation processing using the audio compensation parameter in a case where the environmental sound parameter satisfies a noise equalization condition after the second determination unit 1205 determines the audio compensation parameter based on the hearing detection information.
As an alternative embodiment, the third determining unit may specifically be configured to:
windowing and dividing the environmental sound signal according to the unit window length to obtain at least one frame of environmental sound signal;
performing Fourier transform on each frame of environmental sound signal respectively, and calculating to obtain the frequency spectrum energy corresponding to the environmental sound signal according to each frame of environmental sound signal after transformation;
the spectral energy is compared to the standard energy and the resulting ratio is taken as the ambient sound parameter.
Therefore, by adopting the audio signal processing device described in the above embodiment, the audio signal adjustment parameters in multiple dimensions can be obtained by continuously performing different types of tests according to the difference of the actual scenes such as the ear shape, the hearing characteristics, the wearing state of the earphone and the like of different users wearing the earphone, and accordingly, the target audio signal to be output is correspondingly balanced and compensated, so that the comprehensive adjustment of the target audio signal is realized. In addition, through dividing the ear-shaped self-adaptive equalization and wearing leakage self-adaptive equalization, the method is beneficial to comprehensively compensating the target audio signal to be output by the earphone so as to equalize the influence possibly suffered by the transmission of the target audio signal in the audio system where the earphone is positioned, so that the target audio signal heard by the user is restored to the actual initial sound quality as much as possible. Through realizing the audio signal processing, the earphone can provide personalized audio equalization and compensation processing for different users, and the effect of the adaptive processing of the audio signal of the earphone is improved, so that the effectiveness of the earphone in optimizing and adjusting the audio signal can be improved. In addition, further noise equalization processing is realized outside the audio equalization and compensation, noise interference is reduced, and tone quality experience of a user is further improved.
Referring to fig. 13, fig. 13 is a schematic diagram of a headset according to an embodiment of the application. As shown in fig. 13, the earphone may include:
a memory 1301 storing executable program code;
a processor 1302 coupled to the memory 1301;
wherein the processor 1302 invokes executable program code stored in the memory 1301, which may perform all or part of the steps of any of the audio signal processing methods described in the above embodiments.
Further, the embodiment of the present application further discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program makes a computer execute all or part of the steps of any one of the audio signal processing methods described in the above embodiments.
Furthermore, embodiments of the present application further disclose a computer program product which, when run on a computer, enables the computer to perform all or part of the steps of any of the audio signal processing methods described in the above embodiments.
Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the above embodiments may be implemented by a program that instructs associated hardware, the program may be stored in a computer readable storage medium including Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disk Memory, magnetic disk Memory, tape Memory, or any other medium that can be used for carrying or storing data that is readable by a computer.
The above describes in detail an audio signal processing method and apparatus, an earphone, and a storage medium disclosed in the embodiments of the present application, and specific examples are applied to illustrate the principles and implementations of the present application, where the descriptions of the above embodiments are only used to help understand the method and core ideas of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims (15)

1. An audio signal processing method, applied to a headset, the headset including a speaker and a feedback microphone, the method comprising:
outputting a first test audio signal through the speaker;
collecting a received audio signal corresponding to the first test audio signal through the feedback microphone;
determining an audio equalization parameter according to the received audio signal and the first test audio signal;
according to the audio equalization parameters, performing equalization processing on second test audio to be output, and outputting an equalized second test audio signal through the loudspeaker;
Acquiring hearing test information fed back for the second test audio signal;
and determining an audio compensation parameter according to the hearing detection information, wherein the audio compensation parameter is used for performing adaptive equalization processing matched with the ear shape and/or the wearing state of the earphone of a user on a target audio signal to be output, and the audio compensation parameter is used for performing hearing compensation on the target audio signal subjected to the adaptive equalization processing by using the audio compensation parameter.
2. The method of claim 1, wherein the first test audio signal comprises a base audio signal, wherein the audio equalization parameters comprise first equalization parameters, wherein the determining the audio equalization parameters from the received audio signal and the first test audio signal comprises:
determining a first equalization parameter according to the received audio signal and the basic audio signal, wherein the first equalization parameter is used for performing ear-shaped adaptive equalization matched with the ear shape of a user on the audio signal to be output; and/or the number of the groups of groups,
the first test audio signal comprises a low frequency signal, the frequency of the low frequency signal is lower than a wearing leakage threshold, the audio equalization parameter comprises a second equalization parameter, and the audio equalization parameter is determined according to the received audio signal and the first test audio signal, and the method comprises the following steps:
And determining a second equalization parameter according to the received audio signal and the low-frequency signal, wherein the second equalization parameter is used for carrying out wearing leakage adaptive equalization matched with the wearing state of the earphone on the audio signal to be output.
3. The method of claim 1, wherein the first test audio signal is obtained by mixing a base audio signal filtered by a high pass filter with a low frequency signal, the low frequency signal having a frequency below a wear leakage threshold, and the high pass filter having a cut-off frequency that is higher than a frequency corresponding to the low frequency signal.
4. The method of claim 1, wherein the audio equalization parameters include first equalization parameters corresponding to a user's ear shape, wherein the determining the audio equalization parameters from the received audio signal and the first test audio signal comprises:
determining a test ear transfer function corresponding to the received audio signal according to the received audio signal and the first test audio signal;
and calculating the first equalization parameter according to the test ear transfer function and the target ear transfer function.
5. The method of claim 4, wherein calculating the first equalization parameter based on the test and target ear transfer functions comprises:
And calculating the first equalization parameters corresponding to all frequency domain sub-bands according to the test ear transfer function and the target ear transfer function corresponding to each frequency domain sub-band, wherein each frequency domain sub-band is divided into all sub-frequency ranges in the frequency range of the first test audio signal.
6. The method of claim 1, wherein the audio equalization parameters include a second equalization parameter corresponding to a state of wear of the headset, wherein the determining the audio equalization parameters from the received audio signal and the first test audio signal comprises:
performing signal cancellation processing on the received audio signal based on the first test audio signal to obtain an error audio signal;
and calculating the second equalization parameter according to the error audio signal.
7. The method of claim 6, wherein performing signal cancellation processing on the received audio signal based on the first test audio signal to obtain an error audio signal, comprises:
filtering the first test audio signal through an output equalizer to obtain an output audio signal corresponding to the first test audio signal, wherein the output equalizer is obtained through initial second equalization parameter configuration;
Filtering the output audio signal through a transfer function filter to obtain a transfer audio signal corresponding to the output audio signal, wherein the transfer function filter is used for representing the transmission influence of an audio transmission system where the earphone is positioned on the output audio signal;
and calculating the error between the received audio signal and the transmitted audio signal to obtain an error audio signal.
8. The method of claim 7, wherein said calculating said second equalization parameter from said error audio signal comprises:
calculating a mean square error between the received audio signal and the delivered audio signal from the error audio signal;
updating the output equalizer according to the mean square error, and re-executing the filtering of the first test audio signal by the output equalizer to obtain an output audio signal corresponding to the first test audio signal, and filtering the output audio signal by a transfer function filter to obtain a transfer audio signal corresponding to the output audio signal until an update stop condition is met, and updating the second equalization parameter based on the output equalizer obtained when the update stop condition is met.
9. The method according to claims 1 to 8, wherein equalizing the second test audio to be output according to the audio equalization parameters and outputting the equalized second test audio signal through the speaker, comprises:
performing audio loudness correction on a second test audio signal to be output on the target test frequency point to obtain a corrected second test audio signal;
and carrying out equalization processing on the corrected second test audio signal according to the audio equalization parameters, and outputting the equalized second test audio signal through the loudspeaker.
10. The method of claims 1 to 8, wherein the audio equalization parameters include equalization filter parameters, the audio compensation parameters include compensation filter parameters, and wherein after the determining of audio compensation parameters from the hearing test information, the method further comprises:
configuring a target equalization filter through the equalization filter parameters, wherein the target equalization filter is used for carrying out self-adaptive equalization on a target audio signal to be output;
and configuring a target compensation filter through the compensation filter parameters, wherein the target compensation filter is used for compensating the target audio signal subjected to self-adaptive equalization by using the target equalization filter.
11. The method of claims 1-8, wherein the headset further comprises a feed-forward microphone, the method further comprising:
collecting an ambient sound signal by the feedforward microphone;
according to the environmental sound signal, calculating to obtain an environmental sound parameter;
after the determining of the audio compensation parameters from the hearing test information, the method further comprises:
and under the condition that the environmental sound parameters meet the noise balance condition, performing noise balance processing on the target audio signal subjected to hearing compensation processing by utilizing the audio compensation parameters.
12. The method of claim 11, wherein said calculating environmental sound parameters from said environmental sound signals comprises:
windowing and dividing the environmental sound signal according to the unit window length to obtain at least one frame of environmental sound signal;
performing Fourier transform on each frame of environmental sound signal respectively, and calculating to obtain the frequency spectrum energy corresponding to the environmental sound signal according to each frame of environmental sound signal after transformation;
the spectral energy is compared with standard energy and the resulting ratio is taken as the ambient sound parameter.
13. An audio signal processing device, characterized by being applied to an earphone, the earphone including a speaker and a feedback microphone, the audio signal processing device comprising:
An audio output unit for outputting a first test audio signal through the speaker;
an audio receiving unit, configured to collect, by using the feedback microphone, a received audio signal corresponding to the first test audio signal;
a first determining unit, configured to determine an audio equalization parameter according to the received audio signal and the first test audio signal;
the audio output unit is further configured to perform equalization processing on a second test audio to be output according to the audio equalization parameter, and output an equalized second test audio signal through the speaker;
the information acquisition unit is used for acquiring hearing detection information fed back for the second test audio signal;
and the second determining unit is used for determining audio compensation parameters according to the hearing detection information, wherein the audio compensation parameters are used for performing adaptive equalization processing matched with the ear shape and/or the wearing state of the earphone of the user on the target audio signal to be output, and the audio compensation parameters are used for performing hearing compensation on the target audio signal subjected to the adaptive equalization processing by using the audio compensation parameters.
14. A headset comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to implement the method of any of claims 1 to 12.
15. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any one of claims 1 to 12.
CN202210216457.0A 2022-03-07 2022-03-07 Audio signal processing method and device, earphone and storage medium Pending CN116782084A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210216457.0A CN116782084A (en) 2022-03-07 2022-03-07 Audio signal processing method and device, earphone and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210216457.0A CN116782084A (en) 2022-03-07 2022-03-07 Audio signal processing method and device, earphone and storage medium

Publications (1)

Publication Number Publication Date
CN116782084A true CN116782084A (en) 2023-09-19

Family

ID=87990055

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210216457.0A Pending CN116782084A (en) 2022-03-07 2022-03-07 Audio signal processing method and device, earphone and storage medium

Country Status (1)

Country Link
CN (1) CN116782084A (en)

Similar Documents

Publication Publication Date Title
CN113676803B (en) Active noise reduction method and device
US10382092B2 (en) Method and system for full duplex enhanced audio
US9432766B2 (en) Audio processing device comprising artifact reduction
US8315400B2 (en) Method and device for acoustic management control of multiple microphones
US20100142714A1 (en) Method and system for acoustic shock protection
US20130156208A1 (en) Hearing aid and method of detecting vibration
WO2009136953A1 (en) Method and device for acoustic management control of multiple microphones
Westermann et al. Binaural dereverberation based on interaural coherence histograms
JP6312826B2 (en) Hearing aid system operating method and hearing aid system
CN109327789A (en) Headphone response measurement and equilibrium
EP2752848B1 (en) Method and apparatus for generating a noise reduced audio signal using a microphone array
CN116601701A (en) Dual mode ANC environmental detector
US10204637B2 (en) Noise reduction methodology for wearable devices employing multitude of sensors
WO2022247494A1 (en) Audio signal compensation method and apparatus, earphones, and storage medium
KR20240007168A (en) Optimizing speech in noisy environments
Premananda et al. Speech enhancement algorithm to reduce the effect of background noise in mobile phones
US20240040325A1 (en) Audio signal compensation method and apparatus, earphone and storage medium
US9503815B2 (en) Perceptual echo gate approach and design for improved echo control to support higher audio and conversational quality
JP2008522511A (en) Method and apparatus for adaptive speech processing parameters
Sørensen et al. Semi-non-intrusive objective intelligibility measure using spatial filtering in hearing aids
EP3830823A1 (en) Forced gap insertion for pervasive listening
CN116782084A (en) Audio signal processing method and device, earphone and storage medium
Puder Adaptive signal processing for interference cancellation in hearing aids
CN115412824A (en) Detection method, detection device, earphone and computer readable storage medium
WO2023020208A1 (en) Audio signal processing method and apparatus, earphones, and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination