WO2022016533A1 - 音频处理方法和电子设备 - Google Patents
音频处理方法和电子设备 Download PDFInfo
- Publication number
- WO2022016533A1 WO2022016533A1 PCT/CN2020/104517 CN2020104517W WO2022016533A1 WO 2022016533 A1 WO2022016533 A1 WO 2022016533A1 CN 2020104517 W CN2020104517 W CN 2020104517W WO 2022016533 A1 WO2022016533 A1 WO 2022016533A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- audio signal
- main
- microphone
- signal
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/22—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired frequency characteristic only
- H04R1/222—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired frequency characteristic only for microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers
- H04R3/005—Circuits for transducers for combining the signals of two or more microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers
- H04R3/02—Circuits for transducers for preventing acoustic reaction, i.e. acoustic oscillatory feedback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers
- H04R3/04—Circuits for transducers for correcting frequency response
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
- H04R2410/00—Microphones
- H04R2410/07—Mechanical or electrical reduction of wind noise generated by wind passing a microphone
Definitions
- the embodiments of the present application relate to the field of audio and video technologies, and in particular, to an audio processing method and an electronic device.
- Audio is critical data for media processing and should be of high quality.
- a microphone is mounted on the electronic device.
- the microphone is generally placed close to the surface of the electronic device, and the surface of the electronic device is provided with a hole for the sound pickup channel, so that the microphone can better receive the vibration in the air, and then convert the sound and electricity to form an audio signal.
- Embodiments of the present application provide an audio processing method and electronic device, which improve the quality of audio signals collected by the electronic device under abnormal conditions.
- an embodiment of the present application provides an audio processing method, which is applied to an electronic device.
- the electronic device includes a main microphone and an auxiliary microphone, and the sound pickup cavity of the main microphone and the sound pickup cavity of the auxiliary microphone are both the same as the sound pickup cavity.
- the external environment where the electronic device is located is communicated;
- the electronic device includes a sound pickup protection structure, and the sound pickup protection structure is configured to reduce the airflow entering the sound pickup cavity of the auxiliary microphone from the external environment, and/or, blocking non-gaseous substances from entering the sound pickup cavity of the auxiliary microphone;
- the method includes:
- a target audio signal is synthesized from the main audio signal and the auxiliary audio signal.
- an embodiment of the present application provides an electronic device, including a main microphone and an auxiliary microphone, and both the sound pickup cavity of the main microphone and the sound pickup cavity of the auxiliary microphone are in communication with the external environment where the electronic device is located ;
- the electronic device further comprises a sound pickup protection structure, the sound pickup protection structure is configured to reduce the airflow entering the pickup cavity of the auxiliary microphone from the external environment, and/or, to block the entry of non-gaseous substances the pickup cavity of the auxiliary microphone;
- the electronic device also includes a memory and a processor
- the memory for storing program codes
- the processor calls the program code, and when the program code is executed, is configured to perform the following operations:
- a target audio signal is synthesized from the main audio signal and the auxiliary audio signal.
- an audio processing method including:
- the main audio signal and the auxiliary audio signal are acquired from the same audio source at the same time, and the amplitude and/or phase of the auxiliary audio signal and the main audio signal at a specific frequency different;
- the target tuning parameter adjusts the amplitude parameter and/or the phase parameter of the audio component
- a target audio signal is synthesized according to the main audio signal and the adjusted auxiliary audio signal.
- an embodiment of the present application provides an electronic device, including: a memory and a processor;
- the memory for storing program codes
- the processor calls the program code, and when the program code is executed, is configured to perform the following operations:
- the main audio signal and the auxiliary audio signal are acquired from the same audio source at the same time, and the amplitude and/or phase of the auxiliary audio signal and the main audio signal at a specific frequency different;
- the target tuning parameter adjusts the amplitude parameter and/or the phase parameter of the audio component
- a target audio signal is synthesized according to the main audio signal and the adjusted auxiliary audio signal.
- an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the readable storage medium, and when the computer program is executed, the method provided in the first aspect or the third aspect is implemented.
- Embodiments of the present application provide an audio processing method and an electronic device.
- the electronic device includes a main microphone and an auxiliary microphone, and the electronic device further includes a sound pickup protection structure.
- the auxiliary microphone with the sound pickup protection structure has better sound pickup performance.
- the electronic equipment records based on the main microphone, and corrects the main audio signal collected by the main microphone through the auxiliary audio signal collected by the auxiliary microphone with the sound pickup protection structure to generate the target audio signal, which improves the audio signal collected by the electronic equipment under abnormal conditions. the quality of.
- FIG. 1 is a schematic structural diagram of an electronic device to which an embodiment of the application is applicable;
- FIG. 2 is another schematic structural diagram of an electronic device to which an embodiment of the present application is applicable
- FIG. 3 is a flowchart of an audio processing method provided by an embodiment of the present application.
- FIG. 4 is a schematic diagram of an audio processing method in a wind noise scene provided by an embodiment of the present application.
- FIG. 5 is a schematic diagram of a mapping function relationship provided by an embodiment of the present application.
- FIG. 6 is a schematic diagram of an audio processing method in an overload scenario provided by an embodiment of the present application.
- FIG. 7 is a schematic diagram of an audio processing method in a microphone blocking scenario provided by an embodiment of the present application.
- FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
- the audio processing methods provided in the embodiments of the present application can be applied to electronic devices.
- the electronic device includes a main microphone and an auxiliary microphone, and the sound pickup cavity of the main microphone and the sound pickup cavity of the auxiliary microphone are both communicated with the external environment where the electronic device is located. Both the primary and secondary microphones can capture sound from the external environment in which the electronic device is located to generate audio signals.
- the audio signal collected by the main microphone may be referred to as the main audio signal
- the audio signal collected by the auxiliary microphone may be referred to as the auxiliary audio signal.
- the electronic device further includes a pickup protection structure configured to reduce airflow from the external environment into the pickup cavity of the auxiliary microphone, and/or to block non-gaseous substances from entering the pickup cavity of the auxiliary microphone.
- the main microphone may be understood as a microphone without a sound pickup protection structure in the electronic device
- the auxiliary microphone may be understood as a microphone with a sound pickup protection structure in the electronic device.
- the sound pickup protection structure will have different effects on the sound pickup performance of the auxiliary microphone.
- the sound pickup environment in which the electronic equipment is located is relatively normal, for example, the external environment where the electronic equipment is located is less windy and the air is relatively clean.
- the main microphone and the sound pickup protection structure will reduce the frequency response and sensitivity of the auxiliary microphone during the sound collection process, resulting in the natural distortion of the auxiliary audio signal collected by the auxiliary microphone.
- the electronic equipment is located in an abnormal sound pickup environment, for example, the external environment where the electronic equipment is located is relatively windy or has a lot of dust in the air, such as high-speed sports scenes, outdoor strong wind scenes, dusty weather, etc.
- the structure can weaken the airflow entering the sound pickup cavity of the auxiliary microphone from the external environment, and/or block non-gaseous substances from entering the sound pickup cavity of the auxiliary microphone.
- the auxiliary microphone with the sound pickup protection structure is compared with no sound pickup
- the main microphone of the protective structure is more resistant to abnormal conditions, and the pickup performance is more stable. Therefore, according to the environment where the electronic equipment is located, the main microphone is used for normal recording.
- the main audio signal collected by the main microphone is abnormal, the main audio signal can be repaired and adjusted through the auxiliary audio signal collected by the auxiliary microphone, so as to achieve better Robust, higher-quality pickup.
- FIG. 1 is a schematic structural diagram of an electronic device to which an embodiment of the present application is applied
- FIG. 2 is another structural schematic diagram of an electronic device to which an embodiment of the present application is applied.
- the electronic device 100 includes two main microphones and one auxiliary microphone 13 .
- the two main microphones may be referred to as the main microphone 11 and the main microphone 12 respectively.
- the main microphone 11 and the main microphone 12 may be located on opposite sides of the electronic device to collect sounds in different directions.
- the main microphone 11 and the main microphone 12 may also be referred to as a left channel main microphone and a right channel main microphone.
- the auxiliary microphone 13 has a sound pickup protection structure 14 .
- the electronic device 200 includes a main microphone 21 and an auxiliary microphone 13 , and the auxiliary microphone 13 has a sound pickup protection structure 14 .
- the primary microphone may be close to the housing of the electronic device relative to the secondary microphone.
- the main audio signal collected by the main microphone is more realistic.
- the auxiliary audio signal collected by the auxiliary microphone has a stronger ability to resist abnormal sound pickup, so as to assist in repairing the main audio in abnormal scenarios. signal, and improve the pickup effect of electronic equipment.
- the embodiment of the present application does not limit the sound pickup application scenarios of the electronic device, for example, it may include but not limited to at least one of the following scenarios: a wind noise scenario, an overload scenario, or a microphone blocking scenario.
- Wind noise refers to the noise generated by high-speed moving airflow. for example.
- high-speed airflow created eddy noise around the electronics while the electronics were recording.
- the high-speed wind impacts the electronic device itself and generates wind noise.
- wind produces eddy current noise at the microphone's sound inlet hole (eg, the hole on the surface of the electronic device that communicates with the pickup cavity of the main microphone).
- Overload also known as clipping distortion
- clipping distortion refers to the abnormal scene in which the maximum value of the audio signal exceeds the maximum value that the audio track can record, resulting in automatic clipping of part of the high sound pressure waveform.
- the microphone blocking scenario also known as the silent scenario, refers to an abnormal scenario in which the maximum value of the audio signal collected by the microphone is small due to the blocking of the radio channel.
- the embodiments of the present application do not limit the structures of the sound pickup cavity of the main microphone and the sound pickup cavity of the auxiliary microphone.
- the main microphone may include a first diaphragm and a first casing, and the first diaphragm and the first casing form a sound pickup cavity of the main microphone.
- the auxiliary microphone may include a second diaphragm and a second casing, and the second diaphragm and the second casing form a sound pickup cavity of the auxiliary microphone.
- the embodiments of the present application do not limit the implementation of the sound pickup protection structure, and in different application scenarios, the sound pickup protection structure may have different structures.
- the sound pickup protection structure may include a windproof structure, and the auxiliary microphone may be arranged in the windproof structure, which improves the ability of the auxiliary microphone to resist wind noise, and improves the sound pickup performance of the auxiliary microphone in a wind noise scenario.
- the windproof structure may include a hollow windproof cover body and a support for supporting the windproof cover body, and the auxiliary microphone may be arranged in the cavity of the windproof cover body.
- the sound pickup protection structure may include a dust-proof structure, and the auxiliary microphone may be arranged in the dust-proof structure, which reduces the probability of the auxiliary microphone being blocked by pollutants such as dust and water droplets, and improves the pickup of the auxiliary microphone in the microphone blocking scenario. sound performance.
- the dustproof structure may include at least one layer of filter screen covering the outer side of the auxiliary microphone.
- the filtering functions of different filters can be the same or different.
- the dustproof structure may include two layers of filter nets covering the outer side of the auxiliary microphone, and the two layers of filter nets are respectively used to filter solid pollutants such as dust and gaseous pollutants such as water vapor.
- the sound pickup protection structure may include a sound insulation structure
- the auxiliary microphone may be arranged in the sound insulation structure, which reduces the sensitivity of the auxiliary microphone and improves the sound pickup performance of the auxiliary microphone in an overload scenario.
- the sound pickup protection structure can realize one or more of the above windproof function, dustproof function and anti-overload function.
- Each main microphone corresponds to wind noise level information, which is used to indicate the degree to which the main microphone is affected by wind noise.
- the wind noise level information may be a value within a preset value range.
- the magnitude of the numerical value of the wind noise level information may indicate the degree to which the main microphone is affected by wind noise. This embodiment of the present application does not limit the value range of the wind noise level information.
- Each main microphone has corresponding information on the degree of microphone blocking, which is used to indicate whether the main microphone is abnormally blocked or used to indicate the degree of microphone blocking of the main microphone.
- the wheat blocking degree information may be a value within a preset value range.
- the magnitude of the value of the microphone blocking degree information may indicate the degree of microphone blocking of the main microphone.
- the value range of the wheat blocking degree information is not limited in this embodiment of the present application.
- Each main microphone corresponds to overload degree information, which is used to indicate whether the main microphone is overloaded abnormally or to indicate the degree of overload of the main microphone.
- the overload degree information may be a value within a preset value range.
- the magnitude of the numerical value of the overload degree information may indicate the degree of overload of the main microphone. This embodiment of the present application does not limit the value range of the overload degree information.
- FIG. 3 is a flowchart of an audio processing method provided by an embodiment of the present application.
- the execution body may be an electronic device, and the structure of the electronic device may refer to the above description, and details are not repeated here.
- the audio processing method provided in this embodiment may include:
- the auxiliary microphone has a sound pickup protection structure
- the main audio signal collected by the main microphone and the auxiliary audio signal collected by the auxiliary microphone are different.
- the auxiliary microphone with the pickup protection structure has better pickup performance.
- the audio processing method provided in this embodiment can be applied to an electronic device.
- the electronic device includes a main microphone and an auxiliary microphone, and the electronic device further includes a pickup protection structure, and the pickup protection structure is configured to reduce the airflow entering the pickup cavity of the auxiliary microphone from the external environment, and/or block the entry of non-gaseous substances The pickup cavity of the auxiliary microphone.
- the auxiliary microphone with the sound pickup protection structure has better sound pickup performance. Therefore, on the basis of using the main microphone for normal recording, the main audio signal is adjusted through the auxiliary audio signal collected by the auxiliary microphone with the pickup protection structure to generate the target audio signal, which improves the quality of the audio signal collected by the electronic device. , to improve the pickup effect of electronic equipment.
- the audio processing method provided in this embodiment may further include:
- a plurality of audio components in different frequency bands are determined in the auxiliary audio signal.
- the target tuning parameter of the frequency response parameter of the audio component is determined, and the audio component is adjusted according to the target tuning parameter frequency response parameters.
- the frequency response parameters include amplitude parameters and/or phase parameters, and the corresponding relationship is determined based on the deviation of the frequency response parameters of the sample audio obtained by the main microphone and the auxiliary microphone in the electronic device respectively collecting the sample audio source.
- synthesizing the target audio signal according to the main audio signal and the auxiliary audio signal may include:
- the target audio signal is synthesized according to the main audio signal and the adjusted auxiliary audio signal.
- this embodiment does not limit the division manner of different frequency bands.
- different frequency bands may include multiple frequency bands with continuous frequency ranges.
- the entire frequency band range may be expressed as f min ⁇ f max , and according to the preset number of frequency bands or the preset frequency band interval or other frequency band division rules, the entire frequency band f min ⁇ f max may be divided into the following five frequency bands: f min to f 1 , f 1 to f 2 , f 2 to f 3 , f 3 to f 4 , and f 4 to f max , where f min ⁇ f 1 ⁇ f 2 ⁇ f 3 ⁇ f 4 ⁇ f max . It should be noted that this embodiment does not limit the number of frequency bands and the frequency range of each frequency band.
- different frequency bands may include multiple frequency bands with non-overlapping frequency ranges, each frequency band includes a center frequency point f and a frequency offset F, and the frequency range of the frequency band is (fF) ⁇ ( f+F).
- the frequency ranges of two adjacent frequency bands can be continuous or discontinuous.
- the entire frequency range can be expressed as f min ⁇ f ma x
- the values of each center frequency point and frequency offset are not limited in this embodiment.
- the frequency response also known as the frequency response, refers to the fact that when an audio signal output at a constant voltage is connected to the system, the sound pressure generated by the speaker increases or attenuates with the change of frequency, and the phase changes with the frequency.
- the phenomenon, the relative change of sound pressure and/or phase and frequency, is called frequency response. Since the frequency response is related to the frequency, in this step, for the auxiliary audio signal, according to the corresponding relationship between the preset frequency band and the tuning parameter of the frequency response parameter, the target tuning of the frequency response parameter corresponding to each audio component is determined by sub-band. Parameter. For each audio component, the frequency response of the auxiliary audio signal is modified according to the target adjustment parameter sub-band of the frequency response parameter. The corresponding relationship between the preset frequency band and the adjustment parameter of the frequency response parameter is determined based on the deviation of the frequency response parameter of the sample audio obtained by the main microphone and the auxiliary microphone in the electronic device respectively collecting the sample audio source.
- neural networks include, but are not limited to, convolutional neural network (CNN), recurrent neural network (RNN), and long short term memory (LSTM).
- CNN convolutional neural network
- RNN recurrent neural network
- LSTM long short term memory
- the auxiliary audio signal collected by the auxiliary microphone with the sound pickup protection structure and after the frequency response is repaired can be used for the main microphone.
- the audio signal is adjusted to generate a target audio signal, which improves the quality of the audio signal collected by the electronic device and improves the sound pickup effect of the electronic device.
- the audio processing method provided in this embodiment may further include:
- the feature information may include one or more of the following information: wind noise level information, wheat jam level information, or overload level information.
- synthesizing the target audio signal according to the main audio signal and the adjusted auxiliary audio signal may include:
- the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal.
- the characteristic information of the main microphone is used to indicate the degree to which the main microphone is affected by the abnormal situation. Therefore, according to the characteristic information of the main microphone, the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal, which improves the quality of the target audio signal.
- the electronic device may perform audio processing for a single scene, or may perform audio processing for multiple application scenarios, and the application does not limit the combination.
- the audio processing method provided by the present application is described in conjunction with a wind noise scene.
- the feature information of the main microphone may be wind noise level information of the main microphone.
- acquiring feature information of the main microphone may include:
- the characteristic information is determined according to the main audio signal and the auxiliary audio signal.
- the wind noise has a greater impact on the main audio signal collected by the main microphone, and has a greater impact on the auxiliary microphone.
- the effect of the secondary audio signal is less.
- the wind noise level information of the main microphone is determined according to the main audio signal and the auxiliary audio signal, which improves the accuracy of the wind noise level information of the main microphone.
- the wind noise level information is determined according to the signal correlation between the main audio signal and the auxiliary audio signal.
- signal correlation reflects the degree of association or similarity between two signals. The greater the correlation of the signals, the more similar the two signals are; on the contrary, the smaller the correlation of the signals, the greater the difference between the two signals.
- the signal correlation between the main audio signal and the auxiliary audio signal is strong, it means that the main microphone is less affected by the wind noise.
- the signal correlation between the main audio signal and the auxiliary audio signal The weaker it is, the more the main mic is affected by wind noise.
- the electronic device includes two main microphones, referred to as the left-channel main microphone and the right-channel main microphone.
- the main audio signal collected by the main microphone of the left channel is denoted as x L
- the corresponding frequency domain signal is denoted as XL
- the main audio signal collected by the main microphone of the right channel is denoted as x R
- the corresponding frequency domain signal is denoted as X R .
- the electronic device includes an auxiliary microphone, the auxiliary audio signal collected by the auxiliary microphone is denoted as x Ref , and the corresponding frequency domain signal is denoted as X Ref .
- the wind noise level information of the main microphone of the left channel can be determined by the signal correlation between the main audio signal x L and the auxiliary audio signal x Ref
- the wind noise level information of the main microphone of the right channel can be determined by the main audio signal x R and the auxiliary audio signal x Ref.
- the signal correlation between the audio signals x Ref is determined.
- the correlation calculation may adopt a classical cross-spectral calculation method, see formula 1.
- the value range of the correlation is between 0 and 1. The closer the value is to 1, the better the correlation, indicating that the main microphone is less affected by wind noise.
- the frequency domain signal corresponding to the main audio signal may also be referred to as the main frequency domain signal
- the frequency domain signal corresponding to the auxiliary audio signal may also be referred to as the auxiliary frequency domain signal.
- the correlation calculation may also adopt other calculation methods, for example, obtaining the signal correlation between the main audio signal and the auxiliary audio signal in the time domain according to the two.
- acquiring feature information of the main microphone may include:
- This implementation is applicable to the case where the electronic device includes multiple main microphones.
- the main microphone does not have a sound pickup protection structure, the wind noise has a greater impact on the main audio signal collected by the main microphone.
- the wind noise level information of each main microphone is determined according to the main audio signals collected by different main microphones, which improves the accuracy of determining the wind noise level information of the main microphones.
- the electronic device includes at least two main microphones, and acquiring characteristic information according to the main audio signal may include:
- a first frequency domain signal corresponding to the first main audio signal and a second frequency domain signal corresponding to the second main audio signal are acquired.
- the first main audio signal and the second main audio signal are main audio signals respectively collected by any two main microphones in the at least two main microphones.
- Wind noise level information is determined based on the correlation between the first frequency domain signal and the second frequency domain signal.
- the degree of wind noise microphone left channel or right channel main primary microphone signal correlation information can be defined between the main audio signal and the main audio signal x L x R.
- the correlation calculation may adopt a classical cross-spectral calculation method, see formula 2. The value range of the correlation is between 0 and 1. The closer the value is to 1, the better the correlation, indicating that the main microphone is less affected by wind noise.
- At least two main microphones may be located on different sides of the electronic device respectively.
- the two main microphones may be located on the first side and the opposite side of the first side of the electronic device, respectively.
- This embodiment does not limit the specific position of the first side on the electronic device, which may be set according to the shape of the electronic device and the sound collection requirements.
- the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal, which may include:
- the repair coefficient is determined according to the feature information.
- the repair coefficient includes a first weight corresponding to the main audio signal, and/or a second weight corresponding to the adjusted auxiliary audio signal.
- the target audio signal is synthesized according to the first weight value and/or the second weight value and the adjusted auxiliary audio signal.
- the wind noise level information of the main microphone may indicate the degree to which the main microphone is affected by wind noise.
- the repair coefficients corresponding to the main microphone and/or the auxiliary microphone are determined according to the wind noise level information of the main microphone, and the target audio signal is synthesized according to the repair coefficient, the main audio signal and the adjusted auxiliary audio signal, which improves the electronic equipment based on the main microphone and the auxiliary microphone.
- the quality of the audio collected by the microphone improves the pickup effect.
- the repair coefficient may include a first weight and a second weight.
- synthesizing the target audio signal according to the first weight and/or the second weight and the adjusted auxiliary audio signal may include:
- the main frequency domain signal corresponding to the main audio signal and the auxiliary frequency domain signal corresponding to the adjusted auxiliary audio signal are acquired.
- the sum of the first modified signal and the second modified signal is determined as the target audio signal.
- the first modified signal is the product of the main frequency domain signal and the first weight
- the second modified signal is the product of the auxiliary frequency domain signal and the second weight.
- a certain proportion of the main audio signal is combined with a certain proportion of the adjusted auxiliary audio signal to synthesize the final target audio signal, which improves the quality of the audio collected by the electronic device based on the main microphone and the auxiliary microphone.
- the adjusted auxiliary audio signal may be denoted as x Ref'
- the corresponding frequency domain signal may be denoted as X Ref'
- the wind noise level information of the main microphone of the left channel is denoted as R L
- the wind noise level information of the main microphone of the right channel is denoted as R R , for reference to the above formula 1 or formula 2.
- the first weight corresponding to the left channel main microphone determined according to the wind noise level information of the left channel main microphone is denoted as ratio L
- the determined second weight corresponding to the auxiliary microphone is denoted as ratio Ref1
- the first weight corresponding to the right channel main microphone determined according to the wind noise level information of the right channel main microphone is denoted as ratio R
- the determined second weight corresponding to the auxiliary microphone is denoted as ratio Ref2 .
- the target audio signals corresponding to the main microphone of the left channel and the main microphone of the right channel respectively may refer to formula 3.
- the wind noise level information of the main microphone has a mapping function relationship with the first weight value, the sum of the first weight value and the second weight value is equal to 1, and the mapping function relationship includes any one of the following: a linear function relationship, Exponential function relationship and logarithmic function relationship.
- ratio Ref1 1-ratio L
- ratio Ref2 1-ratio R .
- mapping function relationship is exemplarily described below with reference to FIG. 5 .
- 5 is a schematic diagram of a mapping function relationship provided by an embodiment of the present application, showing three mapping function relationships between wind noise level information of the main microphone and a weight (specifically, a first weight). Among them, map 1 shows a logarithmic function relationship, map 2 shows a linear function relationship, and map 3 shows an exponential function relationship.
- the first weight of the main audio signal in the target audio signal is larger, and correspondingly, the second weight of the adjusted auxiliary audio signal is smaller;
- the first weight of the main audio signal in the target audio signal is smaller, and correspondingly, the second weight of the adjusted auxiliary audio signal is larger;
- the linear function relationship the main audio signal in the target audio signal is larger.
- the first weight of and the adjusted second weight of the auxiliary audio signal are in a fixed ratio.
- the mapping function relationship can be determined according to different audio acquisition requirements. For example, when it is desired to restore the real environment of audio acquisition, the logarithmic function relationship can be used.
- the audio processing method provided by the present application is described in conjunction with an overload scenario.
- the characteristic information of the main microphone may be overload degree information of the main microphone.
- acquiring characteristic information of the main microphone may include:
- the microphone is overloaded according to the audio signal collected by the microphone itself.
- the overload level information of the main microphone is determined according to the main audio signal, and the implementation is simple and easy.
- acquiring feature information according to the main audio signal may include:
- the overload degree information is determined according to the signal amplitude within the first preset time period.
- the value of the first preset time period is not limited in this embodiment.
- the signal amplitude of the main audio signal collected by the main microphone changes from time to time and has volatility.
- the overload degree information may be determined according to the maximum value, the average value or the weighted average value of the signal amplitudes of the main audio signal within the first preset time period.
- the overload degree information is determined according to the maximum value of the absolute value of the signal amplitude within the first preset time period.
- the main audio signal collected by the main microphone is denoted as x M
- the corresponding frequency domain signal is denoted as X M .
- the absolute value of the signal amplitude of the main audio signal is denoted as
- the maximum value of the absolute value of the signal amplitude of the main audio signal within the first preset time period is denoted as max
- the overload level information of the main microphone can be determined according to max
- the overload degree information of the main microphone may be max
- the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal, which may include:
- the repair coefficient is determined according to the feature information.
- the repair coefficient includes a first weight corresponding to the main audio signal, and/or a second weight corresponding to the adjusted auxiliary audio signal.
- the target audio signal is synthesized according to the first weight value and/or the second weight value and the adjusted auxiliary audio signal.
- the overload degree information of the main microphone may indicate whether the main microphone is overloaded or the degree of overloading.
- the greater the degree of overloading of the main microphone the greater the degree of need to repair the main audio signal collected by the main microphone.
- the repair coefficients corresponding to the main microphone and/or the auxiliary microphone are determined according to the overload degree information of the main microphone, and the target audio signal is synthesized according to the repair coefficient, the main audio signal and the adjusted auxiliary audio signal, which improves the electronic equipment based on the main microphone and the auxiliary microphone. The quality of the captured audio improves the pickup effect.
- the repair coefficient may include a second weight.
- synthesizing the target audio signal according to the first weight and/or the second weight and the adjusted auxiliary audio signal may include:
- the product of the adjusted auxiliary audio signal and the second weight is determined as the target audio signal.
- the implementation principle can be seen in FIG. 6 .
- the adjusted auxiliary audio signal with a certain proportion is used to synthesize the final target audio signal, and the second weight corresponding to the adjusted auxiliary audio signal is based on the main microphone. If the overload level information is determined, the quality of the audio collected by the electronic device based on the main microphone and the auxiliary microphone is improved.
- determining whether the main microphone is overloaded according to the overload degree information may include:
- overload level information is greater than the first preset threshold, it is determined that the main microphone is overloaded.
- overload level information is less than or equal to the first preset threshold, it is determined that the main microphone is not overloaded.
- the value of the first preset threshold is not limited in this embodiment.
- the first preset threshold may be related to the number of quantization bits of the recorded audio signal.
- the overload degree information of the main microphone is the maximum value of the absolute value of the signal amplitude of the main audio signal within the first preset time period, which is denoted as max
- the first preset threshold may be 32767. If max
- the second weight can be any of the following:
- A 2 m-1
- m represents the quantization bit of the main audio signal.
- B represents the maximum value of the absolute value of the signal amplitude of the adjusted auxiliary audio signal within the third preset time period.
- E represents the mean square value of the signal amplitude of the main audio signal within the third preset time period.
- ratio indicates a signal scaling factor for setting according to user requirements, ratio>0.
- this embodiment does not limit the values of m, ratio, and the third preset time period.
- the main audio signal collected by the main microphone is denoted as x M
- the corresponding frequency domain signal is denoted as X M
- the auxiliary audio signal collected by the auxiliary microphone is denoted as x Ref
- the corresponding frequency domain signal is denoted as X Ref
- the adjusted auxiliary audio signal is denoted as x Ref'
- the corresponding frequency domain signal is denoted as X Ref'
- the second weight may be target audio signal
- the adjusted auxiliary audio signal is directly amplified to the limit as the target audio signal.
- the second weight may be target audio signal
- the second weight may be target audio signal
- the second weight is
- the value of R ranges from 0 to 1, and the larger the value of R is, the higher the overload degree of the main microphone is. If the value of R is greater than or equal to the second preset threshold, the adjusted auxiliary audio signal can be directly amplified to the limit as the target audio signal.
- the value of the second preset threshold is not limited in this embodiment.
- the audio processing method provided by the present application is described in conjunction with a microphone blocking scenario.
- the characteristic information of the main microphone may be information on the degree of microphone blocking of the main microphone.
- acquiring characteristic information of the main microphone may include:
- the characteristic information is determined according to the main audio signal and the auxiliary audio signal.
- the microphone blocking degree information of the main microphone is determined according to the main audio signal and the auxiliary audio signal, which improves the accuracy of determining the microphone blocking degree information of the main microphone.
- the microphone blocking degree information may be determined according to the magnitude relationship between the signal energy of the main audio signal and the signal energy of the auxiliary audio signal.
- the microphone blocking degree information of the main microphone is determined according to the magnitude relationship between the signal energy of the main audio signal and the signal energy of the auxiliary audio signal, which improves the accuracy of determining the microphone blocking degree information of the main microphone.
- this embodiment does not limit the implementation manner of acquiring the signal energy of the main audio signal and acquiring the signal energy of the auxiliary audio signal.
- the signal energy can be acquired in the time domain according to the main audio signal or the auxiliary audio signal, or the main audio signal
- the frequency domain signal corresponding to the signal or the frequency domain signal corresponding to the auxiliary audio signal obtains the signal energy in the frequency domain.
- the microphone blocking degree information may be determined according to the ratio between the signal energy of the main audio signal and the signal energy of the auxiliary audio signal.
- the signal energy of the main audio signal and the signal energy of the auxiliary audio signal can be continuously acquired to obtain a ratio between the two.
- the ratio may be the ratio of the signal energy of the main audio signal to the signal energy of the auxiliary audio signal, or may be the ratio of the signal energy of the auxiliary audio signal to the signal energy of the main audio signal.
- the ratio may be a ratio between the average value of the signal energy of the main audio signal and the average value of the signal energy of the auxiliary audio signal in the same time period. If the ratio suddenly changes, for example, the ratio of the signal energy of the main audio signal to the signal energy of the auxiliary audio signal suddenly decreases, the main microphone may be blocked.
- the wheat blocking degree information may be determined according to the ratio, for example, may be directly the ratio, or an average or weighted average of the ratio within a period of time, which is not limited in this embodiment.
- the signal energy of the main audio signal collected by the main microphone can be expressed as Eng main
- the signal energy of the auxiliary audio signal collected by the auxiliary microphone can be expressed as Eng ref
- the information on the degree of wheat blocking is expressed as ratio, which can be found in formula 4.
- the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal, which may include:
- the repair coefficient is determined according to the feature information.
- the repair coefficient includes a first weight corresponding to the main audio signal, and/or a second weight corresponding to the adjusted auxiliary audio signal.
- the target audio signal is synthesized according to the first weight value and/or the second weight value and the adjusted auxiliary audio signal.
- the microphone blocking degree information of the main microphone may indicate whether the main microphone is blocked or the degree of the microphone blocking of the main microphone.
- the repair coefficients corresponding to the main microphone and/or the auxiliary microphone are determined according to the microphone blocking information of the main microphone, and the target audio signal is synthesized according to the repair coefficient, the main audio signal and the adjusted auxiliary audio signal, which improves the electronic equipment based on the main microphone and the auxiliary microphone. The quality of the captured audio.
- the repair coefficient may include a second weight.
- synthesizing the target audio signal according to the first weight and/or the second weight and the adjusted auxiliary audio signal may include:
- the product of the adjusted auxiliary audio signal and the second weight is determined as the target audio signal.
- the implementation principle can be seen in FIG. 7 .
- the adjusted auxiliary audio signal with a certain proportion is used to synthesize the final target audio signal, and the second weight corresponding to the adjusted auxiliary audio signal is based on the main microphone.
- the information on the blocking degree of the microphone is determined, which improves the quality of the audio collected by the electronic device based on the main microphone and the auxiliary microphone.
- the second weight may be 1.
- the adjusted auxiliary audio signal is directly used as the target audio signal, and the method is simple and easy to implement.
- determining whether the main microphone is blocked according to the degree of microphone blocking may include:
- the average value is greater than or equal to the preset average value, it is determined that the main microphone is not blocked.
- this embodiment does not limit the values of the second preset time period and the preset mean value.
- another embodiment of the present application provides an audio processing method based on the embodiment shown in FIG. 3 or the embodiment applicable to a wind noise scenario and a microphone blocking scenario.
- characteristic information corresponding to the main microphone in different frequency bands can be obtained, so that the target audio signal can be synthesized by frequency bands based on the characteristic information of the main microphone in each frequency band.
- acquiring the feature information of the main microphone may include:
- the target signal includes a main audio signal, or the target signal includes a main audio signal and an auxiliary audio signal.
- the characteristic information of the main microphone in each frequency band is obtained.
- the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal, which may include:
- the frequency components of the main audio signal in this frequency band and the frequency components of the adjusted auxiliary audio signal in this frequency band are synthesized into the target audio The frequency components of the signal in this frequency band.
- this embodiment does not limit the division manner of the multiple frequency bands. For example, reference may be made to the relevant description in S302, which will not be repeated here.
- the characteristic information of the main microphone on each frequency band is obtained, and reference may be made to the above-mentioned relevant description on obtaining the characteristic information of the main microphone. The principle is similar, and will not be repeated here.
- the frequency component of the main audio signal in this frequency band and the frequency component of the adjusted auxiliary audio signal in this frequency band are synthesized into the target audio signal in this frequency band.
- the frequency components in the frequency band please refer to the above description about synthesizing the main audio signal and the adjusted auxiliary audio signal into the target audio signal according to the characteristic information of the main microphone. The principles are similar and will not be repeated here.
- the characteristic information of the main microphone in different frequency bands is obtained in a microphone blocking scenario as an example for illustration.
- the audio signal has different attenuation characteristics in the high frequency and low frequency bands.
- the signal energy of the audio signal can be determined by frequency bands, and the main microphone can be determined according to the signal energy of the audio signal in different frequency bands. Mic blocking level information on different frequency bands.
- energy detection can be divided into two frequency bands: high frequency and low frequency.
- This embodiment does not limit the division of high frequency and low frequency.
- Above 2kHz is the high frequency band.
- the signal energy of the main audio signal collected by the main microphone in the high frequency band can be expressed as Eng main,H
- the signal energy of the auxiliary audio signal collected by the auxiliary microphone in the high frequency band can be expressed as Eng ref,H .
- the signal energy of the main audio signal collected by the main microphone in the low frequency band can be expressed as Eng main,L
- the signal energy of the auxiliary audio signal collected by the auxiliary microphone in the low frequency band can be expressed as Eng ref,L .
- For the wheat blocking degree information ratio H corresponding to the high frequency band and the wheat blocking degree information ratio L corresponding to the low frequency band reference may be made to formula 5.
- another embodiment of the present application provides an audio processing method.
- the characteristic information of the main microphone acquired in the current time period can be combined with the historical information between the current time periods. Correction, thereby improving the accuracy of the acquired characteristic information of the main microphone, and then synthesizing the target audio signal based on the more accurate characteristic information of the main microphone, thereby improving the quality of the target audio signal.
- the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal, which may include:
- the first feature information is modified according to the second feature information.
- the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal.
- correcting the first feature information according to the second feature information may include:
- the first feature information is modified according to the first feature information, the weight of the first feature information, the second feature information, and the weight of the second feature information.
- the first wind noise level information of the main microphone in the current time period may be recorded as R1
- the second wind noise level information of the main microphone in the previous time period adjacent to the current time period may be recorded as R0.
- the weight of the first wind noise level information R1 may be recorded as a1
- the weight of the second wind noise level information R0 may be recorded as a0.
- a0 1-a1.
- the corrected first wind noise level information R1' may be a1*R1+(1-a1)*R0.
- FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
- the electronic device provided in this embodiment may include a main microphone (not shown) and an auxiliary microphone (not shown), a sound pickup cavity (not shown) of the main microphone and the auxiliary microphone
- the sound pickup cavities (not shown) are all communicated with the external environment where the electronic device is located; the electronic device further includes a sound pickup protection structure (not shown), and the sound pickup protection structure is configured to weaken the sound.
- the electronic device further includes a memory 82 and a processor 81;
- the memory 82 is used to store program codes
- the processor 81 calls the program code, and when the program code is executed, is used to perform the following operations:
- a target audio signal is synthesized from the main audio signal and the auxiliary audio signal.
- the processor 81 is also used for:
- the target tuning parameter adjusts the frequency response parameter of the audio component; wherein, the frequency response parameter includes an amplitude parameter and/or a phase parameter, and the corresponding relationship is based on the main microphone and the auxiliary microphone in the electronic device Determined by the deviation of the frequency response parameters of the sample audio obtained by the audio collection of the sample audio source respectively;
- the processor 81 is specifically used for:
- a target audio signal is synthesized according to the main audio signal and the adjusted auxiliary audio signal.
- the processor 81 is also used for:
- the characteristic information includes one or more of the following information: wind noise level information, microphone blocking level information or overload level information;
- the processor 81 is specifically used for:
- the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal.
- the processor 81 is specifically used for:
- the feature information is determined according to the primary audio signal and the secondary audio signal.
- the feature information includes wind noise level information
- the wind noise level information is determined according to a signal correlation between the primary audio signal and the secondary audio signal.
- the feature information includes microphone blocking degree information, and the microphone blocking degree information is determined according to the magnitude relationship between the signal energy of the main audio signal and the signal energy of the auxiliary audio signal.
- the microphone blocking degree information is determined according to a ratio between the signal energy of the main audio signal and the signal energy of the auxiliary audio signal.
- the processor 81 is specifically used for:
- the characteristic information is acquired according to the main audio signal.
- the electronic device includes at least two main microphones, and the feature information includes wind noise level information;
- the processor 81 is specifically used for:
- first main audio signal and the second main audio signal are the at least two The main audio signal collected by any two main microphones in the main microphones;
- the wind noise level information is determined based on the correlation between the first frequency domain signal and the second frequency domain signal.
- the at least two main microphones are respectively located on different sides of the electronic device.
- the number of the at least two main microphones is two, and the two main microphones are respectively located on the first side of the electronic device and on the opposite side of the first side.
- the feature information includes overload degree information
- the processor 81 is specifically used for:
- the overload degree information is determined according to the signal amplitude within the first preset time period.
- the overload degree information is determined according to the maximum value of the absolute value of the signal amplitude within the first preset time period.
- the processor 81 is specifically used for:
- a repair coefficient is determined according to the feature information; the repair coefficient includes a first weight corresponding to the main audio signal, and/or a second weight corresponding to the adjusted auxiliary audio signal;
- the target audio signal is synthesized according to the first weight and/or the second weight and the adjusted auxiliary audio signal.
- the feature information includes wind noise level information
- the repair coefficient includes the first weight and the second weight
- the processor 81 is specifically used for:
- the first modified signal is the product of the main frequency domain signal and the first weight
- the second modified signal The signal is the product of the secondary frequency domain signal and the second weight.
- the wind noise level information and the first weight have a mapping function relationship, the sum of the first weight and the second weight is equal to 1, and the mapping function relationship includes any of the following: One item: linear functional relationship, exponential functional relationship, and logarithmic functional relationship.
- the feature information includes wheat blocking degree information, and the repair coefficient includes the second weight;
- the processor 81 is specifically used for:
- the product of the adjusted auxiliary audio signal and the second weight is determined as the target audio signal.
- the processor 81 is specifically used for:
- the average value is greater than or equal to the preset average value, it is determined that the main microphone is not blocked.
- the second weight is 1.
- the feature information includes overload degree information
- the repair coefficient includes the second weight
- the processor 81 is specifically used for:
- the product of the adjusted auxiliary audio signal and the second weight is determined as the target audio signal.
- the processor 81 is specifically used for:
- overload level information is greater than the first preset threshold, determine that the main microphone is overloaded
- overload level information is less than or equal to the first preset threshold, it is determined that the main microphone is not overloaded.
- the processor 81 is specifically used for:
- the target signal includes the main audio signal, or the target signal includes the main audio signal and the auxiliary audio signal;
- the processor 81 is specifically used for:
- the frequency component of the main audio signal in the frequency band and the adjusted auxiliary audio signal are The frequency components in the frequency band are synthesized to the frequency components of the target audio signal in the frequency band.
- the processor 81 is specifically used for:
- the target audio signal is synthesized from the main audio signal and the adjusted auxiliary audio signal according to the modified first feature information.
- the processor 81 is specifically used for:
- the first feature information is modified according to the first feature information, the weight of the first feature information, the second feature information, and the weight of the second feature information.
- the main microphone is close to the housing of the electronic device relative to the auxiliary microphone.
- the sound pickup protection structure includes a windproof structure, and the auxiliary microphone is arranged in the windproof structure.
- the windproof structure includes a hollow windproof cover body and a support for supporting the windproof cover body, and the auxiliary microphone is arranged in a cavity of the windproof cover body.
- the sound pickup protection structure includes a dustproof structure, and the auxiliary microphone is arranged in the dustproof structure.
- the dustproof structure includes at least one layer of filter screen covering the outer side of the auxiliary microphone.
- the electronic device provided in this embodiment can execute the audio processing method provided by the embodiments shown in FIG. 3 to FIG. 7 of the present application, and the technical principle and technical effect are similar, which will not be repeated here.
- another embodiment of the present application further provides an audio processing method.
- the audio processing method may include:
- the main audio signal and the auxiliary audio signal are collected from the same audio source at the same time, and the amplitude and/or phase of the auxiliary audio signal and the main audio signal at a specific frequency different;
- the target tuning parameter adjusts the amplitude parameter and/or the phase parameter of the audio component
- a target audio signal is synthesized according to the main audio signal and the adjusted auxiliary audio signal.
- the primary audio signal is collected by a primary microphone provided on the electronic device
- the secondary audio signal is collected by a secondary microphone provided on the electronic device.
- the corresponding relationship is determined based on the deviation of the frequency response parameters of the sample audio obtained by the main microphone and the auxiliary microphone in the electronic device respectively collecting the sample audio source.
- the method further includes:
- the characteristic information includes one or more of the following information: wind noise level information, microphone blocking level information or overload level information;
- the synthesizing target audio signal according to the main audio signal and the adjusted auxiliary audio signal includes:
- the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal.
- the acquiring feature information of the main microphone includes:
- the feature information is determined according to the primary audio signal and the secondary audio signal.
- the feature information includes wind noise level information
- the wind noise level information is determined according to a signal correlation between the primary audio signal and the secondary audio signal.
- the feature information includes microphone blocking degree information, and the microphone blocking degree information is determined according to the magnitude relationship between the signal energy of the main audio signal and the signal energy of the auxiliary audio signal.
- the microphone blocking degree information is determined according to a ratio between the signal energy of the main audio signal and the signal energy of the auxiliary audio signal.
- the acquiring feature information of the main microphone includes:
- the characteristic information is acquired according to the main audio signal.
- the electronic device includes at least two main microphones, and the feature information includes wind noise level information;
- the acquiring the feature information according to the main audio signal includes:
- first main audio signal and the second main audio signal are the at least two The main audio signal collected by any two main microphones in the main microphones;
- the wind noise level information is determined based on the correlation between the first frequency domain signal and the second frequency domain signal.
- the feature information includes overload degree information
- the acquiring the feature information according to the main audio signal includes:
- the overload degree information is determined according to the signal amplitude within the first preset time period.
- the overload degree information is determined according to the maximum value of the absolute value of the signal amplitude within the first preset time period.
- synthesizing the main audio signal and the adjusted auxiliary audio signal into the target audio signal according to the characteristic information of the main microphone includes:
- a repair coefficient is determined according to the feature information; the repair coefficient includes a first weight corresponding to the main audio signal, and/or a second weight corresponding to the adjusted auxiliary audio signal;
- the target audio signal is synthesized according to the first weight and/or the second weight and the adjusted auxiliary audio signal.
- the feature information includes wind noise level information
- the repair coefficient includes the first weight and the second weight
- the synthesizing the target audio signal according to the first weight and/or the second weight and the adjusted auxiliary audio signal includes:
- the first modified signal is the product of the main frequency domain signal and the first weight
- the second modified signal The signal is the product of the secondary frequency domain signal and the second weight.
- the wind noise level information and the first weight have a mapping function relationship, the sum of the first weight and the second weight is equal to 1, and the mapping function relationship includes any of the following: One item: linear functional relationship, exponential functional relationship, and logarithmic functional relationship.
- the feature information includes wheat blocking degree information, and the repair coefficient includes the second weight;
- the synthesizing the target audio signal according to the first weight and/or the second weight and the adjusted auxiliary audio signal includes:
- the product of the adjusted auxiliary audio signal and the second weight is determined as the target audio signal.
- the determining whether the main microphone is blocked according to the information on the degree of microphone blocking includes:
- the average value is greater than or equal to the preset average value, it is determined that the main microphone is not blocked.
- the second weight is 1.
- the feature information includes overload degree information
- the repair coefficient includes the second weight
- the synthesizing the target audio signal according to the first weight and/or the second weight and the adjusted auxiliary audio signal includes:
- the product of the adjusted auxiliary audio signal and the second weight is determined as the target audio signal.
- the determining whether the main microphone is overloaded according to the overload degree information includes:
- overload level information is greater than the first preset threshold, determine that the main microphone is overloaded
- overload level information is less than or equal to the first preset threshold, it is determined that the main microphone is not overloaded.
- the acquiring feature information of the main microphone includes:
- the target signal includes the main audio signal, or the target signal includes the main audio signal and the auxiliary audio signal;
- the frequency components of the main audio signal in the frequency band and the adjusted auxiliary audio signal are The frequency components in the frequency band are synthesized to the frequency components of the target audio signal in the frequency band.
- synthesizing the main audio signal and the adjusted auxiliary audio signal into the target audio signal according to the characteristic information of the main microphone includes:
- the target audio signal is synthesized from the main audio signal and the adjusted auxiliary audio signal according to the modified first feature information.
- the modifying the first feature information according to the second feature information includes:
- the first feature information is modified according to the first feature information, the weight of the first feature information, the second feature information, and the weight of the second feature information.
- the present application further provides an electronic device, and the structure of the electronic device may refer to FIG. 8 .
- the electronic device may include: a memory 82 and a processor 81;
- the memory for storing program codes
- the processor calls the program code, and when the program code is executed, is configured to perform the following operations:
- the main audio signal and the auxiliary audio signal are collected from the same audio source at the same time, and the amplitude and/or phase of the auxiliary audio signal and the main audio signal at a specific frequency different;
- the target tuning parameter adjusts the amplitude parameter and/or the phase parameter of the audio component
- a target audio signal is synthesized according to the main audio signal and the adjusted auxiliary audio signal.
- the primary audio signal is collected by a primary microphone provided on the electronic device
- the secondary audio signal is collected by a secondary microphone provided on the electronic device.
- the corresponding relationship is determined based on the deviation of the frequency response parameters of the sample audio obtained by the main microphone and the auxiliary microphone in the electronic device respectively collecting the sample audio source.
- the processor is also used for:
- the characteristic information includes one or more of the following information: wind noise level information, microphone blocking level information or overload level information;
- the processor is specifically used for:
- the main audio signal and the adjusted auxiliary audio signal are synthesized into the target audio signal.
- the processor is specifically used for:
- the feature information is determined according to the primary audio signal and the secondary audio signal.
- the feature information includes wind noise level information
- the wind noise level information is determined according to a signal correlation between the primary audio signal and the secondary audio signal.
- the feature information includes microphone blocking degree information, and the microphone blocking degree information is determined according to the magnitude relationship between the signal energy of the main audio signal and the signal energy of the auxiliary audio signal.
- the microphone blocking degree information is determined according to a ratio between the signal energy of the main audio signal and the signal energy of the auxiliary audio signal.
- the processor is specifically used for:
- the characteristic information is acquired according to the main audio signal.
- the electronic device includes at least two main microphones, and the feature information includes wind noise level information;
- the processor is specifically used for:
- first main audio signal and the second main audio signal are the at least two The main audio signal collected by any two main microphones in the main microphones;
- the wind noise level information is determined based on the correlation between the first frequency domain signal and the second frequency domain signal.
- the feature information includes overload degree information
- the processor is specifically used for:
- the overload degree information is determined according to the signal amplitude within the first preset time period.
- the overload degree information is determined according to the maximum value of the absolute value of the signal amplitude within the first preset time period.
- the processor is specifically used for:
- a repair coefficient is determined according to the feature information; the repair coefficient includes a first weight corresponding to the main audio signal, and/or a second weight corresponding to the adjusted auxiliary audio signal;
- the target audio signal is synthesized according to the first weight and/or the second weight and the adjusted auxiliary audio signal.
- the feature information includes wind noise level information
- the repair coefficient includes the first weight and the second weight
- the processor is specifically used for:
- the first modified signal is the product of the main frequency domain signal and the first weight
- the second modified signal The signal is the product of the secondary frequency domain signal and the second weight.
- the wind noise level information and the first weight have a mapping function relationship, the sum of the first weight and the second weight is equal to 1, and the mapping function relationship includes any of the following: One item: linear functional relationship, exponential functional relationship, and logarithmic functional relationship.
- the feature information includes wheat blocking degree information, and the repair coefficient includes the second weight;
- the processor is specifically used for:
- the product of the adjusted auxiliary audio signal and the second weight is determined as the target audio signal.
- the processor is specifically used for:
- the average value is greater than or equal to the preset average value, it is determined that the main microphone is not blocked.
- the second weight is 1.
- the feature information includes overload degree information
- the repair coefficient includes the second weight
- the processor is specifically used for:
- the product of the adjusted auxiliary audio signal and the second weight is determined as the target audio signal.
- the processor is specifically used for:
- overload level information is greater than the first preset threshold, determine that the main microphone is overloaded
- overload level information is less than or equal to the first preset threshold, it is determined that the main microphone is not overloaded.
- the processor is specifically used for:
- the target signal includes the main audio signal, or the target signal includes the main audio signal and the auxiliary audio signal;
- the processor is specifically used for:
- the frequency components of the main audio signal in the frequency band and the adjusted auxiliary audio signal are The frequency components in the frequency band are synthesized to the frequency components of the target audio signal in the frequency band.
- the processor is specifically used for:
- the target audio signal is synthesized from the main audio signal and the adjusted auxiliary audio signal according to the modified first feature information.
- the processor is specifically used for:
- the first feature information is modified according to the first feature information, the weight of the first feature information, the second feature information, and the weight of the second feature information.
- the various implementation manners provided in the embodiments of the present application may be combined with each other, and the present application does not limit the combining manners.
- the present application does not limit the types of electronic devices, processors, and memories, and the implementation manners and electrical connection relationships of the processors.
- the processor may be electrically connected to the microphone through pins, obtain the main audio signal collected by the main microphone and the auxiliary audio signal collected by the auxiliary microphone, and perform corresponding processing.
- the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, which can implement or execute the implementation of the present application.
- a general purpose processor may be a microprocessor or any conventional processor or the like.
- the steps of the methods disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.
- the memory may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or a volatile memory (volatile memory), such as Random access memory (RAM).
- Memory is, but is not limited to, any medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
- the memory in this embodiment of the present application may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and/or data.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- General Health & Medical Sciences (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
本申请实施例提供一种音频处理方法和电子设备,其中,音频处理方法应用于电子设备,电子设备包括主麦克风、辅麦克风和拾音保护结构,拾音保护结构被设置为用于减弱从外部环境进入辅麦克风的拾音腔的气流,和/或,阻隔非气体物质进入辅麦克风的拾音腔;音频处理方法包括:获取主麦克风采集的主音频信号和辅麦克风采集的辅音频信号;根据主音频信号和辅音频信号合成目标音频信号。提升了电子设备采集到的音频信号的质量。
Description
本申请实施例涉及声象技术领域,尤其涉及一种音频处理方法和电子设备。
音频是媒体处理的关键数据,应该有较高的质量。相关技术中,电子设备上搭载有麦克风。麦克风一般被设置在靠近电子设备表面,电子设备表面开设拾音通道的孔洞,使麦克风能较好的接收到空气中的振动,进而进行声电转化,形成音频信号。
随着拾音场景的越来越复杂化以及越来越多样化,电子设备在拾音过程中可能遇到各种异常情况,例如,电子设备表面有气流、污染物,导致经拾音通道的空气振动受到影响,采集到的音频信号质量不高,给后续处理带来难度。
发明内容
本申请实施例提供一种音频处理方法和电子设备,提升了异常情况下电子设备采集到的音频信号的质量。
第一方面,本申请实施例提供一种音频处理方法,应用于电子设备,所述电子设备包括主麦克风和辅麦克风,所述主麦克风的拾音腔和所述辅麦克风的拾音腔均与所述电子设备所处的外部环境连通;所述电子设备包括拾音保护结构,所述拾音保护结构被设置为用于减弱从所述外部环境进入所述辅麦克风的拾音腔的气流,和/或,阻隔非气体物质进入所述辅麦克风的拾音腔;
所述方法包括:
获取所述主麦克风采集的主音频信号和所述辅麦克风采集的辅音频信号;
根据所述主音频信号和所述辅音频信号合成目标音频信号。
第二方面,本申请实施例提供一种电子设备,包括主麦克风和辅麦克风,所述主麦克风的拾音腔和所述辅麦克风的拾音腔均与所述电子设备所处的外 部环境连通;所述电子设备还包括拾音保护结构,所述拾音保护结构被设置为用于减弱从所述外部环境进入所述辅麦克风的拾音腔的气流,和/或,阻隔非气体物质进入所述辅麦克风的拾音腔;
所述电子设备还包括存储器和处理器;
所述存储器,用于存储程序代码;
所述处理器,调用所述程序代码,当所述程序代码被执行时,用于执行以下操作:
获取所述主麦克风采集的主音频信号和所述辅麦克风采集的辅音频信号;
根据所述主音频信号和所述辅音频信号合成目标音频信号。
第三方面,本申请实施例提供一种音频处理方法,包括:
获取主音频信号和辅音频信号,所述主音频信号和所述辅音频信号是同时对同一音源采集得到的,所述辅音频信号与所述主音频信号在特定频率的幅值和/或相位不同;
在所述辅音频信号中确定处于不同频段的多个音频分量;
根据所述多个音频分量中任一所述音频分量对应的频段,以及预设的频段和频响参数的调参量的对应关系,确定所述音频分量的频响参数的目标调参量,并根据所述目标调参量调整所述音频分量的幅值参数和/或相位参数;
根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号。
第四方面,本申请实施例提供一种电子设备,包括:存储器和处理器;
所述存储器,用于存储程序代码;
所述处理器,调用所述程序代码,当所述程序代码被执行时,用于执行以下操作:
获取主音频信号和辅音频信号,所述主音频信号和所述辅音频信号是同时对同一音源采集得到的,所述辅音频信号与所述主音频信号在特定频率的幅值和/或相位不同;
在所述辅音频信号中确定处于不同频段的多个音频分量;
根据所述多个音频分量中任一所述音频分量对应的频段,以及预设的频段和频响参数的调参量的对应关系,确定所述音频分量的频响参数的目标调参量,并根据所述目标调参量调整所述音频分量的幅值参数和/或相位参数;
根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号。
第五方面,本申请实施例提供一种计算机可读存储介质,所述可读存储介质中存储计算机程序,所述计算机程序在执行时实现如第一方面或第三方面提供的方法。
本申请实施例提供一种音频处理方法和电子设备,电子设备包括主麦克风和辅麦克风,电子设备还包括拾音保护结构,拾音保护结构被设置为用于减弱从外部环境进入辅麦克风的拾音腔的气流,和/或,阻隔非气体物质进入辅麦克风的拾音腔。这样,在拾音异常场景中,具有拾音保护结构的辅麦克风具有更好的拾音性能。电子设备基于主麦克风进行录音,通过具有拾音保护结构的辅麦克风采集的辅音频信号对主麦克风采集的主音频信号进行修正,生成目标音频信号,提升了异常情况下电子设备采集到的音频信号的质量。
图1为本申请实施例适用的电子设备的一种结构示意图;
图2为本申请实施例适用的电子设备的另一种结构示意图;
图3为本申请实施例提供的音频处理方法的一种流程图;
图4为本申请实施例提供的风噪场景中音频处理方法的一种原理图;
图5为本申请实施例提供的映射函数关系的示意图;
图6为本申请实施例提供的过载场景中音频处理方法的一种原理图;
图7为本申请实施例提供的堵麦场景中音频处理方法的一种原理图;
图8为本申请实施例提供的电子设备的一种结构示意图。
为使本申请实施例的目的、技术方案和优点更加清楚,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。
本申请实施例提供的音频处理方法,可以应用于电子设备。电子设备包括主麦克风和辅麦克风,主麦克风的拾音腔和辅麦克风的拾音腔均与电子设备所处的外部环境连通。主麦克风和辅麦克风均可以采集电子设备所处的外 部环境中的声音,生成音频信号。为了便于描述,主麦克风采集的音频信号可以称为主音频信号,辅麦克风采集的音频信号可以称为辅音频信号。
电子设备还包括拾音保护结构,拾音保护结构被设置为用于减弱从外部环境进入辅麦克风的拾音腔的气流,和/或,阻隔非气体物质进入辅麦克风的拾音腔。在本申请实施例中,主麦克风可以理解为电子设备中不具有拾音保护结构的麦克风,辅麦克风可以理解为电子设备中具有拾音保护结构的麦克风。当电子设备所处的外部环境不同时,拾音保护结构对辅麦克风的收音性能将产生不同的影响。当电子设备所处的拾音环境较为正常,例如,电子设备所处的外部环境中风较小、空气较为洁净,此时,具有拾音保护结构的辅麦克风相比于不具有拾音保护结构的主麦克风,拾音保护结构会降低辅麦克风在收音过程中的频响和灵敏度,导致辅麦克风采集到的辅音频信号天然就存在失真。而当电子设备所处的拾音环境较为异常,例如,电子设备所处的外部环境中风较大或空气中粉尘较多,比如,高速运动场景、室外大风场景、扬尘天气等,由于拾音保护结构可以减弱从外部环境进入辅麦克风的拾音腔的气流,和/或,阻隔非气体物质进入辅麦克风的拾音腔,此时,具有拾音保护结构的辅麦克风相比于不具有拾音保护结构的主麦克风,对异常情况的抵抗能力更强,拾音性能更加稳健。因此,根据电子设备所处的环境的不同,利用主麦克风进行正常录音,当主麦克风采集的主音频信号出现异常时,可以通过辅麦克风采集的辅音频信号对主音频信号进行修复调整,从而实现更稳健、更高音质的拾音效果。
需要说明,本申请实施例对主麦克风、辅麦克风的数量以及在电子设备中的位置不做限定。示例性的,图1为本申请实施例适用的电子设备的一种结构示意图,图2为本申请实施例适用的电子设备的另一种结构示意图。如图1所示,电子设备100包括2个主麦克风和1个辅麦克风13。2个主麦克风可以分别称为主麦克风11和主麦克风12。可选的,主麦克风11和主麦克风12可以位于电子设备中相对的两侧,采集不同方向的声音。主麦克风11和主麦克风12也可以称为左声道主麦克风和右声道主麦克风。辅麦克风13具有拾音保护结构14。如图2所示,电子设备200包括1个主麦克风21和1个辅麦克风13,辅麦克风13具有拾音保护结构14。
可选的,主麦克风相对于辅麦克风可以靠近电子设备的外壳。通过将主 麦克风设置在更加靠近电子设备外壳的位置,使得主麦克风采集到的主音频信号更加真实。通过将辅麦克风设置在更加远离电子设备外壳的位置,而且通过拾音保护结构,使得辅麦克风采集到的辅音频信号抵抗拾音异常情况的能力更强,从而在异常场景中可以辅助修复主音频信号,提升电子设备的拾音效果。
需要说明,本申请实施例对电子设备的拾音应用场景不做限定,例如,可以包括但不限于下列中的至少一种场景:风噪场景、过载场景或堵麦场景。风噪是指高速运动的气流产生的噪声。举例说明。在一个场景中,电子设备录音时,高速运动的气流在电子设备的周围产生了涡流噪声。在另一个场景中,电子设备搭载在汽车或摩托车等高速行驶的车辆进行录音时,高速风对电子设备本身形成冲击而产生风噪。在又一个场景中,风在麦克风的入声孔(例如,电子设备表面上与主麦克风的拾音腔连通的孔洞)处产生涡流噪声。过载也称为削峰失真,是指音频信号的最大值超出了音轨能够记录的最大值,导致自动削除一部分高声压波形的异常场景。堵麦场景也称为无声场景,是指麦克风由于收音通道的堵塞导致采集的音频信号的最大值较小的异常场景。
需要说明,本申请实施例对主麦克风的拾音腔和辅麦克风的拾音腔的结构不做限定。例如,在一种实现方式中,主麦克风可以包括第一振膜和第一壳体,第一振膜与第一壳体形成主麦克风的拾音腔。辅麦克风可以包括第二振膜和第二壳体,第二振膜和第二壳体形成辅麦克风的拾音腔。
需要说明,本申请实施例对拾音保护结构的实现方式不做限定,在不同的应用场景中,拾音保护结构可以具有不同的结构。
可选的,拾音保护结构可以包括防风结构,辅麦克风可以设置在防风结构中,提升了辅麦克风抵抗风噪的能力,在风噪场景中提升了辅麦克风的拾音性能。
可选的,防风结构可以包括中空的防风罩体和用于支撑防风罩体的支撑件,辅麦克风可以设置在防风罩体的空腔中。
可选的,拾音保护结构可以包括防尘结构,辅麦克风可以设置在防尘结构中,降低了辅麦克风受到灰尘、水滴等污染物堵塞的概率,在堵麦场景中提升了辅麦克风的拾音性能。
可选的,防尘结构可以包括至少一层罩设在辅麦克风外侧的过滤网。可 选的,当过滤网的数量为多个时,不同的过滤网的过滤功能可以相同,也可以不同。例如,防尘结构可以包括两层罩设在辅麦克风外侧的过滤网,两层过滤网分别用于过滤灰尘等固态污染物和水汽等气态污染物。
可选的,拾音保护结构可以包括隔音结构,辅麦克风可以设置在隔音结构中,降低了辅麦克风的灵敏度,在过载场景中提升了辅麦克风的拾音性能。
需要说明的是,拾音保护结构可以实现上述防风功能、防尘功能和防过载功能中的一项或多项。
需要说明的是,本申请实施例对拾音保护结构的形状和材料不做限定。
下面,对本申请实施例涉及的概念进行说明。
1、风噪程度信息
每个主麦克风均对应有风噪程度信息,用于指示主麦克风受风噪影响的程度。可选的,风噪程度信息可以是预设取值范围内的数值。风噪程度信息的数值的大小可以指示主麦克风受风噪影响的程度。本申请实施例对风噪程度信息的取值范围不做限定。
2、堵麦程度信息
每个主麦克风均对应有堵麦程度信息,用于指示主麦克风是否发生堵麦异常或用于指示主麦克风堵麦的程度。可选的,堵麦程度信息可以是预设取值范围内的数值。堵麦程度信息的数值的大小可以指示主麦克风堵麦的程度。本申请实施例对堵麦程度信息的取值范围不做限定。
3、过载程度信息
每个主麦克风均对应有过载程度信息,用于指示主麦克风是否发生过载异常或用于指示主麦克风过载的程度。可选的,过载程度信息可以是预设取值范围内的数值。过载程度信息的数值的大小可以指示主麦克风过载的程度。本申请实施例对过载程度信息的取值范围不做限定。
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。
图3为本申请实施例提供的音频处理方法的一种流程图。本实施例提供 的音频处理方法,执行主体可以为电子设备,电子设备的结构可以参见上面描述,此处不再赘述。如图3所示,本实施例提供的音频处理方法,可以包括:
S301、获取主麦克风采集的主音频信号和辅麦克风采集的辅音频信号。
具体的,由于主麦克风和辅麦克风的位置不同,而且辅麦克风具有拾音保护结构,通常,主麦克风采集的主音频信号和辅麦克风采集的辅音频信号是不同的。在风噪、过载或堵麦等异常场景中,具有拾音保护结构的辅麦克风具有更好的拾音性能。
S302、根据主音频信号和辅音频信号合成目标音频信号。
可见,本实施例提供的音频处理方法,可以应用于电子设备。电子设备包括主麦克风和辅麦克风,电子设备还包括拾音保护结构,拾音保护结构被设置为用于减弱从外部环境进入辅麦克风的拾音腔的气流,和/或,阻隔非气体物质进入辅麦克风的拾音腔。这样,在风噪、过载或堵麦等异常场景中,具有拾音保护结构的辅麦克风具有更好的拾音性能。因此,在利用主麦克风进行正常录音的基础上,通过具有拾音保护结构的辅麦克风采集的辅音频信号对主音频信号进行调整,生成目标音频信号,提升了电子设备采集到的音频信号的质量,提升了电子设备的拾音效果。
可选的,本实施例提供的音频处理方法,还可以包括:
在辅音频信号中确定处于不同频段的多个音频分量。
根据多个音频分量中任一音频分量对应的频段,以及预设的频段和频响参数的调参量的对应关系,确定音频分量的频响参数的目标调参量,并根据目标调参量调整音频分量的频响参数。其中,频响参数包括幅值参数和/或相位参数,对应关系是基于电子设备中主麦克风和辅麦克风分别对样本音源进行音频采集得到的样本音频的频响参数的偏差确定的。
相应的,S302中,根据主音频信号和辅音频信号合成目标音频信号,可以包括:
根据主音频信号和调整后的辅音频信号合成目标音频信号。
其中,本实施例对不同频段的划分方式不做限定。
可选的,在一种实现方式中,不同频段可以包括多个频率范围连续的频段。比如,整个频段范围可以表示为f
min~f
max,按照预设的频段个数或者预设 的频段间隔或者其他的频段划分规则,可以将整个频段f
min~f
max划分为如下5个频段:f
min~f
1、f
1~f
2、f
2~f
3、f
3~f
4和f
4~f
max,其中,f
min<f
1<f
2<f
3<f
4<f
max。需要说明,本实施例对频段的个数和每个频段的频率范围不做限定。
可选的,在另一种实现方式中,不同频段可以包括多个频率范围不重叠的频段,每个频段包括中心频点f和频率偏置F,该频段的频率范围为(f-F)~(f+F)。相邻两个频段的频率范围可以连续,也可以不连续。比如,整个频段范围可以表示为f
min~f
max,整个频段f
min~f
max可以划分为如下5个频段:(f
1-F
1)~(f
1+F
1)、(f
2-F
2)~(f
2+F2)、(f
3-F
3)~(f
3+F
3)、(f
4-F
4)~(f
4+F
4)和(f
5-F
5)~(f
5+F
5),其中,f
1<f
2<f
3<f
4<f
5,f
1-F
1=f
min,(f
1+F
1)<(f
2-F
2),(f
2+F
2)<(f
3-F
3),(f
3+F
3)=(f
4-F
4),(f
5+F
5)<f
max。其中,本实施例对各个中心频点和频率偏置的取值不做限定。
其中,频响也称为频率响应,是指将一个以恒电压输出的音频信号与系统相连接时,音箱产生的声压随频率的变化而发生增大或衰减、相位随频率而发生变化的现象,这种声压和/或相位与频率的相关联的变化关系称为频响。由于频响与频率相关,在本步骤中,对于辅音频信号,根据预设的频段与频响参数的调参量之间的对应关系,分频段确定每个音频分量对应的频响参数的目标调参量。针对每个音频分量,根据频响参数的目标调参量分频段对辅音频信号的频响进行修正。其中,预设的频段和频响参数的调参量的对应关系,是基于电子设备中主麦克风和辅麦克风分别对样本音源进行音频采集得到的样本音频的频响参数的偏差确定的。
其中,基于电子设备中主麦克风和辅麦克风分别对样本音源进行音频采集得到的样本音频的频响参数的偏差确定预设的频段和频响参数的调参量的对应关系,可以通过神经网络模型实现,本实施例不限定神经网络模型的类型。例如,神经网络包括但不限于卷积神经网络(convolutional neural network,CNN),循环神经网络(recurrent neural network,RNN)以及长短期记忆网络(long short term memory,LSTM)。
通过分频段对辅音频信号的频响进行修正,可以使得辅麦克风采集的辅音频信号的频率响应与主麦克风采集的主音频信号的频率响应之间的偏差降低,为后续信号处理的准确性提供了保障。这样,在风噪、过载或堵麦等异常场景中,在利用主麦克风进行正常录音的基础上,通过具有拾音保护结构的 辅麦克风采集的、且进行频响修复后的辅音频信号对主音频信号进行调整,生成目标音频信号,提升了电子设备采集到的音频信号的质量,提升了电子设备的拾音效果。
可选的,本实施例提供的音频处理方法,还可以包括:
获取主麦克风的特征信息。特征信息可以包括以下一种或多种信息:风噪程度信息、堵麦程度信息或过载程度信息。
相应的,根据主音频信号和调整后的辅音频信号合成目标音频信号,可以包括:
根据主麦克风的特征信息,将主音频信号和调整后的辅音频信号合成目标音频信号。
其中,主麦克风的特征信息用于指示主麦克风受异常情况影响的程度。因此,根据主麦克风的特征信息,将主音频信号和调整后的辅音频信号合成目标音频信号,提升了目标音频信号的质量。
下面,在图3所示实施例的基础上,结合不同的应用场景对本申请提供的音频处理方法进行说明。其中,电子设备可以针对单一场景进行音频处理,也可以针对多个应用场景进行音频处理,本申请对结合方式不作限定。
可选的,在本申请的另一个实施例中,结合风噪场景对本申请提供的音频处理方法进行说明。在本实施例中,主麦克风的特征信息可以为主麦克风的风噪程度信息。
可选的,在一种实现方式中,获取主麦克风的特征信息,可以包括:
根据主音频信号和辅音频信号,确定特征信息。
具体的,在风噪场景中,由于电子设备中主麦克风和辅麦克风的位置不同,且辅麦克风具有拾音保护结构,风噪对主麦克风采集的主音频信号的影响更大,对辅麦克风采集的辅音频信号的影响更小。根据主音频信号和辅音频信号确定主麦克风的风噪程度信息,提升了主麦克风的风噪程度信息的准确性。
可选的,风噪程度信息是根据主音频信号与辅音频信号之间的信号相关性确定的。其中,信号相关性反映了两个信号之间的关联程度或相似程度。 信号相关性越大,说明两个信号越相似,相反的,信号相关性越小,说明两个信号差别越大。在风噪场景中,如果主音频信号与辅音频信号之间的信号相关性较强,说明主麦克风受风噪的影响越小,相反的,主音频信号与辅音频信号之间的信号相关性较弱,主麦克风受风噪的影响越大。
举例说明,参见图1。电子设备中包括两个主麦克风,分别称为左声道主麦克风和右声道主麦克风。左声道主麦克风采集的主音频信号记为x
L,对应的频域信号记为X
L。右声道主麦克风采集的主音频信号记为x
R,对应的频域信号记为X
R。电子设备中包括一个辅麦克风,辅麦克风采集的辅音频信号记为x
Ref,对应的频域信号记为X
Ref。左声道主麦克风的风噪程度信息可以通过主音频信号x
L与辅音频信号x
Ref之间的信号相关性确定,右声道主麦克风的风噪程度信息可以通过主音频信号x
R与辅音频信号x
Ref之间的信号相关性确定。示例性的,相关性计算可以采用经典的互谱计算方式,参见公式一。相关性的取值范围在0~1之间,取值越接近1,相关性越好,说明主麦克风受风噪的影响越低。需要说明,在本申请各个实施例中,为了便于区分,主音频信号对应的频域信号也可以称为主频域信号,辅音频信号对应的频域信号也可以称为辅频域信号。需要说明,相关性计算也可以采用其他的计算方式,例如,根据主音频信号和辅音频信号在时域上获取两者之间的信号相关性。
可选的,在另一种实现方式中,获取主麦克风的特征信息,可以包括:
根据主音频信号获取特征信息。
该种实现方式适用于电子设备中包括多个主麦克风的情况。在风噪场景中,由于主麦克风不具有拾音保护结构,风噪对主麦克风采集的主音频信号的影响均较大。根据不同主麦克风采集的主音频信号确定各个主麦克风的风噪程度信息,提升了确定主麦克风的风噪程度信息的准确性。
可选的,电子设备包括至少两个主麦克风,根据主音频信号获取特征信息,可以包括:
获取第一主音频信号对应的第一频域信号和第二主音频信号对应的第二频域信号。其中,第一主音频信号和第二主音频信号为至少两个主麦克风中的任意两个主麦克风分别采集的主音频信号。
基于第一频域信号和第二频域信号之间的相关性,确定风噪程度信息。
举例说明,还参见图1。其中,各个信号的含义可以参见上述公式一的相关描述,此处不再赘述。在本示例中,左声道主麦克风或右声道主麦克风的风噪程度信息可以通过主音频信号x
L与主音频信号x
R之间的信号相关性确定。示例性的,相关性计算可以采用经典的互谱计算方式,参见公式二。相关性的取值范围在0~1之间,取值越接近1,相关性越好,说明主麦克风受风噪的影响越小。
可选的,为了采集不同方向上的声音以及提升主麦克风整体抗风噪的性能,至少两个主麦克风可以分别位于电子设备的不同侧。
可选的,若至少两个主麦克风的数量为两个,两个主麦克风可以分别位于电子设备的第一侧和第一侧的对侧。本实施例对第一侧在电子设备的具体位置不做限定,可以根据电子设备的形状和声音采集需求设置。
可选的,根据主麦克风的特征信息,将主音频信号和调整后的辅音频信号合成目标音频信号,可以包括:
根据特征信息确定修复系数。修复系数包括主音频信号对应的第一权值,和/或,调整后的辅音频信号对应的第二权值。
根据第一权值和/或第二权值,以及调整后的辅音频信号,合成目标音频信号。
具体的,主麦克风的风噪程度信息可以指示主麦克风受风噪影响的程度。通常,主麦克风受风噪影响的程度越大,需要对主麦克风采集的主音频信号进行修复的程度也越大。根据主麦克风的风噪程度信息确定主麦克风和/或辅麦克风分别对应的修复系数,根据修复系数、主音频信号和调整后的辅音频信号合成目标音频信号,提升了电子设备基于主麦克风和辅麦克风采集音频的质量,提高了拾音效果。
可选的,修复系数可以包括第一权值和第二权值。
相应的,根据第一权值和/或第二权值,以及调整后的辅音频信号,合成目标音频信号,可以包括:
获取主音频信号对应的主频域信号以及调整后的辅音频信号对应的辅频域信号。
将第一修正信号与第二修正信号的和确定为目标音频信号。其中,第一 修正信号为主频域信号与第一权值的乘积,第二修正信号为辅频域信号与第二权值的乘积。
在该种实现方式中,将一定比重的主音频信号结合一定比重的调整后的辅音频信号合成最终的目标音频信号,提升了电子设备基于主麦克风和辅麦克风采集音频的质量。
举例说明,结合图1和图4。其中,各个信号的含义可以参见上述公式一的相关描述,此处不再赘述。调整后的辅音频信号可以记为x
Ref’,对应的频域信号记为X
Ref’。左声道主麦克风的风噪程度信息记为R
L,右声道主麦克风的风噪程度信息记为R
R,可以参见上述公式一或公式二。根据左声道主麦克风的风噪程度信息确定的左声道主麦克风对应的第一权值记为ratio
L,确定的辅麦克风对应的第二权值记为ratio
Ref1。根据右声道主麦克风的风噪程度信息确定的右声道主麦克风对应的第一权值记为ratio
R,确定的辅麦克风对应的第二权值记为ratio
Ref2。左声道主麦克风和右声道主麦克风分别对应的目标音频信号可以参见公式三。
X′
L=ratio
LX
L+ratio
Ref1X′
Ref 公式三
X′
R=ratio
RX
R+ratio
Ref2X′
Ref
可选的,主麦克风的风噪程度信息与第一权值具有映射函数关系,第一权值与第二权值的和等于1,映射函数关系包括下列中的任意一项:线性函数关系、指数函数关系和对数函数关系。
比如,在上述公式三中,ratio
Ref1=1-ratio
L,ratio
Ref2=1-ratio
R。
下面结合图5对映射函数关系进行示例性说明。图5为本申请实施例提供的映射函数关系的示意图,示出了主麦克风的风噪程度信息与权值(具体为第一权值)之间的三种映射函数关系。其中,映射1示出了对数函数关系,映射2示出了线性函数关系,映射3示出了指数函数关系。当主麦克风的风噪程度信息相同时,在对数函数关系中,目标音频信号中主音频信号的第一权值更大,相应的,调整后的辅音频信号的第二权值更小;在指数函数关系中,目标音频信号中主音频信号的第一权值更小,相应的,调整后的辅音频信号的第二权值更大;在线性函数关系中,目标音频信号中主音频信号的第一权值与调整后的辅音频信号的第二权值为固定比例。可以根据不同的音频采集需求确定映射函数关系,比如,期望还原音频采集真实环境时,可以采 用对数函数关系。
可选的,在本申请的又一个实施例中,结合过载场景对本申请提供的音频处理方法进行说明。在本实施例中,主麦克风的特征信息可以为主麦克风的过载程度信息。
可选的,获取主麦克风的特征信息,可以包括:
根据主音频信号获取特征信息。
具体的,在过载场景中,通常可以根据麦克风自身采集到的音频信号确定麦克风是否发生过载。根据主音频信号确定主麦克风的过载程度信息,实现方式简单易行。
可选的,根据主音频信号获取特征信息,可以包括:
获取主音频信号在第一预设时间段内的信号幅值。
根据第一预设时间段内的信号幅值,确定过载程度信息。
其中,本实施例对第一预设时间段的取值不做限定。
具体的,主麦克风采集的主音频信号的信号幅值是时刻变化的,具有波动性。通过获取主音频信号在第一预设时间段内的信号幅值,根据第一预设时间段内的信号幅值确定过载程度信息,减弱了信号幅值波动对判断准确性的影响,提升了确定过载程度信息的准确性。比如,可以根据第一预设时间段内主音频信号的信号幅值的最大值、平均值或加权平均值,确定过载程度信息。
可选的,过载程度信息是根据第一预设时间段内信号幅值的绝对值的最大值确定的。
举例说明,可以参见图1或图2。以电子设备中的任意一个主麦克风为例,主麦克风采集的主音频信号记为x
M,对应的频域信号记为X
M。主音频信号的信号幅值的绝对值记为|x
M|,第一预设时间段内主音频信号的信号幅值的绝对值的最大值记为max|x
M|。主麦克风的过载程度信息可以根据max|x
M|确定。可选的,在一种实现方式中,主麦克风的过载程度信息可以为max|x
M|。
可选的,根据主麦克风的特征信息,将主音频信号和调整后的辅音频信号合成目标音频信号,可以包括:
根据特征信息确定修复系数。修复系数包括主音频信号对应的第一权值,和/或,调整后的辅音频信号对应的第二权值。
根据第一权值和/或第二权值,以及调整后的辅音频信号,合成目标音频信号。
具体的,主麦克风的过载程度信息可以指示主麦克风是否发生过载或过载的程度。通常,主麦克风过载的程度越大,需要对主麦克风采集的主音频信号进行修复的程度越大。根据主麦克风的过载程度信息确定主麦克风和/或辅麦克风分别对应的修复系数,根据修复系数、主音频信号和调整后的辅音频信号合成目标音频信号,提升了电子设备基于主麦克风和辅麦克风采集音频的质量,提高了拾音效果。
可选的,修复系数可以包括第二权值。
相应的,根据第一权值和/或第二权值,以及调整后的辅音频信号,合成目标音频信号,可以包括:
根据过载程度信息确定主麦克风是否过载。
若确定主麦克风过载,则将调整后的辅音频信号与第二权值的乘积确定为目标音频信号。
示例性的,实现原理可以参见图6。在该种实现方式中,如果主麦克风发生过载,则采用一定比重的调整后的辅音频信号合成最终的目标音频信号,而且,调整后的辅音频信号对应的第二权值是根据主麦克风的过载程度信息确定的,提升了电子设备基于主麦克风和辅麦克风采集音频的质量。
可选的,根据过载程度信息确定主麦克风是否过载,可以包括:
确定过载程度信息是否大于第一预设阈值。
若过载程度信息大于第一预设阈值,则确定主麦克风过载。
若过载程度信息小于或等于第一预设阈值,则确定主麦克风没有过载。
其中,本实施例对第一预设阈值的取值不做限定。可选的,第一预设阈值可以与记录音频信号的量化位数相关。量化位数是指模拟量转换成数字量之后的数据位数,它决定了模拟信号数字化以后的动态范围。比如,量化位数为16位时,音频信号的幅值范围在-32768~32767之间,第一预设阈值可以为2^15-1=32767。相似的,量化位数为8位时,第一预设阈值可以为2^7-1=127。量化位数为32位时,第一预设阈值可以为2^31-1。其中,^表示幂运算。
举例说明。比如,主麦克风的过载程度信息为第一预设时间段内主音频信号的信号幅值的绝对值的最大值,记为max|x
M|。以16bits录制声音为例,第一预设阈值可以为32767。如果max|x
M|>32767,可以确定主麦克风过载。如果max|x
M|≤32767,则可以确定主麦克风没有过载。
下面对调整后的辅音频信号对应的第二权值的实现方式进行说明。
可选的,第二权值可以为下列中的任意一项:
其中,A=2
m-1,m表示主音频信号的量化位数。B表示调整后的辅音频信号在第三预设时间段内的信号幅值的绝对值的最大值。
E表示主音频信号在第三预设时间段内的信号幅值的均方值。ratio表示用于根据用户需求设定的信号缩放系数,ratio>0。
其中,本实施例对m、ratio、第三预设时间段的取值不做限定。
举例说明。主麦克风采集的主音频信号记为x
M,对应的频域信号记为X
M。辅麦克风采集的辅音频信号记为x
Ref,对应的频域信号记为X
Ref。调整后的辅音频信号记为x
Ref’,对应的频域信号记为X
Ref’。目标音频信号可以记为x′
M。假设,量化位数m为16,A=2^15=32768。B记为max
Ref′或者max|x
Ref’|,E记为
R的取值范围为0~1,R越接近1,说明主麦克风的过载程度越高。
在第一种实现方式中,第二权值可以为
目标音频信号
在该种实现方式中,直接将调整后的辅音频信号放大到极限作为目标音频信号。在第二种实现方式中,第二权值可以为
目标音频信号
在第三种实现方式中,第 二权值可以为
目标音频信号
具体的,R的取值范围为0~1,R的取值越大,说明主麦克风的过载程度越高。如果R的取值大于或等于第二预设阈值,则可以直接将调整后的辅音频信号放大到极限作为目标音频信号。
其中,本实施例对第二预设阈值的取值不做限定。
可选的,在本申请的又一个实施例中,结合堵麦场景对本申请提供的音频处理方法进行说明。在本实施例中,主麦克风的特征信息可以为主麦克风的堵麦程度信息。
可选的,获取主麦克风的特征信息,可以包括:
根据主音频信号和辅音频信号,确定特征信息。
具体的,在堵麦场景中,如果主麦克风发生堵麦,而辅麦克风具有拾音保护结构,因此,主麦克风采集的主音频信号和辅麦克风采集的辅音频信号之间的差别会较大。根据主音频信号和辅音频信号确定主麦克风的堵麦程度信息,提升了确定主麦克风的堵麦程度信息的准确性。
可选的,堵麦程度信息可以是根据主音频信号的信号能量和辅音频信号的信号能量之间的大小关系确定的。
具体的,在堵麦场景中,如果主麦克风发生堵麦,那么,主麦克风采集的主音频信号的信号能量将较小。根据主音频信号的信号能量和辅音频信号的信号能量之间的大小关系确定主麦克风的堵麦程度信息,提升了确定主麦克风的堵麦程度信息的准确性。
其中,本实施例对获取主音频信号的信号能量和获取辅音频信号的信号能量的实现方式不做限定,可以根据主音频信号或辅音频信号在时域上获取信号能量,也可以根据主音频信号对应的频域信号或辅音频信号对应的频域信号在频域上获取信号能量。
可选的,堵麦程度信息可以是根据主音频信号的信号能量和辅音频信号 的信号能量之间的比值确定的。
具体的,可以持续获取主音频信号的信号能量和辅音频信号的信号能量,得到两者之间的比值。可选的,该比值可以是主音频信号的信号能量与辅音频信号的信号能量的比值,也可以是辅音频信号的信号能量与主音频信号的信号能量的比值。可选的,该比值可以是相同时间段内主音频信号的信号能量的平均值与辅音频信号的信号能量的平均值之间的比值。如果比值突然发生变化,例如,主音频信号的信号能量与辅音频信号的信号能量的比值突然变小,则主麦克风可能发生堵麦。堵麦程度信息可以根据该比值确定,例如,可以直接为该比值,或者一段时间内该比值的平均值或加权平均值,本实施例对此不作限定。
举例说明。对于电子设备中的任意一个主麦克风,主麦克风采集的主音频信号的信号能量可以表示为Eng
main,辅麦克风采集的辅音频信号的信号能量可以表示为Eng
ref。堵麦程度信息表示为ratio,可以参见公式四。
可选的,根据主麦克风的特征信息,将主音频信号和调整后的辅音频信号合成目标音频信号,可以包括:
根据特征信息确定修复系数。修复系数包括主音频信号对应的第一权值,和/或,调整后的辅音频信号对应的第二权值。
根据第一权值和/或第二权值,以及调整后的辅音频信号,合成目标音频信号。
具体的,主麦克风的堵麦程度信息可以指示主麦克风是否堵麦或者主麦克风堵麦的程度。根据主麦克风的堵麦信息确定主麦克风和/或辅麦克风分别对应的修复系数,根据修复系数、主音频信号和调整后的辅音频信号合成目标音频信号,提升了电子设备基于主麦克风和辅麦克风采集音频的质量。
可选的,修复系数可以包括第二权值。
相应的,根据第一权值和/或第二权值,以及调整后的辅音频信号,合成目标音频信号,可以包括:
根据堵麦程度信息确定主麦克风是否发生堵麦。
若确定主麦克风发生堵麦,则将调整后的辅音频信号与第二权值的乘积确定为目标音频信号。
示例性的,实现原理可以参见图7。在该种实现方式中,如果主麦克风发生堵麦,则采用一定比重的调整后的辅音频信号合成最终的目标音频信号,而且,调整后的辅音频信号对应的第二权值是根据主麦克风的堵麦程度信息确定的,提升了电子设备基于主麦克风和辅麦克风采集音频的质量。
可选的,第二权值可以为1。
由于主麦克风发生堵麦,则直接将调整后的辅音频信号作为目标音频信号,方法简单易行。
可选的,根据堵麦程度信息确定主麦克风是否发生堵麦,可以包括:
获取第二预设时间段内的多个堵麦程度信息。
获取多个堵麦程度信息的平均值。
确定平均值是否小于预设均值。
若平均值小于预设均值,则确定主麦克风发生堵麦。
若平均值大于或等于预设均值,则确定主麦克风没有发生堵麦。
其中,本实施例对第二预设时间段、预设均值的取值不做限定。
可选的,在上述图3所示实施例或者在适用于风噪场景、堵麦场景的实施例的基础上,本申请又一个实施例提供一种音频处理方法。在本实施例中,为了进一步提升合成目标音频信号的质量,可以获取主麦克风在不同频段上分别对应的特征信息,从而基于主麦克风在每个频段上的特征信息,分频段合成目标音频信号。
在本实施例中,获取主麦克风的特征信息,可以包括:
对每个目标信号进行频域变换,得到目标信号分别在多个频段上的频率分量。其中,目标信号包括主音频信号,或者,目标信号包括主音频信号和辅音频信号。
根据目标信号分别在多个频段上的频率分量,获得主麦克风在每个频段上的特征信息。
相应的,根据主麦克风的特征信息,将主音频信号和调整后的辅音频信号合成目标音频信号,可以包括:
获取调整后的辅音频信号分别在多个频段上的频率分量。
针对多个频段中的每个频段,根据主麦克风在该频段上的特征信息,将 主音频信号在该频段上的频率分量和调整后的辅音频信号在该频段上的频率分量,合成目标音频信号在该频段上的频率分量。
其中,本实施例对多个频段的划分方式不做限定。例如,可以参见S302中的相关说明,此处不再赘述。
其中,针对每个频段,获得主麦克风在每个频段上的特征信息,可以参见上述关于获得主麦克风的特征信息的相关说明,原理相似,此处不再赘述。
其中,针对每个频段,根据主麦克风在该频段上的特征信息,将主音频信号在该频段上的频率分量和调整后的辅音频信号在该频段上的频率分量,合成目标音频信号在该频段上的频率分量,可以参见上述关于根据主麦克风的特征信息,将主音频信号和调整后的辅音频信号合成目标音频信号的相关说明,原理相似,此处不再赘述。
举例说明,以堵麦场景中获得主麦克风在不同频段上的特征信息为例进行示例性说明。
音频信号在高频段和低频段具有不同的衰减特性,为了提高确定主麦克风的堵麦程度信息的准确性,可以分频段确定音频信号的信号能量,根据不同频段的音频信号的信号能量确定主麦克风在不同频段上的堵麦程度信息。
在本示例中,能量检测可以分为高频与低频两个频段,本实施例对高频段和低频段的划分不做限定,例如,可以选择以2kHz为分界线,低于2kHz为低频段,高于2kHz为高频段。主麦克风采集的主音频信号在高频段的信号能量可以表示为Eng
main,H,辅麦克风采集的辅音频信号在高频段的信号能量可以表示为Eng
ref,H。主麦克风采集的主音频信号在低频段的信号能量可以表示为Eng
main,L,辅麦克风采集的辅音频信号在低频段的信号能量可以表示为Eng
ref,L。高频段对应的堵麦程度信息ratio
H和低频段对应的堵麦程度信息ratio
L可以参见公式五。
可选的,在上述各个实施例的基础上,本申请又一个实施例提供一种音频处理方法。在本实施例中,为了进一步提升合成目标音频信号的质量,对于当前时间段内获取的主麦克风的特征信息,可以结合当前时间段之间的历 史信息,对当前时间段主麦克风的特征信息进行修正,从而提升获取的主麦克风的特征信息的准确性,进而基于更准确的主麦克风的特征信息合成目标音频信号,提升目标音频信号的质量。
在本实施例中,根据主麦克风的特征信息,将主音频信号和调整后的辅音频信号合成目标音频信号,可以包括:
获取当前时间段内主麦克风的第一特征信息,以及与当前时间段相邻的上一个时间段内主麦克风的第二特征信息。
根据第二特征信息修正第一特征信息。
根据修正后的第一特征信息,将主音频信号和调整后的辅音频信号合成目标音频信号。
可选的,根据第二特征信息修正第一特征信息,可以包括:
获取第一特征信息的权值和第二特征信息的权值。
根据第一特征信息、第一特征信息的权值、第二特征信息和第二特征信息的权值,修正第一特征信息。
举例说明,以风噪场景作为示例。当前时间段内主麦克风的第一风噪程度信息可以记为R1,与当前时间段相邻的上一个时间段内主麦克风的第二风噪程度信息可以记为R0。第一风噪程度信息R1的权值可以记为a1,第二风噪程度信息R0的权值可以记为a0。比如,a0=1-a1。那么,修正后的第一风噪程度信息R1’可以为a1*R1+(1-a1)*R0。
图8为本申请实施例提供的电子设备的一种结构示意图。如图8所示,本实施例提供的电子设备,可以包括主麦克风(未示出)和辅麦克风(未示出),所述主麦克风的拾音腔(未示出)和所述辅麦克风的拾音腔(未示出)均与所述电子设备所处的外部环境连通;所述电子设备还包括拾音保护结构(未示出),所述拾音保护结构被设置为用于减弱从所述外部环境进入所述辅麦克风的拾音腔的气流,和/或,阻隔非气体物质进入所述辅麦克风的拾音腔;
所述电子设备还包括存储器82和处理器81;
所述存储器82,用于存储程序代码;
所述处理器81,调用所述程序代码,当所述程序代码被执行时,用于执 行以下操作:
获取所述主麦克风采集的主音频信号和所述辅麦克风采集的辅音频信号;
根据所述主音频信号和所述辅音频信号合成目标音频信号。
可选的,所述处理器81还用于:
在所述辅音频信号中确定处于不同频段的多个音频分量;
根据多个音频分量中任一所述音频分量对应的频段,以及预设的频段和频响参数的调参量的对应关系,确定所述音频分量的频响参数的目标调参量,并根据所述目标调参量调整所述音频分量的频响参数;其中,所述频响参数包括幅值参数和/或相位参数,所述对应关系是基于所述电子设备中所述主麦克风和所述辅麦克风分别对样本音源进行音频采集得到的样本音频的频响参数的偏差确定的;
所述处理器81具体用于:
根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号。
可选的,所述处理器81还用于:
获取所述主麦克风的特征信息;所述特征信息包括以下一种或多种信息:风噪程度信息、堵麦程度信息或过载程度信息;
所述处理器81具体用于:
根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号。
可选的,所述处理器81具体用于:
根据所述主音频信号和所述辅音频信号,确定所述特征信息。
可选的,所述特征信息包括风噪程度信息,所述风噪程度信息是根据所述主音频信号与所述辅音频信号之间的信号相关性确定的。
可选的,所述特征信息包括堵麦程度信息,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的大小关系确定的。
可选的,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的比值确定的。
可选的,所述处理器81具体用于:
根据所述主音频信号获取所述特征信息。
可选的,所述电子设备包括至少两个主麦克风,所述特征信息包括风噪程度信息;
所述处理器81具体用于:
获取第一主音频信号对应的第一频域信号和第二主音频信号对应的第二频域信号;其中,所述第一主音频信号和所述第二主音频信号为所述至少两个主麦克风中的任意两个主麦克风分别采集的主音频信号;
基于所述第一频域信号和所述第二频域信号之间的相关性,确定所述风噪程度信息。
可选的,所述至少两个主麦克风分别位于所述电子设备的不同侧。
可选的,所述至少两个主麦克风的数量为两个,两个所述主麦克风分别位于所述电子设备的第一侧和所述第一侧的对侧。
可选的,所述特征信息包括过载程度信息;
所述处理器81具体用于:
获取所述主音频信号在第一预设时间段内的信号幅值;
根据所述第一预设时间段内的信号幅值,确定所述过载程度信息。
可选的,所述过载程度信息是根据所述第一预设时间段内所述信号幅值的绝对值的最大值确定的。
可选的,所述处理器81具体用于:
根据所述特征信息确定修复系数;所述修复系数包括所述主音频信号对应的第一权值,和/或,调整后的所述辅音频信号对应的第二权值;
根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号。
可选的,所述特征信息包括风噪程度信息,所述修复系数包括所述第一权值和所述第二权值;
所述处理器81具体用于:
获取所述主音频信号对应的主频域信号以及调整后的所述辅音频信号对应的辅频域信号;
将第一修正信号与第二修正信号的和确定为所述目标音频信号;其中,所述第一修正信号为所述主频域信号与所述第一权值的乘积,所述第二修正信号为所述辅频域信号与所述第二权值的乘积。
可选的,所述风噪程度信息与所述第一权值具有映射函数关系,所述第一权值与所述第二权值的和等于1,所述映射函数关系包括下列中的任意一项:线性函数关系、指数函数关系和对数函数关系。
可选的,所述特征信息包括堵麦程度信息,所述修复系数包括所述第二权值;
所述处理器81具体用于:
根据所述堵麦程度信息确定所述主麦克风是否发生堵麦;
若确定所述主麦克风发生堵麦,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
可选的,所述处理器81具体用于:
获取第二预设时间段内的多个堵麦程度信息;
获取所述多个堵麦程度信息的平均值;
确定所述平均值是否小于预设均值;
若所述平均值小于所述预设均值,则确定所述主麦克风发生堵麦;
若所述平均值大于或等于所述预设均值,则确定所述主麦克风没有发生堵麦。
可选的,所述第二权值为1。
可选的,所述特征信息包括过载程度信息,所述修复系数包括所述第二权值;
所述处理器81具体用于:
根据所述过载程度信息确定所述主麦克风是否过载;
若确定所述主麦克风过载,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
可选的,所述处理器81具体用于:
确定所述过载程度信息是否大于第一预设阈值;
若所述过载程度信息大于所述第一预设阈值,则确定所述主麦克风过载;
若所述过载程度信息小于或等于所述第一预设阈值,则确定所述主麦克风没有过载。
可选的,所述处理器81具体用于:
对每个目标信号进行频域变换,得到所述目标信号分别在多个频段上的 频率分量;其中,所述目标信号包括所述主音频信号,或者,所述目标信号包括所述主音频信号和所述辅音频信号;
根据所述目标信号分别在多个频段上的频率分量,获得所述主麦克风在每个频段上的特征信息;
所述处理器81具体用于:
获取所述调整后的所述辅音频信号分别在所述多个频段上的频率分量;
针对所述多个频段中的每个频段,根据所述主麦克风在该频段上的特征信息,将所述主音频信号在该频段上的频率分量和所述调整后的所述辅音频信号在该频段上的频率分量,合成所述目标音频信号在该频段上的频率分量。
可选的,所述处理器81具体用于:
获取当前时间段内所述主麦克风的第一特征信息,以及与所述当前时间段相邻的上一个时间段内所述主麦克风的第二特征信息;
根据所述第二特征信息修正所述第一特征信息;
根据修正后的所述第一特征信息,将所述主音频信号和所述调整后的所述辅音频信号合成所述目标音频信号。
可选的,所述处理器81具体用于:
获取所述第一特征信息的权值和所述第二特征信息的权值;
根据所述第一特征信息、所述第一特征信息的权值、所述第二特征信息和所述第二特征信息的权值,修正所述第一特征信息。
可选的,所述主麦克风相对于所述辅麦克风靠近所述电子设备的外壳。
可选的,所述拾音保护结构包括防风结构,所述辅麦克风设置在所述防风结构中。
可选的,所述防风结构包括中空的防风罩体和用于支撑所述防风罩体的支撑件,所述辅麦克风设置在所述防风罩体的空腔中。
可选的,所述拾音保护结构包括防尘结构,所述辅麦克风设置在所述防尘结构中。
可选的,所述防尘结构包括至少一层罩设在所述辅麦克风外侧的过滤网。
本实施例提供的电子设备,可以执行本申请图3-图7所示实施例提供的音频处理方法,技术原理和技术效果相似,此处不再赘述。
可选的,本申请的另一个实施例还提供一种音频处理方法。在本实施例中,音频处理方法可以包括:
获取主音频信号和辅音频信号,所述主音频信号和所述辅音频信号是同时对同一音源采集得到的,所述辅音频信号与所述主音频信号在特定频率的幅值和/或相位不同;
在所述辅音频信号中确定处于不同频段的多个音频分量;
根据所述多个音频分量中任一所述音频分量对应的频段,以及预设的频段和频响参数的调参量的对应关系,确定所述音频分量的频响参数的目标调参量,并根据所述目标调参量调整所述音频分量的幅值参数和/或相位参数;
根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号。
可选的,所述主音频信号由设置在电子设备的主麦克风采集,所述辅音频信号由设置在所述电子设备的辅麦克风采集。
可选的,所述对应关系是基于所述电子设备中所述主麦克风和所述辅麦克风分别对样本音源进行音频采集得到的样本音频的频响参数的偏差确定的。
可选的,所述方法还包括:
获取所述主麦克风的特征信息;所述特征信息包括以下一种或多种信息:风噪程度信息、堵麦程度信息或过载程度信息;
所述根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号,包括:
根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号。
可选的,所述获取所述主麦克风的特征信息,包括:
根据所述主音频信号和所述辅音频信号,确定所述特征信息。
可选的,所述特征信息包括风噪程度信息,所述风噪程度信息是根据所述主音频信号与所述辅音频信号之间的信号相关性确定的。
可选的,所述特征信息包括堵麦程度信息,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的大小关系确定的。
可选的,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的比值确定的。
可选的,所述获取所述主麦克风的特征信息,包括:
根据所述主音频信号获取所述特征信息。
可选的,所述电子设备包括至少两个主麦克风,所述特征信息包括风噪程度信息;
所述根据所述主音频信号获取所述特征信息,包括:
获取第一主音频信号对应的第一频域信号和第二主音频信号对应的第二频域信号;其中,所述第一主音频信号和所述第二主音频信号为所述至少两个主麦克风中的任意两个主麦克风分别采集的主音频信号;
基于所述第一频域信号和所述第二频域信号之间的相关性,确定所述风噪程度信息。
可选的,所述特征信息包括过载程度信息;
所述根据所述主音频信号获取所述特征信息,包括:
获取所述主音频信号在第一预设时间段内的信号幅值;
根据所述第一预设时间段内的信号幅值,确定所述过载程度信息。
可选的,所述过载程度信息是根据所述第一预设时间段内所述信号幅值的绝对值的最大值确定的。
可选的,所述根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号,包括:
根据所述特征信息确定修复系数;所述修复系数包括所述主音频信号对应的第一权值,和/或,调整后的所述辅音频信号对应的第二权值;
根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号。
可选的,所述特征信息包括风噪程度信息,所述修复系数包括所述第一权值和所述第二权值;
所述根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号,包括:
获取所述主音频信号对应的主频域信号以及调整后的所述辅音频信号对应的辅频域信号;
将第一修正信号与第二修正信号的和确定为所述目标音频信号;其中,所述第一修正信号为所述主频域信号与所述第一权值的乘积,所述第二修正 信号为所述辅频域信号与所述第二权值的乘积。
可选的,所述风噪程度信息与所述第一权值具有映射函数关系,所述第一权值与所述第二权值的和等于1,所述映射函数关系包括下列中的任意一项:线性函数关系、指数函数关系和对数函数关系。
可选的,所述特征信息包括堵麦程度信息,所述修复系数包括所述第二权值;
所述根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号,包括:
根据所述堵麦程度信息确定所述主麦克风是否发生堵麦;
若确定所述主麦克风发生堵麦,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
可选的,所述根据所述堵麦程度信息确定所述主麦克风是否发生堵麦,包括:
获取第二预设时间段内的多个堵麦程度信息;
获取所述多个堵麦程度信息的平均值;
确定所述平均值是否小于预设均值;
若所述平均值小于所述预设均值,则确定所述主麦克风发生堵麦;
若所述平均值大于或等于所述预设均值,则确定所述主麦克风没有发生堵麦。
可选的,所述第二权值为1。
可选的,所述特征信息包括过载程度信息,所述修复系数包括所述第二权值;
所述根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号,包括:
根据所述过载程度信息确定所述主麦克风是否过载;
若确定所述主麦克风过载,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
可选的,所述根据所述过载程度信息确定所述主麦克风是否过载,包括:
确定所述过载程度信息是否大于第一预设阈值;
若所述过载程度信息大于所述第一预设阈值,则确定所述主麦克风过载;
若所述过载程度信息小于或等于所述第一预设阈值,则确定所述主麦克风没有过载。
可选的,所述获取所述主麦克风的特征信息,包括:
对每个目标信号进行频域变换,得到所述目标信号分别在多个频段上的频率分量;其中,所述目标信号包括所述主音频信号,或者,所述目标信号包括所述主音频信号和所述辅音频信号;
根据所述目标信号分别在多个频段上的频率分量,获得所述主麦克风在每个频段上的特征信息;
所述根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号,包括:
获取所述调整后的所述辅音频信号分别在所述多个频段上的频率分量;
针对所述多个频段中的每个频段,根据所述主麦克风在该频段上的特征信息,将所述主音频信号在该频段上的频率分量和所述调整后的所述辅音频信号在该频段上的频率分量,合成所述目标音频信号在该频段上的频率分量。
可选的,所述根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号,包括:
获取当前时间段内所述主麦克风的第一特征信息,以及与所述当前时间段相邻的上一个时间段内所述主麦克风的第二特征信息;
根据所述第二特征信息修正所述第一特征信息;
根据修正后的所述第一特征信息,将所述主音频信号和所述调整后的所述辅音频信号合成所述目标音频信号。
可选的,所述根据所述第二特征信息修正所述第一特征信息,包括:
获取所述第一特征信息的权值和所述第二特征信息的权值;
根据所述第一特征信息、所述第一特征信息的权值、所述第二特征信息和所述第二特征信息的权值,修正所述第一特征信息。
本实施例提供的音频处理方法,相关描述可以参见上述图3-图7所示实施例中的相关描述,技术原理和技术效果相似,此处不再赘述。
可选的,本申请的另一个实施例还提供一种电子设备,电子设备的结构可以参见图8。在本实施例中,电子设备可以包括:存储器82和处理器81;
所述存储器,用于存储程序代码;
所述处理器,调用所述程序代码,当所述程序代码被执行时,用于执行以下操作:
获取主音频信号和辅音频信号,所述主音频信号和所述辅音频信号是同时对同一音源采集得到的,所述辅音频信号与所述主音频信号在特定频率的幅值和/或相位不同;
在所述辅音频信号中确定处于不同频段的多个音频分量;
根据所述多个音频分量中任一所述音频分量对应的频段,以及预设的频段和频响参数的调参量的对应关系,确定所述音频分量的频响参数的目标调参量,并根据所述目标调参量调整所述音频分量的幅值参数和/或相位参数;
根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号。
可选的,所述主音频信号由设置在电子设备的主麦克风采集,所述辅音频信号由设置在所述电子设备的辅麦克风采集。
可选的,所述对应关系是基于所述电子设备中所述主麦克风和所述辅麦克风分别对样本音源进行音频采集得到的样本音频的频响参数的偏差确定的。
可选的,所述处理器还用于:
获取所述主麦克风的特征信息;所述特征信息包括以下一种或多种信息:风噪程度信息、堵麦程度信息或过载程度信息;
所述处理器具体用于:
根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号。
可选的,所述处理器具体用于:
根据所述主音频信号和所述辅音频信号,确定所述特征信息。
可选的,所述特征信息包括风噪程度信息,所述风噪程度信息是根据所述主音频信号与所述辅音频信号之间的信号相关性确定的。
可选的,所述特征信息包括堵麦程度信息,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的大小关系确定的。
可选的,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的比值确定的。
可选的,所述处理器具体用于:
根据所述主音频信号获取所述特征信息。
可选的,所述电子设备包括至少两个主麦克风,所述特征信息包括风噪程度信息;
所述处理器具体用于:
获取第一主音频信号对应的第一频域信号和第二主音频信号对应的第二频域信号;其中,所述第一主音频信号和所述第二主音频信号为所述至少两个主麦克风中的任意两个主麦克风分别采集的主音频信号;
基于所述第一频域信号和所述第二频域信号之间的相关性,确定所述风噪程度信息。
可选的,所述特征信息包括过载程度信息;
所述处理器具体用于:
获取所述主音频信号在第一预设时间段内的信号幅值;
根据所述第一预设时间段内的信号幅值,确定所述过载程度信息。
可选的,所述过载程度信息是根据所述第一预设时间段内所述信号幅值的绝对值的最大值确定的。
可选的,所述处理器具体用于:
根据所述特征信息确定修复系数;所述修复系数包括所述主音频信号对应的第一权值,和/或,调整后的所述辅音频信号对应的第二权值;
根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号。
可选的,所述特征信息包括风噪程度信息,所述修复系数包括所述第一权值和所述第二权值;
所述处理器具体用于:
获取所述主音频信号对应的主频域信号以及调整后的所述辅音频信号对应的辅频域信号;
将第一修正信号与第二修正信号的和确定为所述目标音频信号;其中,所述第一修正信号为所述主频域信号与所述第一权值的乘积,所述第二修正信号为所述辅频域信号与所述第二权值的乘积。
可选的,所述风噪程度信息与所述第一权值具有映射函数关系,所述第 一权值与所述第二权值的和等于1,所述映射函数关系包括下列中的任意一项:线性函数关系、指数函数关系和对数函数关系。
可选的,所述特征信息包括堵麦程度信息,所述修复系数包括所述第二权值;
所述处理器具体用于:
根据所述堵麦程度信息确定所述主麦克风是否发生堵麦;
若确定所述主麦克风发生堵麦,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
可选的,所述处理器具体用于:
获取第二预设时间段内的多个堵麦程度信息;
获取所述多个堵麦程度信息的平均值;
确定所述平均值是否小于预设均值;
若所述平均值小于所述预设均值,则确定所述主麦克风发生堵麦;
若所述平均值大于或等于所述预设均值,则确定所述主麦克风没有发生堵麦。
可选的,所述第二权值为1。
可选的,所述特征信息包括过载程度信息,所述修复系数包括所述第二权值;
所述处理器具体用于:
根据所述过载程度信息确定所述主麦克风是否过载;
若确定所述主麦克风过载,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
可选的,所述处理器具体用于:
确定所述过载程度信息是否大于第一预设阈值;
若所述过载程度信息大于所述第一预设阈值,则确定所述主麦克风过载;
若所述过载程度信息小于或等于所述第一预设阈值,则确定所述主麦克风没有过载。
可选的,所述处理器具体用于:
对每个目标信号进行频域变换,得到所述目标信号分别在多个频段上的频率分量;其中,所述目标信号包括所述主音频信号,或者,所述目标信号包 括所述主音频信号和所述辅音频信号;
根据所述目标信号分别在多个频段上的频率分量,获得所述主麦克风在每个频段上的特征信息;
所述处理器具体用于:
获取所述调整后的所述辅音频信号分别在所述多个频段上的频率分量;
针对所述多个频段中的每个频段,根据所述主麦克风在该频段上的特征信息,将所述主音频信号在该频段上的频率分量和所述调整后的所述辅音频信号在该频段上的频率分量,合成所述目标音频信号在该频段上的频率分量。
可选的,所述处理器具体用于:
获取当前时间段内所述主麦克风的第一特征信息,以及与所述当前时间段相邻的上一个时间段内所述主麦克风的第二特征信息;
根据所述第二特征信息修正所述第一特征信息;
根据修正后的所述第一特征信息,将所述主音频信号和所述调整后的所述辅音频信号合成所述目标音频信号。
可选的,所述处理器具体用于:
获取所述第一特征信息的权值和所述第二特征信息的权值;
根据所述第一特征信息、所述第一特征信息的权值、所述第二特征信息和所述第二特征信息的权值,修正所述第一特征信息。
需要说明,本申请实施例提供的各种实现方式可以相互结合,本申请对结合方式不作限定。本申请对电子设备、处理器和存储器的类型、对处理器的实现方式和电连接关系不做限定。例如,处理器可以通过引脚与麦克风电连接,获取主麦克风采集的主音频信号和辅麦克风采集的辅音频信号,并进行相应的处理。
应理解,处理器可以是通用处理器、数字信号处理器、专用集成电路、现场可编程门阵列或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件,可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成,或者用处理器中的硬件及软件模块组合执行完成。
在本申请实施例中,存储器可以是非易失性存储器,比如硬盘(hard disk drive,HDD)或固态硬盘(solid-state drive,SSD)等,还可以是易失性存储器(volatile memory),例如随机存取存储器(random access memory,RAM)。存储器是能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何介质,但不限于此。本申请实施例中的存储器还可以是电路或者其它任意能够实现存储功能的装置,用于存储程序指令和/或数据。
本领域普通技术人员可以理解:实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时,执行包括上述各方法实施例的步骤;而前述的存储介质包括:ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是:以上各实施例仅用以说明本申请实施例的技术方案,而非对其限制;尽管参照前述各实施例对本申请实施例进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分或者全部技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本申请实施例技术方案的范围。
Claims (105)
- 一种音频处理方法,其特征在于,应用于电子设备,所述电子设备包括主麦克风和辅麦克风,所述主麦克风的拾音腔和所述辅麦克风的拾音腔均与所述电子设备所处的外部环境连通;所述电子设备包括拾音保护结构,所述拾音保护结构被设置为用于减弱从所述外部环境进入所述辅麦克风的拾音腔的气流,和/或,阻隔非气体物质进入所述辅麦克风的拾音腔;所述方法包括:获取所述主麦克风采集的主音频信号和所述辅麦克风采集的辅音频信号;根据所述主音频信号和所述辅音频信号合成目标音频信号。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:在所述辅音频信号中确定处于不同频段的多个音频分量;根据所述多个音频分量中任一所述音频分量对应的频段,以及预设的频段和频响参数的调参量的对应关系,确定所述音频分量的频响参数的目标调参量,并根据所述目标调参量调整所述音频分量的频响参数;其中,所述频响参数包括幅值参数和/或相位参数,所述对应关系是基于所述电子设备中所述主麦克风和所述辅麦克风分别对样本音源进行音频采集得到的样本音频的频响参数的偏差确定的;所述根据所述主音频信号和所述辅音频信号合成目标音频信号,包括:根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号。
- 根据权利要求2所述的方法,其特征在于,所述方法还包括:获取所述主麦克风的特征信息;所述特征信息包括以下一种或多种信息:风噪程度信息、堵麦程度信息或过载程度信息;所述根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号,包括:根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号。
- 根据权利要求3所述的方法,其特征在于,所述获取所述主麦克风的特征信息,包括:根据所述主音频信号和所述辅音频信号,确定所述特征信息。
- 根据权利要求4所述的方法,其特征在于,所述特征信息包括风噪程 度信息,所述风噪程度信息是根据所述主音频信号与所述辅音频信号之间的信号相关性确定的。
- 根据权利要求4所述的方法,其特征在于,所述特征信息包括堵麦程度信息,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的大小关系确定的。
- 根据权利要求6所述的方法,其特征在于,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的比值确定的。
- 根据权利要求3所述的方法,其特征在于,所述获取所述主麦克风的特征信息,包括:根据所述主音频信号获取所述特征信息。
- 根据权利要求8所述的方法,其特征在于,所述电子设备包括至少两个主麦克风,所述特征信息包括风噪程度信息;所述根据所述主音频信号获取所述特征信息,包括:获取第一主音频信号对应的第一频域信号和第二主音频信号对应的第二频域信号;其中,所述第一主音频信号和所述第二主音频信号为所述至少两个主麦克风中的任意两个主麦克风分别采集的主音频信号;基于所述第一频域信号和所述第二频域信号之间的相关性,确定所述风噪程度信息。
- 根据权利要求9所述的方法,其特征在于,所述至少两个主麦克风分别位于所述电子设备的不同侧。
- 根据权利要求10所述的方法,其特征在于,所述至少两个主麦克风的数量为两个,两个所述主麦克风分别位于所述电子设备的第一侧和所述第一侧的对侧。
- 根据权利要求8所述的方法,其特征在于,所述特征信息包括过载程度信息;所述根据所述主音频信号获取所述特征信息,包括:获取所述主音频信号在第一预设时间段内的信号幅值;根据所述第一预设时间段内的信号幅值,确定所述过载程度信息。
- 根据权利要求12所述的方法,其特征在于,所述过载程度信息是根据所述第一预设时间段内所述信号幅值的绝对值的最大值确定的。
- 根据权利要求3所述的方法,其特征在于,所述根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号,包括:根据所述特征信息确定修复系数;所述修复系数包括所述主音频信号对应的第一权值,和/或,调整后的所述辅音频信号对应的第二权值;根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号。
- 根据权利要求14所述的方法,其特征在于,所述特征信息包括风噪程度信息,所述修复系数包括所述第一权值和所述第二权值;所述根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号,包括:获取所述主音频信号对应的主频域信号以及调整后的所述辅音频信号对应的辅频域信号;将第一修正信号与第二修正信号的和确定为所述目标音频信号;其中,所述第一修正信号为所述主频域信号与所述第一权值的乘积,所述第二修正信号为所述辅频域信号与所述第二权值的乘积。
- 根据权利要求15所述的方法,其特征在于,所述风噪程度信息与所述第一权值具有映射函数关系,所述第一权值与所述第二权值的和等于1,所述映射函数关系包括下列中的任意一项:线性函数关系、指数函数关系和对数函数关系。
- 根据权利要求14所述的方法,其特征在于,所述特征信息包括堵麦程度信息,所述修复系数包括所述第二权值;所述根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号,包括:根据所述堵麦程度信息确定所述主麦克风是否发生堵麦;若确定所述主麦克风发生堵麦,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
- 根据权利要求17所述的方法,其特征在于,所述根据所述堵麦程度信息确定所述主麦克风是否发生堵麦,包括:获取第二预设时间段内的多个堵麦程度信息;获取所述多个堵麦程度信息的平均值;确定所述平均值是否小于预设均值;若所述平均值小于所述预设均值,则确定所述主麦克风发生堵麦;若所述平均值大于或等于所述预设均值,则确定所述主麦克风没有发生堵麦。
- 根据权利要求17所述的方法,其特征在于,所述第二权值为1。
- 根据权利要求14所述的方法,其特征在于,所述特征信息包括过载程度信息,所述修复系数包括所述第二权值;所述根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号,包括:根据所述过载程度信息确定所述主麦克风是否过载;若确定所述主麦克风过载,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
- 根据权利要求20所述的方法,其特征在于,所述根据所述过载程度信息确定所述主麦克风是否过载,包括:确定所述过载程度信息是否大于第一预设阈值;若所述过载程度信息大于所述第一预设阈值,则确定所述主麦克风过载;若所述过载程度信息小于或等于所述第一预设阈值,则确定所述主麦克风没有过载。
- 根据权利要求3-11、14-19任一项所述的方法,其特征在于,所述获取所述主麦克风的特征信息,包括:对每个目标信号进行频域变换,得到所述目标信号分别在多个频段上的频率分量;其中,所述目标信号包括所述主音频信号,或者,所述目标信号包括所述主音频信号和所述辅音频信号;根据所述目标信号分别在多个频段上的频率分量,获得所述主麦克风在每个频段上的特征信息;所述根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号,包括:获取所述调整后的所述辅音频信号分别在所述多个频段上的频率分量;针对所述多个频段中的每个频段,根据所述主麦克风在该频段上的特征 信息,将所述主音频信号在该频段上的频率分量和所述调整后的所述辅音频信号在该频段上的频率分量,合成所述目标音频信号在该频段上的频率分量。
- 根据权利要求3-21任一项所述的方法,其特征在于,所述根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号,包括:获取当前时间段内所述主麦克风的第一特征信息,以及与所述当前时间段相邻的上一个时间段内所述主麦克风的第二特征信息;根据所述第二特征信息修正所述第一特征信息;根据修正后的所述第一特征信息,将所述主音频信号和所述调整后的所述辅音频信号合成所述目标音频信号。
- 根据权利要求23所述的方法,其特征在于,所述根据所述第二特征信息修正所述第一特征信息,包括:获取所述第一特征信息的权值和所述第二特征信息的权值;根据所述第一特征信息、所述第一特征信息的权值、所述第二特征信息和所述第二特征信息的权值,修正所述第一特征信息。
- 根据权利要求1-21任一项所述的方法,其特征在于,所述主麦克风相对于所述辅麦克风靠近所述电子设备的外壳。
- 根据权利要求1-21任一项所述的方法,其特征在于,所述拾音保护结构包括防风结构,所述辅麦克风设置在所述防风结构中。
- 根据权利要求26所述的方法,其特征在于,所述防风结构包括中空的防风罩体和用于支撑所述防风罩体的支撑件,所述辅麦克风设置在所述防风罩体的空腔中。
- 根据权利要求1-21任一项所述的方法,其特征在于,所述拾音保护结构包括防尘结构,所述辅麦克风设置在所述防尘结构中。
- 根据权利要求28所述的方法,其特征在于,所述防尘结构包括至少一层罩设在所述辅麦克风外侧的过滤网。
- 一种电子设备,其特征在于,包括主麦克风和辅麦克风,所述主麦克风的拾音腔和所述辅麦克风的拾音腔均与所述电子设备所处的外部环境连通;所述电子设备还包括拾音保护结构,所述拾音保护结构被设置为用于减弱从 所述外部环境进入所述辅麦克风的拾音腔的气流,和/或,阻隔非气体物质进入所述辅麦克风的拾音腔;所述电子设备还包括存储器和处理器;所述存储器,用于存储程序代码;所述处理器,调用所述程序代码,当所述程序代码被执行时,用于执行以下操作:获取所述主麦克风采集的主音频信号和所述辅麦克风采集的辅音频信号;根据所述主音频信号和所述辅音频信号合成目标音频信号。
- 根据权利要求30所述的电子设备,其特征在于,所述处理器还用于:在所述辅音频信号中确定处于不同频段的多个音频分量;根据多个音频分量中任一所述音频分量对应的频段,以及预设的频段和频响参数的调参量的对应关系,确定所述音频分量的频响参数的目标调参量,并根据所述目标调参量调整所述音频分量的频响参数;其中,所述频响参数包括幅值参数和/或相位参数,所述对应关系是基于所述电子设备中所述主麦克风和所述辅麦克风分别对样本音源进行音频采集得到的样本音频的频响参数的偏差确定的;所述处理器具体用于:根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号。
- 根据权利要求31所述的电子设备,其特征在于,所述处理器还用于:获取所述主麦克风的特征信息;所述特征信息包括以下一种或多种信息:风噪程度信息、堵麦程度信息或过载程度信息;所述处理器具体用于:根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号。
- 根据权利要求32所述的电子设备,其特征在于,所述处理器具体用于:根据所述主音频信号和所述辅音频信号,确定所述特征信息。
- 根据权利要求33所述的电子设备,其特征在于,所述特征信息包括风噪程度信息,所述风噪程度信息是根据所述主音频信号与所述辅音频信号之间的信号相关性确定的。
- 根据权利要求33所述的电子设备,其特征在于,所述特征信息包括堵麦程度信息,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的大小关系确定的。
- 根据权利要求35所述的电子设备,其特征在于,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的比值确定的。
- 根据权利要求32所述的电子设备,其特征在于,所述处理器具体用于:根据所述主音频信号获取所述特征信息。
- 根据权利要求37所述的电子设备,其特征在于,所述电子设备包括至少两个主麦克风,所述特征信息包括风噪程度信息;所述处理器具体用于:获取第一主音频信号对应的第一频域信号和第二主音频信号对应的第二频域信号;其中,所述第一主音频信号和所述第二主音频信号为所述至少两个主麦克风中的任意两个主麦克风分别采集的主音频信号;基于所述第一频域信号和所述第二频域信号之间的相关性,确定所述风噪程度信息。
- 根据权利要求38所述的电子设备,其特征在于,所述至少两个主麦克风分别位于所述电子设备的不同侧。
- 根据权利要求39所述的电子设备,其特征在于,所述至少两个主麦克风的数量为两个,两个所述主麦克风分别位于所述电子设备的第一侧和所述第一侧的对侧。
- 根据权利要求37所述的电子设备,其特征在于,所述特征信息包括过载程度信息;所述处理器具体用于:获取所述主音频信号在第一预设时间段内的信号幅值;根据所述第一预设时间段内的信号幅值,确定所述过载程度信息。
- 根据权利要求41所述的电子设备,其特征在于,所述过载程度信息是根据所述第一预设时间段内所述信号幅值的绝对值的最大值确定的。
- 根据权利要求32所述的电子设备,其特征在于,所述处理器具体用 于:根据所述特征信息确定修复系数;所述修复系数包括所述主音频信号对应的第一权值,和/或,调整后的所述辅音频信号对应的第二权值;根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号。
- 根据权利要求43所述的电子设备,其特征在于,所述特征信息包括风噪程度信息,所述修复系数包括所述第一权值和所述第二权值;所述处理器具体用于:获取所述主音频信号对应的主频域信号以及调整后的所述辅音频信号对应的辅频域信号;将第一修正信号与第二修正信号的和确定为所述目标音频信号;其中,所述第一修正信号为所述主频域信号与所述第一权值的乘积,所述第二修正信号为所述辅频域信号与所述第二权值的乘积。
- 根据权利要求44所述的电子设备,其特征在于,所述风噪程度信息与所述第一权值具有映射函数关系,所述第一权值与所述第二权值的和等于1,所述映射函数关系包括下列中的任意一项:线性函数关系、指数函数关系和对数函数关系。
- 根据权利要求43所述的电子设备,其特征在于,所述特征信息包括堵麦程度信息,所述修复系数包括所述第二权值;所述处理器具体用于:根据所述堵麦程度信息确定所述主麦克风是否发生堵麦;若确定所述主麦克风发生堵麦,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
- 根据权利要求46所述的电子设备,其特征在于,所述处理器具体用于:获取第二预设时间段内的多个堵麦程度信息;获取所述多个堵麦程度信息的平均值;确定所述平均值是否小于预设均值;若所述平均值小于所述预设均值,则确定所述主麦克风发生堵麦;若所述平均值大于或等于所述预设均值,则确定所述主麦克风没有发生 堵麦。
- 根据权利要求46所述的电子设备,其特征在于,所述第二权值为1。
- 根据权利要求43所述的电子设备,其特征在于,所述特征信息包括过载程度信息,所述修复系数包括所述第二权值;所述处理器具体用于:根据所述过载程度信息确定所述主麦克风是否过载;若确定所述主麦克风过载,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
- 根据权利要求49所述的电子设备,其特征在于,所述处理器具体用于:确定所述过载程度信息是否大于第一预设阈值;若所述过载程度信息大于所述第一预设阈值,则确定所述主麦克风过载;若所述过载程度信息小于或等于所述第一预设阈值,则确定所述主麦克风没有过载。
- 根据权利要求32-40、43-48任一项所述的电子设备,其特征在于,所述处理器具体用于:对每个目标信号进行频域变换,得到所述目标信号分别在多个频段上的频率分量;其中,所述目标信号包括所述主音频信号,或者,所述目标信号包括所述主音频信号和所述辅音频信号;根据所述目标信号分别在多个频段上的频率分量,获得所述主麦克风在每个频段上的特征信息;所述处理器具体用于:获取所述调整后的所述辅音频信号分别在所述多个频段上的频率分量;针对所述多个频段中的每个频段,根据所述主麦克风在该频段上的特征信息,将所述主音频信号在该频段上的频率分量和所述调整后的所述辅音频信号在该频段上的频率分量,合成所述目标音频信号在该频段上的频率分量。
- 根据权利要求32-50任一项所述的电子设备,其特征在于,所述处理器具体用于:获取当前时间段内所述主麦克风的第一特征信息,以及与所述当前时间段相邻的上一个时间段内所述主麦克风的第二特征信息;根据所述第二特征信息修正所述第一特征信息;根据修正后的所述第一特征信息,将所述主音频信号和所述调整后的所述辅音频信号合成所述目标音频信号。
- 根据权利要求52所述的电子设备,其特征在于,所述处理器具体用于:获取所述第一特征信息的权值和所述第二特征信息的权值;根据所述第一特征信息、所述第一特征信息的权值、所述第二特征信息和所述第二特征信息的权值,修正所述第一特征信息。
- 根据权利要求32-50任一项所述的电子设备,其特征在于,所述主麦克风相对于所述辅麦克风靠近所述电子设备的外壳。
- 根据权利要求32-50任一项所述的电子设备,其特征在于,所述拾音保护结构包括防风结构,所述辅麦克风设置在所述防风结构中。
- 根据权利要求55所述的电子设备,其特征在于,所述防风结构包括中空的防风罩体和用于支撑所述防风罩体的支撑件,所述辅麦克风设置在所述防风罩体的空腔中。
- 根据权利要求32-50任一项所述的电子设备,其特征在于,所述拾音保护结构包括防尘结构,所述辅麦克风设置在所述防尘结构中。
- 根据权利要求57所述的电子设备,其特征在于,所述防尘结构包括至少一层罩设在所述辅麦克风外侧的过滤网。
- 一种音频处理方法,其特征在于,包括:获取主音频信号和辅音频信号,所述主音频信号和所述辅音频信号是同时对同一音源采集得到的,所述辅音频信号与所述主音频信号在特定频率的幅值和/或相位不同;在所述辅音频信号中确定处于不同频段的多个音频分量;根据所述多个音频分量中任一所述音频分量对应的频段,以及预设的频段和频响参数的调参量的对应关系,确定所述音频分量的频响参数的目标调参量,并根据所述目标调参量调整所述音频分量的幅值参数和/或相位参数;根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号。
- 根据权利要求59所述的方法,其特征在于,所述主音频信号由设置 在电子设备的主麦克风采集,所述辅音频信号由设置在所述电子设备的辅麦克风采集。
- 根据权利要求60所述的方法,其特征在于,所述对应关系是基于所述电子设备中所述主麦克风和所述辅麦克风分别对样本音源进行音频采集得到的样本音频的频响参数的偏差确定的。
- 根据权利要求60所述的方法,其特征在于,所述方法还包括:获取所述主麦克风的特征信息;所述特征信息包括以下一种或多种信息:风噪程度信息、堵麦程度信息或过载程度信息;所述根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号,包括:根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号。
- 根据权利要求62所述的方法,其特征在于,所述获取所述主麦克风的特征信息,包括:根据所述主音频信号和所述辅音频信号,确定所述特征信息。
- 根据权利要求63所述的方法,其特征在于,所述特征信息包括风噪程度信息,所述风噪程度信息是根据所述主音频信号与所述辅音频信号之间的信号相关性确定的。
- 根据权利要求63所述的方法,其特征在于,所述特征信息包括堵麦程度信息,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的大小关系确定的。
- 根据权利要求65所述的方法,其特征在于,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的比值确定的。
- 根据权利要求62所述的方法,其特征在于,所述获取所述主麦克风的特征信息,包括:根据所述主音频信号获取所述特征信息。
- 根据权利要求67所述的方法,其特征在于,所述电子设备包括至少两个主麦克风,所述特征信息包括风噪程度信息;所述根据所述主音频信号获取所述特征信息,包括:获取第一主音频信号对应的第一频域信号和第二主音频信号对应的第二频域信号;其中,所述第一主音频信号和所述第二主音频信号为所述至少两个主麦克风中的任意两个主麦克风分别采集的主音频信号;基于所述第一频域信号和所述第二频域信号之间的相关性,确定所述风噪程度信息。
- 根据权利要求67所述的方法,其特征在于,所述特征信息包括过载程度信息;所述根据所述主音频信号获取所述特征信息,包括:获取所述主音频信号在第一预设时间段内的信号幅值;根据所述第一预设时间段内的信号幅值,确定所述过载程度信息。
- 根据权利要求69所述的方法,其特征在于,所述过载程度信息是根据所述第一预设时间段内所述信号幅值的绝对值的最大值确定的。
- 根据权利要求62所述的方法,其特征在于,所述根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号,包括:根据所述特征信息确定修复系数;所述修复系数包括所述主音频信号对应的第一权值,和/或,调整后的所述辅音频信号对应的第二权值;根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号。
- 根据权利要求71所述的方法,其特征在于,所述特征信息包括风噪程度信息,所述修复系数包括所述第一权值和所述第二权值;所述根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号,包括:获取所述主音频信号对应的主频域信号以及调整后的所述辅音频信号对应的辅频域信号;将第一修正信号与第二修正信号的和确定为所述目标音频信号;其中,所述第一修正信号为所述主频域信号与所述第一权值的乘积,所述第二修正信号为所述辅频域信号与所述第二权值的乘积。
- 根据权利要求72所述的方法,其特征在于,所述风噪程度信息与所述第一权值具有映射函数关系,所述第一权值与所述第二权值的和等于1,所 述映射函数关系包括下列中的任意一项:线性函数关系、指数函数关系和对数函数关系。
- 根据权利要求71所述的方法,其特征在于,所述特征信息包括堵麦程度信息,所述修复系数包括所述第二权值;所述根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号,包括:根据所述堵麦程度信息确定所述主麦克风是否发生堵麦;若确定所述主麦克风发生堵麦,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
- 根据权利要求74所述的方法,其特征在于,所述根据所述堵麦程度信息确定所述主麦克风是否发生堵麦,包括:获取第二预设时间段内的多个堵麦程度信息;获取所述多个堵麦程度信息的平均值;确定所述平均值是否小于预设均值;若所述平均值小于所述预设均值,则确定所述主麦克风发生堵麦;若所述平均值大于或等于所述预设均值,则确定所述主麦克风没有发生堵麦。
- 根据权利要求74所述的方法,其特征在于,所述第二权值为1。
- 根据权利要求71所述的方法,其特征在于,所述特征信息包括过载程度信息,所述修复系数包括所述第二权值;所述根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号,包括:根据所述过载程度信息确定所述主麦克风是否过载;若确定所述主麦克风过载,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
- 根据权利要求77所述的方法,其特征在于,所述根据所述过载程度信息确定所述主麦克风是否过载,包括:确定所述过载程度信息是否大于第一预设阈值;若所述过载程度信息大于所述第一预设阈值,则确定所述主麦克风过载;若所述过载程度信息小于或等于所述第一预设阈值,则确定所述主麦克 风没有过载。
- 根据权利要求62-68、71-76任一项所述的方法,其特征在于,所述获取所述主麦克风的特征信息,包括:对每个目标信号进行频域变换,得到所述目标信号分别在多个频段上的频率分量;其中,所述目标信号包括所述主音频信号,或者,所述目标信号包括所述主音频信号和所述辅音频信号;根据所述目标信号分别在多个频段上的频率分量,获得所述主麦克风在每个频段上的特征信息;所述根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号,包括:获取所述调整后的所述辅音频信号分别在所述多个频段上的频率分量;针对所述多个频段中的每个频段,根据所述主麦克风在该频段上的特征信息,将所述主音频信号在该频段上的频率分量和所述调整后的所述辅音频信号在该频段上的频率分量,合成所述目标音频信号在该频段上的频率分量。
- 根据权利要求62-78任一项所述的方法,其特征在于,所述根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号,包括:获取当前时间段内所述主麦克风的第一特征信息,以及与所述当前时间段相邻的上一个时间段内所述主麦克风的第二特征信息;根据所述第二特征信息修正所述第一特征信息;根据修正后的所述第一特征信息,将所述主音频信号和所述调整后的所述辅音频信号合成所述目标音频信号。
- 根据权利要求80所述的方法,其特征在于,所述根据所述第二特征信息修正所述第一特征信息,包括:获取所述第一特征信息的权值和所述第二特征信息的权值;根据所述第一特征信息、所述第一特征信息的权值、所述第二特征信息和所述第二特征信息的权值,修正所述第一特征信息。
- 一种电子设备,其特征在于,包括:存储器和处理器;所述存储器,用于存储程序代码;所述处理器,调用所述程序代码,当所述程序代码被执行时,用于执行以下操作:获取主音频信号和辅音频信号,所述主音频信号和所述辅音频信号是同时对同一音源采集得到的,所述辅音频信号与所述主音频信号在特定频率的幅值和/或相位不同;在所述辅音频信号中确定处于不同频段的多个音频分量;根据所述多个音频分量中任一所述音频分量对应的频段,以及预设的频段和频响参数的调参量的对应关系,确定所述音频分量的频响参数的目标调参量,并根据所述目标调参量调整所述音频分量的幅值参数和/或相位参数;根据所述主音频信号和调整后的所述辅音频信号合成目标音频信号。
- 根据权利要求82所述的电子设备,其特征在于,所述主音频信号由设置在电子设备的主麦克风采集,所述辅音频信号由设置在所述电子设备的辅麦克风采集。
- 根据权利要求83所述的电子设备,其特征在于,所述对应关系是基于所述电子设备中所述主麦克风和所述辅麦克风分别对样本音源进行音频采集得到的样本音频的频响参数的偏差确定的。
- 根据权利要求83所述的电子设备,其特征在于,所述处理器还用于:获取所述主麦克风的特征信息;所述特征信息包括以下一种或多种信息:风噪程度信息、堵麦程度信息或过载程度信息;所述处理器具体用于:根据所述主麦克风的所述特征信息,将所述主音频信号和调整后的所述辅音频信号合成所述目标音频信号。
- 根据权利要求85所述的电子设备,其特征在于,所述处理器具体用于:根据所述主音频信号和所述辅音频信号,确定所述特征信息。
- 根据权利要求86所述的电子设备,其特征在于,所述特征信息包括风噪程度信息,所述风噪程度信息是根据所述主音频信号与所述辅音频信号之间的信号相关性确定的。
- 根据权利要求86所述的电子设备,其特征在于,所述特征信息包括堵麦程度信息,所述堵麦程度信息是根据所述主音频信号的信号能量和所述 辅音频信号的信号能量之间的大小关系确定的。
- 根据权利要求88所述的电子设备,其特征在于,所述堵麦程度信息是根据所述主音频信号的信号能量和所述辅音频信号的信号能量之间的比值确定的。
- 根据权利要求85所述的电子设备,其特征在于,所述处理器具体用于:根据所述主音频信号获取所述特征信息。
- 根据权利要求90所述的电子设备,其特征在于,所述电子设备包括至少两个主麦克风,所述特征信息包括风噪程度信息;所述处理器具体用于:获取第一主音频信号对应的第一频域信号和第二主音频信号对应的第二频域信号;其中,所述第一主音频信号和所述第二主音频信号为所述至少两个主麦克风中的任意两个主麦克风分别采集的主音频信号;基于所述第一频域信号和所述第二频域信号之间的相关性,确定所述风噪程度信息。
- 根据权利要求90所述的电子设备,其特征在于,所述特征信息包括过载程度信息;所述处理器具体用于:获取所述主音频信号在第一预设时间段内的信号幅值;根据所述第一预设时间段内的信号幅值,确定所述过载程度信息。
- 根据权利要求92所述的电子设备,其特征在于,所述过载程度信息是根据所述第一预设时间段内所述信号幅值的绝对值的最大值确定的。
- 根据权利要求85所述的电子设备,其特征在于,所述处理器具体用于:根据所述特征信息确定修复系数;所述修复系数包括所述主音频信号对应的第一权值,和/或,调整后的所述辅音频信号对应的第二权值;根据所述第一权值和/或所述第二权值,以及调整后的所述辅音频信号,合成所述目标音频信号。
- 根据权利要求94所述的电子设备,其特征在于,所述特征信息包括风噪程度信息,所述修复系数包括所述第一权值和所述第二权值;所述处理器具体用于:获取所述主音频信号对应的主频域信号以及调整后的所述辅音频信号对应的辅频域信号;将第一修正信号与第二修正信号的和确定为所述目标音频信号;其中,所述第一修正信号为所述主频域信号与所述第一权值的乘积,所述第二修正信号为所述辅频域信号与所述第二权值的乘积。
- 根据权利要求95所述的电子设备,其特征在于,所述风噪程度信息与所述第一权值具有映射函数关系,所述第一权值与所述第二权值的和等于1,所述映射函数关系包括下列中的任意一项:线性函数关系、指数函数关系和对数函数关系。
- 根据权利要求94所述的电子设备,其特征在于,所述特征信息包括堵麦程度信息,所述修复系数包括所述第二权值;所述处理器具体用于:根据所述堵麦程度信息确定所述主麦克风是否发生堵麦;若确定所述主麦克风发生堵麦,则将调整后的所述辅音频信号与所述第二权值的乘积确定为所述目标音频信号。
- 根据权利要求97所述的电子设备,其特征在于,所述处理器具体用于:获取第二预设时间段内的多个堵麦程度信息;获取所述多个堵麦程度信息的平均值;确定所述平均值是否小于预设均值;若所述平均值小于所述预设均值,则确定所述主麦克风发生堵麦;若所述平均值大于或等于所述预设均值,则确定所述主麦克风没有发生堵麦。
- 根据权利要求97所述的电子设备,其特征在于,所述第二权值为1。
- 根据权利要求94所述的电子设备,其特征在于,所述特征信息包括过载程度信息,所述修复系数包括所述第二权值;所述处理器具体用于:根据所述过载程度信息确定所述主麦克风是否过载;若确定所述主麦克风过载,则将调整后的所述辅音频信号与所述第二权 值的乘积确定为所述目标音频信号。
- 根据权利要求100所述的电子设备,其特征在于,所述处理器具体用于:确定所述过载程度信息是否大于第一预设阈值;若所述过载程度信息大于所述第一预设阈值,则确定所述主麦克风过载;若所述过载程度信息小于或等于所述第一预设阈值,则确定所述主麦克风没有过载。
- 根据权利要求85-91、94-99任一项所述的电子设备,其特征在于,所述处理器具体用于:对每个目标信号进行频域变换,得到所述目标信号分别在多个频段上的频率分量;其中,所述目标信号包括所述主音频信号,或者,所述目标信号包括所述主音频信号和所述辅音频信号;根据所述目标信号分别在多个频段上的频率分量,获得所述主麦克风在每个频段上的特征信息;所述处理器具体用于:获取所述调整后的所述辅音频信号分别在所述多个频段上的频率分量;针对所述多个频段中的每个频段,根据所述主麦克风在该频段上的特征信息,将所述主音频信号在该频段上的频率分量和所述调整后的所述辅音频信号在该频段上的频率分量,合成所述目标音频信号在该频段上的频率分量。
- 根据权利要求85-101任一项所述的电子设备,其特征在于,所述处理器具体用于:获取当前时间段内所述主麦克风的第一特征信息,以及与所述当前时间段相邻的上一个时间段内所述主麦克风的第二特征信息;根据所述第二特征信息修正所述第一特征信息;根据修正后的所述第一特征信息,将所述主音频信号和所述调整后的所述辅音频信号合成所述目标音频信号。
- 根据权利要求103所述的电子设备,其特征在于,所述处理器具体用于:获取所述第一特征信息的权值和所述第二特征信息的权值;根据所述第一特征信息、所述第一特征信息的权值、所述第二特征信息 和所述第二特征信息的权值,修正所述第一特征信息。
- 一种计算机可读存储介质,其特征在于,所述可读存储介质上存储有计算机程序;所述计算机程序在被执行时,实现如权利要求1-29任一项所述的方法,或者,实现如权利要求59-81任一项所述的方法。
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/104517 WO2022016533A1 (zh) | 2020-07-24 | 2020-07-24 | 音频处理方法和电子设备 |
| CN202080033633.6A CN114145025B (zh) | 2020-07-24 | 2020-07-24 | 音频处理方法和电子设备 |
| US18/095,390 US12395777B2 (en) | 2020-07-24 | 2023-01-10 | Audio processing method and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/104517 WO2022016533A1 (zh) | 2020-07-24 | 2020-07-24 | 音频处理方法和电子设备 |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US18/095,390 Continuation US12395777B2 (en) | 2020-07-24 | 2023-01-10 | Audio processing method and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2022016533A1 true WO2022016533A1 (zh) | 2022-01-27 |
Family
ID=79729726
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2020/104517 Ceased WO2022016533A1 (zh) | 2020-07-24 | 2020-07-24 | 音频处理方法和电子设备 |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US12395777B2 (zh) |
| CN (1) | CN114145025B (zh) |
| WO (1) | WO2022016533A1 (zh) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114554330A (zh) * | 2022-04-20 | 2022-05-27 | 广州市星康科技有限公司 | 一种可拆卸式具有防尘清灰结构的扬声器 |
| EP4439556A1 (en) * | 2023-03-27 | 2024-10-02 | Harman International Industries, Inc. | Method for detecting distortions of speech signals and inpainting distorted speech signals |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116015993B (zh) * | 2022-12-13 | 2024-10-22 | 南京大鱼半导体有限公司 | 一种音频信号处理方法及终端 |
| CN117215516B (zh) * | 2023-09-12 | 2024-07-16 | 深圳市品声科技有限公司 | 一种基于麦克风的交互方法及装置 |
| CN117153365B (zh) * | 2023-10-30 | 2024-03-12 | 中国人民解放军总医院第二医学中心 | 一种基于音频识别的透析设备运行状态预警方法及系统 |
| CN117998254B (zh) * | 2024-04-07 | 2024-07-30 | 腾讯科技(深圳)有限公司 | 破音修复方法、装置及存储介质 |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110164770A1 (en) * | 2010-01-06 | 2011-07-07 | Apple Inc. | Processing a multi-channel signal for output to a mono speaker |
| CN102137318A (zh) * | 2010-01-22 | 2011-07-27 | 华为终端有限公司 | 拾音控制方法和装置 |
| CN102209988A (zh) * | 2008-09-11 | 2011-10-05 | 弗劳恩霍夫应用研究促进协会 | 基于麦克风信号来提供空间提示集合的设备、方法和计算机程序以及用于提供双声道音频信号和空间提示集合的设备 |
| CN102761808A (zh) * | 2011-04-28 | 2012-10-31 | 索尼公司 | 噪声消除单元 |
| CN103219012A (zh) * | 2013-04-23 | 2013-07-24 | 中国人民解放军总后勤部军需装备研究所 | 一种基于声源距离的双麦克风噪声消除方法及装置 |
| CN106487971A (zh) * | 2015-08-25 | 2017-03-08 | 黑莓有限公司 | 用于减轻在设备的麦克风处生成的语音信号中的风噪声的方法和设备 |
| CN206210386U (zh) * | 2016-12-02 | 2017-05-31 | 上海木爷机器人技术有限公司 | 音频信号处理系统 |
| CN109068251A (zh) * | 2018-07-10 | 2018-12-21 | 深圳市亿鑫数字科技有限公司 | 一种智能助听器的自动均衡方法及系统 |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8488803B2 (en) * | 2007-05-25 | 2013-07-16 | Aliphcom | Wind suppression/replacement component for use with electronic systems |
| US8861745B2 (en) * | 2010-12-01 | 2014-10-14 | Cambridge Silicon Radio Limited | Wind noise mitigation |
| JP5926490B2 (ja) * | 2011-02-10 | 2016-05-25 | キヤノン株式会社 | 音声処理装置 |
| CN106452384A (zh) * | 2015-08-10 | 2017-02-22 | 西安中兴新软件有限责任公司 | 一种音频调节方法及终端设备 |
| GB2555139A (en) * | 2016-10-21 | 2018-04-25 | Nokia Technologies Oy | Detecting the presence of wind noise |
| GB201715824D0 (en) * | 2017-07-06 | 2017-11-15 | Cirrus Logic Int Semiconductor Ltd | Blocked Microphone Detection |
| WO2019112468A1 (en) * | 2017-12-08 | 2019-06-13 | Huawei Technologies Co., Ltd. | Multi-microphone noise reduction method, apparatus and terminal device |
| KR102652553B1 (ko) * | 2019-02-08 | 2024-03-29 | 삼성전자 주식회사 | 전자 장치 및 전자 장치의 마이크로폰 블록 검출 방법 |
| CN109982179B (zh) * | 2019-04-19 | 2023-08-11 | 努比亚技术有限公司 | 音频信号输出方法、装置、可穿戴设备及存储介质 |
-
2020
- 2020-07-24 WO PCT/CN2020/104517 patent/WO2022016533A1/zh not_active Ceased
- 2020-07-24 CN CN202080033633.6A patent/CN114145025B/zh active Active
-
2023
- 2023-01-10 US US18/095,390 patent/US12395777B2/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102209988A (zh) * | 2008-09-11 | 2011-10-05 | 弗劳恩霍夫应用研究促进协会 | 基于麦克风信号来提供空间提示集合的设备、方法和计算机程序以及用于提供双声道音频信号和空间提示集合的设备 |
| US20110164770A1 (en) * | 2010-01-06 | 2011-07-07 | Apple Inc. | Processing a multi-channel signal for output to a mono speaker |
| CN102137318A (zh) * | 2010-01-22 | 2011-07-27 | 华为终端有限公司 | 拾音控制方法和装置 |
| CN102761808A (zh) * | 2011-04-28 | 2012-10-31 | 索尼公司 | 噪声消除单元 |
| CN103219012A (zh) * | 2013-04-23 | 2013-07-24 | 中国人民解放军总后勤部军需装备研究所 | 一种基于声源距离的双麦克风噪声消除方法及装置 |
| CN106487971A (zh) * | 2015-08-25 | 2017-03-08 | 黑莓有限公司 | 用于减轻在设备的麦克风处生成的语音信号中的风噪声的方法和设备 |
| CN206210386U (zh) * | 2016-12-02 | 2017-05-31 | 上海木爷机器人技术有限公司 | 音频信号处理系统 |
| CN109068251A (zh) * | 2018-07-10 | 2018-12-21 | 深圳市亿鑫数字科技有限公司 | 一种智能助听器的自动均衡方法及系统 |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114554330A (zh) * | 2022-04-20 | 2022-05-27 | 广州市星康科技有限公司 | 一种可拆卸式具有防尘清灰结构的扬声器 |
| CN114554330B (zh) * | 2022-04-20 | 2022-07-12 | 广州市星康科技有限公司 | 一种可拆卸式具有防尘清灰结构的扬声器 |
| EP4439556A1 (en) * | 2023-03-27 | 2024-10-02 | Harman International Industries, Inc. | Method for detecting distortions of speech signals and inpainting distorted speech signals |
Also Published As
| Publication number | Publication date |
|---|---|
| US20230164480A1 (en) | 2023-05-25 |
| CN114145025B (zh) | 2024-04-12 |
| CN114145025A (zh) | 2022-03-04 |
| US12395777B2 (en) | 2025-08-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114145025B (zh) | 音频处理方法和电子设备 | |
| JP4449987B2 (ja) | 音声処理装置、音声処理方法およびプログラム | |
| CN111554317A (zh) | 一种语音播报方法、设备、计算机存储介质及系统 | |
| JP5081245B2 (ja) | 指向性マイクロホン装置 | |
| US8498429B2 (en) | Acoustic correction apparatus, audio output apparatus, and acoustic correction method | |
| JP6838649B2 (ja) | 収音装置および収音方法 | |
| CN109257675A (zh) | 一种防风噪方法、耳机及存储介质 | |
| CN113949968A (zh) | 一种频响校正方法、电子设备及信号处理方法 | |
| WO2022007239A1 (zh) | 音质自适应调节方法、相关系统和设备及存储介质 | |
| CN114422893A (zh) | 一种降噪处理方法、装置、耳机及存储介质 | |
| CN114724575A (zh) | 一种啸叫检测方法、装置以及系统 | |
| CN115171637B (zh) | 主动降噪滤波器参数的确定方法及装置 | |
| JP7158976B2 (ja) | 収音装置、収音プログラム及び収音方法 | |
| CN118400650B (zh) | 一种麦克风防啸叫方法及装置 | |
| CN112581974A (zh) | 波束设计方法、装置、设备及存储介质 | |
| JP2001352594A (ja) | 風音低減方法及び装置 | |
| CN1031444C (zh) | 收听空间特性校正装置 | |
| CN116634330A (zh) | 一种扬声器振膜位移的控制方法和控制系统 | |
| CN113573205B (zh) | 信号处理方法、设备及计算机存储介质 | |
| JP5762797B2 (ja) | 信号処理装置及び信号処理方法 | |
| CN119811348B (zh) | 空间降噪方法、装置、电子设备和计算机可读存储介质 | |
| TWI876539B (zh) | 收音的動態校正方法及電子系統 | |
| TWI878997B (zh) | 寬動態範圍壓縮曲線調整方法及電子裝置 | |
| US12556861B2 (en) | Apparatus, methods and computer programs for controlling audibility of sound sources | |
| JPH06133388A (ja) | マイクロホン装置 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20946010 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 20946010 Country of ref document: EP Kind code of ref document: A1 |






