CN113613143A - Audio processing method and device suitable for mobile terminal and storage medium - Google Patents
Audio processing method and device suitable for mobile terminal and storage medium Download PDFInfo
- Publication number
- CN113613143A CN113613143A CN202110770953.6A CN202110770953A CN113613143A CN 113613143 A CN113613143 A CN 113613143A CN 202110770953 A CN202110770953 A CN 202110770953A CN 113613143 A CN113613143 A CN 113613143A
- Authority
- CN
- China
- Prior art keywords
- acoustic feedback
- audio
- mobile terminal
- voice
- feedback loop
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/02—Circuits for transducers, loudspeakers or microphones for preventing acoustic reaction, i.e. acoustic oscillatory feedback
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10K11/00—Methods or devices for transmitting, conducting or directing sound in general; Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/16—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/175—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound
- G10K11/178—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase
- G10K11/1781—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase characterised by the analysis of input or output signals, e.g. frequency range, modes, transfer functions
- G10K11/17813—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase characterised by the analysis of input or output signals, e.g. frequency range, modes, transfer functions characterised by the analysis of the acoustic paths, e.g. estimating, calibrating or testing of transfer functions or cross-terms
- G10K11/17819—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase characterised by the analysis of input or output signals, e.g. frequency range, modes, transfer functions characterised by the analysis of the acoustic paths, e.g. estimating, calibrating or testing of transfer functions or cross-terms between the output signals and the reference signals, e.g. to prevent howling
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10K11/00—Methods or devices for transmitting, conducting or directing sound in general; Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/16—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/175—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound
- G10K11/178—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase
- G10K11/1785—Methods, e.g. algorithms; Devices
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- Signal Processing (AREA)
- Telephone Function (AREA)
Abstract
The invention provides an audio processing method, an audio processing device and a storage medium suitable for a mobile terminal, wherein the audio processing method comprises the following steps: acquiring the characteristics of an acoustic feedback loop in real time; carrying out human voice beautifying processing on the input first multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain a second multi-channel audio; and carrying out master tape mixing and shrinking processing on the second multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain single-channel audio output. The technical scheme provided by the invention can dynamically adjust the parameters of reverberation and balance according to the characteristics of the acoustic feedback loop, and reduce the difference of sound effect and audibility under different environments. In the dynamic adjustment, the adjustment can be performed according to different pose information of the mobile terminal, so that the scheme provided by the invention has stronger adaptability. In addition, the voice of the person holding the mobile terminal is separated in the voice beautifying process, environmental noise and other voice are suppressed, and voice quality is compensated.
Description
Technical Field
The present invention relates to the field of audio data processing technologies, and in particular, to an audio processing method and apparatus suitable for a mobile terminal, and a storage medium.
Background
With the rapid development of science and technology and music culture, the way of singing (singing) is more and more diversified. The mode of carrying out K song based on mobile terminal has the advantage of convenience, easy operation, and is most favored by everyone. Most of the current karaoke software has a function of beautifying the voice, but the current karaoke software can only be passively beautified when beautifying the voice, namely the beautifying function of each voice is similar, and different beautifying modes cannot be adopted according to the difference of the environment.
Disclosure of Invention
The embodiment of the invention provides an audio processing method, an audio processing device and a storage medium suitable for a mobile terminal, which can adopt different beautifying modes for human voice based on different environments and scenes and have higher pertinence.
In a first aspect of the embodiments of the present invention, an audio processing method suitable for a mobile terminal is provided, including:
acquiring the characteristics of an acoustic feedback loop of the mobile terminal in real time;
carrying out human voice beautifying processing on the input first multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain a second multi-channel audio;
and carrying out master tape mixing and shrinking processing on the second multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain single-channel audio output.
Optionally, in a possible implementation manner of the first aspect, the performing human voice beautification processing on the input first multi-channel audio based on the acoustic feedback loop feature to obtain a second multi-channel audio includes:
acquiring an environment reverberation pulse response in the characteristics of an acoustic feedback loop, and generating a reverse signal based on the environment reverberation pulse response;
the first multi-channel audio comprises accompaniment audio and a single-channel voice signal, and the single-channel voice signal is subjected to signal cancellation processing based on the reverse signal to obtain the single-channel voice signal without environmental reverberation;
and beautifying the single-path voice signal after the offset processing to obtain a stereo voice frequency.
Optionally, in a possible implementation manner of the first aspect, performing a mastering process on the second multi-channel audio based on the acoustic feedback loop characteristic to obtain a single-channel audio output includes:
obtaining an environment reverberation pulse response, an acoustic feedback loop frequency response curve and a spatial mode standing wave frequency in the acoustic feedback loop characteristic;
mixing the stereo human voice audio and the accompaniment audio to obtain a single-channel audio;
and processing the single-channel audio based on the environment reverberation pulse response, the acoustic feedback loop frequency response curve and the spatial mode standing wave frequency to obtain the processed stereo audio.
Optionally, in a possible implementation manner of the first aspect, the enhancing frequency point sequence and the gain amount sequence that change with time are calculated according to a frequency response of the acoustic feedback loop, a real-time change of a frequency response curve of the acoustic feedback loop, and an impulse response of a room environment of echo tracking;
configuring an environment equalization filter bank, wherein the environment equalization filter bank is used for inputting a mixed signal of stereo human voice audio and accompaniment audio into the filter bank for equalization compensation, and the compensation mode is based on the enhancement frequency point sequence and the gain quantity sequence for compensation;
and configuring an exciter, wherein the exciter is used for generating corresponding higher harmonics to compensate the loss frequency caused by the spatial mode, and the compensation mode is based on the enhancement frequency point sequence and the gain quantity sequence.
Optionally, in a possible implementation manner of the first aspect, the acoustic feedback loop characteristics of the mobile terminal are obtained in real time:
controlling a loudspeaker of the mobile terminal to play at least one acoustic feedback detection signal and recording the acoustic feedback detection signal;
collecting an acoustic feedback detection signal through a microphone of the mobile terminal and recording the collected acoustic feedback detection signal;
and comparing the played acoustic feedback detection signal with the collected acoustic feedback detection signal to obtain an acoustic feedback detection result, wherein the acoustic feedback detection result comprises the characteristics of an acoustic feedback loop.
Optionally, in a possible implementation manner of the first aspect, the comparing the played acoustic feedback detection signal with the collected acoustic feedback detection signal to obtain an acoustic feedback detection result includes:
comparing the played acoustic feedback detection signal with the collected acoustic feedback detection signal in an echo power dimension, a frequency response dimension and an environment reverberation dimension respectively to obtain an acoustic feedback detection result;
the acoustic feedback detection result includes echo power information, frequency response information, and ambient reverberation impulse response information.
Optionally, in a possible implementation manner of the first aspect, the method further includes:
acquiring any one or more of attitude information, position information and speed information of the mobile terminal in real time;
obtaining pose information of the mobile terminal at the current moment based on the attitude information, the position information and the speed information;
and updating the acoustic feedback detection result based on the pose information at the current moment.
In a second aspect of the embodiments of the present invention, an audio processing apparatus suitable for a mobile terminal is provided, including:
the acquisition module is used for acquiring the characteristics of an acoustic feedback loop of the mobile terminal in real time;
the voice beautifying module is used for carrying out voice beautifying processing on the input first multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain a second multi-channel audio;
and the master tape mixing and compressing module is used for carrying out master tape mixing and compressing processing on the second multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain single-channel audio output.
Optionally, in a possible implementation manner of the second aspect, the voice beautification module is further configured to perform the following steps, including:
receiving audio features of a target user;
extracting audio information corresponding to the audio features of the target user in the first multi-channel audio as target voice information of only the voice of the target user;
and performing sound quality enhancement processing on the target voice information to generate a second multi-channel audio.
In a third aspect of the embodiments of the present invention, a readable storage medium is provided, in which a computer program is stored, which, when being executed by a processor, is adapted to carry out the method according to the first aspect of the present invention and various possible designs of the first aspect of the present invention.
The audio processing method, the device and the storage medium suitable for the mobile terminal provided by the invention can dynamically adjust the parameters of reverberation and balance according to the characteristics of the acoustic feedback loop, and reduce the difference of sound effect and audibility under different environments. In the dynamic adjustment, the adjustment can be performed according to different pose information of the mobile terminal, so that the scheme provided by the invention has stronger adaptability. In addition, the voice of the person holding the mobile terminal is separated in the voice beautifying process, environmental noise and other voice are suppressed, and voice quality is compensated.
Drawings
Fig. 1 is a flowchart of a first embodiment of an audio processing method suitable for a mobile terminal;
FIG. 2 is a schematic diagram of a first embodiment of a voiceprint processing and mastering process;
FIG. 3 is a schematic diagram of a second embodiment of a voiceprint enhancement process and a mastering process;
FIG. 4 is a schematic diagram of a measurement phase of an acoustic feedback loop characteristic;
FIG. 5 is a schematic diagram of a tracking phase of an acoustic feedback loop feature;
fig. 6 is a block diagram of a first embodiment of an audio processing method applied to a mobile terminal.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein.
It should be understood that, in various embodiments of the present invention, the sequence numbers of the processes do not mean the execution sequence, and the execution sequence of the processes should be determined by the functions and the internal logic of the processes, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
It should be understood that in the present application, "comprising" and "having" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that, in the present invention, "a plurality" means two or more. "and/or" is merely an association describing an associated object, meaning that three relationships may exist, for example, and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "comprises A, B and C" and "comprises A, B, C" means that all three of A, B, C comprise, "comprises A, B or C" means that one of A, B, C comprises, "comprises A, B and/or C" means that any 1 or any 2 or 3 of A, B, C comprises.
It should be understood that in the present invention, "B corresponding to a", "a corresponds to B", or "B corresponds to a" means that B is associated with a, and B can be determined from a. Determining B from a does not mean determining B from a alone, but may be determined from a and/or other information. And the matching of A and B means that the similarity of A and B is greater than or equal to a preset threshold value.
As used herein, "if" may be interpreted as "at … …" or "when … …" or "in response to a determination" or "in response to a detection", depending on the context.
The technical solution of the present invention will be described in detail below with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
The present invention provides an audio processing method suitable for a mobile terminal, as shown in fig. 1 and fig. 2, comprising:
and step S110, acquiring the acoustic feedback loop characteristics of the mobile terminal in real time. Because different influences can be generated on audio frequency under different environments and scenes, the method can acquire the characteristics of the acoustic feedback loop in real time so as to be suitable for different environments and scenes.
And S120, performing human voice beautification on the input first multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain a second multi-channel audio. After the characteristics of the acoustic feedback loop are obtained, the voice beautification treatment is firstly carried out on the input multi-channel audio. Because the accompaniment is invariable during singing and is mainly used for beautifying the human voice, the human voice needs to be processed before sound mixing, and the adjustment can be carried out in real time according to the characteristics of an acoustic feedback loop in the process of human voice processing so as to prevent the human voice from being interfered under different environments.
And S130, carrying out master tape mixing and shrinking processing on the second multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain single-channel audio output. After the beautified human voice is obtained, multi-channel audio is subjected to audio mixing processing to obtain single-channel audio output, and the single-channel audio is adjusted in real time according to the characteristics of an acoustic feedback loop in the audio mixing process to prevent the single-channel audio from being interfered in different environments.
Further, as shown in fig. 3, performing human voice beautification on the input first multi-channel audio based on the acoustic feedback loop characteristic to obtain a second multi-channel audio includes:
acquiring an environment reverberation pulse response in the characteristics of an acoustic feedback loop, and generating a reverse signal based on the environment reverberation pulse response;
the first multi-channel audio comprises accompaniment audio and a single-channel voice signal, and the single-channel voice signal is subjected to signal cancellation processing based on the reverse signal to obtain the single-channel voice signal without environmental reverberation;
and beautifying the single-path voice signal after the offset processing to obtain a stereo voice frequency.
The acoustic feedback loop detection module provided by the invention can transmit the environment reverberation pulse response to the human voice beautifying module. The voice beautifying module generates a reverse signal according to the environment reverberation pulse response, cancels the signal generated by the environment reverberation pulse response in the recording signal to obtain pure dry voice (without reverberation), namely a one-way voice signal without environment reverberation, and then performs traditional voice beautification, such as equalization, compression, reverberation and the like, to obtain voice with sound effect. The invention can remove the interference of reverberation introduced by the environment to the voice beautifying work in the first multi-channel audio, and improves the voice beautifying effect.
Further, performing master band mixing processing on the second multi-channel audio based on the characteristics of the acoustic feedback loop to obtain a single-channel audio output includes:
obtaining an environment reverberation pulse response, an acoustic feedback loop frequency response curve and a spatial mode standing wave frequency in the acoustic feedback loop characteristic;
mixing the stereo human voice audio and the accompaniment audio to obtain a single-channel audio;
and processing the single-channel audio based on the environment reverberation pulse response, the acoustic feedback loop frequency response curve and the spatial mode standing wave frequency to obtain the processed stereo audio.
The acoustic feedback loop detection module transmits the ambient reverberation pulse response, the real-time change of the acoustic feedback loop frequency response curve and the spatial mode standing wave frequency of the spatial position of the mobile end to the master band mixing and compressing module. The mother tape mixing can mix the stereo human voice audio processed by the human voice beautifying module with the accompaniment audio, and the audio after mixing is processed according to the characteristics input by the acoustic feedback loop detection module, so that certain frequency bands in the auditory sense caused by the environmental reverberation can be balanced and enhanced after processing, and meanwhile, the frequency loss caused by the space mode can be compensated, and the effect of reducing the difference of the sound effect auditory sense in different environments is achieved.
Further, according to the frequency response of the acoustic feedback loop, the real-time change of the frequency response curve of the acoustic feedback loop and the room environment impulse response of echo tracking, an enhanced frequency point sequence and a gain quantity sequence which change along with time are calculated.
And configuring an environment equalization filter bank, wherein the environment equalization filter bank is used for inputting mixed signals of stereo human voice audio and accompaniment audio into the filter bank for equalization compensation, and the compensation mode is based on the enhancement frequency point sequence and the gain quantity sequence for compensation.
According to the frequency response of the acoustic feedback loop, the real-time change of the frequency response curve of the acoustic feedback loop and the room environment impulse response of echo tracking, an enhanced frequency point sequence and a gain quantity sequence which change along with time are calculated, an environment gain equalization cascade filter bank is designed, the parameters of the filter bank change along with the time change, the acting frequency band is consistent with the enhanced frequency point sequence, and the gain parameters of the filter are opposite to the gain quantity sequence caused by the environment impulse response. And the mixed signal of the voice and the accompaniment passes through the filter bank to obtain an audio signal after balanced compensation.
And configuring an exciter, wherein the exciter is used for generating corresponding higher harmonics to compensate the loss frequency caused by the spatial mode, and the compensation mode is based on the enhancement frequency point sequence and the gain quantity sequence.
According to the loss frequency caused by the space mode, the traditional exciter is adopted to generate the higher harmonic wave of the frequency band, so that the existence of the energy of the frequency band is simulated on the hearing sense, and the frequency loss caused by the space mode is compensated.
Further, step S110 includes:
and controlling a loudspeaker of the mobile terminal to play at least one acoustic feedback detection signal and recording the acoustic feedback detection signal. In the present invention, as shown in fig. 4, an acoustic detection signal library may be configured in advance, and the acoustic feedback detection signal is a signal stored in advance in the acoustic detection signal library. Wherein the acoustic probe signal bank stores signals at least for predicting echo power, frequency response and ambient reverberation.
And acquiring the acoustic feedback detection signal through a microphone of the mobile terminal and recording the acquired acoustic feedback detection signal. After the mobile terminal plays the acoustic feedback detection signal, factors such as environment and the like can enhance and \ or weaken the acoustic feedback detection signal, and the enhanced and \ or weakened acoustic feedback detection signal is obtained through the microphone.
And comparing the played acoustic feedback detection signal with the collected acoustic feedback detection signal to obtain an acoustic feedback detection result, wherein the acoustic feedback detection result comprises the characteristics of an acoustic feedback loop. In the step, the enhanced and/or weakened acoustic feedback detection signal is compared with the original acoustic feedback detection signal to obtain an acoustic feedback detection result, and the acoustic feedback detection result can reflect the enhancement and/or weakening condition of the acoustic feedback detection signal, wherein the acoustic feedback detection result comprises the characteristics of an acoustic feedback loop.
Further, comparing the played sound feedback detection signal with the collected sound feedback detection signal to obtain a sound feedback detection result includes:
and comparing the played acoustic feedback detection signal with the collected acoustic feedback detection signal in an echo power dimension, a frequency response dimension and an environment reverberation dimension respectively to obtain an acoustic feedback detection result. According to the technical scheme provided by the invention, the acoustic feedback detection result can be obtained in the echo power dimension, the frequency response dimension and the environment reverberation dimension, so that the effect of carrying out multi-dimensional processing on human voice is achieved.
The acoustic feedback detection result includes echo power information, frequency response information, and ambient reverberation impulse response information.
In one possible embodiment, the method further comprises:
and acquiring any one or more of attitude information, position information and speed information of the mobile terminal in real time. As shown in fig. 5, the pose information in the present invention includes the pose information of the mobile terminal when the user holds the mobile terminal, the positions of the user and the mobile terminal, and the speed at which the user controls the mobile terminal, so that the present invention is more comprehensive when considering the situation of acoustic feedback, thereby ensuring that the present invention comprehensively suppresses and avoids the generation of howling.
And obtaining the pose information of the mobile terminal at the current moment based on the attitude information, the position information and the speed information. At each time, one or more of the pose information, position information, and velocity information may change, so that the pose information is adjusted when any one of the information changes.
And updating the acoustic feedback detection result based on the pose information at the current moment. By the method, the dynamic update of the acoustic feedback detection result is realized, and the audio is dynamically processed based on the characteristics of the acoustic feedback loop.
An embodiment of the present invention further provides an audio processing apparatus for a mobile terminal, as shown in fig. 6, including:
the acquisition module is used for acquiring the characteristics of an acoustic feedback loop of the mobile terminal in real time;
the voice beautifying module is used for carrying out voice beautifying processing on the input first multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain a second multi-channel audio;
and the master tape mixing and compressing module is used for carrying out master tape mixing and compressing processing on the second multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain single-channel audio output.
In one embodiment, the voice beautification module is further configured to perform steps comprising:
receiving audio features of a target user;
extracting audio information corresponding to the audio features of the target user in the first multi-channel audio as target voice information of only the voice of the target user;
and performing sound quality enhancement processing on the target voice information to generate a second multi-channel audio.
The readable storage medium may be a computer storage medium or a communication medium. Communication media includes any medium that facilitates transfer of a computer program from one place to another. Computer storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, a readable storage medium is coupled to the processor such that the processor can read information from, and write information to, the readable storage medium. Of course, the readable storage medium may also be an integral part of the processor. The processor and the readable storage medium may reside in an Application Specific Integrated Circuits (ASIC). Additionally, the ASIC may reside in user equipment. Of course, the processor and the readable storage medium may also reside as discrete components in a communication device. The readable storage medium may be a read-only memory (ROM), a random-access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
The present invention also provides a program product comprising execution instructions stored in a readable storage medium. The at least one processor of the device may read the execution instructions from the readable storage medium, and the execution of the execution instructions by the at least one processor causes the device to implement the methods provided by the various embodiments described above.
In the above embodiments of the terminal or the server, it should be understood that the Processor may be a Central Processing Unit (CPU), other general-purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. An audio processing method suitable for a mobile terminal, comprising:
acquiring the characteristics of an acoustic feedback loop of the mobile terminal in real time;
carrying out human voice beautifying processing on the input first multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain a second multi-channel audio;
and carrying out master tape mixing and shrinking processing on the second multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain single-channel audio output.
2. The audio processing method applicable to a mobile terminal according to claim 1,
the step of carrying out voice beautification processing on the input first multi-channel audio based on the characteristics of the acoustic feedback loop to obtain a second multi-channel audio comprises the following steps:
acquiring an environment reverberation pulse response in the characteristics of an acoustic feedback loop, and generating a reverse signal based on the environment reverberation pulse response;
the first multi-channel audio comprises accompaniment audio and a single-channel voice signal, and the single-channel voice signal is subjected to signal cancellation processing based on the reverse signal to obtain the single-channel voice signal without environmental reverberation;
and beautifying the single-path voice signal after the offset processing to obtain a stereo voice frequency.
3. The audio processing method applicable to a mobile terminal according to claim 2,
the processing of the second multi-channel audio through the master tape mixing and shrinking based on the characteristics of the acoustic feedback loop to obtain the single-channel audio output comprises the following steps:
obtaining an environment reverberation pulse response, an acoustic feedback loop frequency response curve and a spatial mode standing wave frequency in the acoustic feedback loop characteristic;
mixing the stereo human voice audio and the accompaniment audio to obtain a single-channel audio;
and processing the single-channel audio based on the environment reverberation pulse response, the acoustic feedback loop frequency response curve and the spatial mode standing wave frequency to obtain the processed stereo audio.
4. The audio processing method applicable to a mobile terminal according to claim 3,
calculating an enhanced frequency point sequence and a gain quantity sequence which change along with time according to the frequency response of the acoustic feedback loop, the real-time change of the frequency response curve of the acoustic feedback loop and the room environment impulse response of echo tracking;
configuring an environment equalization filter bank, wherein the environment equalization filter bank is used for inputting a mixed signal of stereo human voice audio and accompaniment audio into the filter bank for equalization compensation, and the compensation mode is based on the enhancement frequency point sequence and the gain quantity sequence for compensation;
and configuring an exciter, wherein the exciter is used for generating corresponding higher harmonics to compensate the loss frequency caused by the spatial mode, and the compensation mode is based on the enhancement frequency point sequence and the gain quantity sequence.
5. The audio processing method applicable to a mobile terminal according to claim 1,
acquiring the characteristics of an acoustic feedback loop of the mobile terminal in real time:
controlling a loudspeaker of the mobile terminal to play at least one acoustic feedback detection signal and recording the acoustic feedback detection signal;
collecting an acoustic feedback detection signal through a microphone of the mobile terminal and recording the collected acoustic feedback detection signal;
and comparing the played acoustic feedback detection signal with the collected acoustic feedback detection signal to obtain an acoustic feedback detection result, wherein the acoustic feedback detection result comprises the characteristics of an acoustic feedback loop.
6. The audio processing method applicable to a mobile terminal according to claim 5,
the step of comparing the played acoustic feedback detection signal with the collected acoustic feedback detection signal to obtain an acoustic feedback detection result comprises:
comparing the played acoustic feedback detection signal with the collected acoustic feedback detection signal in an echo power dimension, a frequency response dimension and an environment reverberation dimension respectively to obtain an acoustic feedback detection result;
the acoustic feedback detection result includes echo power information, frequency response information, and ambient reverberation impulse response information.
7. The audio processing method suitable for mobile terminals according to claim 5, further comprising:
acquiring any one or more of attitude information, position information and speed information of the mobile terminal in real time;
obtaining pose information of the mobile terminal at the current moment based on the attitude information, the position information and the speed information;
and updating the acoustic feedback detection result based on the pose information at the current moment.
8. An audio processing apparatus adapted for use in a mobile terminal, comprising:
the acquisition module is used for acquiring the characteristics of an acoustic feedback loop of the mobile terminal in real time;
the voice beautifying module is used for carrying out voice beautifying processing on the input first multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain a second multi-channel audio;
and the master tape mixing and compressing module is used for carrying out master tape mixing and compressing processing on the second multi-channel audio in real time based on the characteristics of the acoustic feedback loop to obtain single-channel audio output.
9. The audio processing apparatus suitable for mobile terminal according to claim 8,
the voice beautification module is further configured to perform the following steps, including:
receiving audio features of a target user;
extracting audio information corresponding to the audio features of the target user in the first multi-channel audio as target voice information of only the voice of the target user;
and performing sound quality enhancement processing on the target voice information to generate a second multi-channel audio.
10. A readable storage medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110770953.6A CN113613143B (en) | 2021-07-08 | 2021-07-08 | Audio processing method, device and storage medium suitable for mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110770953.6A CN113613143B (en) | 2021-07-08 | 2021-07-08 | Audio processing method, device and storage medium suitable for mobile terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113613143A true CN113613143A (en) | 2021-11-05 |
| CN113613143B CN113613143B (en) | 2023-06-13 |
Family
ID=78304181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110770953.6A Active CN113613143B (en) | 2021-07-08 | 2021-07-08 | Audio processing method, device and storage medium suitable for mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113613143B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102165798A (en) * | 2008-09-25 | 2011-08-24 | 杜比实验室特许公司 | Binaural filters for monophonic compatibility and loudspeaker compatibility |
| CN102543060A (en) * | 2011-12-27 | 2012-07-04 | 瑞声声学科技(深圳)有限公司 | Active noise control system and design method thereof |
| CN103475980A (en) * | 2013-07-19 | 2013-12-25 | 杭州联汇数字科技有限公司 | Self-adaptive acoustic-feedback-eliminating method |
| CN106548782A (en) * | 2016-10-31 | 2017-03-29 | 维沃移动通信有限公司 | The processing method and mobile terminal of acoustical signal |
| CN108430003A (en) * | 2018-03-30 | 2018-08-21 | 广东欧珀移动通信有限公司 | Audio compensation method and device, readable storage medium storing program for executing, terminal |
| CN109767777A (en) * | 2019-01-31 | 2019-05-17 | 迅雷计算机（深圳）有限公司 | A kind of sound mixing method that software is broadcast live |
| CN110177317A (en) * | 2019-05-17 | 2019-08-27 | 腾讯科技（深圳）有限公司 | Echo cancel method, device, computer readable storage medium and computer equipment |
| CN209657794U (en) * | 2018-12-20 | 2019-11-19 | 孙卫平 | A kind of Karaoke control circuit, Karaoke control device and mobile terminal |
| CN110868674A (en) * | 2019-12-17 | 2020-03-06 | 广州优谷信息技术有限公司 | Audio signal processing system of reading pavilion |
| CN111798864A (en) * | 2020-07-02 | 2020-10-20 | 杭州海康威视数字技术股份有限公司 | Echo cancellation method and device, electronic equipment and storage medium |
| CN112767908A (en) * | 2020-12-29 | 2021-05-07 | 安克创新科技股份有限公司 | Active noise reduction method based on key sound recognition, electronic equipment and storage medium |
-
2021
- 2021-07-08 CN CN202110770953.6A patent/CN113613143B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102165798A (en) * | 2008-09-25 | 2011-08-24 | 杜比实验室特许公司 | Binaural filters for monophonic compatibility and loudspeaker compatibility |
| CN102543060A (en) * | 2011-12-27 | 2012-07-04 | 瑞声声学科技(深圳)有限公司 | Active noise control system and design method thereof |
| CN103475980A (en) * | 2013-07-19 | 2013-12-25 | 杭州联汇数字科技有限公司 | Self-adaptive acoustic-feedback-eliminating method |
| CN106548782A (en) * | 2016-10-31 | 2017-03-29 | 维沃移动通信有限公司 | The processing method and mobile terminal of acoustical signal |
| CN108430003A (en) * | 2018-03-30 | 2018-08-21 | 广东欧珀移动通信有限公司 | Audio compensation method and device, readable storage medium storing program for executing, terminal |
| US20190306622A1 (en) * | 2018-03-30 | 2019-10-03 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Method for Sound Effect Compensation, Non-Transitory Computer-Readable Storage Medium, and Terminal Device |
| CN209657794U (en) * | 2018-12-20 | 2019-11-19 | 孙卫平 | A kind of Karaoke control circuit, Karaoke control device and mobile terminal |
| CN109767777A (en) * | 2019-01-31 | 2019-05-17 | 迅雷计算机（深圳）有限公司 | A kind of sound mixing method that software is broadcast live |
| CN110177317A (en) * | 2019-05-17 | 2019-08-27 | 腾讯科技（深圳）有限公司 | Echo cancel method, device, computer readable storage medium and computer equipment |
| CN110868674A (en) * | 2019-12-17 | 2020-03-06 | 广州优谷信息技术有限公司 | Audio signal processing system of reading pavilion |
| CN111798864A (en) * | 2020-07-02 | 2020-10-20 | 杭州海康威视数字技术股份有限公司 | Echo cancellation method and device, electronic equipment and storage medium |
| CN112767908A (en) * | 2020-12-29 | 2021-05-07 | 安克创新科技股份有限公司 | Active noise reduction method based on key sound recognition, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113613143B (en) | 2023-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020108614A1 (en) | Audio recognition method, and target audio positioning method, apparatus and device | |
| JP3670562B2 (en) | Stereo sound signal processing method and apparatus, and recording medium on which stereo sound signal processing program is recorded | |
| EP3163902A1 (en) | Information-processing device, information processing method, and program | |
| CN102131136A (en) | Adaptive ambient sound suppression and speech tracking | |
| KR20050115857A (en) | System and method for speech processing using independent component analysis under stability constraints | |
| CN103137136B (en) | Sound processing device | |
| KR20170053623A (en) | Method and apparatus for enhancing sound sources | |
| CN110956976B (en) | Echo cancellation method, device and equipment and readable storage medium | |
| CN112017681A (en) | Directional voice enhancement method and system | |
| CN112712816A (en) | Training method and device of voice processing model and voice processing method and device | |
| US11380312B1 (en) | Residual echo suppression for keyword detection | |
| CN112399302B (en) | Audio playing method and device of wearable audio playing device | |
| WO2024067782A1 (en) | Sound field expansion method, audio device, and computer readable storage medium | |
| CN116612778B (en) | Echo and noise suppression method, related device and medium | |
| CN113270082A (en) | Vehicle-mounted KTV control method and device and vehicle-mounted intelligent networking terminal | |
| CN113613143B (en) | Audio processing method, device and storage medium suitable for mobile terminal | |
| JP2002247699A (en) | Stereophonic signal processing method and device, and program and recording medium | |
| CN111370017B (en) | Voice enhancement method, device and system | |
| CN114678038A (en) | Audio noise detection method, computer device and computer program product | |
| CN114627889A (en) | Multi-sound-source sound signal processing method and device, storage medium and electronic equipment | |
| CN114067817A (en) | Bass enhancement method, bass enhancement device, electronic equipment and storage medium | |
| CN113611271B (en) | Digital volume augmentation method and device suitable for mobile terminal and storage medium | |
| JP2015070292A (en) | Sound collection/emission device and sound collection/emission program | |
| JP6790659B2 (en) | Sound processing equipment and sound processing method | |
| JP2015070291A (en) | Sound collection/emission device, sound source separation unit and sound source separation program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |