EP3839951B1 - Procédé et dispositif de traitement de signal audio, terminal et support d'enregistrement - Google Patents

Procédé et dispositif de traitement de signal audio, terminal et support d'enregistrement Download PDF

Info

Publication number
EP3839951B1
EP3839951B1 EP20180826.8A EP20180826A EP3839951B1 EP 3839951 B1 EP3839951 B1 EP 3839951B1 EP 20180826 A EP20180826 A EP 20180826A EP 3839951 B1 EP3839951 B1 EP 3839951B1
Authority
EP
European Patent Office
Prior art keywords
frequency
domain
domain estimation
matrix
signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP20180826.8A
Other languages
German (de)
English (en)
Other versions
EP3839951A1 (fr
Inventor
Haining HOU
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Xiaomi Intelligent Technology Co Ltd
Original Assignee
Beijing Xiaomi Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Xiaomi Intelligent Technology Co Ltd filed Critical Beijing Xiaomi Intelligent Technology Co Ltd
Publication of EP3839951A1 publication Critical patent/EP3839951A1/fr
Application granted granted Critical
Publication of EP3839951B1 publication Critical patent/EP3839951B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0272—Voice signal separating
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L21/0232—Processing in the frequency domain
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/04—Circuits for transducers, loudspeakers or microphones for correcting frequency response
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02165—Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00—Signal processing covered by H04R, not provided for in its groups
    • H04R2430/03—Synergistic effects of band splitting and sub-band processing

Definitions

  • the present disclosure generally relates to the technical field of communication, and particularly to a method and device for processing an audio signal, a terminal and a storage medium.
  • an intelligent product device mostly adopts a microphone array for recording voices, and a microphone-based beamforming technology may be adopted to improve voice signal processing quality to increase a voice recognition rate in a real environment.
  • a microphone-based beamforming technology is sensitive to a position error of the microphones, resulting in great influence on performance.
  • an increase in the number of microphones may also increase product cost.
  • the two microphones usually adopt a blind source separation technology different from the multi-microphone-based beamforming technology for voice enhancement. How to obtain high voice quality of a signal separated based on the blind source separation technology is a problem urgent to be solved at present.
  • the present disclosure provides a method for processing an audio signal, a terminal and a storage medium.
  • a method for processing an audio signal is provided, which may include operations as follows.
  • Audio signals sent by at least two sound sources are acquired by at least two microphones to obtain multiple frames of original noisy signals of each of the at least two microphones on a time domain.
  • frequency-domain estimation signals of each of the at least two sound sources are acquired according to the original noisy signals of the at least two microphones.
  • the frequency-domain estimation signals are divided into multiple frequency-domain estimation components on a frequency domain.
  • Each frequency-domain estimation component corresponds to a frequency-domain sub-band and includes multiple pieces of frequency point data.
  • feature decomposition is performed on a related matrix of each frequency-domain estimation component, to obtain a target feature vector corresponding to the frequency-domain estimation component.
  • a separation matrix of each frequency point is obtained based on target feature vectors and the frequency-domain estimation signals of each sound source.
  • the audio signals of sounds produced by the at least two sound sources are obtained based on the separation matrixes and the original noisy signals.
  • the separation matrix obtained in the embodiments of the present disclosure is determined based on the target feature vectors decomposed from the related matrixes of the frequency-domain estimation components in different frequency-domain sub-bands. Therefore, according to the embodiments of the present disclosure, signals may be decomposed based on subspaces corresponding to the target feature vectors, thereby suppressing a noise signal in each original noisy signal, and improving quality of the separated audio signal.
  • the method for processing an audio signal in the embodiment of the present disclosure can obtain accurate separation for audio signals of sounds produced by the sound sources without considering positions of these microphones.
  • a device for processing an audio signal may include an acquisition module, a conversion module, a division module, a decomposition module, a first processing module and a second processing module.
  • the acquisition module is configured to acquire, through at least two microphones, audio signals sent by at least two sound sources, to obtain multiple frames of original noisy signals of each of the at least two microphones on a time domain.
  • the conversion module is configured to, for each frame of original noisy signal on the time domain, acquire frequency-domain estimation signals of each of the at least two sound sources according to the original noisy signals of the at least two microphones.
  • the division module is configured to, for each of the at least two sound sources, divide the frequency-domain estimation signals into multiple frequency-domain estimation components on a frequency domain.
  • Each frequency-domain estimation component corresponds to a frequency-domain sub-band and includes a plurality of pieces of frequency point data.
  • the decomposition module is configured to, for each of the at least two sound sources, perform feature decomposition on a related matrix of each of the frequency-domain estimation components to obtain a target feature vector corresponding to the frequency-domain estimation component.
  • the first processing module is configured to, for each of the at least two sound sources, obtain a separation matrix of each of frequency points based on the target feature vectors and the frequency-domain estimation signals of the sound source.
  • the second processing module configured to obtain the audio signals of sounds produced by the at least two sound sources based on the separation matrixes and the original noisy signals.
  • a terminal may include a processor and a memory configured to store instructions executable by the processor.
  • the processor may be configured to execute the executable instructions to implement the method for processing an audio signal of any embodiment of the present disclosure.
  • a computer-readable storage medium which stores an executable program.
  • the executable program is executed by a processor to implement the method for processing an audio signal of any embodiment of the present disclosure.
  • FIG. 1 is a flow chart of a method for processing an audio signal according to an exemplary embodiment. As shown in FIG. 1 , the method includes the following operations.
  • audio signals sent by at least two sound sources are acquired by at least two microphones to obtain multiple frames of original noisy signals of each of the at least two microphones on a time domain.
  • the time domain may be a time period for a frame of audio signals that include noises from each of the microphones.
  • the original noisy signals may be audio signals including noises that can be collected via a microphone.
  • frequency-domain estimation signals of each of the at least two sound sources are acquired according to the original noisy signals of the at least two microphones.
  • the frequency-domain estimation signals are divided into multiple frequency-domain estimation components on a frequency domain.
  • the frequency domain may be a frequency range for the frequency-domain estimate component.
  • Each frequency-domain estimation component corresponds to a frequency-domain sub-band and includes multiple pieces of frequency point data.
  • the audio signals of sounds produced by the at least two sound sources are obtained based on the separation matrixes and the original noisy signals.
  • the terminal is an electronic device integrated with two or more than two microphones.
  • the terminal may be an on-vehicle terminal, a computer or a server.
  • the terminal may also be an electronic device connected with a predetermined device integrated with two or more than two microphones, and the electronic device receives an audio signal acquired by the predetermined device based on the connection and sends the processed audio signal to the predetermined device based on the connection.
  • the predetermined device is a speaker.
  • the terminal includes at least two microphones, and the at least two microphones simultaneously detect the audio signals sent by the at least two sound sources, to obtain the original noisy signals of the at least two microphones.
  • the at least two microphones synchronously detect the audio signals sent by the two sound sources.
  • audio signals of audio frames in a predetermined time are separated after original noisy signals of the audio frames in the predetermined time are acquired.
  • the microphones include two or more than two microphones
  • the sound sources include two or more than two sound sources.
  • the original noisy signal is a mixed signal of sounds produced by the at least two sound sources.
  • the original noisy signal of the microphone 1 includes audio signals of the sound source 1 and the sound source 2
  • the original noisy signal of the microphone 2 also includes audio signals of the sound source 1 and the sound source 2.
  • the original noisy signal of the microphone 1 includes audio signals of the sound source 1, the sound source 2 and the sound source 3
  • the original noisy signal of each of the microphone 2 and the microphone 3 also includes audio signals of the sound source 1, the sound source 2 and the sound source 3.
  • a signal of a sound produced by a sound source is an audio signal in a microphone
  • a signal of other sound source in the microphone is a noise signal.
  • the audio signals produced by the at least two sound sources are recovered from the at least two microphones.
  • the number of the sound sources is usually the same as the number of the microphones. In some embodiments, if the number of the microphones is smaller than the number of the sound sources, a dimension of the number of the sound sources may be reduced to a dimension equal to the number of the microphones.
  • the frequency-domain estimation signals may be divided into at least two frequency-domain estimation components in at least two frequency-domain sub-bands.
  • the number of the frequency-domain estimation signals in the frequency-domain estimation components in any two frequency-domain sub-bands may be the same with each other or different from each other.
  • an audio frame may be an audio band with a preset time length.
  • the frequency-domain estimation signals are divided into frequency-domain estimation components in three frequency-domain sub-bands.
  • the frequency-domain estimation components of the first frequency-domain sub-band, the second frequency-domain sub-band and the third frequency-domain sub-band include 25, 35 and 40 frequency-domain estimation signals respectively.
  • there are 100 frequency-domain estimation signals and the frequency-domain estimation signals are divided into frequency-domain estimation components in four frequency-domain sub-bands, each of the frequency-domain estimation components in the four frequency-domain sub-bands includes 25 frequency-domain estimation signals.
  • S14 includes an operation as follows.
  • Feature decomposition is performed on a related matrix of the frequency-domain estimation component to obtain a maximum feature value.
  • a target feature vector corresponding to the maximum feature value is obtained based on the maximum feature value.
  • feature decomposition may be performed on one frequency-domain estimation component to obtain multiple feature values, and one feature vector may be obtained based on one feature value.
  • one target feature vector corresponds to one subspace, and the subspaces corresponding to target feature vectors of the frequency-domain estimation components form a space.
  • signal to noise ratios of the original noisy signal in different subspaces of the space are different.
  • the signal to noise ratio refers to a ratio of the audio signal to the noise signal.
  • the signal to noise ratio of the subspace corresponding to the maximum target feature vector is maximum.
  • the frequency-domain estimation components of the at least two sound sources may be obtained based on the acquired multiple frames of original noisy signals, the frequency-domain estimation signals are divided into at least two frequency-domain estimation components in different frequency-domain sub-bands, feature separation is performed on the related matrix of the frequency-domain estimation component to obtain the target feature vector. Furthermore, the separation matrix of each frequency point is obtained based on the target feature vectors. In this way, the separation matrixes obtained in the embodiment of the present disclosure are determined based on the target feature vectors decomposed from the related matrixes of the frequency-domain estimation components of different frequency-domain sub-bands. Therefore, according to the embodiment of the present disclosure, signals may be decomposed based on subspaces corresponding to the target feature vectors, thereby suppressing a noise signal in each original noisy signal, and improving quality of the separated audio signal.
  • the separation matrix in the embodiment of the present disclosure is determined based on the related matrix of the frequency-domain estimation component of each of the frequency-domain sub-bands. Compared with the separation matrix which is determined based on all the frequency-domain estimation signals of the whole band, the present disclosure takes into consideration that the frequency-domain estimation signals between the frequency-domain sub-bands have the same dependence without considering that all the frequency-domain estimation signals of the whole band have the same dependent, thereby having higher separation performance.
  • the positions of the microphones are not considered in the method for processing an audio signal provided in the embodiment of the present disclosure, thereby implementing high accurate separation for the audio signals of the sounds produced by the sound sources.
  • the method for processing an audio signal is applied to a terminal device with two microphones, compared with the conventional art that voice quality is improved by use of a beamforming technology based on at least more than three microphones, the number of microphones can be greatly reduced in the method, thereby reducing hardware cost of the terminal.
  • separating the original noisy signals by use of the separation matrix obtained based on the maximum target feature vector is implemented by separating the original noisy signals based on the subspace corresponding to the maximum signal to noise ratio, thereby further improving the separation performance, and improving the quality of the separated audio signal.
  • S11 includes an operation as follows.
  • the audio signals sent by the at least two sound sources are simultaneously detected through at least two microphones to obtain each frame of original noisy signal acquired by the at least two microphones on the time domain.
  • S 12 includes an operation as follows.
  • the original noisy signal on the time domain is converted into original noisy signal on the frequency domain, and the original noisy signal on the frequency domain is converted into the frequency-domain estimation signal.
  • frequency-domain transform may be performed on the time-domain signal based on Fast Fourier Transform (FFT).
  • FFT Fast Fourier Transform
  • STFT Short-Time Fourier Transform
  • frequency-domain transform may also be performed on the time-domain signal based on other Fourier transform.
  • each frame of original noisy signal on the frequency domain may be obtained by conversion from the time domain to the frequency domain.
  • each frame of original noisy signal may also be obtained based on another Fourier transform formula, which is not limited herein.
  • the method further includes operations as follows.
  • a first matrix of the cth frequency-domain estimation component is obtained based on a product of the cth frequency-domain estimation component and a conjugate transpose of the cth frequency-domain estimation component.
  • the related matrix of the cth frequency-domain estimation component is acquired based on the first matrixes of the cth frequency-domain estimation components of the first frame to the Nth frame.
  • N denotes the frame number of the original noisy signals
  • c is a positive integer less than or equal to C
  • C denotes the number of the frequency-domain sub-bands.
  • the cth frequency-domain estimation component is denoted as Y c ( n )
  • the conjugate transpose of the cth frequency-domain estimation component of the pth sound source is denoted as Y c (n ) H
  • the obtained first matrix of the cth frequency-domain estimation component is denoted as Y c ( n ) Y c (n ) H
  • the cth frequency-domain estimation component of the pth sound source is denoted as Y p c n
  • the conjugate transpose of the cth frequency-domain estimation component of the pth sound source is denoted as Y p c n H
  • the obtained first matrix of the cth frequency-domain estimation component of the pth sound source is denoted as Y ⁇ p c n Y ⁇ p c n H
  • c is a positive integer less than or equal to C
  • C denotes the number of the frequency-domain sub-bands
  • p is a positive integer less than or equal to P
  • P is the number of the sound sources.
  • the related matrix of the frequency-domain estimation component may be obtained based on the frequency-domain sub-band, and the separation matrix is obtained based on the related matrix. Therefore, the present disclosure takes into consideration that the frequency-domain estimation signals between the frequency-domain sub-bands have the same dependence without considering that all the frequency-domain estimation signals of the whole band have the same dependent, thereby having higher separation performance.
  • S15 includes operations as follows.
  • mapping data of the cth frequency-domain estimation component mapped into a preset space is obtained based on a product of a transposed matrix of the target feature vector of the cth frequency-domain estimation component and the cth frequency-domain estimation component.
  • the separation matrixes are obtained based on the mapping data and iterative operations of the first frame to the Nth frames of original noisy signals.
  • the preset space is the subspace corresponding to the maximum target feature vector.
  • the maximum target feature vector is a target feature vector corresponding to the maximum feature value
  • the preset space is the subspace corresponding to the target feature vector of the maximum feature value
  • the operation that the mapping data of the cth frequency-domain estimation component mapped into the preset space is obtained based on the product of the transposed matrix of the target feature vector of the cth frequency-domain estimation component and the cth frequency-domain estimation component includes operations as follows.
  • mapping data is obtained based on the product of the transposed matrix of the target feature vector of the cth frequency-domain estimation component and the cth frequency-domain estimation component.
  • the mapping data of the cth frequency-domain estimation component mapped into the preset space is obtained based on the alternative mapping data and a first numerical value.
  • the first numerical value is a value obtained by rooting the feature value corresponding to the target feature vector.
  • the mapping data of a frequency-domain estimation component in the corresponding subspace may be obtained based on the product of the transposed matrix of the target feature vector of the frequency-domain estimation component and the frequency-domain estimation component, the mapping data may represent mapping data of the original noisy signal projected into the subspace. Furthermore, the mapping data of the maximum target feature vector projected into the corresponding subspace is obtained based on a product of a transposed matrix of the target feature vector corresponding to the maximum feature value of each frequency-domain estimation component and the frequency-domain estimation component. In this way, the separation matrix obtained based on the mapping data has higher separation performance, thereby improving the quality of the separated audio signal.
  • the method further includes an operation as follows.
  • Nonlinear transform is performed on the mapping data according to a logarithmic function to obtain updated mapping data.
  • nonlinear transform may be performed on the mapping data based on the logarithmic function, for estimating a signal entropy of the mapping data.
  • the separation matrix obtained based on the updated mapping data has higher separation performance, thereby improving the voice quality of the acquired audio signal.
  • the operation that the separation matrix is obtained based on the mapping data and the iterative operations of the first frame to the Nth frames of original noisy signals includes operations as follows.
  • Gradient iteration is performed based on the updated mapping data of the cth frequency-domain estimation component, the frequency-domain estimation signal, the original noisy signal and an (x-1)th alternative matrix, to obtain an xth alternative matrix.
  • a first alternative matrix is a known identity matrix, and x is a positive integer more than or equal to 2.
  • the cth separation matrix is determined based on the xth alternative matrix.
  • gradient iteration may be performed on the alternative matrix.
  • the alternative matrix gets approximate to the required separation matrix every time when gradient iteration is performed.
  • meeting the iteration stopping condition refers to the xth alternative matrix and the (x-1)th alternative matrix meeting a convergence condition.
  • that the xth alternative matrix and the (x-1)th alternative matrix meeting the convergence condition refers to a product of the xth alternative matrix and the (x-1)th alternative matrix being in a predetermined numerical range.
  • the predetermined numerical range is (0.9, 1.1).
  • the operation that gradient iteration is performed based on the updated mapping data of the cth frequency-domain estimation component, the frequency-domain estimation signal, the original noisy signal and the (x-1)th alternative matrix to obtain the xth alternative matrix includes operations as follows.
  • First derivation is performed on the updated mapping data of the cth frequency-domain estimation component to obtain a first derivative.
  • Second derivation is performed on the updated mapping data of the cth frequency-domain estimation component to obtain a second derivative.
  • Gradient iteration is performed based on the first derivative, the second derivative, the frequency-domain estimation signal, the original noisy signal and the (x-1)th alternative matrix to obtain the xth alternative matrix.
  • the above formula meeting the iteration stopping condition may be represented as
  • the operation that the cth separation matrix is determined based on the xth alternative matrix when the xth alternative matrix meets an iteration stopping condition includes operations as follows.
  • the xth alternative matrix meets the iteration stopping condition, the xth alternative matrix is acquired.
  • the cth separation matrix is obtained based on the xth alternative matrix and a conjugate transpose of the xth alternative matrix.
  • the updated separation matrix may be obtained based on the mapping data of the frequency-domain estimation component of each of frequency-domain sub-bands and each frame of frequency-domain estimation signal and the like, and separation is performed on the original noisy signal based on the updated separation matrix, thereby obtaining better separation performance, and further improving accuracy of the separated audio signal.
  • the operation that the separation matrixes are obtained based on the mapping data and the iterative operations of the first frame to the Nth frames of original noisy signals may also be implemented as follows.
  • Gradient iteration is performed based on the mapping data of the cth frequency-domain estimation component, the frequency-domain estimation signal, the original noisy signal and an (x-1)th alternative matrix, to obtain an xth alternative matrix.
  • a first alternative matrix is a known identity matrix, and x is a positive integer more than or equal to 2.
  • the cth separation matrix is determined based on the xth alternative matrix.
  • the operation that gradient iteration is performed based on the mapping data of the cth frequency-domain estimation component, the frequency-domain estimation signal, the original noisy signal and the (x-1)th alternative matrix to obtain the xth alternative matrix includes operations as follows.
  • First derivation is performed on the mapping data of the cth frequency-domain estimation component to obtain a first derivative.
  • Second derivation is performed on the mapping data of the cth frequency-domain estimation component to obtain a second derivative.
  • Gradient iteration is performed based on the first derivative, the second derivative, the frequency-domain estimation signal, the original noisy signal and the (x-1)th alternative matrix to obtain the xth alternative matrix.
  • the mapping data is non-updated mapping data.
  • the separation matrix may also be acquired based on the non-updated mapping data, and signal decomposition is also performed on the mapping data based on the space corresponding to the target feature vector, thereby suppressing the noise signals in various original noisy signals, and improving the quality of the separated audio signal.
  • mapping data is used, and it is unnecessary to perform nonlinear transform on the mapping data according to the logarithmic function, thereby simplifying calculation for the separation matrix to a certain extent.
  • the operation that the original noisy signal on the frequency domain is converted into the frequency-domain estimation signals includes an operation that the original noisy signal on the frequency domain is converted into the frequency-domain estimation signals based on a known identity matrix.
  • the operation that the original noisy signal on the frequency domain is converted into the frequency-domain estimation signals includes an operation that the original noisy signal on the frequency domain is converted into the frequency-domain estimation signals based on an alternative matrix.
  • the alternative matrix may be the first alternative matrix to the (x-1)th alternative matrix in the abovementioned embodiment.
  • W ( k ) is a known identity matrix or an alternative matrix obtained by (x-1)th iteration.
  • the known identity matrix may be used as a separation matrix during first iteration.
  • the alternative matrix obtained by the previous iteration may be used as a separation matrix for the subsequent iteration, so that a basis is provided for acquisition of the separation matrix.
  • the operation that the audio signals of the sounds produced by the at least two sound sources are obtained based on the separation matrixes and the original noisy signals includes operations as follows.
  • n is a positive integer less than N.
  • the audio signals of the pth sound source in the nth frame of original noisy signal corresponding to the frequency-domain estimation signals are combined to obtain a nth frame of audio signal of the pth sound source, where p is a positive integer less than or equal to P, and P is the number of the sound sources.
  • the microphone 1 and the microphone 2 acquires three frames of original noisy signals.
  • separation matrixes corresponding to a first frequency-domain estimation signal to a Cth frequency-domain estimation signal are calculated.
  • the separation matrix of the first frequency-domain estimation signal is a first separation matrix
  • the separation matrix of the second frequency-domain estimation signal is a second separation matrix
  • the separation matrix of the Cth frequency-domain estimation signal is a Cth separation matrix.
  • an audio signal of the first frequency-domain estimation signal is acquired based on a noise signal corresponding to the first frequency-domain estimation signal and the first separation matrix
  • an audio signal of the second frequency-domain estimation signal is obtained based on a noise signal corresponding to the second frequency-domain estimation signal and the second separation matrix
  • an audio signal of the Cth frequency-domain estimation signal is obtained based on a noise signal corresponding to the Cth frequency-domain estimation signal and the Cth separation matrix.
  • the audio signal of the first frequency-domain estimation signal, the audio signal of the second frequency-domain estimation signal and the audio signal of the third frequency-domain estimation signal are combined to obtain first frame audio signals of the microphone 1 and the microphone 2.
  • the audio signals of frequency-domain estimation signals in the frame may be obtained based on the noise signals and separation matrixes corresponding to the frequency-domain estimation signals in the frame, and then the audio signals of the frequency-domain estimation signals in the frame are combined to obtain a first frame audio signal.
  • time-domain transform may further be performed on the audio signal to obtain the audio signal of each sound source on the time domain.
  • time-domain transform may be performed on the frequency-domain signal based on Inverse Fast Fourier Transform (IFFT).
  • IFFT Inverse Fast Fourier Transform
  • ISTFT Inverse Short-Time Fourier Transform
  • time-domain transform may also be performed on the frequency-domain signal based on other Inverse Fourier transform.
  • the method further includes an operation that the first frame audio signal to the Nth frame audio signal of the pth sound source are combined in time chorological to obtain N frames of original noisy signals comprising the audio signal of the pth sound source.
  • two microphones i.e., a microphone 1 and a microphone 2
  • two sound sources i.e., a sound source 1 and a sound source 2
  • Each of the microphone 1 and the microphone 2 acquires three frames of original noisy signals, the three frames include a first frame, a second frame and a third frame in chronological order.
  • the first frame audio signal, the second frame audio signal and the third frame audio signal of the sound source 1 are obtained by calculation, and the audio signal of the sound source 1 is obtained by combining the first frame audio signal, the second frame audio signal and the third frame audio signal of the sound source 1 in chronological order.
  • the first frame audio signal, the second frame audio signal and the third frame audio signal of the sound source 2 are obtained, and the audio signal of the sound source 2 is obtained by combining the first frame audio signal, the second frame audio signal and the third frame audio signal of the sound source 2 in chronological order.
  • the audio signals of all audio frames of the sound source may be combined, to obtain the complete audio signal of the sound source.
  • a terminal includes a speaker A
  • the speaker A includes two microphones, i.e., a microphone 1 and a microphone 2 respectively
  • two sound sources i.e., a sound source 1 and a sound source 2 are included.
  • Signals sent by the sound source 1 and the sound source 2 may be acquired by the microphone 1 and the microphone 2.
  • the signals of the two sound sources are mixed in each microphone.
  • FIG. 3 is a flow chart of a method for processing an audio signal according to an exemplary embodiment.
  • sound sources include a sound source 1 and a sound source 2
  • microphones include a microphone 1 and a microphone 2.
  • the sound source 1 and the sound source 2 are recovered from signals of the microphone 1 and the microphone 2.
  • the method includes the following operations.
  • a separation matrix of each frequency point is initialized.
  • the time-domain signal is an original noisy signal.
  • the priori frequency-domain estimation is the frequency-domain estimation signal in the abovementioned embodiment.
  • the whole band is divided into at least two frequency-domain sub-bands.
  • the whole band is divided into C frequency-domain sub-bands.
  • mapping data of projection in a subspace is acquired.
  • mapping data q p c ⁇ ⁇ p c T Y ⁇ p c n of a frequency-domain estimation component of the cth frequency-domain sub-band mapped into a subspace corresponding to the target feature vector is obtained based on ⁇ p c , where ⁇ p c T is a transposed matrix of ⁇ p c .
  • mapping data is implemented by performing nonlinear transform on the mapping data according to a logarithmic function.
  • ⁇ is a value less than or equal to (1/10 6 ).
  • the point k is in the cth frequency-domain sub-band.
  • gradient iteration is performed according to a sequence from high frequency to low frequency. Therefore, the separation matrix of each frequency of each frequency-domain sub-band may be updated.
  • pseudo codes for sequentially acquiring the separation matrix of each frequency-domain estimation signal are provided below.
  • converged [m] [k] 1, it indicates that the frequency point has been converged, otherwise it is not converged.
  • ⁇ denotes a threshold for determining convergence of W ( k ), and ⁇ is (1/10 6 ).
  • Y p ( k ,m) W p ( k )
  • time-domain transform is performed on the audio signal on a frequency domain.
  • Time-domain transform is performed on the audio signal on the frequency domain to obtain an audio signal on a time domain.
  • the mapping data of the maximum target feature vector projected into the corresponding subspace may be obtained based on a product of a transposed matrix of the target feature vector corresponding to the maximum feature value of each frequency-domain estimation component and the frequency-domain estimation component.
  • the original noisy signals are decomposed based on the subspace corresponding to the maximum signal to noise ratio, thereby suppressing a noise signal in each original noisy signal, improving separation performance, and further improving quality of the separated audio signal.
  • the method for processing an audio signal provided in the embodiment of the present disclosure can realize high-accurate separation for the audio signals of the sounds produced by the sound sources without considering the positions of these microphones.
  • only two microphones are used in the embodiment of the present disclosure, thereby greatly reducing the number of microphones and reducing hardware cost of the terminal, compared with the conventional art that voice quality is improved by use of a beamforming technology based on at least more than three microphones.
  • FIG. 4 is a block diagram of a device for processing an audio signal according to an exemplary embodiment.
  • the device includes an acquisition module 41, a conversion module 42, a division module 43, a decomposition module 44, a first processing module 45 and a second processing module 46.
  • the acquisition module 41 is configured to acquire audio signals sent by at least two sound sources through at least two microphones, to obtain multiple frames of original noisy signals of each of the at least two microphones on a time domain.
  • the conversion module 42 is configured to, for each frame on the time domain, acquire frequency-domain estimation signals of each of the at least two sound sources according to the original noisy signals of the at least two microphones.
  • the division module 43 is configured to, for each of the at least two sound sources, divide the frequency-domain estimation signals into multiple frequency-domain estimation components on a frequency domain.
  • Each frequency-domain estimation component corresponds to a frequency-domain sub-band and includes multiple pieces of frequency point data.
  • the decomposition module 44 is configured to, for each sound source, perform feature decomposition on a related matrix of each of the frequency-domain estimation components to obtain a target feature vector corresponding to the frequency-domain estimation component.
  • the first processing module 45 is configured to, for each sound source, obtain a separation matrix of each frequency point based on the target feature vectors and the frequency-domain estimation signals of the sound source.
  • the second processing module 46 is configured to obtain the audio signals of sounds produced by the at least two sound sources based on the separation matrixes and the original noisy signals.
  • the acquisition module 41 is configured to, for each sound source, obtain a first matrix of the cth frequency-domain estimation component based on a product of the cth frequency-domain estimation component and a conjugate transpose of the cth frequency-domain estimation component; acquire the related matrix of the cth frequency-domain estimation component based on the first matrixes of the cth frequency-domain estimation component in the first frame to the Nth frame, N being the number of frames of the original noisy signals, c being a positive integer less than or equal to C and C being the number of the frequency-domain sub-bands.
  • the first processing module 45 is configured to, for each sound source, obtain mapping data of the cth frequency-domain estimation component mapped into a preset space based on a product of a transposed matrix of the target feature vector of the cth frequency-domain estimation component and the cth frequency-domain estimation component; and obtain the separation matrixes based on the mapping data and iterative operations of the first frame original noisy signal to the Nth frame original noisy signal.
  • the first processing module 45 is further configured to perform nonlinear transform on the mapping data according to a logarithmic function to obtain updated mapping data.
  • the first processing module 45 is configured to perform gradient iteration based on the updated mapping data of the cth frequency-domain estimation component, the frequency-domain estimation signal, the original noisy signal and an (x-1)th alternative matrix to obtain an xth alternative matrix.
  • a first alternative matrix is a known identity matrix and x is a positive integer more than or equal to 2, and when the xth alternative matrix meets an iteration stopping condition, determine the cth separation matrix based on the xth alternative matrix.
  • the first processing module 45 is configured to perform first derivation on the updated mapping data of the cth frequency-domain estimation component to obtain a first derivative, perform second derivation on the updated mapping data of the cth frequency-domain estimation component to obtain a second derivative and perform gradient iteration based on the first derivative, the second derivative, the frequency-domain estimation signal, the original noisy signal and the (x-1)th alternative matrix to obtain the xth alternative matrix.
  • the second processing module 46 is configured to perform separation on the nth frame of original noisy signal corresponding to each of the frequency-domain estimation signals based on the first separation matrix to the Cth separation matrix, to obtain audio signals of different sound sources in the nth frame of original noisy signal corresponding to the frequency-domain estimation signal, where n being a positive integer less than N; and combine the audio signals of the pth sound source in the nth frame of original noisy signal corresponding to the frequency-domain estimation signals to obtain a nth frame audio signal of the pth sound source, wherein p being a positive integer less than or equal to P and P being the number of the sound sources.
  • the second processing module 46 is further configured to combine first frame audio signal to Nth frame audio signal of the pth sound source in chronological order to obtain N frames of original noisy signals comprising the audio signal of the pth sound source.
  • the embodiments of the present disclosure also provide a terminal, which includes a processor; and a memory configured to store an instruction executable for a processor.
  • the processor is configured to execute the executable instruction to implement the method for processing an audio signal of any embodiment of the present disclosure.
  • the memory may include various types of storage mediums, and the storage medium is a non-transitory computer storage medium and may store information in a communication device after the communication device powers down.
  • the processor may be connected with the memory through a bus and the like, and is configured to read an executable program stored in the memory to implement, for example, at least one of the methods illustrated in FIG. 1 and FIG. 3 .
  • the embodiments of the present disclosure also provide a computer-readable storage medium, which stores an executable program.
  • the executable program is executed by a processor to implement the method for processing an audio signal according to any embodiment of the present disclosure, for implementing, for example, at least one of the methods illustrated in FIG. 1 and FIG. 3 .
  • FIG. 5 is a block diagram of a terminal 800 according to an exemplary embodiment.
  • the terminal 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a gaming console, a tablet, a medical device, exercise equipment, a personal digital assistant and the like.
  • the terminal 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an Input/Output (I/O) interface 812, a sensor component 814, and a communication component 816.
  • a processing component 802 a memory 804
  • a power component 806 a multimedia component 808, an audio component 810, an Input/Output (I/O) interface 812, a sensor component 814, and a communication component 816.
  • I/O Input/Output
  • the processing component 802 typically controls overall operations of the terminal 800, such as the operations associated with display, telephone calls, data communications, camera operations, and recording operations.
  • the processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps in the abovementioned method.
  • the processing component 802 may include one or more modules which facilitate interaction between the processing component 802 and the other components.
  • the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
  • the memory 804 is configured to store various types of data to support the operation of the device 800. Examples of such data include instructions for any application programs or methods operated on the terminal 800, contact data, phonebook data, messages, pictures, video, etc.
  • the memory 804 may be implemented by any type of volatile or non-volatile memory devices, or a combination thereof, such as an Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic memory, a flash memory, and a magnetic or optical disk.
  • SRAM Static Random Access Memory
  • EEPROM Electrically Erasable Programmable Read-Only Memory
  • EPROM Erasable Programmable Read-Only Memory
  • PROM Programmable Read-Only Memory
  • ROM Read-Only Memory
  • magnetic memory a magnetic memory
  • flash memory and a magnetic or optical disk.
  • the power component 806 provides power for various components of the terminal 800.
  • the power component 806 may include a power management system, one or more power supplies, and other components associated with generation, management and distribution of power for the terminal 800.
  • the multimedia component 808 includes a screen providing an output interface between the terminal 800 and a user.
  • the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes the TP, the screen may be implemented as a touch screen to receive an input signal from the user.
  • the TP includes one or more touch sensors to sense touches, swipes and gestures on the TP. The touch sensors may not only sense a boundary of a touch or swipe action but also detect a duration and pressure associated with the touch or swipe action.
  • the multimedia component 808 includes a front camera and/or a rear camera.
  • the front camera and/or the rear camera may receive external multimedia data when the device 800 is in an operation mode, such as a photographing mode or a video mode.
  • an operation mode such as a photographing mode or a video mode.
  • Each of the front camera and the rear camera may be a fixed optical lens system or have focusing and optical zooming capabilities.
  • the audio component 810 is configured to output and/or input an audio signal.
  • the audio component 810 includes a microphone (MIC), and the MIC is configured to receive an external audio signal when the terminal 800 is in the operation mode, such as a call mode, a recording mode and a voice recognition mode.
  • the received audio signal may further be stored in the memory 804 or sent through the communication component 816.
  • the audio component 810 further includes a speaker configured to output the audio signal.
  • the I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, and the peripheral interface module may be a keyboard, a click wheel, a button and the like.
  • the button may include, but be not limited to: a home button, a volume button, a starting button and a locking button.
  • the sensor component 814 includes one or more sensors configured to provide status assessment in various aspects for the terminal 800. For instance, the sensor component 814 may detect an on/off status of the device 800 and relative positioning of components, such as a display and small keyboard of the terminal 800, and the sensor component 814 may further detect a change in a position of the terminal 800 or a component of the terminal 800, presence or absence of contact between the user and the terminal 800, orientation or acceleration/deceleration of the terminal 800 and a change in temperature of the terminal 800.
  • the sensor component 814 may include a proximity sensor configured to detect presence of an object nearby without any physical contact.
  • the sensor component 814 may also include a light sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, configured for use in an imaging application.
  • CMOS Complementary Metal Oxide Semiconductor
  • CCD Charge Coupled Device
  • the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.
  • the communication component 816 is configured to facilitate wired or wireless communication between the terminal 800 and another device.
  • the terminal 800 may access a communication-standard-based wireless network, such as a Wireless Fidelity (WiFi) network, a 2nd-Generation (2G) or 3rd-Generation (3G) network or a combination thereof.
  • WiFi Wireless Fidelity
  • 2G 2nd-Generation
  • 3G 3rd-Generation
  • the communication component 816 receives a broadcast signal or broadcast associated information from an external broadcast management system through a broadcast channel.
  • the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication.
  • NFC Near Field Communication
  • the NFC module may be implemented based on a Radio Frequency Identification (RFID) technology, an Infrared Data Association (IrDA) technology, an Ultra-Wide Band (UWB) technology, a Bluetooth (BT) technology and another technology.
  • RFID Radio Frequency Identification
  • IrDA Infrared Data Association
  • UWB Ultra-Wide Band
  • BT Bluetooth
  • the terminal 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components, and is configured to execute the abovementioned method.
  • ASICs Application Specific Integrated Circuits
  • DSPs Digital Signal Processors
  • DSPDs Digital Signal Processing Devices
  • PLDs Programmable Logic Devices
  • FPGAs Field Programmable Gate Arrays
  • controllers micro-controllers, microprocessors or other electronic components, and is configured to execute the abovementioned method.
  • a non-transitory computer-readable storage medium including an instruction is further provided, such as the memory 804 including an instruction, and the instruction may be executed by the processor 820 of the terminal 800 to implement the abovementioned method.
  • the non-transitory computer-readable storage medium may be an ROM, a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disc, an optical data storage device and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Quality & Reliability (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Circuit For Audible Band Transducer (AREA)

Claims (13)

  1. Procédé de traitement d'un signal audio, le procédé comprenant :
    l'acquisition, par l'intermédiaire d'au moins deux microphones, des signaux audio envoyés par au moins deux sources sonores, pour obtenir une pluralité de trames de signaux bruités d'origine de chacun des au moins deux microphones sur un domaine temporel (S11) ;
    pour chaque trame des signaux bruités d'origine sur le domaine temporel, l'acquisition des signaux d'estimation de domaine fréquentiel de chacune des au moins deux sources sonores en fonction des signaux bruités d'origine des au moins deux microphones (S12) ;
    pour chacune des au moins deux sources sonores, la division des signaux d'estimation de domaine fréquentiel en une pluralité de composants d'estimation de domaine fréquentiel sur la base d'un domaine fréquentiel (S13), dans lequel chaque composant d'estimation de domaine fréquentiel correspond à une sous-bande de domaine fréquentiel et comprend une pluralité d'éléments de données de point fréquentiel ;
    pour chacune des au moins deux sources sonores, la réalisation d'une décomposition de caractéristiques sur une matrice connexe de chacun des composants d'estimation de domaine fréquentiel pour obtenir un vecteur caractéristique cible correspondant au composant d'estimation de domaine fréquentiel (S14) ;
    pour chacune des au moins deux sources sonores, l'obtention d'une matrice de séparation de chacun des points de fréquence sur la base des vecteurs de caractéristiques cibles et des signaux d'estimation de domaine fréquentiel de la source sonore (S15) ; et
    l'obtention des signaux audio de sons produits par au moins deux sources sonores sur la base des matrices de séparation et des signaux bruités d'origine (S16),
    dans lequel pour chacune des au moins deux sources sonores, l'obtention des matrices de séparation des points de fréquence sur la base des vecteurs de caractéristiques cibles et des signaux d'estimation de domaine fréquentiel de la source sonore (S15) comprend :
    pour chacune des au moins deux sources sonores, l'obtention de données de mappage du cième composant d'estimation de domaine fréquentiel mis en correspondance dans un espace prédéfini sur la base d'un produit d'une matrice transposée du vecteur caractéristique cible du cième composant d'estimation de domaine fréquentiel et du cième composant d'estimation de domaine fréquentiel ; et
    l'obtention des matrices de séparation sur la base des données de mappage et des opérations itératives du premier signal bruité d'origine de trame au Nième signal bruité d'origine de trame.
  2. Procédé selon la revendication 1, comprenant en outre :
    pour chacune des au moins deux sources sonores, l'obtention d'une première matrice d'un cième composant d'estimation de domaine fréquentiel sur la base d'un produit du cième composant d'estimation de domaine fréquentiel et d'une transposée conjuguée du cième composant d'estimation de domaine fréquentiel ; et
    l'acquisition d'une matrice connexe du cième composant d'estimation de domaine fréquentiel sur la base de premières matrices du cième composant d'estimation de domaine fréquentiel dans un premier signal bruité d'origine de trame à un Nième signal bruité d'origine de trame, N étant un nombre de trames des signaux bruités d'origine, c étant un nombre entier positif inférieur ou égal à C et C étant le nombre des sous-bandes de domaine fréquentiel.
  3. Procédé selon l'une quelconque des revendications 1 à 2, comprenant en outre :
    la réalisation d'une transformation non linéaire sur les données de mappage selon une fonction logarithmique pour obtenir des données de mappage mises à jour.
  4. Procédé selon l'une quelconque des revendications 1 à 3, dans lequel l'obtention des matrices de séparation sur la base des données de mappage et des opérations itératives du premier signal bruité d'origine de trame au Nième signal bruité d'origine de trame comprend :
    la réalisation d'une itération de gradient sur la base des données de mappage mises à jour du cième composant d'estimation de domaine fréquentiel, du signal d'estimation de domaine fréquentiel, du signal bruité d'origine et d'une (x-1 )ième matrice alternative pour obtenir une xième matrice alternative, dans lequel une première matrice alternative est une matrice d'identité connue et x est un nombre entier positif supérieur ou égal à 2 ; et
    la détermination d'une cième matrice de séparation sur la base de la xième matrice alternative lorsque la xième matrice alternative satisfait une condition d'arrêt d'itération.
  5. Procédé selon la revendication 4, dans lequel l'exécution de l'itération de gradient sur la base des données de mappage mises à jour du cième composant d'estimation de domaine fréquentiel, du signal d'estimation de domaine fréquentiel, du signal bruité d'origine et de la (x-1)ème matrice alternative pour obtenir la xième matrice alternative comprend :
    la réalisation d'une première dérivation sur les données de mappage mises à jour du cième composant d'estimation de domaine fréquentiel pour obtenir une première dérivée ;
    la réalisation d'une seconde dérivation sur les données de mappage mises à jour du cième composant d'estimation de domaine fréquentiel pour obtenir une seconde dérivée ; et
    la réalisation de l'itération de gradient sur la base de la première dérivée, de la seconde dérivée, du signal d'estimation de domaine fréquentiel, du signal bruité d'origine et de la (x-1)ième matrice alternative pour obtenir la xième matrice alternative.
  6. Procédé selon l'une quelconque des revendications 1 à 5, dans lequel l'obtention des signaux audio de sons produits par les au moins deux sources sonores sur la base des matrices de séparation et des signaux bruités d'origine (S16) comprend :
    pour chacun des signaux d'estimation de domaine fréquentiel, la réalisation d'une séparation sur un nième signal bruité d'origine de trame correspondant au signal d'estimation de domaine fréquentiel sur la base d'une première matrice de séparation à une Cième matrice de séparation, pour obtenir des signaux audio de sources sonores différentes dans le Nième signal bruité d'origine de trame correspondant au signal d'estimation de domaine fréquentiel, n étant un entier positif inférieur à N ; et
    la combinaison des signaux audio d'une pième source sonore dans le nième signal bruité d'origine de trame correspondant à tous les signaux d'estimation de domaine fréquentiel pour obtenir un nième signal audio de trame de la pième source sonore, p étant un entier positif inférieur ou égal à P et P étant le nombre des sources sonores.
  7. Procédé selon l'une quelconque des revendications 1 à 6, comprenant en outre :
    la combinaison d'un premier signal audio de trame à un Nième signal audio de trame de la pième source sonore en ordre chronologique pour obtenir N trames de signaux bruités originaux comprenant le signal audio de la pième source sonore.
  8. Dispositif de traitement d'un signal audio, comprenant :
    un module d'acquisition (41) configuré pour acquérir, à travers au moins deux microphones, des signaux audio envoyés par au moins deux sources sonores, pour obtenir une pluralité de trames de signaux bruités d'origine de chacun des au moins deux microphones sur un domaine temporel ;
    un module de conversion (42) configuré pour, pour chaque trame du signal bruité d'origine sur le domaine temporel, acquérir des signaux d'estimation de domaine fréquentiel de chacune des au moins deux sources sonores selon les signaux bruités d'origine des au moins deux microphones ;
    un module de division (43) configuré pour, pour chacune des au moins deux sources sonores, diviser les signaux d'estimation de domaine fréquentiel en une pluralité de composants d'estimation de domaine fréquentiel sur un domaine fréquentiel, dans lequel chaque composant d'estimation de domaine fréquentiel correspond à une sous-bande de domaine fréquentiel et comprend une pluralité d'éléments de données de point fréquentiel ;
    un module de décomposition (44) configuré pour, pour chacune des au moins deux sources sonores, réaliser une décomposition de caractéristiques sur une matrice connexe de chacun des composants d'estimation de domaine fréquentiel pour obtenir un vecteur caractéristique cible correspondant au composant d'estimation de domaine fréquentiel ;
    un premier module de traitement (45) configuré pour, pour chacune des au moins deux sources sonores, obtenir une matrice de séparation de chacun des points de fréquence sur la base des vecteurs de caractéristiques cibles et des signaux d'estimation du domaine fréquentiel de la source sonore ; et
    un deuxième module de traitement (46) configuré pour obtenir les signaux audio de sons produits par les au moins deux sources sonores sur la base des matrices de séparation et des signaux bruités d'origine,
    dans lequel le premier module de traitement (45) est configuré pour :
    pour chacune des au moins deux sources sonores, l'obtention des données de mappage du cième composant d'estimation de domaine fréquentiel mis en correspondance dans un espace prédéfini sur la base d'un produit d'une matrice transposée du vecteur caractéristique cible du cième composant d'estimation de domaine fréquentiel et du cième composant d'estimation de domaine fréquentiel ; et
    l'obtention des matrices de séparation sur la base des données de mappage et des opérations itératives du premier signal bruité d'origine de trame au Nième signal bruité d'origine de trame,
    dans lequel le premier module de traitement (45) est en outre configuré pour effectuer une transformation non linéaire sur les données de mappage selon une fonction logarithmique pour obtenir des données de mappage mises à jour.
  9. Dispositif selon la revendication 8, dans lequel le module d'acquisition (41) est configuré pour :
    pour chacune des au moins deux sources sonores, obtenir une première matrice d'un cième composant d'estimation de domaine fréquentiel sur la base d'un produit du cième composant d'estimation de domaine fréquentiel et d'une transposée conjuguée du cième composant d'estimation de domaine fréquentiel ; et
    acquérir une matrice connexe du cième composant d'estimation de domaine fréquentiel sur la base des premières matrices du cième composant d'estimation de domaine fréquentiel dans un premier signal bruité d'origine de trame à un Nième signal bruité d'origine de trame, N étant un nombre de trames des signaux bruités d'origine, c étant un entier positif inférieur ou égal à C et C étant un nombre des sous-bandes de domaine fréquentiel.
  10. Dispositif selon l'une quelconque des revendications 8 à 9, dans lequel le premier module de traitement (45) est configuré pour : effectuer une itération de gradient sur la base des données de mappage mises à jour du cième composant d'estimation de domaine fréquentiel, du signal d'estimation de domaine fréquentiel, du signal bruité d'origine et d'une (x-1)ème matrice alternative pour obtenir une xième matrice alternative, dans lequel une première matrice alternative est une matrice d'identité connue et x étant un entier positif supérieur ou égal à 2 ; et
    déterminer une cième matrice de séparation sur la base de la xième matrice alternative lorsque la xième matrice alternative respecte une condition d'arrêt d'itération,
    dans lequel le premier module de traitement (45) est configuré pour :
    effectuer une première dérivation sur les données de mappage mises à jour du cième composant d'estimation de domaine fréquentiel pour obtenir une première dérivée ;
    effectuer une seconde dérivation sur les données de mappage mises à jour du cième composant d'estimation de domaine fréquentiel pour obtenir une seconde dérivée ; et
    effectuer une itération de gradient sur la base de la première dérivée, de la seconde dérivée, du signal d'estimation de domaine fréquentiel, du signal bruité d'origine et de la (x-1) ième matrice alternative pour obtenir la xième matrice alternative.
  11. Dispositif selon l'une quelconque des revendications 8 à 10, dans lequel le second module de traitement (46) est configuré pour : pour chacun des signaux d'estimation de domaine fréquentiel, réaliser une séparation sur le nième signal bruité d'origine de trame correspondant au signal d'estimation de domaine fréquentiel sur la base d'une première matrice de séparation à une Cième matrice de séparation, pour obtenir des signaux audio de sources sonores différentes dans le nième signal bruité d'origine de trame correspondant au signal d'estimation de domaine fréquentiel, n étant un entier positif inférieur à N ; et
    combiner les signaux audio d'une pième source sonore dans le nième signal bruité d'origine de trame correspondant à tous les signaux d'estimation de domaine fréquentiel pour obtenir un nième signal audio de trame de la pième source sonore, p étant un entier positif inférieur ou égal à P et P étant le nombre des sources sonores, dans lequel le deuxième module de traitement (46) est en outre configuré pour :
    combiner un premier signal audio de trame à un Nième signal audio de trame de la pième source sonore en ordre chronologique pour obtenir N trames de signaux bruités originaux comprenant le signal audio de la pième source sonore.
  12. Terminal, comprenant :
    un processeur ; et
    une mémoire configurée pour stocker des instructions exécutables par le processeur,
    dans lequel le processeur est configuré pour exécuter les instructions exécutables afin de mettre en oeuvre le procédé de traitement d'un signal audio selon l'une quelconque des revendications 1 à 7.
  13. Support de stockage lisible par ordinateur stockant un programme exécutable, le programme exécutable étant exécuté par un processeur pour mettre en oeuvre le procédé de traitement d'un signal audio selon l'une quelconque des revendications 1 à 7.
EP20180826.8A 2019-12-17 2020-06-18 Procédé et dispositif de traitement de signal audio, terminal et support d'enregistrement Active EP3839951B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911301727.2A CN111009256B (zh) 2019-12-17 2019-12-17 一种音频信号处理方法、装置、终端及存储介质

Publications (2)

Publication Number Publication Date
EP3839951A1 EP3839951A1 (fr) 2021-06-23
EP3839951B1 true EP3839951B1 (fr) 2024-01-24

Family

ID=70116520

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20180826.8A Active EP3839951B1 (fr) 2019-12-17 2020-06-18 Procédé et dispositif de traitement de signal audio, terminal et support d'enregistrement

Country Status (3)

Country Link
US (1) US11284190B2 (fr)
EP (1) EP3839951B1 (fr)
CN (1) CN111009256B (fr)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111724801B (zh) * 2020-06-22 2024-07-30 北京小米松果电子有限公司 音频信号处理方法及装置、存储介质
CN111916075A (zh) * 2020-07-03 2020-11-10 北京声智科技有限公司 一种音频信号的处理方法、装置、设备及介质
CN112599144B (zh) * 2020-12-03 2023-06-06 Oppo（重庆）智能科技有限公司 音频数据处理方法、音频数据处理装置、介质与电子设备
CN112750455A (zh) * 2020-12-29 2021-05-04 苏州思必驰信息科技有限公司 音频处理方法及装置
CN112863537B (zh) * 2021-01-04 2024-06-04 北京小米松果电子有限公司 一种音频信号处理方法、装置及存储介质
CN113053406B (zh) * 2021-05-08 2024-06-18 北京小米移动软件有限公司 声音信号识别方法及装置
CN113314135B (zh) * 2021-05-25 2024-04-26 北京小米移动软件有限公司 声音信号识别方法及装置
CN113409813B (zh) * 2021-05-26 2023-06-06 北京捷通华声科技股份有限公司 语音分离方法及装置
CN113096684A (zh) * 2021-06-07 2021-07-09 成都启英泰伦科技有限公司 一种基于双麦克风阵列的目标语音提取方法
CN113362848B (zh) * 2021-06-08 2022-10-04 北京小米移动软件有限公司 音频信号处理方法、装置及存储介质
CN113362864B (zh) * 2021-06-16 2022-08-02 北京字节跳动网络技术有限公司 音频信号处理的方法、装置、存储介质及电子设备
CN117172135B (zh) * 2023-11-02 2024-02-06 山东省科霖检测有限公司 一种智能噪声监测管理方法与系统

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4449871B2 (ja) * 2005-01-26 2010-04-14 ソニー株式会社 音声信号分離装置及び方法
JP4675177B2 (ja) * 2005-07-26 2011-04-20 株式会社神戸製鋼所 音源分離装置，音源分離プログラム及び音源分離方法
JP4897519B2 (ja) * 2007-03-05 2012-03-14 株式会社神戸製鋼所 音源分離装置，音源分離プログラム及び音源分離方法
WO2009151578A2 (fr) * 2008-06-09 2009-12-17 The Board Of Trustees Of The University Of Illinois Procédé et appareil de récupération de signal aveugle dans des environnements bruyants et réverbérants
CN102890936A (zh) * 2011-07-19 2013-01-23 联想(北京)有限公司 一种音频处理方法、终端设备及系统
JP5568530B2 (ja) * 2011-09-06 2014-08-06 日本電信電話株式会社 音源分離装置とその方法とプログラム
EP2912660B1 (fr) * 2012-11-21 2017-01-11 Huawei Technologies Co., Ltd. Procede pour determiner une dictionnaire de composants de base d'un signal audio
CN106405501B (zh) * 2015-07-29 2019-05-17 中国科学院声学研究所 一种基于相位差回归的单声源定位方法
JP6434657B2 (ja) * 2015-12-02 2018-12-05 日本電信電話株式会社 空間相関行列推定装置、空間相関行列推定方法および空間相関行列推定プログラム
JP6622159B2 (ja) * 2016-08-31 2019-12-18 株式会社東芝 信号処理システム、信号処理方法およびプログラム
JP6454916B2 (ja) * 2017-03-28 2019-01-23 本田技研工業株式会社 音声処理装置、音声処理方法及びプログラム
EP3392882A1 (fr) * 2017-04-20 2018-10-24 Thomson Licensing Procédé de traitement d'un signal audio et dispositif électronique correspondant, produit-programme lisible par ordinateur non transitoire et support d'informations lisible par ordinateur
CN110473565A (zh) * 2019-07-04 2019-11-19 中国人民解放军63892部队 一种无需源数估计的独立向量分析信号分离方法

Also Published As

Publication number Publication date
US20210185438A1 (en) 2021-06-17
EP3839951A1 (fr) 2021-06-23
CN111009256A (zh) 2020-04-14
CN111009256B (zh) 2022-12-27
US11284190B2 (en) 2022-03-22

Similar Documents

Publication Publication Date Title
EP3839951B1 (fr) Procédé et dispositif de traitement de signal audio, terminal et support d'enregistrement
EP3839949A1 (fr) Procédé et dispositif de traitement de signal audio, terminal et support d'enregistrement
CN111128221B (zh) 一种音频信号处理方法、装置、终端及存储介质
US11490200B2 (en) Audio signal processing method and device, and storage medium
CN111429933B (zh) 音频信号的处理方法及装置、存储介质
CN111179960B (zh) 音频信号处理方法及装置、存储介质
EP3657497B1 (fr) Procédé et dispositif de sélection de données de faisceau cible à partir d'une pluralité de faisceaux
CN110133594B (zh) 一种声源定位方法、装置和用于声源定位的装置
EP3929920B1 (fr) Procédé et dispositif de traitement de signal audio et support d'enregistrement
CN113223553B (zh) 分离语音信号的方法、装置及介质
CN112863537B (zh) 一种音频信号处理方法、装置及存储介质
EP3779985B1 (fr) Procédé d'estimation du bruit de signal audio, et dispositif et support d'enregistrement
CN111667842A (zh) 音频信号处理方法及装置
CN111429934B (zh) 音频信号处理方法及装置、存储介质
CN114724578A (zh) 一种音频信号处理方法、装置及存储介质
CN115767346A (zh) 耳机风噪处理方法、装置及存储介质
CN114566175A (zh) 一种语音增强及模型训练方法、装置和电子设备

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20211216

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20230922

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20231108

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602020024706

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20240124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240524

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240621

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240619

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240425

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1652821

Country of ref document: AT

Kind code of ref document: T

Effective date: 20240124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240424

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240424

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240424

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240524

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240425

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240628

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240524

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240524

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240124