CN111292758A - Voice activity detection method and device and readable storage medium - Google Patents
Voice activity detection method and device and readable storage medium Download PDFInfo
- Publication number
- CN111292758A CN111292758A CN201910184966.8A CN201910184966A CN111292758A CN 111292758 A CN111292758 A CN 111292758A CN 201910184966 A CN201910184966 A CN 201910184966A CN 111292758 A CN111292758 A CN 111292758A
- Authority
- CN
- China
- Prior art keywords
- voice
- voice signal
- voice activity
- preset
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000000694 effects Effects 0.000 title claims abstract description 341
- 238000001514 detection method Methods 0.000 title claims abstract description 176
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 159
- 230000009467 reduction Effects 0.000 claims abstract description 130
- 230000000875 corresponding effect Effects 0.000 claims description 267
- 238000001228 spectrum Methods 0.000 claims description 199
- 238000000034 method Methods 0.000 claims description 54
- 238000001914 filtration Methods 0.000 claims description 30
- 238000004364 calculation method Methods 0.000 claims description 24
- 230000006870 function Effects 0.000 claims description 19
- 230000002596 correlated effect Effects 0.000 claims description 12
- 230000003595 spectral effect Effects 0.000 claims description 7
- 238000005311 autocorrelation function Methods 0.000 claims description 6
- 230000005236 sound signal Effects 0.000 claims 2
- 238000010586 diagram Methods 0.000 description 17
- 238000012545 processing Methods 0.000 description 10
- 238000007664 blowing Methods 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 230000004913 activation Effects 0.000 description 5
- 238000001994 activation Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000001052 transient effect Effects 0.000 description 4
- 230000000737 periodic effect Effects 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 101100134058 Caenorhabditis elegans nth-1 gene Proteins 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/78—Detection of presence or absence of voice signals
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Noise Elimination (AREA)
- Telephone Function (AREA)
Abstract
A voice activity detection method and device and a readable storage medium are provided, wherein the voice activity detection method comprises the following steps: acquiring an acquired voice signal; respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm to judge whether voice activity exists in the voice signal; determining that voice activity is detected from the speech signal when both the speech noise reduction algorithm and the harmonic detection algorithm determine that voice activity is present in the speech signal. By adopting the scheme, the voice activity can be accurately detected.
Description
Technical Field
The invention belongs to the technical field of voice, and particularly relates to a voice activity detection method and device and a readable storage medium.
Background
Traditional voice activity detection methods, such as detection methods based on volume or microphone level, often cause false detection in a noisy environment, which further causes false activation of voice equipment, not only has the problem of power consumption, but also can disturb others.
Most of the existing voice activity detection methods are based on sound energy judgment, and mainly have the following two defects: firstly, speech cannot be accurately distinguished in a noise environment, such as wind sound and the like which often occur in a noisy public place and outdoors; secondly, in a quiet environment, some sudden non-voice sounds such as telephone ring, door closing sound and the like are easy to be mistakenly detected.
Disclosure of Invention
The embodiment of the invention solves the problem of accurately detecting the voice activity.
To solve the foregoing technical problem, an embodiment of the present invention provides a voice activity detection method, where the voice activity detection method includes: acquiring an acquired voice signal; respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm to judge whether voice activity exists in the voice signal; determining that voice activity is detected from the speech signal when both the speech noise reduction algorithm and the harmonic detection algorithm determine that voice activity is present in the speech signal.
Optionally, the determining whether the voice signal has voice activity by respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm includes: performing voice noise reduction calculation on the voice signal to obtain a noise-reduced voice signal; calculating the energy corresponding to the voice signal and the energy corresponding to the voice signal after noise reduction to obtain the energy ratio of the voice signal before and after noise reduction; and when the energy ratio is smaller than a preset first energy ratio threshold, judging that voice activity exists in the voice signal.
Optionally, the determining whether the voice signal has voice activity by respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm includes: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; performing voice noise reduction calculation on the voice signal by adopting a wiener filtering noise reduction algorithm to obtain a noise-reduced voice signal frequency domain amplitude spectrum; calculating the energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction according to the wiener filter function, the voice signal frequency domain amplitude spectrum after noise reduction and the voice signal frequency domain amplitude spectrum; the wiener filter function is obtained by calculation according to the wiener filter noise reduction algorithm and the noise estimation value of the voice signal; the noise estimation value is obtained by adopting a noise estimation algorithm; and when the energy ratio is smaller than a preset second energy ratio threshold, judging that voice activity exists in the voice signal.
Optionally, the energy ratio of the frequency domain amplitude spectrum of the speech signal before and after noise reduction is calculated by using the following formula:wherein E iswThe energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction is shown, ks is a preset frequency point starting point, ke is a preset frequency point terminal point, Y (k) is the voice signal frequency domain amplitude spectrum, and S' (k) is the voice signal frequency domain amplitude spectrum after noise reduction.
Optionally, the preset second energy ratio threshold is positively correlated with the following value: a ratio of an energy corresponding to the speech signal to an energy corresponding to the noise estimate.
Optionally, the determining whether the voice signal has voice activity by respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm includes: and when the voice signal is in a preset voice fundamental frequency range and contains harmonic features, judging that voice activity exists in the voice signal.
Optionally, the determining whether the voice signal has voice activity by respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm includes: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; determining the number of peak values of the frequency domain amplitude spectrum of the voice signal; the peak value of the voice signal frequency domain amplitude spectrum is determined by adopting the following method: when the frequency domain amplitude spectrum corresponding to the ith frequency point in the voice signal frequency domain amplitude spectrum is larger than the maximum value in the frequency domain amplitude spectrum corresponding to the (i + 1) th frequency point, the frequency domain amplitude spectrum corresponding to the (i-1) th frequency point and a preset amplitude threshold corresponding to the ith frequency point, determining the frequency domain amplitude spectrum corresponding to the ith frequency point as a peak value of the voice signal voice frequency domain amplitude spectrum; and when the number of the peak values exceeds a preset threshold value of the number of the peak values, judging that voice activity exists in the voice signal.
Optionally, the preset amplitude threshold corresponding to the ith frequency point is obtained by the following method: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; selecting a noise estimation value of the voice signal, a mean value of the voice signal frequency domain amplitude spectrum, and a maximum value in a corresponding minimum voice frequency domain amplitude spectrum from the i-1 frequency point to the i +1 frequency point as a preset amplitude threshold corresponding to the i frequency point; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
Optionally, after determining the number of peaks of the frequency domain amplitude spectrum of the speech signal, the method further includes: sequentially taking the frequency index value corresponding to each peak value as a fundamental frequency, and calculating the frequency doubling deviation between the frequency index value corresponding to each peak value after the peak value corresponding to the fundamental frequency and the fundamental frequency; when the frequency multiplication deviation is larger than a preset deviation threshold value, the peak value is excluded; sequentially calculating weighted values of all residual peak values according to the frequency multiplication deviation and the residual peak values; comparing the weighted value corresponding to each fundamental frequency, and selecting the maximum weighted value; and when the maximum weighted value is larger than a preset weighted threshold value, judging that voice activity exists in the voice signal.
Optionally, the weighted values of all remaining peaks are calculated by using the following formula: eh=∑αnY[pn](ii) a Wherein E ishIs a weighted value, p, of said total remaining peak valuenFor the frequency index value corresponding to the nth remaining peak, Y [ p ]n]For the nth remaining peak pairCorresponding frequency domain amplitude spectrum, αnα for the preset weight coefficientn∈(0,1]。
Optionally, the frequency multiplication deviation is calculated by using the following formula:where Δ f is the frequency multiplication deviation, pnFor the frequency index value, p, corresponding to the nth peakbbIs a frequency index value that is a fundamental frequency.
Optionally, the preset weighting threshold is positively correlated with the following value: a ratio of energy corresponding to the speech signal to energy corresponding to the noise estimate; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
Optionally, the following formula is adopted to calculate a preset amplitude threshold corresponding to the ith frequency point: y isthr=2max(mean(Y),D[k],min(Y[k-2]……Y[k+2]) (ii) a Wherein, YthrA preset amplitude threshold corresponding to the ith frequency point, Y is the frequency domain amplitude spectrum of the voice signal, Dk]For the noise estimate, k is the frequency point, Y [ k-2 ]]……Y[k+2]And the spectrum is the voice signal frequency domain amplitude spectrum corresponding to the k-2 to k +2 frequency points.
Optionally, the determining that voice activity is detected from the voice signal includes: calculating the energy corresponding to the voice signal; and when the energy corresponding to the voice signal is larger than a preset energy threshold value, judging that voice activity is detected from the voice signal.
Optionally, the energy corresponding to the speech signal is calculated by using the following formula: eabs=∑(Y[k])2(ii) a Wherein E isabsFor the corresponding energy of the speech signal, Y [ k ]]A frequency domain amplitude spectrum of the voice signal is obtained, and k is a preset frequency range; the voice signal frequency domain magnitude spectrum is obtained by performing fast Fourier transform on the voice signal.
Optionally, the calculating the energy corresponding to the speech signal includes: carrying out noise estimation on the voice signal to obtain a noise estimation value; calculating the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value; and when the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value is larger than a preset third energy ratio threshold value, judging that voice activity is detected from the voice signal.
Optionally, the following formula is adopted to calculate a ratio of energy corresponding to the speech signal to energy corresponding to the noise estimation value: evs=log(Eabs)-log(En) (ii) a Wherein E isvsIs the ratio of the energy corresponding to the speech signal to the energy corresponding to the noise estimate, EabsFor the corresponding energy of the speech signal, EnFor the energy corresponding to said noise estimate, En=∑(D[k])2,D[k]Is the noise estimate.
Optionally, after determining that voice activity is detected from the voice signal, the method further includes: when the detected voice activity appears after continuous non-voice activity and the number of the continuous non-voice activity frames exceeds a preset first frame number threshold, caching the voice activity, and when the number of the voice frames of the voice activity exceeds a preset second frame number threshold, outputting a voice signal corresponding to the voice activity; when the detected non-voice activity appears after continuous voice activity, and the frame number of the continuous voice activity exceeds a preset third frame number threshold value, continuing voice activity detection, and when the frame number of the non-voice activity exceeds a preset fourth frame number threshold value, stopping outputting a voice signal corresponding to the voice activity.
Optionally, after determining that voice activity is detected from the voice signal, the method further includes: within a preset mixed frame number threshold range, when the detected voice activity and non-voice activity occur alternately, calculating the proportion of the voice activity frame number to the sum of the voice activity frame number and the non-voice activity frame number; and outputting a voice signal corresponding to the voice activity when the proportion is larger than a preset proportion threshold value.
Optionally, the speech noise reduction algorithm is at least one of the following algorithms: LMS, NLMS, spectral subtraction, and wiener filtering algorithms.
Optionally, the harmonic detection algorithm is at least one of the following algorithms: autocorrelation function methods, cepstrum methods, linear prediction methods, and wavelet methods.
In order to solve the above technical problem, an embodiment of the present invention further discloses a voice activity detection apparatus, including: the acquisition unit is used for acquiring the acquired voice signals; the first judging unit is used for judging whether voice activity exists in the voice signal by respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm; a second determining unit, configured to determine that voice activity is detected from the voice signal when the voice noise reduction algorithm and the harmonic detection algorithm both determine that voice activity exists in the voice signal.
Optionally, the first judging unit is configured to: performing voice noise reduction calculation on the voice signal to obtain a noise-reduced voice signal; calculating the energy corresponding to the voice signal and the energy corresponding to the voice signal after noise reduction to obtain the energy ratio of the voice signal before and after noise reduction; and when the energy ratio is smaller than a preset first energy ratio threshold, judging that voice activity exists in the voice signal.
Optionally, the first judging unit is configured to: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; performing voice noise reduction calculation on the voice signal by adopting a wiener filtering noise reduction algorithm to obtain a noise-reduced voice signal frequency domain amplitude spectrum; calculating the energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction according to the wiener filter function, the voice signal frequency domain amplitude spectrum after noise reduction and the voice signal frequency domain amplitude spectrum; the wiener filter function is obtained by calculation according to the wiener filter noise reduction algorithm and the noise estimation value of the voice signal; the noise estimation value is obtained by adopting a noise estimation algorithm; and when the energy ratio is smaller than a preset second energy ratio threshold, judging that voice activity exists in the voice signal.
Optionally, the energy ratio of the frequency domain amplitude spectrum of the speech signal before and after noise reduction is calculated by using the following formula:wherein E iswThe energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction is shown, ks is a preset frequency point starting point, ke is a preset frequency point terminal point, Y (k) is the voice signal frequency domain amplitude spectrum, and S' (k) is the voice signal frequency domain amplitude spectrum after noise reduction.
Optionally, the preset second energy ratio threshold is positively correlated with the following value: a ratio of an energy corresponding to the speech signal to an energy corresponding to the noise estimate.
Optionally, the first judging unit is configured to: and when the voice signal is in a preset voice fundamental frequency range and contains harmonic features, judging that voice activity exists in the voice signal.
Optionally, the first judging unit is configured to: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; determining the number of peak values of the frequency domain amplitude spectrum of the voice signal; the peak value of the voice signal frequency domain amplitude spectrum is determined by adopting the following method: when the frequency domain amplitude spectrum corresponding to the ith frequency point in the voice signal frequency domain amplitude spectrum is larger than the maximum value in the frequency domain amplitude spectrum corresponding to the (i + 1) th frequency point, the frequency domain amplitude spectrum corresponding to the (i-1) th frequency point and a preset amplitude threshold corresponding to the ith frequency point, determining the frequency domain amplitude spectrum corresponding to the ith frequency point as a peak value of the voice signal voice frequency domain amplitude spectrum; and when the number of the peak values exceeds a preset threshold value of the number of the peak values, judging that voice activity exists in the voice signal.
Optionally, the preset amplitude threshold corresponding to the ith frequency point is obtained by the following method: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; selecting a noise estimation value of the voice signal, a mean value of the voice signal frequency domain amplitude spectrum, and a maximum value in a corresponding minimum voice frequency domain amplitude spectrum from the i-1 frequency point to the i +1 frequency point as a preset amplitude threshold corresponding to the i frequency point; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
Optionally, the first determining unit is further configured to: sequentially taking the frequency index value corresponding to each peak value as a fundamental frequency, and calculating the frequency doubling deviation between the frequency index value corresponding to each peak value after the peak value corresponding to the fundamental frequency and the fundamental frequency; when the frequency multiplication deviation is larger than a preset deviation threshold value, the peak value is excluded; sequentially calculating weighted values of all residual peak values according to the frequency multiplication deviation and the residual peak values; comparing the weighted value corresponding to each fundamental frequency, and selecting the maximum weighted value; and when the maximum weighted value is larger than a preset weighted threshold value, judging that voice activity exists in the voice signal.
Optionally, the weighted values of all remaining peaks are calculated by using the following formula: eh=∑αnY[pn](ii) a Wherein E ishIs a weighted value, p, of said total remaining peak valuenFor the frequency index value corresponding to the nth remaining peak, Y [ p ]n]For the frequency domain amplitude spectrum corresponding to the nth residual peak, αnα for the preset weight coefficientn∈(0,1]。
Optionally, the frequency multiplication deviation is calculated by using the following formula:where Δ f is the frequency multiplication deviation, pnFor the frequency index value, p, corresponding to the nth peakbbIs a frequency index value that is a fundamental frequency.
Optionally, the preset weighting threshold is positively correlated with the following value: a ratio of energy corresponding to the speech signal to energy corresponding to the noise estimate; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
Optionally, the following formula is adopted to calculate a preset amplitude threshold corresponding to the ith frequency point: y isthr=2max(mean(Y),D[k],min(Y[k-2]……Y[k+2]) (ii) a Wherein, YthrA preset amplitude threshold corresponding to the ith frequency point, Y is the frequency domain amplitude spectrum of the voice signal, Dk]For the noise estimate, k is the frequency point, Y [ k-2 ]]……Y[k+2]And the spectrum is the voice signal frequency domain amplitude spectrum corresponding to the k-2 to k +2 frequency points.
Optionally, the second judging unit is configured to: calculating the energy corresponding to the voice signal; and when the energy corresponding to the voice signal is larger than a preset energy threshold value, judging that voice activity is detected from the voice signal.
Optionally, the energy corresponding to the speech signal is calculated by using the following formula: eabs=∑(Y[k])2(ii) a Wherein E isabsFor the corresponding energy of the speech signal, Y [ k ]]A frequency domain amplitude spectrum of the voice signal is obtained, and k is a preset frequency range; the voice signal frequency domain magnitude spectrum is obtained by performing fast Fourier transform on the voice signal.
Optionally, the second judging unit is configured to: carrying out noise estimation on the voice signal to obtain a noise estimation value; calculating the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value; and when the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value is larger than a preset third energy ratio threshold value, judging that voice activity is detected from the voice signal.
Optionally, the following formula is adopted to calculate a ratio of energy corresponding to the speech signal to energy corresponding to the noise estimation value: evs=log(Eabs)-log(En) (ii) a Wherein E isvsIs the ratio of the energy corresponding to the speech signal to the energy corresponding to the noise estimate, EabsFor the corresponding energy of the speech signal, EnFor the energy corresponding to said noise estimate, En=∑(D[k])2,D[k]Is the noise estimate.
Optionally, the second determining unit is further configured to: when the detected voice activity appears after continuous non-voice activity and the number of the continuous non-voice activity frames exceeds a preset first frame number threshold, caching the voice activity, and when the number of the voice frames of the voice activity exceeds a preset second frame number threshold, outputting a voice signal corresponding to the voice activity; when the detected non-voice activity appears after continuous voice activity, and the frame number of the continuous voice activity exceeds a preset third frame number threshold value, continuing voice activity detection, and when the frame number of the non-voice activity exceeds a preset fourth frame number threshold value, stopping outputting a voice signal corresponding to the voice activity.
Optionally, the second determining unit is further configured to: within a preset mixed frame number threshold range, when the detected voice activity and non-voice activity occur alternately, calculating the proportion of the voice activity frame number to the sum of the voice activity frame number and the non-voice activity frame number; and outputting a voice signal corresponding to the voice activity when the proportion is larger than a preset proportion threshold value.
Optionally, the speech noise reduction algorithm is at least one of the following algorithms: LMS, NLMS, spectral subtraction, and wiener filtering algorithms.
Optionally, the harmonic detection algorithm is at least one of the following algorithms: autocorrelation function methods, cepstrum methods, linear prediction methods, and wavelet methods.
The embodiment of the invention also discloses a computer-readable storage medium, which is a nonvolatile storage medium or a non-transient storage medium, and is stored with computer instructions, and the computer instructions execute the steps of any one of the voice activity detection methods when running.
The embodiment of the present invention further provides a voice activity detection apparatus, which includes a memory and a processor, where the memory stores a computer instruction that can be executed on the processor, and the processor executes any of the steps of the voice activity detection method when executing the computer instruction.
Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:
and respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm to judge whether the voice signal has voice activity. Determining that voice activity is detected from the speech signal when both the speech noise reduction algorithm and the harmonic detection algorithm determine that voice activity is present in the speech signal. And filtering the interference of steady-state noise to voice activity detection by adopting a voice noise reduction algorithm, and filtering the interference of unsteady-state noise to voice activity detection by adopting a harmonic detection algorithm. When the voice noise reduction algorithm and the harmonic detection algorithm judge that voice activity exists in the voice signal, the voice activity is judged to be detected from the voice signal, and the accuracy of voice activity detection is greatly improved.
Further, when the energy corresponding to the voice signal is larger than a preset energy threshold value, the voice activity is judged to be detected from the voice signal. The accuracy of voice activity detection can be improved by energy determination of the voice signal.
Further, when the detected voice activity occurs after continuous non-voice activity and the number of the continuous non-voice activity frames exceeds a preset first frame number threshold, caching the voice activity, and when the number of the voice frames of the voice activity exceeds a preset second frame number threshold, outputting a voice signal corresponding to the voice activity; when the detected non-voice activity appears after continuous voice activity, and the frame number of the continuous voice activity exceeds a preset third frame number threshold value, continuing voice activity detection, and when the frame number of the non-voice activity exceeds a preset fourth frame number threshold value, stopping outputting a voice signal corresponding to the voice activity. The false detection of individual voice frames can be eliminated, and the interference of some transient voices to the voice activity detection is also filtered.
Drawings
FIG. 1 is a flow chart of a method of voice activity detection in an embodiment of the present invention;
FIG. 2 is a waveform diagram of a noisy speech signal in an embodiment of the present invention;
FIG. 3 is a diagram illustrating a voice activity detection result of noisy speech using a speech noise reduction algorithm in an embodiment of the present invention;
FIG. 4 is a schematic diagram of a voice activity detection result of noisy speech using a harmonic detection algorithm in an embodiment of the present invention;
FIG. 5 is a waveform diagram of the blowing sound and the wind sound in the embodiment of the present invention;
FIG. 6 is a diagram illustrating the detection results of the voice activity of the blowing sound and the wind sound by using the wiener filtering algorithm in the embodiment of the present invention;
FIG. 7 is a diagram illustrating the detection results of the voice activity of the blowing sound and the wind sound using the peak detection algorithm in an embodiment of the present invention;
FIG. 8 is a waveform of an alarm sound in an embodiment of the present invention;
FIG. 9 is a schematic diagram illustrating the detection of an alarm voice activity using a wiener filtering algorithm in an embodiment of the present invention;
FIG. 10 is a schematic representation of the detection of voice activity for an alarm using a peak detection algorithm in an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a voice activity detection apparatus according to an embodiment of the present invention.
Detailed Description
In the prior art, most voice activity detection methods are based on sound energy judgment, and mainly have the following two disadvantages: firstly, speech cannot be accurately distinguished in a noise environment, such as wind sound and the like which often occur in a noisy public place and outdoors; secondly, in a quiet environment, some sudden non-voice sounds such as telephone ring, door closing sound and the like are easy to be mistakenly detected.
In the embodiment of the invention, a voice noise reduction algorithm and a harmonic detection algorithm are respectively adopted to judge whether voice activity exists in the voice signal. Determining that voice activity is detected from the speech signal when both the speech noise reduction algorithm and the harmonic detection algorithm determine that voice activity is present in the speech signal. And filtering the interference of steady-state noise to voice activity detection by adopting a voice noise reduction algorithm, and filtering the interference of unsteady-state noise to voice activity detection by adopting a harmonic detection algorithm. When the voice noise reduction algorithm and the harmonic detection algorithm judge that voice activity exists in the voice signal, the voice activity is judged to be detected from the voice signal, and the accuracy of voice activity detection is greatly improved.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
An embodiment of the present invention provides a voice activity detection method, which is described in detail below with reference to fig. 1 through specific steps.
And step S101, acquiring the collected voice signal.
In particular implementations, the speech signal may be captured by a device having audio capture capabilities. In practical application, it can be known that the collected voice signal usually carries a noise signal, so that voice activity detection processing can be performed on the collected voice signal to determine whether the voice signal has corresponding voice activity.
In practical applications, the collected voice signal may have a certain dc drift, which may affect the subsequent processing. Therefore, the collected voice signals can be subjected to direct current removing processing to obtain the voice signals with direct current drift interference removed.
As described above, the first-order filtering algorithm may be used to perform dc removal processing on the collected voice signal, and the specific processing procedure is as follows: and according to the collected voice signals and the filter coefficients, obtaining direct current component estimated values, and then removing the direct current component estimated values in the collected voice signals to obtain the voice signals with the direct current drift interference removed.
In the embodiment of the present invention, the dc component estimated value may be calculated by using the following formula (1):
dc(n)=αy(n)+(1-α)dc(n-1); (1)
dc (n) is an estimated value of a direct current component of an nth frame, an initial value of dc (1) is 0, dc (n-1) is an estimated value of a direct current component of an nth-1 frame, y (n) is an acquired voice signal, α is a filter coefficient, α ∈ (0,1) the smaller the filter coefficient α is, the smoother the estimated value of the direct current component is, but the sensitivity is correspondingly reduced, the larger the filter coefficient α is, the higher the sensitivity is, the more unstable the corresponding estimated value of the direct current component is, therefore, a user can determine an acceptable sensitivity range and a smoothness in actual use according to own actual requirements, and select an appropriate filter coefficient α, which is not limited herein.
It is understood that the dc removal process described above is described using a first order filtering algorithm. In a specific application, a user may select different methods to perform dc removal processing on the collected voice signal according to the own requirements, which is not described herein.
Step S102, whether voice activity exists in the voice signal is judged by respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm.
In specific implementation, windowing and framing processing can be performed on a voice signal to perform fast fourier transform, so as to obtain a short-time frequency domain magnitude spectrum of the voice signal; and judging whether voice activity exists in the voice signal or not by respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm according to the voice signal and the short-time frequency domain amplitude spectrum of the voice signal.
Because the collected voice signal usually contains various types of noise, effective voice activity needs to be detected from the voice signal, so that the voice equipment can perform subsequent operations such as extraction or output on the voice activity. In the embodiment of the present invention, the voice activity may refer to: speech output by a user using the speech device.
In specific implementation, a speech noise reduction algorithm can be adopted to calculate the ratio of the energies corresponding to the speech signals before and after noise reduction, so as to judge whether the speech signals have speech activity; meanwhile, a harmonic detection algorithm can be adopted to calculate whether the harmonic energy of the voice signal conforms to the characteristics of voice activity, and then whether the voice activity exists in the voice signal is judged.
In a specific implementation, the noise reduction processing may be performed on the acquired voice signal first. In the embodiment of the present invention, the collected voice signal may be subjected to noise reduction processing by using at least one of the following voice noise reduction algorithms: LMS, NLMS, spectral subtraction, and wiener filtering algorithms. The user can select one or more algorithm combination modes to perform noise reduction processing according to the actual situation of the user, and then after voice activity is detected in the voice signal, the frequency domain amplitude spectrum of the voice signal subjected to noise reduction is converted back to time domain data, the voice signal subjected to noise reduction is output, and the definition of voice is improved.
In a specific implementation, the harmonic detection algorithm may include at least one of the following algorithms: autocorrelation function methods, cepstrum methods, linear prediction methods, and wavelet methods. The user can select one or more algorithms to be combined for harmonic detection according to the actual situation of the user, which is not described herein.
In a specific implementation, the determining whether voice activity exists in the voice signal by using a voice noise reduction algorithm may include the following processes: firstly, carrying out voice noise reduction calculation on a voice signal to obtain a noise-reduced voice signal; then calculating the energy corresponding to the voice signal and the energy corresponding to the voice signal after noise reduction to obtain the energy ratio of the voice signal before and after noise reduction; and when the energy ratio is smaller than a preset first energy ratio threshold, judging that voice activity exists in the voice signal.
In a specific implementation, a wiener filtering algorithm may be used to determine whether voice activity is present in the voice signal.
The wiener filtering algorithm designs a digital filter h (n), when a voice signal y (n) containing noise is input, the output of the filter is y (n) h (n), and the mean square error E [ { y (n) h (n) -s (n) }of the output of the filter y (n) h (n) and the voice signal s (n) without noise is enabled to be in accordance with the minimum mean square error criterion2]To a minimum.
The frequency domain magnitude spectrum estimator h (k) of the wiener filter can be represented by the following equation (2):
wherein, Ps[k]For noise-free speech power spectrum, Pd[k]For the noise power spectrum, the frequency domain amplitude spectrum estimator h (k) of the wiener filter is obtained by performing fast fourier transform on the digital filter h (n), and the voice signal power spectrum is obtained by performing autocorrelation budget on the corresponding voice signal and then performing fast fourier transform, which is not described in detail in the following.
Defining a priori signal-to-noise ratio SNRprior[k]For speech power spectrum P without noises[k]And the noise power spectrum Pd[k]Can convert equation (2) to equation (3) below:
defining the A posteriori SNRpost[k]For the power spectrum (without noise) P of speechy[k]And the noise power spectrum Pd[k]Is expressed by the following formula (4)post[k]:
Since the speech signal includes a noise-free speech signal and a noise signal, the following formula (5) is used to represent the speech signal power spectrum Py[k]:
Py[k]=Ps[k]+Pd[k]; (5)
Therefore, equation (6) and equation (7) are obtained using the following derivation procedure:
wherein the SNRprior[k]=SNRpost[k]-1;SNRprior[k]tA priori signal-to-noise ratio, SNR, at t timepost[k]tA posteriori signal-to-noise ratio, SNR, at t timeprior[k](t-1)Is the prior signal-to-noise ratio at t-1 time, Y [ k ]]For the frequency domain amplitude spectrum of the speech signal, d (k) is the noise estimate of the speech signal, t is time, α is the smoothing parameter, α ∈ (0,1), and the larger α is, the greater the suppression effect on noise is.
The voice signal frequency domain amplitude spectrum Y [ k ] can be obtained by collecting voice signals through voice collecting equipment, and the noise estimation value of the voice signals can be obtained through calculation of a noise estimation algorithm.
In practical applications, the noise estimation algorithm may include: a continuous spectrum minimum tracking method, recursive average noise estimation, histogram noise estimation, etc., and the invention does not limit the noise estimation algorithm for noise estimation.
Taking the continuous spectrum minimum tracking method as an example, the method combines a spectrum short-time minimum algorithm and a time recursive average algorithm, and calculates the frequency domain amplitude spectrum d (k) of the noise estimation value by adopting the following formula (8):
wherein, D (k)tAnd D [ k ]]t-1Respectively representing the frequency domain amplitude spectrum of the noise estimation values of the current frame and the previous frame at the k-th frequency point, Y k]tAnd Y [ k ]]t-1Respectively representing the frequency domain amplitude spectrum of the speech signal containing noise at the k-th frequency point of the current frame and the previous frame, and defaulting to the pure noise signal of the initial frame, namely D [ k [ ]]0=Y[k]0,S[k]tRepresenting the frequency domain amplitude spectrum, SM, of the noisy speech signal of the current frame at the k-th frequency point1、SM2、SM3Are smoothing factors, and are all larger than 0 and smaller than 1.
After obtaining the frequency domain amplitude spectrum D (k) of the noise estimation value, combining the formulas (3), (4) and (6), the filter function H (k) of the current frame can be obtained by recursion according to the filter function of the previous frametThen, the frequency domain amplitude spectrum of the noise-reduced speech signal can be calculated by using the following formula (9):
S'(k)=Y(k)tH(k)t; (9)
in a specific implementation, a speech noise reduction algorithm, wiener filtering, is used to determine whether speech activity exists in the speech signal. Firstly, carrying out fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; performing voice noise reduction calculation on the voice signal by adopting a wiener filtering noise reduction algorithm to obtain a noise-reduced voice signal frequency domain amplitude spectrum; calculating the energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction according to the wiener filter function, the voice signal frequency domain amplitude spectrum after noise reduction and the voice signal frequency domain amplitude spectrum; the wiener filter function is obtained by calculation according to the wiener filter noise reduction algorithm and the noise estimation value of the voice signal; the noise estimation value is obtained by adopting a noise estimation algorithm; and when the energy ratio is smaller than a preset second energy ratio threshold, judging that voice activity exists in the voice signal.
In a specific implementation, the energy ratio of the frequency domain amplitude spectrum of the speech signal before and after the noise reduction is calculated by using the following formula (10):
wherein E iswThe energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction is shown, ks is a preset frequency point starting point, ke is a preset frequency point terminal point, Y (k) is the voice signal frequency domain amplitude spectrum, and S' (k) is the voice signal frequency domain amplitude spectrum after noise reduction.
In practical application, the difference value of the frequency domain amplitude spectrum of the voice signal before and after noise reduction in a logarithmic domain can be directly calculated to serve as an energy ratio value, so that the calculated amount is reduced; the square error of the frequency domain amplitude spectrum of the voice signal before and after noise reduction can be calculated to serve as an energy ratio, and then when the square error is smaller than a preset noise threshold value, the voice signal is judged to have voice activity, namely, the purpose of calculating the energy ratio corresponding to the voice signal before and after noise reduction can be achieved, and a specific calculation mode is not limited.
In practical application, the energy ratio of the frequency domain amplitude spectrum of the voice signal before and after noise reduction is calculated, and a preset second energy ratio threshold value can be stabilized within a certain range through the function characteristic of logarithm, so that the aim of accurately judging whether voice activity exists in the voice signal is fulfilled. It can be understood that the user may also set the difference or ratio of the energy of the amplitude spectrum of the frequency domain of the speech signal before and after the noise reduction in different function domains according to the self condition, which is not limited herein.
In a specific implementation, the preset second energy ratio threshold may be set as a fixed value according to a previous voice activity detection result, may also be set as a dynamic threshold according to a specific situation of a voice signal, and may also be positively correlated with the following value: a ratio of an energy corresponding to the speech signal to an energy corresponding to the noise estimate. The user may select one or more preset setting manners of the second energy ratio threshold according to different requirements of the user, which is not described herein again.
In a specific implementation, since the speech signal is composed of a fundamental frequency signal and a series of harmonic signals, when the speech signal is within a preset speech fundamental frequency range and contains harmonic features, it can be determined that speech activity exists in the speech signal. The harmonic features may be a complete harmonic series including fundamental, first harmonic, second harmonic, etc., or may be a continuous harmonic segment including second and third harmonics. The voice activity is judged through the harmonic wave characteristics, the requirement on the quality of the voice signal is low, and the voice activity detection method has the capability of resisting various noise interferences.
Formants are important features that reflect the resonance characteristics of vocal tract, and appear as periodic maxima in the speech spectral envelope. Because the frequency of the harmonic signal is approximate to the integral multiple of the fundamental frequency signal, the maximum value of the frequency domain amplitude spectrum of a frame of voice and the corresponding frequency thereof can be detected by searching the mode with periodic harmonic energy, namely peak detection.
In practical applications, a low frequency band of a speech signal generally has more noise, and a peak detection search range above 3000Hz increases a large amount of calculation, and false detection is easily caused due to a small amplitude. Therefore, in consideration of noise interference and computational complexity, the search range for peak detection is not as large as possible, and a search should be performed in a main frequency range of human voice.
In one embodiment of the present invention, the range of 80Hz-3000Hz is selected for peak detection. The index range is 80N/fsTo 3000N/fsWhere N is the fast Fourier transform length, fsIs the sampling rate. The user can adjust the search range according to the self requirement, and the invention is not limited herein. After the search range is determined, detection is performed sequentially from the index range. And if the amplitude of the current index is larger than the amplitudes of the two points before and after the current index and is also larger than a preset amplitude threshold value, judging that the current index is a peak value, and adding one to the total number of the peak values. And after the detection is finished in the index range, judging that the probability of no voice activity exists in the voice signal is higher when the number of the detected peak values is less than two.
In specific implementation, the voice signal is subjected to fast fourier transform to obtain a frequency domain amplitude spectrum of the voice signal, and then the number of peaks of the frequency domain amplitude spectrum of the voice signal is determined. And when the number of the peak values exceeds a preset threshold value of the number of the peak values, judging that voice activity exists in the voice signal.
The peak value of the frequency domain amplitude spectrum of the speech signal is determined as follows: and when the frequency domain amplitude spectrum corresponding to the ith frequency point in the frequency domain amplitude spectrum of the voice signal is larger than the maximum value of the frequency domain amplitude spectrum corresponding to the (i + 1) th frequency point, the frequency domain amplitude spectrum corresponding to the (i-1) th frequency point and a preset amplitude threshold corresponding to the ith frequency point, determining that the frequency domain amplitude spectrum corresponding to the ith frequency point is a peak value of the voice frequency domain amplitude spectrum of the voice signal.
In a specific implementation, the preset amplitude threshold corresponding to the ith frequency point may be obtained by: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; selecting a noise estimation value of the voice signal, a mean value of the voice signal frequency domain amplitude spectrum, and a maximum value in a corresponding minimum voice frequency domain amplitude spectrum from the i-1 frequency point to the i +1 frequency point as a preset amplitude threshold corresponding to the i frequency point; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
In a specific implementation, the following formula (11) may be adopted to calculate the preset amplitude threshold corresponding to the ith frequency point:
Ythr=2max(mean(Y),D[k],min(Y[k-2]……Y[k+2]); (11)
wherein, YthrA preset amplitude threshold corresponding to the ith frequency point, Y is the frequency domain amplitude spectrum of the voice signal, Dk]For the noise estimate, k is the frequency point, Y [ k-2 ]]……Y[k+2]And the spectrum is the voice signal frequency domain amplitude spectrum corresponding to the k-2 to k +2 frequency points.
In a specific implementation, after determining the number of peaks of the frequency domain magnitude spectrum of the speech signal, the method further includes: sequentially taking the frequency index value corresponding to each peak value as a fundamental frequency, and calculating the frequency doubling deviation between the frequency index value corresponding to each peak value after the peak value corresponding to the fundamental frequency and the fundamental frequency; when the frequency multiplication deviation is larger than a preset deviation threshold value, the peak value is excluded; sequentially calculating weighted values of all residual peak values according to the frequency multiplication deviation and the residual peak values; comparing the weighted value corresponding to each fundamental frequency, and selecting the maximum weighted value; and when the maximum weighted value is larger than a preset weighted threshold value, judging that voice activity exists in the voice signal.
In a specific implementation, the frequency doubling deviation is calculated using the following equation (12):
where Δ f is the frequency multiplication deviation, pnFor the frequency index value, p, corresponding to the nth peakbbIs a frequency index value that is a frequency of the fundamental frequency.
In a specific implementation, the weighted values of all remaining peaks are calculated using the following equation (13):
Eh=∑αnY[pn]; (13)
wherein E ishIs a weighted value, p, of said total remaining peak valuenFor the frequency index value corresponding to the nth remaining peak, Y [ p ]n]For the frequency domain amplitude spectrum corresponding to the nth residual peak, αnα for the preset weight coefficientn∈(0,1],αnAssociated with doubling and doubling deviation, α when the doubling is largernThe smaller the frequency doubling deviation, the larger αnThe smaller.
In a specific implementation, the preset weighting threshold may be set as a fixed value according to a previous voice activity detection result, or may be set as a dynamic threshold according to a specific situation of a voice signal, and may be positively correlated with the following values: a ratio of energy corresponding to the speech signal to energy corresponding to the noise estimate; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
In one embodiment of the invention, after all peaks are detected within the index range, the mark is madeAll detected peaks are Y [ p ]1]……Y[pcnt]And sequentially taking the frequency index values corresponding to all the peak values as candidate fundamental frequency or harmonic signals. Since the fundamental frequency of speech is typically between 80Hz and 500Hz, it is first assumed that p1For the fundamental frequency, the frequency p of the candidate harmonic wave above the fundamental frequency is2……pcntMaking a frequency doubling decision, i.e. determining p2To pcntWhether or not to approach an integer multiple of the base frequency signal index. And if the frequency multiplication deviation is greater than a preset deviation threshold value, excluding the corresponding peak value at the index. The remaining harmonics are then amplitude weighted to p1The amplitude of the fundamental frequency is weighted. Then, let it successively assume that p does not exceed the frequency index of 500Hz2……pcnt-1Repeating the above steps to find the maximum weight value for the fundamental frequency. And when the maximum weighted value is larger than a preset weighted threshold value, judging that voice activity exists in the voice signal.
The voice activity detection has higher robustness by adopting the voice noise reduction algorithm and the harmonic detection algorithm, and because various algorithms are calculated on the basis of the frequency domain amplitude spectrum, the data reuse rate in the calculation process is high, and the calculation amount and the calculation cost are reduced.
And S103, when the voice noise reduction algorithm and the harmonic detection algorithm both judge that voice activity exists in the voice signal, judging that the voice activity is detected from the voice signal.
In the process of voice activity detection, the voice of the non-user is often mixed in the voice signal, and then the false detection is caused. Therefore, the energy corresponding to the voice signal can be calculated, and then compared with the preset energy threshold value to judge whether voice activity exists in the voice signal.
In a specific implementation, determining that voice activity is detected from the voice signal may include: calculating the energy corresponding to the voice signal; and when the energy corresponding to the voice signal is larger than a preset energy threshold value, judging that the voice activity is detected from the voice signal, and further improving the accuracy of voice activity detection on the basis of a voice noise reduction algorithm and a harmonic detection algorithm.
In a specific implementation, the energy corresponding to the speech signal is calculated by the following equation (14):
Eabs=∑(Y[k])2; (14)
wherein E isabsFor the corresponding energy of the speech signal, Y [ k ]]A frequency domain amplitude spectrum of the voice signal is obtained, and k is a preset frequency range; the voice signal frequency domain magnitude spectrum is obtained by performing fast Fourier transform on the voice signal.
In an embodiment of the present invention, for a frequency domain amplitude spectrum of a frame of speech signal, energy corresponding to the speech signal is a sum of squares of each frequency point. Meanwhile, the main frequency range of the voice is considered, possible noise interference is eliminated, and energy corresponding to the voice signal with the frequency point of 80Hz to 3000Hz (namely, partial frequency band) is calculated. It can be understood that the user can adjust the selected frequency band range according to different requirements of the user, and the present invention is not described herein.
In specific implementation, energy corresponding to the voice signal is calculated, and noise estimation needs to be performed on the voice signal to obtain a noise estimation value; then calculating the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value; and when the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value is larger than a preset third energy ratio threshold value, judging that voice activity is detected from the voice signal.
In practical application, the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value is calculated, and a preset third energy ratio threshold value can be stabilized within a certain range through the function characteristic of logarithm, so that the purpose of accurately judging whether voice activity exists in the voice signal is achieved. It is understood that the user may also set a ratio of the energy corresponding to the speech signal to the energy corresponding to the noise estimation value in different function domains according to the user's own situation, and the invention is not limited herein.
In a specific implementation, the following formula (15) is used to calculate a ratio of the energy corresponding to the speech signal to the energy corresponding to the noise estimation value:
Evs=log(Eabs)-log(En); (15)
wherein E isvsIs the ratio of the energy corresponding to the speech signal to the energy corresponding to the noise estimate, EabsFor the corresponding energy of the speech signal, EnFor the energy corresponding to said noise estimate, En=∑(D[k])2,D[k]Is the noise estimate.
In practical applications, due To special working requirements of some users, such as engineering maintenance personnel, it is often inconvenient To release both hands To operate a Talk key (Push To Talk, PTT) of the interphone for talking, some interphones have a Voice Operated Exchange (VOX) function, that is, a signal collected by a microphone is used To determine whether there is Voice activity, so as To be used as a basis for starting or sleeping the interphone. When the interphone needs to be activated, the user can directly start the interphone through voice without pressing a PTT key. Therefore, the hands of the user can be effectively released, and the communication conversation is more convenient.
The voice activity detection method provided by the invention can be used for voice intercom equipment, not only can have reliable activation rate and lower false activation rate in a noise environment, but also can prevent other people around from activating the interphone of a user when speaking by detecting the energy corresponding to the voice signal, and is convenient for the user to adjust the volume according to the sound.
In a specific implementation, after determining that voice activity is detected from the voice signal, the method further includes: when the detected voice activity appears after continuous non-voice activity and the number of the continuous non-voice activity frames exceeds a preset first frame number threshold, caching the voice activity, and when the number of the voice frames of the voice activity exceeds a preset second frame number threshold, outputting a voice signal corresponding to the voice activity; when the detected non-voice activity appears after continuous voice activity, and the frame number of the continuous voice activity exceeds a preset third frame number threshold value, continuing voice activity detection, and when the frame number of the non-voice activity exceeds a preset fourth frame number threshold value, stopping outputting a voice signal corresponding to the voice activity. The false detection of individual voice frames can be eliminated, and the interference of some transient voices to the voice activity detection is also filtered.
In a specific implementation, when voice activity is detected from the voice signal, within a preset threshold range of a mixed frame number, when the detected voice activity and non-voice activity occur alternately, calculating the proportion of the voice activity frame number to the sum of the voice activity frame number and the non-voice activity frame number; and when the proportion is larger than a preset proportion threshold value, outputting a voice signal corresponding to the voice activity so as to resist the interference of transient sudden sound on the voice activity detection.
When the voice activity detection method provided by the invention is used on the voice equipment, a certain amount of cache can be set in the voice equipment. Every time the microphone collects a frame of signal, the signal is stored in the buffer. The speech device is not activated immediately when a speech frame is detected in consecutive non-speech frames. After a plurality of frames of voice are continuously accumulated and judged, activation is started, and the data are sent from the buffer memory. When a non-speech frame is detected in continuous speech frames, the transmission is not interrupted immediately, but is interrupted after a plurality of frames of non-speech frames are continuously accumulated and determined.
In the embodiment of the present invention, threshold values voice _ confirm, mute _ confirm, and mix _ confirm may be set, where voice _ confirm indicates the number of frames required for confirming the start of voice and the number of frames required for determining buffering. A voice _ confirm increase may eliminate very brief sounds, reduce false activations, but increase transmission delay. The voice _ confirm reduction may reduce the delay and transmit a shorter duration signal. mute _ confirm indicates the number of frames needed to confirm the end of speech. mix _ confirm indicates the number of frames required to determine a speech candidate segment when speech frames and non-speech frames are mixed, and when set to 2 or 3, detection accuracy can be improved. It should be understood that the three thresholds can be set to different values according to the actual requirements of the user, and the invention is not limited herein.
Next, the initial value of the speech frame number, voice _ cnt, is set to zero, and the initial value of the non-speech frame number, mute _ cnt, is set to zero. Each time a speech frame occurs, voice _ cnt is incremented by 1, otherwise mute _ cnt is incremented by 1. When voice _ cnt is equal to mix _ confirm, let mute _ cnt be zero. When voice _ cnt exceeds voice _ confirm, let mute _ cnt be zero, i.e. it is determined that there is voice activity, and send voice from buffered voice frames.
When the voice frame and the non-voice frame are mixed together alternately in a plurality of frames, calculating the ratio rate of the mixed section of voice frame, if the ratio rate is higher, the voice frame is classified as the voice frame, otherwise, the voice frame is classified as the non-voice frame.
And when the mute _ cnt exceeds the mute _ confirm, the voice _ cnt is enabled to be zero, and the voice signal is judged to have no voice activity and not to be sent.
In summary, a voice noise reduction algorithm and a harmonic detection algorithm are respectively adopted to determine whether voice activity exists in the voice signal. Determining that voice activity is detected from the speech signal when both the speech noise reduction algorithm and the harmonic detection algorithm determine that voice activity is present in the speech signal. And filtering the interference of steady-state noise to voice activity detection by adopting a voice noise reduction algorithm, and filtering the interference of unsteady-state noise to voice activity detection by adopting a harmonic detection algorithm. When the voice noise reduction algorithm and the harmonic detection algorithm judge that voice activity exists in the voice signal, the voice activity is judged to be detected from the voice signal, and the accuracy of voice activity detection is greatly improved.
Referring to fig. 2, a waveform diagram of a noisy speech signal in an embodiment of the present invention is shown. Fig. 3 is a schematic diagram showing a voice activity detection result of noisy speech using a speech noise reduction algorithm in an embodiment of the present invention. Fig. 4 is a schematic diagram showing a voice activity detection result of noisy speech using a harmonic detection algorithm in the embodiment of the present invention. Wherein E iswFor the energy ratio of the frequency-domain amplitude spectrum of the speech signal before and after noise reduction, EhThe maximum weight for all remaining peaks.
In fig. 2, the abscissa unit is the sampling point and the ordinate unit is the normalized amplitude value. In fig. 3 and 4, the abscissa unit is a sampling point, and the ordinate unit is a set value.
Because the background noise contains the voice with small volume, harmonic detection occasionally exceeds a threshold value, but the wiener filtering noise reduction result is judged as a noise frame, the noise is not judged as voice activity, and the voice activity detection has better detection accuracy by combining the harmonic detection and the wiener filtering noise reduction algorithm.
Referring to fig. 5, a waveform diagram of blowing sound and wind sound in the embodiment of the present invention is shown, fig. 6 is a schematic diagram of a voice activity detection result of blowing sound and wind sound using a wiener filtering algorithm in the embodiment of the present invention, and fig. 7 is a schematic diagram of a voice activity detection result of blowing sound and wind sound using a peak detection algorithm in the embodiment of the present invention. EwFor the energy ratio of the frequency-domain amplitude spectrum of the speech signal before and after noise reduction, EhThe maximum weight for all remaining peaks.
In fig. 5, the abscissa unit is the sample point and the ordinate unit is the normalized amplitude value. In fig. 6 and 7, the abscissa unit is a sampling point, and the ordinate unit is a set value.
As can be seen from fig. 5-7, when noise is just present, there is a short misjudgment because the noise estimation detected by wiener filtering is not updated in time, but the harmonic detection portion does not detect continuous speech, and there are only misjudgments of a few frames. Therefore, by combining the two algorithms, the voice activity detection result still has no misjudgment, and has better detection accuracy.
Referring to fig. 8, a waveform diagram of an alarm sound in the embodiment of the present invention is shown, fig. 9 is a schematic diagram of a voice activity detection result of an alarm sound using a wiener filtering algorithm in the embodiment of the present invention, and fig. 10 is a schematic diagram of a voice activity detection result of an alarm sound using a peak detection algorithm in the embodiment of the present invention. EwFor the energy ratio of the frequency-domain amplitude spectrum of the speech signal before and after noise reduction, EhThe maximum weight for all remaining peaks.
In fig. 8, the abscissa unit is the sampling point and the ordinate unit is the normalized amplitude value. In fig. 9 and 10, the abscissa unit is a sampling point, and the ordinate unit is a set value.
As can be seen from fig. 8 to 10, in the voice activity detection process, since the voice fundamental frequency signal or the periodic harmonic signal is not detected, the voice activity detection result does not have a false judgment, and shows a better robustness.
Referring to fig. 11, an embodiment of the present invention further provides a voice activity detection apparatus 100, including: an acquisition unit 1001, a first determination unit 1002, and a second determination unit 1003;
the acquiring unit 1001 is configured to acquire an acquired voice signal;
the first judging unit 1002 is configured to judge whether voice activity exists in the voice signal by using a voice noise reduction algorithm and a harmonic detection algorithm, respectively;
the second determining unit 1003 is configured to determine that voice activity is detected from the voice signal when both the voice noise reduction algorithm and the harmonic detection algorithm determine that voice activity exists in the voice signal.
In a specific implementation, the first determining unit 1002 may be configured to: performing voice noise reduction calculation on the voice signal to obtain a noise-reduced voice signal; calculating the energy corresponding to the voice signal and the energy corresponding to the voice signal after noise reduction to obtain the energy ratio of the voice signal before and after noise reduction; and when the energy ratio is smaller than a preset first energy ratio threshold, judging that voice activity exists in the voice signal.
In a specific implementation, the first determining unit 1002 may be configured to: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; performing voice noise reduction calculation on the voice signal by adopting a wiener filtering noise reduction algorithm to obtain a noise-reduced voice signal frequency domain amplitude spectrum; calculating the energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction according to the wiener filter function, the voice signal frequency domain amplitude spectrum after noise reduction and the voice signal frequency domain amplitude spectrum; the wiener filter function is obtained by calculation according to the wiener filter noise reduction algorithm and the noise estimation value of the voice signal; the noise estimation value is obtained by adopting a noise estimation algorithm; and when the energy ratio is smaller than a preset second energy ratio threshold, judging that voice activity exists in the voice signal.
In a specific implementation, the energy ratio of the frequency domain amplitude spectrum of the speech signal before and after the noise reduction can be calculated by using the following formula:wherein E iswThe energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction is shown, ks is a preset frequency point starting point, ke is a preset frequency point terminal point, Y (k) is the voice signal frequency domain amplitude spectrum, and S' (k) is the voice signal frequency domain amplitude spectrum after noise reduction.
In a specific implementation, the preset second energy ratio threshold may be positively correlated with: a ratio of an energy corresponding to the speech signal to an energy corresponding to the noise estimate.
In a specific implementation, the first determining unit 1002 may be configured to: and when the voice signal is in a preset voice fundamental frequency range and contains harmonic features, judging that voice activity exists in the voice signal.
In a specific implementation, the first determining unit 1002 may be configured to: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; determining the number of peak values of the frequency domain amplitude spectrum of the voice signal; the peak value of the frequency domain amplitude spectrum of the voice signal is determined by adopting the following method: when the frequency domain amplitude spectrum corresponding to the ith frequency point in the frequency domain amplitude spectrum of the voice signal is larger than the maximum value of the frequency domain amplitude spectrum corresponding to the (i + 1) th frequency point, the frequency domain amplitude spectrum corresponding to the (i-1) th frequency point and a preset amplitude threshold corresponding to the ith frequency point, determining the frequency domain amplitude spectrum corresponding to the ith frequency point as a peak value of the voice frequency domain amplitude spectrum of the voice signal; and when the number of the peak values exceeds a preset threshold value of the number of the peak values, judging that voice activity exists in the voice signal.
In a specific implementation, the preset amplitude threshold corresponding to the ith frequency point may be obtained by: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; selecting a noise estimation value of the voice signal, a mean value of the voice signal frequency domain amplitude spectrum, and a maximum value in a corresponding minimum voice frequency domain amplitude spectrum from the i-1 frequency point to the i +1 frequency point as a preset amplitude threshold corresponding to the i frequency point; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
In a specific implementation, the first determining unit 1002 may be further configured to: sequentially taking the frequency index value corresponding to each peak value as a fundamental frequency, and calculating the frequency doubling deviation between the frequency index value corresponding to each peak value after the peak value corresponding to the fundamental frequency and the fundamental frequency; when the frequency multiplication deviation is larger than a preset deviation threshold value, the peak value is excluded; sequentially calculating weighted values of all residual peak values according to the frequency multiplication deviation and the residual peak values; comparing the weighted value corresponding to each fundamental frequency, and selecting the maximum weighted value; and when the maximum weighted value is larger than a preset weighted threshold value, judging that voice activity exists in the voice signal.
In a specific implementation, the weighted values of all the remaining peaks can be calculated using the following formula: eh=∑αnY[pn](ii) a Wherein E ishIs a weighted value, p, of said total remaining peak valuenFor the frequency index value corresponding to the nth remaining peak, Y [ p ]n]For the frequency domain amplitude spectrum corresponding to the nth residual peak, αnα for the preset weight coefficientn∈(0,1]。
In a specific implementation, the frequency doubling deviation can be calculated using the following formula:where Δ f is the frequency multiplication deviation, pnFor the frequency index value, p, corresponding to the nth peakbbIs a frequency index value that is a fundamental frequency.
In a specific implementation, the preset weighted threshold may be positively correlated with the following values: a ratio of energy corresponding to the speech signal to energy corresponding to the noise estimate; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
In a specific implementation, the following formula may be employedCalculating a preset amplitude threshold corresponding to the ith frequency point: y isthr=2max(mean(Y),D[k],min(Y[k-2]……Y[k+2]) (ii) a Wherein, YthrA preset amplitude threshold corresponding to the ith frequency point, Y is the frequency domain amplitude spectrum of the voice signal, Dk]For the noise estimate, k is the frequency point, Y [ k-2 ]]……Y[k+2]And the spectrum is the voice signal frequency domain amplitude spectrum corresponding to the k-2 to k +2 frequency points.
In a specific implementation, the second determining unit 1003 may be configured to: calculating the energy corresponding to the voice signal; and when the energy corresponding to the voice signal is larger than a preset energy threshold value, judging that voice activity is detected from the voice signal.
In a specific implementation, the energy corresponding to the speech signal can be calculated by using the following formula: eabs=∑(Y[k])2(ii) a Wherein E isabsFor the corresponding energy of the speech signal, Y [ k ]]A frequency domain amplitude spectrum of the voice signal is obtained, and k is a preset frequency range; the voice signal frequency domain magnitude spectrum is obtained by performing fast Fourier transform on the voice signal.
In a specific implementation, the second determining unit 1003 may be configured to: carrying out noise estimation on the voice signal to obtain a noise estimation value; calculating the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value; and when the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value is larger than a preset third energy ratio threshold value, judging that voice activity is detected from the voice signal.
In a specific implementation, the following formula may be used to calculate a ratio of the energy corresponding to the speech signal to the energy corresponding to the noise estimation value: evs=log(Eabs)-log(En) (ii) a Wherein E isvsIs the ratio of the energy corresponding to the speech signal to the energy corresponding to the noise estimate, EabsFor the corresponding energy of the speech signal, EnFor the energy corresponding to said noise estimate, En=∑(D[k])2,D[k]Is the noise estimate.
In a specific implementation, the second determining unit 1003 may be further configured to: when the detected voice activity appears after continuous non-voice activity and the number of the continuous non-voice activity frames exceeds a preset first frame number threshold, caching the voice activity, and when the number of the voice frames of the voice activity exceeds a preset second frame number threshold, outputting a voice signal corresponding to the voice activity; when the detected non-voice activity appears after continuous voice activity, and the frame number of the continuous voice activity exceeds a preset third frame number threshold value, continuing voice activity detection, and when the frame number of the non-voice activity exceeds a preset fourth frame number threshold value, stopping outputting a voice signal corresponding to the voice activity.
In a specific implementation, the second determining unit 1003 may be further configured to: within a preset mixed frame number threshold range, when the detected voice activity and non-voice activity occur alternately, calculating the proportion of the voice activity frame number to the sum of the voice activity frame number and the non-voice activity frame number; and outputting a voice signal corresponding to the voice activity when the proportion is larger than a preset proportion threshold value.
In a specific implementation, the speech noise reduction algorithm may be at least one of the following algorithms: LMS, NLMS, spectral subtraction, and wiener filtering algorithms.
In a specific implementation, the harmonic detection algorithm may be at least one of the following algorithms: autocorrelation function methods, cepstrum methods, linear prediction methods, and wavelet methods.
An embodiment of the present invention further provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transitory storage medium, and has stored thereon computer instructions, where the computer instructions, when executed, perform the steps of any one of the voice activity detection methods provided in the foregoing embodiments of the present invention.
An embodiment of the present invention further provides a voice activity detection apparatus, including a memory and a processor, where the memory stores a computer instruction that can be executed on the processor, and when the processor executes the computer instruction, the processor executes any of the steps of the voice activity detection method provided in the above embodiments of the present invention.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in any computer readable storage medium, and the storage medium may include: ROM, RAM, magnetic or optical disks, and the like.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (44)
1. A method of voice activity detection, comprising:
acquiring an acquired voice signal;
respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm to judge whether voice activity exists in the voice signal;
determining that voice activity is detected from the speech signal when both the speech noise reduction algorithm and the harmonic detection algorithm determine that voice activity is present in the speech signal.
2. The method of claim 1, wherein the determining whether voice activity is present in the voice signal using a voice noise reduction algorithm and a harmonic detection algorithm, respectively, comprises:
performing voice noise reduction calculation on the voice signal to obtain a noise-reduced voice signal;
calculating the energy corresponding to the voice signal and the energy corresponding to the voice signal after noise reduction to obtain the energy ratio of the voice signal before and after noise reduction;
and when the energy ratio is smaller than a preset first energy ratio threshold, judging that voice activity exists in the voice signal.
3. The method of claim 1, wherein the determining whether voice activity is present in the voice signal using a voice noise reduction algorithm and a harmonic detection algorithm, respectively, comprises:
performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum;
performing voice noise reduction calculation on the voice signal by adopting a wiener filtering noise reduction algorithm to obtain a noise-reduced voice signal frequency domain amplitude spectrum;
calculating the energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction according to the wiener filter function, the voice signal frequency domain amplitude spectrum after noise reduction and the voice signal frequency domain amplitude spectrum; the wiener filter function is obtained by calculation according to the wiener filter noise reduction algorithm and the noise estimation value of the voice signal; the noise estimation value is obtained by adopting a noise estimation algorithm;
and when the energy ratio is smaller than a preset second energy ratio threshold, judging that voice activity exists in the voice signal.
4. The voice activity detection method according to claim 3, wherein the energy ratio of the voice signal frequency domain magnitude spectrum before and after the noise reduction is calculated by using the following formula:
wherein E iswThe energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction is shown, ks is a preset frequency point starting point, ke is a preset frequency point terminal point, Y (k) is the voice signal frequency domain amplitude spectrum, and S' (k) is the voice signal frequency domain amplitude spectrum after noise reduction.
5. The voice activity detection method according to claim 3, wherein the preset second energy ratio threshold is positively correlated with: a ratio of an energy corresponding to the speech signal to an energy corresponding to the noise estimate.
6. The method of claim 1, wherein the determining whether voice activity is present in the voice signal using a voice noise reduction algorithm and a harmonic detection algorithm, respectively, comprises:
and when the voice signal is in a preset voice fundamental frequency range and contains harmonic features, judging that voice activity exists in the voice signal.
7. The method of claim 1, wherein the determining whether voice activity is present in the voice signal using a voice noise reduction algorithm and a harmonic detection algorithm, respectively, comprises:
performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum;
determining the number of peak values of the frequency domain amplitude spectrum of the voice signal; the peak value of the voice signal frequency domain amplitude spectrum is determined by adopting the following method: when the frequency domain amplitude spectrum corresponding to the ith frequency point in the voice signal frequency domain amplitude spectrum is larger than the maximum value in the frequency domain amplitude spectrum corresponding to the (i + 1) th frequency point, the frequency domain amplitude spectrum corresponding to the (i-1) th frequency point and a preset amplitude threshold corresponding to the ith frequency point, determining the frequency domain amplitude spectrum corresponding to the ith frequency point as a peak value of the voice signal voice frequency domain amplitude spectrum;
and when the number of the peak values exceeds a preset threshold value of the number of the peak values, judging that voice activity exists in the voice signal.
8. The voice activity detection method according to claim 7, wherein the preset amplitude threshold corresponding to the ith frequency point is obtained by:
performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum;
selecting a noise estimation value of the voice signal, a mean value of the voice signal frequency domain amplitude spectrum, and a maximum value in a corresponding minimum voice frequency domain amplitude spectrum from the i-1 frequency point to the i +1 frequency point as a preset amplitude threshold corresponding to the i frequency point; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
9. The voice activity detection method of claim 7, after determining the number of peaks in the frequency domain magnitude spectrum of the voice signal, further comprising:
sequentially taking the frequency index value corresponding to each peak value as a fundamental frequency, and calculating the frequency doubling deviation between the frequency index value corresponding to each peak value after the peak value corresponding to the fundamental frequency and the fundamental frequency;
when the frequency multiplication deviation is larger than a preset deviation threshold value, the peak value is excluded;
sequentially calculating weighted values of all residual peak values according to the frequency multiplication deviation and the residual peak values;
comparing the weighted value corresponding to each fundamental frequency, and selecting the maximum weighted value;
and when the maximum weighted value is larger than a preset weighted threshold value, judging that voice activity exists in the voice signal.
10. The voice activity detection method of claim 9 wherein the weighting of all remaining peaks is calculated using the following equation:
Eh=∑αnY[pn];
wherein E ishIs a weighted value, p, of said total remaining peak valuenFor the frequency index value corresponding to the nth remaining peak, Y [ p ]n]For the frequency domain amplitude spectrum corresponding to the nth residual peak, αnα for the preset weight coefficientn∈(0,1]。
12. The voice activity detection method of claim 9, wherein the predetermined weighted threshold is positively correlated with: a ratio of energy corresponding to the speech signal to energy corresponding to the noise estimate; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
13. The voice activity detection method according to claim 8, wherein the preset amplitude threshold corresponding to the ith frequency point is calculated by using the following formula:
Ythr=2max(mean(Y),D[k],min(Y[k-2]……Y[k+2]);
wherein, YthrA preset amplitude threshold corresponding to the ith frequency point, Y is the frequency domain amplitude spectrum of the voice signal, Dk]For the noise estimate, k is the frequency point, Y [ k-2 ]]……Y[k+2]And the spectrum is the voice signal frequency domain amplitude spectrum corresponding to the k-2 to k +2 frequency points.
14. The voice activity detection method of claim 1, wherein the determining that voice activity is detected from the voice signal comprises:
calculating the energy corresponding to the voice signal;
and when the energy corresponding to the voice signal is larger than a preset energy threshold value, judging that voice activity is detected from the voice signal.
15. The voice activity detection method of claim 14 wherein the corresponding energy of the voice signal is calculated using the following equation:
Eabs=∑(Y[k])2;
wherein E isabsFor the corresponding energy of the speech signal, Y [ k ]]A frequency domain amplitude spectrum of the voice signal is obtained, and k is a preset frequency range; the voice signal frequency domain amplitude spectrum passes through the voiceAnd performing fast Fourier transform on the sound signal to obtain the sound signal.
16. The voice activity detection method of claim 14 wherein the calculating the corresponding energy of the voice signal comprises:
carrying out noise estimation on the voice signal to obtain a noise estimation value;
calculating the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value;
and when the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value is larger than a preset third energy ratio threshold value, judging that voice activity is detected from the voice signal.
17. The voice activity detection method of claim 16 wherein the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimate is calculated using the formula:
Evs=log(Eabs)-log(En);
wherein E isvsIs the ratio of the energy corresponding to the speech signal to the energy corresponding to the noise estimate, EabsFor the corresponding energy of the speech signal, EnFor the energy corresponding to said noise estimate, En=∑(D[k])2,D[k]Is the noise estimate.
18. The voice activity detection method of claim 1, when it is determined that voice activity is detected from the voice signal, further comprising:
when the detected voice activity appears after continuous non-voice activity and the number of the continuous non-voice activity frames exceeds a preset first frame number threshold, caching the voice activity, and when the number of the voice frames of the voice activity exceeds a preset second frame number threshold, outputting a voice signal corresponding to the voice activity;
when the detected non-voice activity appears after continuous voice activity, and the frame number of the continuous voice activity exceeds a preset third frame number threshold value, continuing voice activity detection, and when the frame number of the non-voice activity exceeds a preset fourth frame number threshold value, stopping outputting a voice signal corresponding to the voice activity.
19. The voice activity detection method of claim 18, when it is determined that voice activity is detected from the voice signal, further comprising:
within a preset mixed frame number threshold range, when the detected voice activity and non-voice activity occur alternately, calculating the proportion of the voice activity frame number to the sum of the voice activity frame number and the non-voice activity frame number;
and outputting a voice signal corresponding to the voice activity when the proportion is larger than a preset proportion threshold value.
20. The voice activity detection method of claim 1, wherein the voice noise reduction algorithm is at least one of: LMS, NLMS, spectral subtraction, and wiener filtering algorithms.
21. The voice activity detection method of claim 1, wherein the harmonic detection algorithm is at least one of: autocorrelation function methods, cepstrum methods, linear prediction methods, and wavelet methods.
22. A voice activity detection apparatus, comprising:
the acquisition unit is used for acquiring the acquired voice signals;
the first judging unit is used for judging whether voice activity exists in the voice signal by respectively adopting a voice noise reduction algorithm and a harmonic detection algorithm;
a second determining unit, configured to determine that voice activity is detected from the voice signal when the voice noise reduction algorithm and the harmonic detection algorithm both determine that voice activity exists in the voice signal.
23. The voice activity detection apparatus of claim 22, wherein the first determination unit is configured to: performing voice noise reduction calculation on the voice signal to obtain a noise-reduced voice signal; calculating the energy corresponding to the voice signal and the energy corresponding to the voice signal after noise reduction to obtain the energy ratio of the voice signal before and after noise reduction; and when the energy ratio is smaller than a preset first energy ratio threshold, judging that voice activity exists in the voice signal.
24. The voice activity detection apparatus of claim 22, wherein the first determination unit is configured to: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; performing voice noise reduction calculation on the voice signal by adopting a wiener filtering noise reduction algorithm to obtain a noise-reduced voice signal frequency domain amplitude spectrum; calculating the energy ratio of the voice signal frequency domain amplitude spectrum before and after noise reduction according to the wiener filter function, the voice signal frequency domain amplitude spectrum after noise reduction and the voice signal frequency domain amplitude spectrum; the wiener filter function is obtained by calculation according to the wiener filter noise reduction algorithm and the noise estimation value of the voice signal; the noise estimation value is obtained by adopting a noise estimation algorithm; and when the energy ratio is smaller than a preset second energy ratio threshold, judging that voice activity exists in the voice signal.
25. The voice activity detection apparatus according to claim 24, wherein the energy ratio of the amplitude spectrum of the voice signal before and after the noise reduction is calculated by using the following formula:
wherein E iswIs the energy ratio of the speech signal frequency domain amplitude spectrum before and after noise reduction, ks is the preset frequency point starting point, ke is the preset frequency point end point, Y (k) is the speech signal frequency domain amplitude spectrum, and S' (k) is the speech signal after noise reductionFrequency domain magnitude spectra.
26. The voice activity detection device of claim 24, wherein the predetermined second energy ratio threshold is positively correlated with: a ratio of an energy corresponding to the speech signal to an energy corresponding to the noise estimate.
27. The voice activity detection apparatus of claim 22, wherein the first determination unit is configured to: and when the voice signal is in a preset voice fundamental frequency range and contains harmonic features, judging that voice activity exists in the voice signal.
28. The voice activity detection apparatus of claim 22, wherein the first determination unit is configured to: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; determining the number of peak values of the frequency domain amplitude spectrum of the voice signal; the peak value of the voice signal frequency domain amplitude spectrum is determined by adopting the following method: when the frequency domain amplitude spectrum corresponding to the ith frequency point in the voice signal frequency domain amplitude spectrum is larger than the maximum value in the frequency domain amplitude spectrum corresponding to the (i + 1) th frequency point, the frequency domain amplitude spectrum corresponding to the (i-1) th frequency point and a preset amplitude threshold corresponding to the ith frequency point, determining the frequency domain amplitude spectrum corresponding to the ith frequency point as a peak value of the voice signal voice frequency domain amplitude spectrum; and when the number of the peak values exceeds a preset threshold value of the number of the peak values, judging that voice activity exists in the voice signal.
29. The voice activity detection device according to claim 28, wherein the preset amplitude threshold corresponding to the ith frequency point is obtained by: performing fast Fourier transform on the voice signal to obtain a voice signal frequency domain amplitude spectrum; selecting a noise estimation value of the voice signal, a mean value of the voice signal frequency domain amplitude spectrum, and a maximum value in a corresponding minimum voice frequency domain amplitude spectrum from the i-1 frequency point to the i +1 frequency point as a preset amplitude threshold corresponding to the i frequency point; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
30. The voice activity detection apparatus of claim 28, wherein the first determination unit is further configured to: sequentially taking the frequency index value corresponding to each peak value as a fundamental frequency, and calculating the frequency doubling deviation between the frequency index value corresponding to each peak value after the peak value corresponding to the fundamental frequency and the fundamental frequency; when the frequency multiplication deviation is larger than a preset deviation threshold value, the peak value is excluded; sequentially calculating weighted values of all residual peak values according to the frequency multiplication deviation and the residual peak values; comparing the weighted value corresponding to each fundamental frequency, and selecting the maximum weighted value; and when the maximum weighted value is larger than a preset weighted threshold value, judging that voice activity exists in the voice signal.
31. The voice activity detection device of claim 30 wherein the weighting of all remaining peaks is calculated using the following equation:
Eh=∑αnY[pn];
wherein E ishIs a weighted value, p, of said total remaining peak valuenFor the frequency index value corresponding to the nth remaining peak, Y [ p ]n]For the frequency domain amplitude spectrum corresponding to the nth residual peak, αnα for the preset weight coefficientn∈(0,1]。
32. The voice activity detection device of claim 30, wherein the octave deviation is calculated using the following equation:
where Δ f is the frequency multiplication deviation, pnFor the frequency index value, p, corresponding to the nth peakbbIs a frequency index value that is a fundamental frequency.
33. The voice activity detection device of claim 30, wherein the pre-set weighted threshold is positively correlated with: a ratio of energy corresponding to the speech signal to energy corresponding to the noise estimate; and calculating the noise estimation value of the voice signal by adopting a noise estimation algorithm.
34. The voice activity detection device according to claim 29, wherein the preset amplitude threshold corresponding to the ith frequency point is calculated using the following formula:
Ythr=2max(mean(Y),D[k],min(Y[k-2]……Y[k+2]);
wherein, YthrA preset amplitude threshold corresponding to the ith frequency point, Y is the frequency domain amplitude spectrum of the voice signal, Dk]For the noise estimate, k is the frequency point, Y [ k-2 ]]……Y[k+2]And the spectrum is the voice signal frequency domain amplitude spectrum corresponding to the k-2 to k +2 frequency points.
35. The voice activity detection apparatus of claim 22, wherein the second determination unit is configured to: calculating the energy corresponding to the voice signal; and when the energy corresponding to the voice signal is larger than a preset energy threshold value, judging that voice activity is detected from the voice signal.
36. The voice activity detection device of claim 35 wherein the corresponding energy of the voice signal is calculated using the formula:
Eabs=∑(Y[k])2;
wherein E isabsFor the corresponding energy of the speech signal, Y [ k ]]A frequency domain amplitude spectrum of the voice signal is obtained, and k is a preset frequency range; the voice signal frequency domain magnitude spectrum is obtained by performing fast Fourier transform on the voice signal.
37. The voice activity detection apparatus of claim 35, wherein the second determination unit is configured to: carrying out noise estimation on the voice signal to obtain a noise estimation value; calculating the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value; and when the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimation value is larger than a preset third energy ratio threshold value, judging that voice activity is detected from the voice signal.
38. The voice activity detection device of claim 37 wherein the ratio of the energy corresponding to the voice signal to the energy corresponding to the noise estimate is calculated using the following equation:
Evs=log(Eabs)-log(En);
wherein E isvsIs the ratio of the energy corresponding to the speech signal to the energy corresponding to the noise estimate, EabsFor the corresponding energy of the speech signal, EnFor the energy corresponding to said noise estimate, En=∑(D[k])2,D[k]Is the noise estimate.
39. The voice activity detection apparatus of claim 22, wherein the second determination unit is further configured to: when the detected voice activity appears after continuous non-voice activity and the number of the continuous non-voice activity frames exceeds a preset first frame number threshold, caching the voice activity, and when the number of the voice frames of the voice activity exceeds a preset second frame number threshold, outputting a voice signal corresponding to the voice activity; when the detected non-voice activity appears after continuous voice activity, and the frame number of the continuous voice activity exceeds a preset third frame number threshold value, continuing voice activity detection, and when the frame number of the non-voice activity exceeds a preset fourth frame number threshold value, stopping outputting a voice signal corresponding to the voice activity.
40. The voice activity detection apparatus of claim 39, wherein the second determination unit is further configured to: within a preset mixed frame number threshold range, when the detected voice activity and non-voice activity occur alternately, calculating the proportion of the voice activity frame number to the sum of the voice activity frame number and the non-voice activity frame number; and outputting a voice signal corresponding to the voice activity when the proportion is larger than a preset proportion threshold value.
41. The voice activity detection apparatus of claim 22 wherein the voice noise reduction algorithm is at least one of: LMS, NLMS, spectral subtraction, and wiener filtering algorithms.
42. The voice activity detection apparatus of claim 22, wherein the harmonic detection algorithm is at least one of: autocorrelation function methods, cepstrum methods, linear prediction methods, and wavelet methods.
43. A computer-readable storage medium, being a non-volatile storage medium or a non-transitory storage medium, having computer instructions stored thereon, wherein the computer instructions, when executed, perform the voice activity detection method according to any one of claims 1 to 21.
44. A voice activity detection apparatus comprising a memory and a processor, the memory having stored thereon computer instructions executable on the processor, wherein the processor executes the computer instructions to perform the voice activity detection method of any one of claims 1 to 21.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910184966.8A CN111292758B (en) | 2019-03-12 | 2019-03-12 | Voice activity detection method and device and readable storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910184966.8A CN111292758B (en) | 2019-03-12 | 2019-03-12 | Voice activity detection method and device and readable storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111292758A true CN111292758A (en) | 2020-06-16 |
CN111292758B CN111292758B (en) | 2022-10-25 |
Family
ID=71023303
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910184966.8A Active CN111292758B (en) | 2019-03-12 | 2019-03-12 | Voice activity detection method and device and readable storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111292758B (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112967738A (en) * | 2021-02-01 | 2021-06-15 | 腾讯音乐娱乐科技(深圳)有限公司 | Human voice detection method and device, electronic equipment and computer readable storage medium |
CN113035222A (en) * | 2021-02-26 | 2021-06-25 | 北京安声浩朗科技有限公司 | Voice noise reduction method and device, filter determination method and voice interaction equipment |
CN114242116A (en) * | 2022-01-05 | 2022-03-25 | 成都锦江电子系统工程有限公司 | Comprehensive judgment method for voice and non-voice of voice |
CN116631456A (en) * | 2023-07-21 | 2023-08-22 | 江西红声技术有限公司 | Sound control communication processing method, earphone, storage medium and computer |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060195316A1 (en) * | 2005-01-11 | 2006-08-31 | Sony Corporation | Voice detecting apparatus, automatic image pickup apparatus, and voice detecting method |
US20080281589A1 (en) * | 2004-06-18 | 2008-11-13 | Matsushita Electric Industrail Co., Ltd. | Noise Suppression Device and Noise Suppression Method |
CN104464722A (en) * | 2014-11-13 | 2015-03-25 | 北京云知声信息技术有限公司 | Voice activity detection method and equipment based on time domain and frequency domain |
US20160203833A1 (en) * | 2013-08-30 | 2016-07-14 | Zte Corporation | Voice Activity Detection Method and Device |
CN106971741A (en) * | 2016-01-14 | 2017-07-21 | 芋头科技(杭州)有限公司 | The method and system for the voice de-noising that voice is separated in real time |
CN107833581A (en) * | 2017-10-20 | 2018-03-23 | 广州酷狗计算机科技有限公司 | A kind of method, apparatus and readable storage medium storing program for executing of the fundamental frequency for extracting sound |
KR20180067920A (en) * | 2016-12-13 | 2018-06-21 | 한국전자통신연구원 | System and method for end-point detection of speech based in harmonic component |
-
2019
- 2019-03-12 CN CN201910184966.8A patent/CN111292758B/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080281589A1 (en) * | 2004-06-18 | 2008-11-13 | Matsushita Electric Industrail Co., Ltd. | Noise Suppression Device and Noise Suppression Method |
US20060195316A1 (en) * | 2005-01-11 | 2006-08-31 | Sony Corporation | Voice detecting apparatus, automatic image pickup apparatus, and voice detecting method |
US20160203833A1 (en) * | 2013-08-30 | 2016-07-14 | Zte Corporation | Voice Activity Detection Method and Device |
CN104464722A (en) * | 2014-11-13 | 2015-03-25 | 北京云知声信息技术有限公司 | Voice activity detection method and equipment based on time domain and frequency domain |
CN106971741A (en) * | 2016-01-14 | 2017-07-21 | 芋头科技(杭州)有限公司 | The method and system for the voice de-noising that voice is separated in real time |
KR20180067920A (en) * | 2016-12-13 | 2018-06-21 | 한국전자통신연구원 | System and method for end-point detection of speech based in harmonic component |
CN107833581A (en) * | 2017-10-20 | 2018-03-23 | 广州酷狗计算机科技有限公司 | A kind of method, apparatus and readable storage medium storing program for executing of the fundamental frequency for extracting sound |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112967738A (en) * | 2021-02-01 | 2021-06-15 | 腾讯音乐娱乐科技(深圳)有限公司 | Human voice detection method and device, electronic equipment and computer readable storage medium |
CN113035222A (en) * | 2021-02-26 | 2021-06-25 | 北京安声浩朗科技有限公司 | Voice noise reduction method and device, filter determination method and voice interaction equipment |
CN113035222B (en) * | 2021-02-26 | 2023-10-27 | 北京安声浩朗科技有限公司 | Voice noise reduction method and device, filter determination method and voice interaction equipment |
CN114242116A (en) * | 2022-01-05 | 2022-03-25 | 成都锦江电子系统工程有限公司 | Comprehensive judgment method for voice and non-voice of voice |
CN116631456A (en) * | 2023-07-21 | 2023-08-22 | 江西红声技术有限公司 | Sound control communication processing method, earphone, storage medium and computer |
Also Published As
Publication number | Publication date |
---|---|
CN111292758B (en) | 2022-10-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111292758B (en) | Voice activity detection method and device and readable storage medium | |
Sadjadi et al. | Unsupervised speech activity detection using voicing measures and perceptual spectral flux | |
CN103456310B (en) | Transient noise suppression method based on spectrum estimation | |
Kim et al. | Feature extraction for robust speech recognition based on maximizing the sharpness of the power distribution and on power flooring | |
US8521530B1 (en) | System and method for enhancing a monaural audio signal | |
US7912567B2 (en) | Noise suppressor | |
Soon et al. | Low distortion speech enhancement | |
KR101737824B1 (en) | Method and Apparatus for removing a noise signal from input signal in a noisy environment | |
Lu | Enhancement of single channel speech using perceptual-decision-directed approach | |
CN111554315A (en) | Single-channel voice enhancement method and device, storage medium and terminal | |
Upadhyay et al. | An improved multi-band spectral subtraction algorithm for enhancing speech in various noise environments | |
Morales-Cordovilla et al. | Feature extraction based on pitch-synchronous averaging for robust speech recognition | |
CN112309417A (en) | Wind noise suppression audio signal processing method, device, system and readable medium | |
Wang et al. | Spectral subtraction based on two-stage spectral estimation and modified cepstrum thresholding | |
Zhang et al. | A soft decision based noise cross power spectral density estimation for two-microphone speech enhancement systems | |
WO2022068440A1 (en) | Howling suppression method and apparatus, computer device, and storage medium | |
KR100931487B1 (en) | Noisy voice signal processing device and voice-based application device including the device | |
Lu | Reduction of musical residual noise using block-and-directional-median filter adapted by harmonic properties | |
KR20100009936A (en) | Noise environment estimation/exclusion apparatus and method in sound detecting system | |
Yektaeian et al. | Comparison of spectral subtraction methods used in noise suppression algorithms | |
Asgari et al. | Voice activity detection using entropy in spectrum domain | |
Abramson et al. | Enhancement of speech signals under multiple hypotheses using an indicator for transient noise presence | |
Farahani et al. | Robust feature extraction of speech via noise reduction in autocorrelation domain | |
Xia et al. | A modified spectral subtraction method for speech enhancement based on masking property of human auditory system | |
Selvi et al. | Efficient speech enhancement technique by exploiting the harmonic structure of voiced segments |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |