KR102343811B1 - Method for detecting voice - Google Patents

Abstract

A voice detection method is disclosed. A voice detection method according to an embodiment of the present invention includes receiving an audio signal through a first microphone and a second microphone, and selecting reliable frequency bins from among a plurality of frequency bins corresponding to the audio signal based on a preset condition. selecting, and detecting the presence or absence of speech in the audio signal using the selected confidence frequency bins.

Description

Voice detection method {METHOD FOR DETECTING VOICE}

The present invention relates to a voice detection method for selecting reliable frequencies based on a preset condition and detecting the presence or absence of voice in an audio signal using the trusted frequencies.

Voice activity detection (VAD) is to detect the presence/absence of a human voice.

A voice is a signal that changes with the passage of time, and it is common to divide an audio signal into frames of 20 ms to 30 ms and detect the voice for each frame. And in general, voice activity detection (VAD) aims to determine whether a human voice or a target person's voice is present in a corresponding frame of an audio signal.

This voice activity detection technology can be widely used in pre-processing of voice enhancement (noise removal), noise estimation, pitch extraction, variable voice communication codecs, voice recognition, and the like.

In the conventional voice activity detection technology, after converting an audio signal into a frequency domain, a threshold value is compared with test statistics obtained by averaging values corresponding to all frequency bins in the frequency domain. was determined to exist.

On the other hand, components of speech are sparsely distributed in frequency bins. And since the method of using values corresponding to all frequency bins is a method of using not only reliable frequency bin values but also unreliable frequency bin values, the voice There was a problem in that the accuracy of detecting the presence or absence of

SUMMARY OF THE INVENTION The present invention is to solve the above problems, and an object of the present invention is to provide a voice detection method for selecting reliable frequencies based on a preset condition and detecting the presence or absence of a voice in an audio signal using the trusted frequencies. it's about

A voice detection method according to an embodiment of the present invention includes receiving an audio signal through a first microphone and a second microphone, and selecting reliable frequency bins from among a plurality of frequency bins corresponding to the audio signal based on a preset condition. selecting, and detecting the presence or absence of a voice in the audio signal using the selected confidence frequency bins.

In this case, the selecting of the confidence frequency bins includes determining a frequency bin that satisfies the preset condition as the confidence frequency bin, and the preset condition includes the size of the audio signal and the level difference between channels of the audio signal. , may be based on at least one of a time difference between channels of the audio signal.

In this case, the preset condition may be a condition in which the magnitude of the first audio signal received through the first microphone closer to the expected position of the sound source in a corresponding frequency bin is greater than a preset value.

Meanwhile, the preset condition is a condition in which a level difference between channels of a first audio signal received by the first microphone and a second audio signal received by the second microphone in a corresponding frequency bin is greater than a preset value can

Meanwhile, the preset condition is that, in a corresponding frequency bin, a time difference between channels of the audio signal is within a preset range, the corresponding frequency bin is smaller than a first preset value, or the corresponding frequency bin is a second preset value It may be a condition greater than the value.

Meanwhile, the preset condition includes two or more detailed conditions, and the selecting of the confidence frequency bins may include selecting the corresponding frequency bin as the confidence frequency bin when the corresponding frequency bin satisfies all of the two or more detailed conditions. can be determined as

On the other hand, the step of detecting the presence of voice in the audio signal using the selected confidence frequency bins may include detecting voice activity based on detection statistics using NDPSD, LTIPD, and values corresponding to the selected confidence frequency bins. The presence or absence of a voice in the audio signal may be detected using a voice activity detection algorithm based on at least one of the algorithms.

Meanwhile, the detecting of the presence or absence of a voice in the audio signal using the selected confidence frequency bins may include combining inter-channel time difference information corresponding to the selected reliable frequency bins and inter-channel level difference information to obtain the audio signal. Detecting the presence of a voice in the audio signal, wherein detecting the presence of a voice in the audio signal by combining the inter-channel time difference information corresponding to the selected confidence frequency bins and the inter-channel level difference information includes: Whether a voice is present in the audio signal by using a voice activity detection algorithm based on at least one of algorithms for detecting voice activity based on AND, SVM, and a classifier inputting values corresponding to the selected confidence frequency bins to detect

In this case, the step of detecting the presence of a voice in the audio signal using the selected reliable frequency bins may include, when the number of the selected reliable frequency bins is smaller than a preset number, the voice does not exist in the audio signal. can decide not to

Meanwhile, the mobile terminal according to an embodiment of the present invention receives an audio signal through a receiver including a first microphone and a second microphone, and the first microphone and the second microphone, and based on a preset condition, the and a control unit that selects reliable frequency bins from among a plurality of frequency bins corresponding to the audio signal and detects whether a voice is present in the audio signal using the selected reliable frequency bins.

In this case, the first microphone may be disposed closer to the mouth of the speaker who is talking using the mobile terminal, and the second microphone may be disposed farther from the mouth of the speaker who is talking with the first microphone than the first microphone. .

According to the present invention, there is an advantage in that the accuracy of voice activity detection can be improved by selecting a reliable frequency by applying a condition and detecting the presence or absence of a voice using values corresponding to the reliable frequencies.

1 is a block diagram illustrating a voice detection apparatus.
2 is a diagram for explaining a mobile terminal that is an example of a voice detection device.
3 is a flowchart illustrating a voice detection method.
4 is a diagram for explaining conditions in a test.
5 is a view showing a comprehensive test result.
6 is a diagram illustrating test results for each level of SNR.
7 and 8 are diagrams illustrating experimental results according to another test.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numbers regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including an ordinal number such as 1st, 2nd, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

1 is a block diagram illustrating a voice detection apparatus.

Audio signals may include voice signals and other acoustic signals within the audible range.

In addition, the voice detection device may refer to a device that detects a voice in the audio signal by using the audio signal.

The voice detection apparatus 100 according to an embodiment of the present invention may include an output unit 120 , a reception unit 110 , a control unit 130 , and a memory 140 .

The receiver 110 may receive an audio signal.

Specifically, the receiver 110 may include a multi-channel microphone, and the multi-channel microphone may receive an audio signal from the outside. Here, the multi-channel microphone may include two or more microphones.

The memory 140 may store data supporting various functions of the voice detection apparatus 100 .

The output unit 120 may output whether a voice is present in the audio signal. Specifically, the output unit 120 includes at least one of a display and a device for outputting a voice detection result as an electrical signal, and displays the presence or absence of a voice in the audio signal detected by the controller 130 or outputs it as an electrical signal. can do.

Meanwhile, the controller 130 may control the overall operation of the voice detection apparatus 100 .

In addition, when the audio signal generated from the sound source is received through the multi-channel microphone, the controller 130 may detect the presence of a voice in the audio signal using the received audio signal.

2 is a diagram for explaining a mobile terminal that is an example of a voice detection device.

The voice detection apparatus 100 may be a mobile terminal such as a mobile phone or a smart phone. Therefore, in this specification, the term voice detection apparatus 100 is used in parallel with the term mobile terminal 100 .

The mobile terminal 100 may include a multi-channel microphone. In this specification, it is assumed that the mobile terminal 100 includes two microphones, that is, a first microphone 210 and a second microphone 220 .

The two microphones may be disposed on different sides of the mobile terminal 100 . For example, the first microphone 210 may be disposed on the lower side of the mobile terminal 100 , and the second microphone 220 may be disposed on the upper side of the mobile terminal 100 .

Meanwhile, the first microphone 210 may be disposed closer to the expected position of the sound source.

For example, when the speaker uses the mobile terminal 100 for a general purpose, the first microphone 210 may be disposed closer to the speaker's mouth. Specifically, when the speaker makes a phone call using the mobile terminal 100 , the first microphone 210 may be located closer to the mouth of the speaker than the second microphone 220 . The second microphone 220 may be located farther from the speaker's mouth than the first microphone 210 .

Also, the mobile terminal may include a wireless communication unit and an output unit.

The wireless communication unit includes one or more modules that enable wireless communication between the mobile terminal 100 and the wireless communication system, between the mobile terminal 100 and another mobile terminal 100, or between the mobile terminal 100 and an external server. may include In addition, the wireless communication unit may include one or more modules for connecting the mobile terminal 100 to one or more networks.

The wireless communication unit may include a mobile communication module. In addition, the mobile communication module includes technical standards or communication methods for mobile communication (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), and EV-DO ( Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term (LTE-A) Evolution-Advanced), etc.), transmits and receives radio signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network.

The wireless signal may include various types of data according to transmission/reception of a voice call signal, a video call signal, or a text/multimedia message.

Also, the mobile terminal may include a speaker, and the speaker may output a call sound in a call mode.

3 is a flowchart illustrating a voice detection method.

The control unit 130 of the voice detection apparatus 100 may receive an audio signal through the first microphone and the second microphone (S310).

Here, the audio signal may include noise. In addition, the audio signal may or may not include a human voice for each time section.

Accordingly, the controller 130 may generate a plurality of frames by dividing the audio signal according to a time period (eg, 20-30 ms), and may detect whether a voice is present in the audio signal for each frame.

In this case, the control unit 130 obtains a complex spectrum of an audio signal by applying a short interval Fourier transform (STFT) to each of a plurality of frames, and applies a stage for a plurality of frequency bins present in each of the plurality of frames. Interval Fourier transform coefficients may be obtained.

Hereinafter, an example of the l-th frame among the plurality of frames will be described.

Also, the controller 130 may obtain short-term Fourier transform coefficients for a plurality of frequency bins with respect to the first audio signal received through the first microphone and the second audio signal received through the second microphone. have.

으로 표현될 수 있다.In this case, for the k-th frequency bin of the l-th frame, the short-term Fourier coefficient of the first audio signal received by the first microphone is

can be expressed as

으로 표현될 수 있다.Also, for the k-th frequency bin of the l-th frame, the short-term Fourier coefficient of the second audio signal received by the second microphone is

can be expressed as

The components of speech are sparsely distributed in frequency bins. And since the method of using values corresponding to all frequency bins is a method of using not only reliable frequency bin values but also unreliable frequency bin values, the voice There was a problem in that the accuracy of detecting the presence or absence of

Therefore, in the present specification, a method of selecting a frequency bin having high reliability and detecting the presence or absence of a voice based on spatial information of the selected frequency bins is proposed.

In the present specification, a frequency bin having high reliability is referred to as a reliable frequency bin.

The controller 130 may select reliable frequency bins from among a plurality of frequency bins corresponding to the audio signal based on a preset condition ( S330 ).

In more detail, the controller 130 may determine a frequency bin that satisfies a preset condition as a reliable frequency bin.

Here, the preset condition may be based on at least one of a size of an audio signal, a level difference between channels of the audio signal, and a time difference between channels of the audio signal.

First, the first detailed condition will be described. The first detailed condition may be referred to as a first mask.

Also, the first detailed condition (first mask) may be expressed as m1(l, k). Here, l may mean an l-th frame, and k may mean a k-th frequency bin. Also, m1(l, k)=1 may mean that the k-th frequency bin of the l-th frame is a confidence frequency bin under the first detailed condition. Also, m1(l, k)=0 may mean that the k-th frequency bin of the l-th frame is not a reliable frequency bin under the second detailed condition.

The first detailed condition is based on the size of the audio signal, among others, the size of the first audio signal received through the first microphone closer to the expected location of the sound source (eg, the speaker's mouth).

Specifically, the first detailed condition is a condition in which, in the corresponding frequency bin, the magnitude of the first audio signal received through the first microphone closer to the expected location of the sound source (eg, the speaker's mouth) is greater than a preset value. can

This can be expressed by the following formula.

는 제1 마이크에 의해 수신된 제1 오디오 신호의 l번째 프레임의 k번째 주파수 빈에 대한 단 구간 푸리에 계수,

는 크기,

는 기 설정된 값을 의미할 수 있다.where m1(l, k) is the first mask,

is the short-term Fourier coefficient for the k-th frequency bin of the l-th frame of the first audio signal received by the first microphone,

is the size,

may mean a preset value.

That is, when the speaker speaks, the level of the audio signal received from the first microphone close to the speaker's mouth may increase regardless of the presence or absence of surrounding noise. Accordingly, the first detailed condition (first mask) may be determined based on the power of the first audio signal received through the first microphone close to the speaker's mouth.

In addition, when the magnitude of the first audio signal received through the first microphone closer to the speaker's mouth is greater than a preset value, the controller 130 may determine the corresponding frequency bin as the reliable frequency bin. Accordingly, the k-th frequency bin of the l-th frame may be determined as a reliable frequency or an unreliable frequency.

Next, the second detailed condition will be described. The second detailed condition may be referred to as a second mask.

Also, the second detailed condition (second mask) may be expressed as m2(l, k). Also, m2(l, k)=1 may mean that the k-th frequency bin of the l-th frame is a confidence frequency bin under the second detailed condition. Also, m2(l, k)=0 may mean that the k-th frequency bin of the l-th frame is not a reliable frequency bin under the second detailed condition.

The second detailed condition is based on a level difference between channels of the audio signal.

로 표현될 수 있다. 여기서

는 l번째 프레임의 k번째 주파수 빈에서, 제1 마이크에 의해 수신된 제1 오디오 신호의 단 구간 푸리에 계수를 의미할 수 있다. 또한 여기서

는 l번째 프레임의 k번째 주파수 빈에서, 제2 마이크에 의해 수신된 제2 오디오 신호의 단 구간 푸리에 계수를 의미할 수 있다.An interchannel level difference (ILD) may mean a level difference between audio signals acquired by two microphones. And the interchannel level difference (ILD) is

can be expressed as here

may mean a short-term Fourier coefficient of the first audio signal received by the first microphone in the k-th frequency bin of the l-th frame. also here

may mean a short-term Fourier coefficient of the second audio signal received by the second microphone in the k-th frequency bin of the l-th frame.

In addition, the second detailed condition may be a condition in which a level difference between channels of the first audio signal received by the first microphone and the second audio signal received by the second microphone at a corresponding frequency is greater than a preset value.

This can be expressed by the following formula.

는 l프레임의 k번째 주파수 빈에서의 채널 간 레벨 차,

는 기 설정된 값을 의미할 수 있다.where m2(l, k) is the second mask,

is the level difference between channels in the k-th frequency bin of the l frame,

may mean a preset value.

It is assumed that the speaker is talking on the phone using a mobile terminal and another person is talking from the back. In addition, it is assumed that the first microphone, which is closer to the speaker's mouth, mainly receives the speaker's voice, and the second microphone mainly receives the voice of another person. In this case, there may be a situation in which the magnitude of the first audio signal received through the first microphone and the magnitude of the second audio signal received through the second microphone are not significantly different from each other.

In addition, according to the prior art, the presence or absence of a voice is detected by averaging the amplitudes of signals in all frequency bins. The pitch of each person's voice is different, and it appears in the form of harmonics of the basic pitch frequency in the frequency domain. In the case of averaging the level difference (sign: -) between channels in frequency bins corresponding to the region, the average value is low, so that the presence or absence of the speaker's voice cannot be properly detected.

))보다 클 때, 제어부(130)는 해당하는 주파수 빈을 신뢰 주파수 빈으로 결정할 수 있다. 이에 따라 l번째 프레임의 k번째 주파수 빈은 신뢰 주파수 또는 비 신뢰 주파수로 결정될 수 있다.Accordingly, in the present invention, the level of the first audio signal received from the first microphone close to the expected position of the sound source (eg, the speaker's mouth) is sufficiently greater than the level of the second audio signal received from the second microphone (preset value). (

)), the controller 130 may determine a corresponding frequency bin as a reliable frequency bin. Accordingly, the k-th frequency bin of the l-th frame may be determined as a reliable frequency or an unreliable frequency.

Accordingly, the second mask may improve robustness of voice detection with respect to a noise source close to the second microphone.

Next, the third detailed condition will be described. The third detailed condition may be referred to as a third mask.

Also, the third detailed condition (third mask) may be expressed as m3(l, k). Also, m3(l, k)=1 may mean that the k-th frequency bin of the l-th frame is a confidence frequency bin under the third detailed condition. Also, m3(l, k)=0 may mean that the k-th frequency bin of the l-th frame is not a reliable frequency bin under the third detailed condition.

The third detailed condition is based on a time difference between channels of the audio signal.

Here, an interchannel time difference (ITD) may mean a time difference between audio signals acquired by two or more microphones.

로 표현될 수 있다.That is, the time difference between channels may mean a time difference between the first audio signal received from the first microphone and the second audio signal received from the second microphone when the same audio signal is received,

can be expressed as

And the third detailed condition is a condition that a time difference between channels of an audio signal at a corresponding frequency is within a preset range, a corresponding frequency is less than a first preset value, or a corresponding frequency is greater than a second preset value can

This can be expressed by the following formula.

는 l번째 프레임의 k번째 주파수 빈에서의 채널 간 시간 차를 의미할 수 있다.where m3(l, k) is the third mask,

may mean a time difference between channels in the k-th frequency bin of the l-th frame.

내지

) 이내이거나, 해당하는 주파수 빈(k)이 제1 기 설정된 값(k1)보다 작거나, 해당하는 주파수 빈(k)이 제2 기 설정된 값(k2)보다 큰 조건일 수 있다.In addition, the third detailed condition is a preset range (

inside

), the corresponding frequency bin k is smaller than the first preset value k1, or the corresponding frequency bin k is larger than the second preset value k2.

Here, k1 and k2 may be ranges of frequency bin indices. For example, k1 may be 100 Hz and k2 may be 1000 Hz.

And k1 may be set so that frequency bins in which the direction-of-arrival (DoA) of the sound source is too sensitive to a small error in phase measurement are excluded from the selection using the inter-channel time difference. Accordingly, when the corresponding frequency bin k is smaller than the first preset value k1, the corresponding frequency bin k may be determined as a reliable frequency bin.

Also, k2 can be set to avoid spatial aliasing with a specific margin. Accordingly, when the corresponding frequency bin k is greater than the second preset value k2, the corresponding frequency bin k may be determined as a reliable frequency bin.

On the other hand, when the corresponding frequency bin (k) is smaller than the second preset value (k2) and greater than the first preset value (k1), whether the corresponding frequency bin (k) is a reliable frequency based on the time difference between channels can be determined.

내지

)는 시간을 나타내는 것으로(예를 들어

는 0.008s,

는 0.04s), 예상되는 음원의 위치에 기반하여 결정될 수 있다.In Equation 3, the preset range (

inside

) denotes time (e.g.

is 0.008s,

0.04s), may be determined based on the expected location of the sound source.

내지

)는 화자의 입이 위치할 것이라고 예상되는 범위에 대응하도록 설정될 수 있다.For example, when the speaker uses the mobile terminal 100 to make a phone call, there is a range in which the speaker's mouth is expected to be located. In this case, the preset range (

inside

) may be set to correspond to a range in which the speaker's mouth is expected to be located.

내지

)에 있을 수 있다. 따라서 채널 간 시간 차가 기 설정된 범위 내 (

내지

)에 있는 경우, 제어부(130)는 해당하는 주파수 빈을 신뢰 주파수 빈으로 결정할 수 있다.That is, when the actual sound source is located in the same direction as the expected position of the sound source, the time difference between channels for the received audio signal is within a preset range (

inside

) may be in Therefore, the time difference between channels is within the preset range (

inside

), the controller 130 may determine a corresponding frequency bin as a reliable frequency bin.

내지

) 밖에 있을 수 있다. 따라서 채널 간 시간 차가 기 설정된 범위 (

내지

) 밖에 있는 경우, 제어부(130)는 해당하는 주파수 빈을 비 신뢰 주파수 빈으로 결정할 수 있다.On the other hand, when the actual sound source is located in a different direction from the expected position of the sound source, the time difference between channels for the received audio signal is within a preset range (

inside

) may be outside. Therefore, the time difference between channels is within the preset range (

inside

), the controller 130 may determine the corresponding frequency bin as an unreliable frequency bin.

내지

) 밖에 있는 경우, 해당하는 주파수 빈에 대응하는 공간 정보는, 음성의 존재 여부를 검출하는데 사용되지 않을 수 있다.That is, the time difference between channels is a preset range (

inside

), spatial information corresponding to the corresponding frequency bin may not be used to detect the presence of a voice.

Meanwhile, based on a preset condition, the controller 130 may select reliable frequency bins from among a plurality of frequency bins corresponding to the audio signal.

The first detailed condition, the second detailed condition, and the third detailed condition have been described above as examples of the detailed conditions.

Meanwhile, any one of the detailed conditions may independently constitute a preset condition. However, the present invention is not limited thereto, and two or more detailed conditions may constitute a preset condition.

In this case, two or more detailed conditions may be connected with “and”. That is, the preset condition includes two or more detailed conditions, and when the corresponding frequency bin satisfies all of the two or more detailed conditions, the corresponding frequency bin may be determined as a reliable frequency bin.

In Equation 4, a preset condition (m) including a first detailed condition (m1(l, k)), a second detailed condition (m2(l, k)), and a third detailed condition (m3(l, k)) (l, k)) is shown.

In this case, the first detailed condition (m1(l, k)), the second detailed condition (m2(l, k)), and the third detailed condition (m3(l, k)) must all be satisfied, and the corresponding frequency bin ( The k-th frequency bin of the l-th frame) may be determined as a confidence frequency.

And m(l, k)=1 may mean that the k-th frequency bin of the l-th frame is a confidence frequency bin under a preset condition in which a plurality of detailed conditions are combined. Also, m(l, k)=0 may mean that the k-th frequency bin of the l-th frame is not a reliable frequency bin under a preset condition in which a plurality of detailed conditions are combined.

Meanwhile, the controller 130 may detect the presence of a voice in the audio signal using the selected confidence frequency bins ( S350 ).

In more detail, the controller 130 may detect the presence of a voice in the audio signal by using a voice activity detection algorithm based on at least one of NDPSD, LTIPD, AND, and SVM. In addition, the selected confidence frequency bins may be used to detect the presence of a voice in the audio signal.

First, a voice detection method based on NDSD will be described.

The NDPSD-based voice activity detection algorithm is an algorithm that determines whether a voice is present according to the difference in the size (power difference) of the audio signal between two microphones. Prior Document 1 (Jeub, M.; Herglotz, C.; Nelke, C. ; Beaugeant, C.; Vary, P. Noise reduction for dual-microphone mobile phones exploiting power level differences. 1693?1696.) is the algorithm presented. And this algorithm may be based on the level difference (ILD) between channels.

In this case, the existing NDPSD-based voice activity detection method can be expressed by the following equation.

는 크기,

는 l번째 프레임의 k번째 주파수 빈에서, 제1 마이크로부터 수신된 신호의 단구간 푸리에 변환(STFT) 계수(coefficients),

는 l번째 프레임의 k번째 주파수 빈에서, 제2 마이크로부터 수신된 신호의 단구간 푸리에 변환(STFT) 계수(coefficients),

는 l번째 프레임의 k번째 주파수 빈에서 신호간의 정규화된 크기 차,

는 모든 주파수 빈에 대한 신호간 크기 차들의 평균,

는 l번째 프레임에서의 음성의 존재 유무를 의미한다.here

is the size,

In the k-th frequency bin of the l-th frame, the short-term Fourier transform (STFT) coefficients of the signal received from the first microphone,

In the k-th frequency bin of the l-th frame, the short-term Fourier transform (STFT) coefficients of the signal received from the second microphone,

is the normalized magnitude difference between signals in the k-th frequency bin of the l-th frame,

is the average of the magnitude differences between signals for all frequency bins,

denotes the presence or absence of a voice in the l-th frame.

)를 임계 값

과 비교함으로써 음성의 존재 유무가 검출된다.That is, in the existing NDPSD-based voice detection method, the average (

) to the threshold

By comparing with , the presence or absence of a voice is detected.

Meanwhile, the voice detection method based on the improved NDPSD proposed by the present invention can be expressed by the following equation.

는 개선된 NDPSD 음성 활동 감지 알고리즘에 기반하여 산출된, l번째 프레임에서의 음성의 존재 유무를 의미한다. 또한

는 선별된 신뢰 주파수 빈들(m(l, k)가 1인 주파수 빈들)에 대응하는 값들(신호 간의 정규화된 크기 차)에 대한 평균을 의미할 수 있다.

denotes the presence or absence of a voice in the lth frame calculated based on the improved NDPSD voice activity detection algorithm. Also

may mean an average of values (normalized difference between signals) corresponding to the selected confidence frequency bins (frequency bins in which m(l, k) is 1).

)이 임계 값

보다 큰 경우 해당 프레임에 음성이 존재하는 것으로 결정한 다는 것을 의미할 수 있다.Therefore, Equation 7 is the average (

) this threshold

If it is larger than that, it may mean that it is determined that the voice is present in the corresponding frame.

)이 추가될 수 있다. 여기서

은 신뢰 주파수 빈의 개수를 의미하고,

는 기 설정된 개수를 의미할 수 있다.In addition, in order to determine that voice is present in the frame, a new condition (

) can be added. here

is the number of confidence frequency bins,

may mean a preset number.

)이 임계 값

보다 크거나 같고, 해당 프레임 내에 존재하는 신뢰 주파수 빈의 개수(

)가 기 설정된 개수(

)보다 크거나 같은 경우, 헤당 프레임에 음성이 존재하는 것으로 결정할 수 있다.Accordingly, the control unit 130 calculates the average (

) this threshold

greater than or equal to the number of confidence frequency bins present in the frame (

) is the preset number (

) is greater than or equal to, it may be determined that the voice is present in the corresponding frame.

)이 임계 값

보다 크지만, 해당 프레임 내에 존재하는 신뢰 주파수 빈의 개수(

)가 기 설정된 개수(

)보다 작은 경우, 제어부(130)는 해당 프레임에 음성이 존재하지 않는 것으로 결정할 수 있다.On the other hand, the average (

) this threshold

greater than, but the number of confidence frequency bins present in the frame (

) is the preset number (

), the controller 130 may determine that there is no voice in the corresponding frame.

)가 기 설정된 개수(

)보다 큰 경우에만, 오디오 신호 내 음성이 존재하는 것으로 판단할 수 있다.That is, if the number of confidence frequency bins is too small, an error may occur because the sample is too small. Therefore, the number of confidence frequency bins (

) is the preset number (

), it can be determined that the voice in the audio signal is present.

The following describes a voice detection method based on LTIPD.

The LTIPD-based voice activity detection algorithm is an algorithm that determines that a voice is present when a signal is continuously received from a target direction. Prior Document 2 (Guo, Y.; Li, K.; Fu, Q.; Yan, Y A two microphone based voice activity detection for distant talking speech in wide range of direction of arrival.In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Kyoto, Japan, 25?30 March 2012; ) is presented in And this algorithm may be based on the time difference (ITD) between channels.

That is, the LTIPD-based voice activity detection algorithm measures how much energy is concentrated in frequency bins with the same direction-of-arrival (DoA) estimate (with the same sector direction-of-arrival estimate). It could be a way to

개의 섹터로 분할될 수 있다. In this case, the range of the target direction-of-arrival (DoA) of the sound source has the same width.

It can be divided into sectors.

And the voice detection method based on LTIPD can be expressed by the following equation.

는 l번째 프레임의 k번째 주파수 빈에서, 제1 마이크로부터 수신된 신호의 단구간 푸리에 변환(STFT) 계수(coefficients),

은 채널 간 시간 차가 타겟 범위 내에 있는 주파수 들에 대응하는 파워들의 합산 치,

는 에너지 집중의 임계 값,

는 l번째 프레임에서의 음성의 존재 유무를 의미할 수 있다.where i is the index of the sector, Ci (l, k) is the number of frames in which the DOA estimate for the k-th frequency bin represents the i-th sector among the latest L frames, and ki is the threshold of DOA concentration value,

In the k-th frequency bin of the l-th frame, the short-term Fourier transform (STFT) coefficients of the signal received from the first microphone,

is the sum of powers corresponding to frequencies within the target range with the time difference between channels,

is the threshold of energy concentration,

may mean the presence or absence of a voice in the l-th frame.

)과 비교하여 음성의 존재 유무를 검출하였다.That is, in the existing LTIPD-based voice detection method, after summing the signal sizes of all frequency bins pointing to the target direction angle, the sum of the sectors with the largest sum among the sectors and the threshold value of energy concentration (

) and the presence or absence of a negative was detected.

Meanwhile, the voice detection method based on the improved LTIPD proposed by the present invention can be expressed by the following equation.

가 결정될 수 있다. 그리고 파워들의 합산 치

를 에너지 집중의 임계 값과 비교함으로써, l번째 프레임에서의 음성의 존재 여부(

)가 검출될 수 있다.That is, in the improved LTIPD-based voice detection method, by summing the sizes of frequency bins (reliable frequency bins) that satisfy a preset condition, the sum of powers

can be determined. and the sum of the powers

By comparing with the threshold of energy concentration, the presence of speech in the lth frame (

) can be detected.

Meanwhile, in addition to NDPSD and LTIPD, an algorithm for detecting voice activity based on detection statistics obtained using values corresponding to selected confidence frequency bins may be used.

Hereinafter, a voice detection method based on AND and SVM will be described. AND and SVM may be a method of detecting the presence or absence of voice in an audio signal by combining information on a time difference between channels and information on a level difference between channels. In addition, the improved AND and the improved SVM proposed in the present invention may be a method of detecting the presence or absence of a voice in an audio signal by combining a time difference between channels corresponding to selected confidence frequency bins and a level difference between channels.

First, a voice detection method based on AND will be described.

The AND-based voice activity detection algorithm is based on the inter-channel time difference (ITD) and voice activity detection based on the inter-channel time difference (ITD) because the inter-channel time difference (ITD) and the inter-channel level difference (ILD) have different characteristics for the location of the sound source. As a method of combining the results of voice activity detection based on the level difference (ILD), Prior Document 3 (Park, J.; Jin, YG; Hwang, S.; Shin, JW Dual Microphone Voice Activity Detection Exploiting InterchannelTime and Level Difference. IEEE Signal Process. Lett. 2016, 23, 1335-1339).

Voice activity detection based on inter-channel time difference (ITD) and voice activity detection based on inter-channel level difference (ILD) may be combined with and. Therefore, the AND-based voice activity detection algorithm can be expressed by the following equation.

: 채널 간 시간 차(ITD)에 기초한 음성 활동 감지의 결과,

: 채널 간 레벨 차(ILD)에 기초한 음성 활동 감지의 결과)(

: the result of voice activity detection based on the time difference (ITD) between channels,

: Result of detection of voice activity based on the level difference (ILD) between channels)

That is, in the AND-based voice activity detection algorithm, the result of voice activity detection based on the inter-channel time difference (ITD) is 'existence of voice', and the result of voice activity detection based on the inter-channel level difference (ILD) is 'voice'. In the case of 'existence of ', it is finally determined that the voice exists.

)와 개선된 LTIPD 음성 활동 감지 알고리즘에 기반하여 산출된 음성의 존재 유무(

)를 and 연산 한 것일 수 있다.On the other hand, the voice detection method based on the improved AND proposed in the present invention is the presence or absence of a voice calculated based on the improved NDPSD voice activity detection algorithm.

) and the presence or absence of a voice calculated based on the improved LTIPD voice activity detection algorithm (

) may be an and operation.

That is, when it is determined that the voice is present based on the improved NDPSD voice activity detection algorithm and it is determined that the voice exists based on the improved LTIPD voice activity detection algorithm, the controller 130 determines that the voice is finally present. it can be decided that

The following describes a voice detection method based on SVM.

The voice activity detection algorithm based on SVM trains the classifier using the inter-channel time difference (ITD) and the inter-channel level difference (ILD) as features, and the classifier is the inter-channel time difference (ITD) and the inter-channel level As a method of classifying the presence or absence of voice using ILD as a feature, prior document 3 (Park, J.; Jin, YG; Hwang, S.; Shin, JW Dual Microphone Voice Activity) Detection Exploiting InterchannelTime and Level Difference. IEEE Signal Process. Lett. 2016, 23, 1335-1339).

The existing SVM-based voice activity detection algorithm trains a classifier by providing an inter-channel time difference (ITD) and an inter-channel level difference (ILD) for all frequency bins. In addition, the existing SVM-based voice activity detection algorithm provides the inter-channel time difference (ITD) and the inter-channel level difference (ILD) to the classifier for all frequency bins, so that the classifier outputs the presence or absence of a voice.

Meanwhile, the improved SVM-based voice detection method proposed by the present invention can train a classifier by providing an inter-channel time difference (ITD) and an inter-channel level difference (ILD) corresponding to a confidence frequency bin.

In addition, the improved SVM-based voice detection method provides the inter-channel time difference (ITD) and the inter-channel level difference (ILD) corresponding to the confidence frequency bins to the classifier so that the classifier outputs the presence or absence of a voice.

Specifically, information corresponding to a frequency bin that satisfies a preset condition is provided to the classifier, and information corresponding to a frequency bin that does not satisfy a preset condition is treated as 0 and then provided to the classifier. have.

Meanwhile, in addition to AND and SVM, an algorithm for detecting voice activity based on detection statistics obtained using values corresponding to selected confidence frequency bins may be used.

4 is a diagram for explaining conditions in a test.

The size of the room is 3119 * 3232 * 2080 mm3, and the reverberation time (RT60) is 100-200 ms.

And in the center of the room, the speaker was standing with a cell phone. The distance between the two microphones included in the phone is about 140mm.

On the other hand, omni-directional noise was generated by white, babble, and car noise in the NOISEX-92 database, and four loudspeakers (410, 420, 430, 440) were installed to face the corner of the room to generate complex reflections.

In addition, directional noise is a utterance uttered by four male and female speakers selected from the TIMIT database, and the speaker is uttered by any one of four loudspeakers at a distance of 1000 mm in the directions of 45, 135, 225, and 315 degrees from the speaker. played in the direction toward

Meanwhile, the speaker's voice was mixed with either directional noise or omnidirectional noise from one of the four loudspeakers. In this case, the SNR levels on the first microphone are -5dB, 0dB, 5dB, 10dB, 15dB, and 20dB.

)에서의 주파수 인덱스 범위(k1, k2)는 제1 마이크 및 제2 마이크 사이의 거리가 140mm인 것을 고려하여, (125Hz, 968.75Hz)에 대응하는 (4, 31)으로 설정되었다(수학식 3 참고).3rd mask (

), the frequency index range (k1, k2) was set to (4, 31) corresponding to (125 Hz, 968.75 Hz), considering that the distance between the first microphone and the second microphone was 140 mm (Equation 3) Reference).

And when it is assumed that 0 degrees is the direction of the end fire with respect to the first microphone, the range of the target DOA is set to 10 degrees to 70 degrees.

FIG. 5 is a view showing comprehensive test results, and FIG. 6 is a view showing test results for each level of SNR.

Here, the dotted line indicates the conventional algorithms (PLDR, MNDPSD, LTIPD, SVM, AND), and the solid line indicates the algorithms improved by the present invention (MNDPSD ^FS , LTIPD ^FS , SVM ^FS , AND ^FS ).

5 and 6, it can be seen that the algorithms improved by the present invention exhibit higher performance.

7 and 8 are diagrams showing experimental results according to another test.

This experiment was conducted in a larger lecture room, and the reverberation time (RT60) was 400-500 ms.

In FIG. 8 , a clean waveform denotes a waveform of an audio signal without noise, a noisy waveform denotes a waveform of an audio signal mixed with noise, and a ground truth of voice actively denotes actual voice activity.

And with reference to FIGS. 7 and 8 , it was found that the improved algorithms (MNDPSD ^FS , LTIPD ^FS , SVM ^FS , AND ^FS ) exhibit higher performance than the conventional algorithms (PLDR, MNDPSD, LTIPD, SVM, AND). Able to know.

As described above, according to the present invention, there is an advantage in that the accuracy of voice activity detection can be improved by selecting a reliable frequency by applying a condition and detecting the presence or absence of a voice using values corresponding to the reliable frequencies. .

On the other hand, the control unit is a component in charge of controlling the device in general, and may be used interchangeably with terms such as a central processing unit, a microprocessor, and a processor.

The present invention described above can be implemented as computer-readable codes on a medium in which a program is recorded. The computer-readable medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is this. In addition, the computer may include the control unit 180 of the terminal. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

110: receiving unit 120: output unit
130: control unit

Claims (11)

receiving an audio signal through the first microphone and the second microphone;
Based on a preset condition, a frequency bin that satisfies the preset condition among a plurality of frequency bins corresponding to the audio signal is determined as a reliable frequency bin, and a frequency bin that does not satisfy the preset condition is determined as a non-reliable frequency bin. and selecting the determined confidence frequency bins; and
detecting the presence of speech in the audio signal using the selected confidence frequency bins;
The preset condition is
In a corresponding frequency bin, the first detailed condition is a condition in which the magnitude of the first audio signal received through the first microphone closer to the expected location of the sound source is greater than a preset value;
a second detailed condition in which a level difference between channels of a first audio signal received by the first microphone and a second audio signal received by the second microphone in a corresponding frequency bin is greater than a preset value;
In the corresponding frequency bin, the third condition is that a time difference between channels of the audio signal is within a preset range, the corresponding frequency bin is smaller than a first preset value, or the corresponding frequency bin is greater than a second preset value including detailed conditions;
The step of selecting the confidence frequency bins comprises:
determining the corresponding frequency bin as the confidence frequency bin when the corresponding frequency bin satisfies all two or more detailed conditions among the first detailed condition to the third detailed condition
voice detection method. delete delete delete delete delete The method of claim 1,
Detecting the presence of a voice in the audio signal using the selected confidence frequency bins includes:
Whether a voice is present in the audio signal using a voice activity detection algorithm based on at least one of NDPSD, LTIPD, and algorithms for detecting voice activity based on detection statistics obtained using values corresponding to the selected confidence frequency bins to detect,
voice detection method. The method of claim 1,
Detecting the presence of a voice in the audio signal using the selected confidence frequency bins includes:
detecting the presence of a voice in the audio signal by combining inter-channel time difference information and inter-channel level difference information corresponding to the selected confidence frequency bins;
Detecting the presence of a voice in the audio signal by combining the inter-channel time difference information and the inter-channel level difference information corresponding to the selected confidence frequency bins,
Whether a voice is present in the audio signal by using a voice activity detection algorithm based on at least one of algorithms for detecting voice activity based on AND, SVM, and a classifier inputting values corresponding to the selected confidence frequency bins to detect
voice detection method. 8. The method of claim 7,
Detecting the presence of a voice in the audio signal using the selected confidence frequency bins includes:
When the number of the selected confidence frequency bins is less than a preset number, determining that the voice does not exist in the audio signal
voice detection method. a receiver including a first microphone and a second microphone; and
receiving an audio signal through the first microphone and the second microphone, and using a frequency bin satisfying the preset condition among a plurality of frequency bins corresponding to the audio signal based on a preset condition as a confidence frequency bin, and a control unit configured to determine a frequency bin that does not satisfy a preset condition as an unreliable frequency bin, select the determined reliable frequency bins, and detect the presence or absence of a voice in the audio signal using the selected reliable frequency bins, ,
The preset condition is
In a corresponding frequency bin, the first detailed condition is a condition in which the magnitude of the first audio signal received through the first microphone closer to the expected location of the sound source is greater than a preset value;
a second detailed condition in which a level difference between channels of a first audio signal received by the first microphone and a second audio signal received by the second microphone in a corresponding frequency bin is greater than a preset value;
In the corresponding frequency bin, the time difference between channels of the audio signal is within a preset range, the corresponding frequency bin is smaller than a first preset value, or the corresponding frequency bin is greater than a second preset value. including detailed conditions;
When the corresponding frequency bin satisfies all two or more detailed conditions among the first detailed condition to the third detailed condition, the control unit determines the corresponding frequency bin as the confidence frequency bin
mobile terminal. 11. The method of claim 10,
The first microphone,
disposed closer to the mouth of the speaker who is talking using the mobile terminal,
The second microphone is disposed farther from the mouth of the talker than the first microphone.
mobile terminal.

Images