JP6125953B2 - Voice section detection apparatus, method and program - Google Patents

Description

  The present invention relates to a technique for detecting a voice section from a voice digital signal, and a voice recognition technique performed for the detected voice section.

  Patent Document 1 is known as a technique for detecting a voice section from a multichannel voice digital signal. Technology for detecting the voice section of the main speaker from a digital audio signal that includes noise and the voice of a person other than the target person (hereinafter also referred to as “main speaker”) (hereinafter also referred to as “other speaker”) Patent Document 2 is known. In Patent Document 2, first, a voice digital signal is taken out for each frame of a predetermined length, the voice digital signal of the frame is analyzed, and the target speaker's voice digital signal is analyzed. It is determined whether or not speech is included, and the determination result is obtained as a speech / non-speech determination value. Next, speech recognition is performed on the speech digital signal, and a series of recognition units and speech time information of each recognition unit are obtained from the speech digital signal. Furthermore, using the speech / non-speech judgment value, the recognition unit sequence and the speech time information of each recognition unit, the total value of the speech / non-speech judgment value of the frame corresponding to the speech time of the recognition unit Based on this, it is determined for each recognition unit whether or not the target speaker has spoken.

JP 2009-031604 A JP 2012-048119 A

  Since Patent Document 1 detects all voice segments regardless of the relationship between the main speaker and other speakers and extracts voices other than the main speaker, the voice recognition accuracy for the main speaker may be greatly degraded. Further, in Patent Document 1, it is assumed that a plurality of microphones are used, and cannot be used in speech recognition in a mobile environment where sound collection by a single microphone is assumed. In Patent Document 2, since speech recognition is performed and the recognition result is used, the recognition accuracy is reduced under reverberation. As a result, a section including the voice of the main speaker from the voice digital signal (hereinafter referred to as “main story”). There is a problem that the accuracy of estimating the “personal speech section” is deteriorated.

  The present invention provides a high-accuracy main speaker even for an audio digital signal in which the voices of a plurality of speakers including a main speaker and other speakers are mixed in a real environment including reverberation. It is an object of the present invention to provide a speech section detection technique that can detect a speech of

  In order to solve the above-described problem, according to the first aspect of the present invention, a speech segment detection apparatus estimates a reverberation component included in a speech digital signal, acquires a reverberation signal, and a reverberation signal. And a section-by-speaker detection unit for detecting at least one of the main speaker voice section and the main speaker outside sound section.

  In order to solve the above-described problem, according to another aspect of the present invention, a speech interval detection method estimates a reverberation component included in a speech digital signal and acquires a reverberation signal; And a speaker-by-speaker section detecting step for detecting at least one of the main speaker voice section and the main speaker outside sound section.

  According to the present invention, in a real environment including reverberation, a main microphone with a high accuracy can be obtained even for a voice digital signal in which voices of a plurality of speakers including a main speaker and other speakers are mixed into a single microphone. There is an effect that the voice of the speaker can be detected.

The functional block diagram of the audio | voice area detection apparatus which concerns on 1st embodiment. The figure which shows the processing flow of the audio | voice area detection apparatus which concerns on 1st embodiment. FIG. 3A is a diagram showing an image of an audio analog signal corresponding to the voice of the main speaker, and FIG. 3B is a diagram showing an image of an audio analog signal corresponding to the external sound of the main speaker. 4A is a diagram illustrating an image in which a reverberation signal corresponding to the voice of the main speaker is represented by a vector, and FIG. 4B is a diagram illustrating an image in which a reverberation signal corresponding to the external sound of the main speaker is represented by a vector. FIG. 5A is a diagram illustrating an image in which a difference corresponding to the voice of the main speaker is represented by a vector, and FIG. 5B is a diagram illustrating an image in which a difference corresponding to the external sound from the main speaker is represented by a vector. The functional block diagram of the audio | voice area detection apparatus which concerns on 2nd embodiment. The figure which shows the processing flow of the audio | voice area detection apparatus which concerns on 2nd embodiment. The functional block diagram of the audio | voice area detection apparatus which concerns on 3rd, 4th embodiment. The figure which shows the processing flow of the audio | voice area detection apparatus which concerns on 3rd, 4th embodiment. The functional block diagram of the audio | voice area detection apparatus which concerns on 5th embodiment. The figure which shows the processing flow of the audio | voice area detection apparatus which concerns on 5th embodiment. FIG. 12A is a diagram showing an image representing the power of the reverberation signal of the main speaker's voice as a vector, and FIG. 12B is a diagram showing an image showing the power of the reverberation signal of the main speaker's sound as a vector. The functional block diagram of the audio | voice area detection apparatus which concerns on 6th embodiment. The figure which shows the processing flow of the audio | voice area detection apparatus which concerns on 6th embodiment. FIG. 15A is a diagram showing an image in which the power of the reverberation signal of the main speaker's voice is expressed as a vector when the voice of the other speaker included in the voice digital signal is small and the reverberation component is small, and FIG. The figure showing the image which represented the power of the reverberation signal of the main speaker outside sound by a vector in case the voice of the other speaker contained in is small and the reverberation component is small. The functional block diagram of the audio | voice area detection apparatus which concerns on 7th embodiment. The figure which shows the processing flow of the audio | voice area detection apparatus which concerns on 7th embodiment. The figure for demonstrating arrangement | positioning with either of the speech area detection apparatuses which concern on 7th embodiment from 1st, and the speech recognition apparatus which concerns on 7th embodiment.

  Hereinafter, embodiments of the present invention will be described. In the drawings used for the following description, constituent parts having the same function and steps for performing the same process are denoted by the same reference numerals, and redundant description is omitted. In the following description, the symbol “˜” and the like used in the text should be described immediately above the immediately preceding character, but are described immediately after the character due to restrictions on text notation. In the formula, these symbols are written in their original positions. Further, the processing performed for each element of a vector or matrix is applied to all elements of the vector or matrix unless otherwise specified.

<Points of first embodiment>
In this embodiment, the reverberation component contained in the audio digital signal is estimated, and the main speaker's external sound is emphasized from the reverberation component. In other words, the main speaker external sound section included in the input signal (audio analog signal) to the microphone is extracted from the reverberation component included in the input signal. The “main speaker outside sound” means a main sound other than the voice of the main speaker. As a reverberation component estimation method included in the audio digital signal, for example, the reverberation component estimation method of Reference 1 can be used.
[Reference 1] Pamphlet of International Publication No. 2007/100137

  In the mobile environment, the difference between the reverberation components contained in the main speaker's voice and the main speaker's external sound recorded by a single microphone is large, so that the main speaker's external sound can be emphasized with high accuracy. However, the present embodiment is not limited to a mobile environment and a single microphone, but can also be applied to audio analog signals obtained from other environments and a plurality of microphones.

<Audio section detection device 100 according to the first embodiment>
FIG. 1 is a functional block diagram of the speech section detection apparatus 100 according to the first embodiment, and FIG. 2 shows a processing flow thereof. The speech section detection apparatus 100 includes a speech signal acquisition unit 110, a reverberation estimation unit 120, a gain adjustment unit 130, a main speaker external sound enhancement unit 140, a main speaker external sound section detection unit 160, and a main speaker voice extraction unit 170. Including.
The voice section detection apparatus 100 receives a voice analog signal and outputs a voice digital signal corresponding to the voice of the main speaker.

<Audio signal acquisition unit 110>
Input: Audio analog signal output: Audio digital signal The audio signal acquisition unit 110 receives an analog audio signal (audio signal), converts it into a digital audio signal (audio digital signal) (s110), and outputs it. FIG. 3 shows an image in which the audio signal collected by the microphone 90 is expressed as a vector. 3A and 3B show audio analog signals corresponding to the main speaker's voice and main speaker's external sound, respectively. As shown in FIG. 3A, the sound signal corresponding to the voice of the main speaker has a large direct sound D and a small reflected sound R (reverberation component). On the other hand, as shown in FIG. 3B, the voice signal corresponding to the voice of the other speaker has a small direct sound D and a large reflected sound R (reverberation component). Using such a property, the main speaker outside sound section detection unit 160 described later detects at least one of the main speaker sound section and the main speaker outside sound section.

<Reverberation estimation unit 120>
Input: Audio digital signal output: Reverberation signal The reverberation estimation unit 120 estimates a reverberation component included in the audio digital signal (s120), and acquires the reverberation signal. 4A and 4B show images representing vectors of reverberation signals corresponding to the main speaker's voice and main speaker's external sound, respectively.

  Hereinafter, an outline of a method for estimating reverberation components will be described.

この原音声信号s(z)が空間を伝達し、マイクで観測される信号x(z)は、式(1)より、以下のように表される。

The original audio signal s (z) is obtained by applying a short auto-regressive (hereinafter also referred to as “AR”) process to the white signal u (z) as shown in Expression (1). The Z transformation of the AR process is v (z) = 1 / (1-b (z)), and 1-b (z) is a polynomial.

The original audio signal s (z) is transmitted through the space, and the signal x (z) observed by the microphone is expressed as follows from the equation (1).

Here, h (z) represents an indoor transfer function from the sound source to the microphone. The audio signal has a strong short-term correlation according to v (z). Therefore, by performing pre-whitening processing by linear prediction that removes short-term correlation according to Equation (3), v (z) can be regarded as a substantially white signal, and v (z) ≈1 holds.

Here, b (p) is a linear prediction coefficient for effectively suppressing v (z), and is obtained by Expression (4).

  Here, r (i) represents an autocorrelation coefficient when the signal x (z) observed by the microphone is shifted by i samples. This linear prediction is performed with a filter length of 30 ms, and it is expected that the short-term correlation between the early reflection component and the speech included within 30 ms is removed.

When D is a step size (delay) and L is a filter length, the reverberation signal d (n) can be formulated as follows.

ここで、h_min(z)とh_max(z)は、それぞれh(z)の最小位相成分（Z平面上の単位円内の零点に対応する成分）と最大位相成分（Z平面上の単位円外の零点に対応する成分）を表す。また、min[h_max(z)]は、h_max(z)を最小位相化する関数を表す。 Here, a (l) (Roman letter L) represents a linear prediction coefficient, and x to (n) represent pre-whitening-processed observation sounds obtained by Expression (3). a (z) obtained by z-converting a (l) is obtained by Expression (6).

Where h _min (z) and h _max (z) are the minimum phase component of h (z) (the component corresponding to the zero in the unit circle on the Z plane) and the maximum phase component (unit on the Z plane). Represents the component corresponding to the zero point outside the circle). Also, min [h _max (z)] represents a function that minimizes h _max (z).

In general, D is set to a value corresponding to 10 to 200 ms, and L is set to a value corresponding to 100 ms to 500 ms.
This technique is detailed in Reference Document 1, for example.

  Using the above-described method and other existing reverberation estimation techniques, the reverberation estimation unit 120 estimates a reverberation component included in the audio digital signal x (n) and acquires a reverberation signal d (n).

<Gain adjustment unit 130>
Input: Reverberation signal output: Gain-adjusted reverberation signal The gain adjustment unit 130 receives the reverberation signal, multiplies the reverberation signal by the gain G (s130), and obtains and outputs the gain-adjusted reverberation signal. As the gain G, a value smaller than 1 and larger than 0 is used. For example, a value of 0.8 to 1.0 is used. Thereby, in the main speaker outside sound emphasizing unit 140 described later, it is possible to reduce distortion that occurs when obtaining the difference between the audio digital signal and the reverberation signal.

<Outside sound enhancement unit 140>
Input: voice digital signal, gain-adjusted reverberation signal output: voice digital signal in which main speaker external sound is emphasized The main speaker external sound enhancement unit 140 receives the voice digital signal and the gain-adjusted reverberation signal, The difference between these signals is calculated (s140) and output as a voice digital signal in which the main speaker's external sound is emphasized. FIG. 5A and FIG. 5B represent images in which differences corresponding to the main speaker's voice and main speaker's external sound are represented by vectors. A small arrow in the figure represents a reverberation component R ′ that could not be removed. By performing such processing, the power difference between the main speaker's voice and main speaker's external sound can be obtained, and the main speaker's external sound section in the audio digital signal can be extracted with high accuracy. It becomes like this. Note that the processing of the gain adjusting unit 130 and the main speaker outside sound emphasizing unit 140 can be combined and realized by a known technique called a spectral subtraction method (see Reference 2).
[Reference 2] BOLL, SF, "Suppression of Acoustic Noise in Speech Using Spectral Subtraction", IEEE Trans. Acoust., Speech, Signal Processing, 1979, vol. ASSP-27, pp. 113-120

<Main speaker outer sound section detection unit 160>
Input: Voice digital signal with emphasized main speaker outside sound Output: Section information The main speaker outside sound section detecting unit 160 receives a voice digital signal with emphasized main speaker outside sound, and the main speaker outside sound is received. At least one of the main speaker voice section and the main speaker outside sound section is detected from the power of the emphasized voice digital signal (s160), and section information indicating at least one of the main speaker voice section and the main speaker outside sound section. Is output. For example, the power of the voice digital signal in which the main speaker external sound is emphasized is compared with a threshold value. (1) If it is larger than the threshold, it is determined that it is the main speaker's voice section. The threshold is determined in advance using learning data with correct labels of the main speaker voice section and the main speaker outside sound section. For example, the processing is performed for each section, with the audio digital signal having N samples before and after (number of samples corresponding to 0.1 to 0.3 ms) as one section. As the section information, for example, the start time and the end time of at least one of the main speaker voice section or the main speaker outside sound section can be used. Also, a voice digital signal is input and a signal in which at least one flag of the main speaker voice section or the main speaker outside voice section is added to the voice digital signal (hereinafter also referred to as “flagged voice digital signal”), etc. It may be used as However, the section information is not limited to the exemplified one, and may be any information as long as it is information indicating at least one of the main speaker voice section and the main speaker outside sound section.

<Main speaker voice extraction unit 170>
Input: voice digital signal, section information output: voice digital signal corresponding to the voice of the main speaker The main speaker voice extraction unit 170 receives the voice digital signal and the section information, and uses the section information from the voice digital signal. A portion corresponding to the voice of the main speaker is extracted (s170) and output as an output value of the voice section detecting device 100.

  For example, when the start time and the end time are used as the section information, 1 is added to the sample between the start time and the end time, and in order to secure a margin between the start time and the end time, the main speaker voice section 1 of N samples (sample length corresponding to 0.1 to 0.4ms) before start time to switch to speaker outside sound section, M samples (corresponding to 0.1 to 0.4ms) after end time to switch from 1 to 0 Perform margin processing to add 1 of (sample length). The main speaker speech section (that is, a time sample sequence in which the portion corresponding to the N samples before the start time to the M samples after the end time is 1 and the other portion is 0) subjected to the margin processing is timed to the speech digital signal. The main speaker voice can be extracted by multiplying each sample.

  In addition, when a signal obtained by adding a flag of the main speaker voice section to the voice digital signal is used as the section information, margin processing is performed on the signal (that is, the voice digital signal of N samples and M samples at the start and end, respectively). The voice digital signal corresponding to the portion to which the flag of the main speaker voice section is added is extracted. In addition, when a signal with the flag of the main speaker outside sound section added to the voice digital signal is used as the section information, margin processing is performed on the voice digital signal without the flag of the main speaker outside sound section, A voice digital signal corresponding to a portion to which no external sound section flag is assigned is extracted.

<Effect>
With such a configuration, even in a real environment including reverberation, even in a digital audio signal in which the voices of a plurality of speakers including a main speaker and other speakers are mixed into a single microphone, the main can be performed with high accuracy. The voice of the speaker can be detected. As a result, the number of microphones used can be reduced, and the hardware configuration can be reduced.

<Modification>
The speech section detection apparatus 100 does not necessarily need to include the speech signal acquisition unit 110 when receiving a speech digital signal as an input signal.
The speech segment detection device 100 does not necessarily have to include the gain adjustment unit 130. In this case, the main speaker external sound enhancement unit 140 uses a reverberation signal that has not been gain-adjusted.
The speech segment detection device 100 does not necessarily include the main speaker speech extraction unit 170. The output value (section information) of the main speaker outside sound section detection unit 160 is output as the output value of the speech section detection apparatus 100.

<Second embodiment>
A description will be given centering on differences from the first embodiment.
The sound collected by the microphone 90 may include noise other than voice. Therefore, in this embodiment, noise included in the audio digital signal is suppressed before the processing of the first embodiment. With such a configuration, it is possible to prevent accuracy deterioration due to noise in the main speaker external sound emphasis unit and the main speaker external sound section detection unit, and to extract the main speaker voice with high accuracy even in a noisy environment.

  FIG. 6 is a functional block diagram of the speech segment detection apparatus 200 according to the second embodiment, and FIG. 7 shows the processing flow. The speech segment detection device 200 further includes a noise suppressing unit 210 in addition to the configuration of the speech segment detection device 100.

<Noise Suppression Unit 210>
Input: voice digital signal output: voice digital signal with suppressed noise The noise suppression unit 210 receives the voice digital signal, suppresses noise other than the voice included in the voice digital signal (s210), and has the noise suppressed. Output a digital signal. As a noise suppression method, a known technique can be used. For example, the MMSE-STSA method (see Reference 3) can be considered.
[Reference 3] Y. Ephraim and D. Malah, “Speech enhancement using a minimum mean-square error log-spectral amplitude estimator”, IEEE Trans. Acoust. Speech Signal Process., April 1985, vol.ASSP-33, no .2, pp.443-445

  Note that the speech digital signal used in the reverberation estimation unit 120, the main speaker outside sound section detection unit 160, and the main speaker speech extraction unit 170 is a speech digital signal in which noise is suppressed by the noise suppression unit 210.

<Effect>
With such a configuration, the same effect as that of the first embodiment can be obtained. Furthermore, it is possible to prevent the extraction accuracy degradation of the main speaker voice section due to noise.

<Third embodiment>
A description will be given centering on differences from the first embodiment.
In the present embodiment, the main speaker external sound section is detected also using the power value of the entire voice digital signal including the main speaker's voice and main speaker external sound.

  FIG. 8 is a functional block diagram of the speech section detection apparatus 300 according to the third embodiment, and FIG. 9 shows the processing flow. The speech section detection apparatus 300 includes a speech signal acquisition unit 110, a reverberation estimation unit 120, a gain adjustment unit 130, a main speaker external sound enhancement unit 140, a main speaker external sound section detection unit 360, and a main speaker voice extraction unit 170. And an audio signal power calculator 350.

<Audio signal power calculator 350>
Input: Audio digital signal output: Audio digital signal power The audio signal power calculation unit 350 receives the audio digital signal, calculates its power (s350), and outputs it. Note that the audio signal power calculation unit 350 may include a signal smoothing unit 351 (indicated by a broken line in the drawing). The signal smoothing unit 351 smoothes an audio digital signal by averaging it using N samples before and after (number of samples corresponding to 0.1 to 0.3 ms) (s351, indicated by a broken line in the figure). The audio signal power calculation unit 350 may calculate the power of the smoothed audio digital signal. By performing the smoothing, it is possible to emphasize so that the main speaker voice section or the main speaker external sound can be easily detected.

<Outside sound section detection unit 360>
Input: power of voice digital signal, voice digital signal in which main speaker outside sound is emphasized output: main speaker voice section or main speaker outside sound section The main speaker outside sound section detecting unit 360 is a power of the voice digital signal. And from the power of the “sound digital signal with emphasized external sound of the main speaker” obtained by receiving the audio digital signal with emphasized external sound of the main speaker and normalized by the power of the audio digital signal or At least one of the main speaker outer sound sections is detected (s360), and section information indicating at least one of the main speaker voice section and the main speaker outer sound section is output. For example, the power of the “voice digital signal in which the sound outside the main speaker is emphasized” normalized by the power of the voice digital signal is compared with a threshold value. In place of the voice digital signal in which the main speaker external sound is emphasized, the power of the “voice digital signal in which the main speaker external sound is emphasized” normalized by the power of the voice digital signal is used. The same processing as the processing in the main speaker outside sound section detection unit 160 is performed.

<Effect>
By adopting such a configuration, the reverberation component estimated to be a direct sound component due to an error by the reverberation estimation unit can be attenuated by the power of the audio digital signal, and the same effect as in the first embodiment can be obtained. Can do. In addition, you may combine the modification of 1st embodiment, 2nd embodiment, and this embodiment.

<Fourth embodiment>
A description will be given centering on differences from the third embodiment.
In the present embodiment, a silent section where there is no main speaker voice or main speaker outside sound is estimated using the power of the entire voice digital signal, and the main speaker voice section or main speaker outside sound is outside the silent section. By detecting the section, the detection accuracy is increased.

  FIG. 8 is a functional block diagram of the speech segment detection apparatus 400 according to the fourth embodiment, and FIG. 9 shows the processing flow. The voice section detection device 400 includes a main speaker outside sound section detection unit 460 instead of the main speaker outside sound section detection unit 360.

<Main speaker outside sound section detection unit 460>
Input: power of voice digital signal, voice digital signal with emphasized main speaker sound output: section information The main speaker outer sound section detector 460 emphasizes the power of voice digital signal and the main speaker noise. An audio digital signal is received. The main speaker outside sound section detection unit 460 includes a silent section extraction unit 461 (indicated by a broken line in the figure). The silent section extraction unit 461 receives the power of the audio digital signal and extracts a silent section from the power (s461, indicated by a broken line in the figure). For example, the power of the audio digital signal is compared with a threshold value. When the power is set to P = 10 log 10 (square value of audio digital signal) [dB], the threshold value is set to a value of -10 to 10 [dB], and the magnitude is compared with this threshold value. A section where the power is lower than the threshold value is set as a silent section, and a large section is set as outside the silent section. The silent section is determined for each section, for example, with 0.1 to 0.3 ms as one section.

  Furthermore, the main speaker outer sound section detection unit 460 detects at least one of the main speaker voice section and the main speaker outer sound section from the audio digital signal outside the silent section (s460). For example, outside the silence period, the voice digital signal in which the sound outside the main speaker is emphasized is normalized by the power of the voice digital signal, and outside the silence period, the normalization by the power of the voice digital signal is performed. At least one of the main speaker voice section and the main speaker outer sound section is detected from the power of the “voice digital signal in which the speaker outer sound is emphasized”. The power of the “sound digital signal with emphasized external sound of the main speaker” that is normalized by the power of the audio digital signal outside the silent period instead of just the “sound digital signal with emphasized external sound of the main speaker” Except for the point of using, the same processing as the processing in the main speaker outside sound section detection unit 160 is performed.

  In addition, the main speaker outside sound section detection unit 460 may include a signal smoothing unit 462 (indicated by a broken line in the drawing). The signal smoothing unit 462 performs the same processing as that of the above-described signal smoothing unit 351 instead of a simple “audio digital signal”. By performing the smoothing, it is possible to emphasize so that the main speaker voice section or the main speaker external sound can be easily detected. In this case, the main speaker outside sound section detection unit 460 performs the smoothing of the “sound digital signal in which the main speaker outside sound is emphasized” that is normalized by the power of the sound digital signal outside the silent section. At least one of the main speaker voice section and the main speaker outside sound section is detected from the power.

<Effect>
With such a configuration, the same effect as that of the third embodiment can be obtained. Furthermore, detection accuracy can be improved by detecting the main speaker voice section or the main speaker outside sound section outside the silent section. Note that the silent section extraction unit 461 may not be included, and only the signal smoothing unit 462 may be included.

<Fifth embodiment>
A description will be given centering on differences from the first embodiment.
FIG. 10 is a functional block diagram of a speech section detection apparatus 500 according to the fifth embodiment, and FIG. 11 shows the processing flow. The speech section detection apparatus 500 includes a speech signal acquisition unit 110, a reverberation estimation unit 120, a gain adjustment unit 130, a reverberation signal power calculation unit 540, a main speaker outside sound section detection unit 560, and a main speaker speech extraction unit 170.

<Reverberation signal power calculation unit 540>
Input: reverberation signal with gain adjusted: power of reverberation signal The reverberation signal power calculation unit 540 receives the reverberation signal with gain adjustment, calculates its power (s540), and outputs it. The processing in the reverberation signal power calculation unit 540 is the same as the processing in the audio signal power calculation unit 350 except that a reverberation signal whose gain is adjusted is used instead of the audio digital signal. For example, a signal smoothing unit 351 may be included therein. 12A and 12B show images representing the power of the reverberation signal of the main speaker's voice and main speaker's external sound as vectors, respectively. As can be seen from FIGS. 12A and 12B and FIGS. 4A and 4B, the power of the reverberation signal is small in the voice of the main speaker, and the power of the reverberation signal is large in the sound outside the main speaker. Utilizing such a property, the main speaker outside sound section detection unit 560 described later obtains section information.

<Main speaker outside sound section detection unit 560>
Input: Power output of reverberation signal: Section information The main speaker outer sound section detector 560 receives the power of the reverberation signal, and determines at least one of the main speaker voice section and the main speaker outer sound section from the power of the reverberation signal. Detection is performed (s560), and section information indicating at least one of the main speaker voice section and the main speaker outside sound section is output. For example, the power of the reverberation signal is compared with a threshold value. (1) When larger than the threshold value, it is determined that it is a main speaker outside sound section, and (2) when it is smaller than the threshold value, it is determined as a main speaker voice section. The threshold is determined in advance using learning data with correct labels of the main speaker voice section and the main speaker outside sound section. For example, the processing is performed for each section, with the audio digital signal having N samples before and after (number of samples corresponding to 0.1 to 0.3 ms) as one section.

<Effect>
With such a configuration, the same effect as that of the first embodiment can be obtained. Note that the modification of the first embodiment may be combined with the present embodiment, the noise suppression unit 210 of the second embodiment, the audio signal power calculation unit 350 of the fourth embodiment, the silence interval extraction unit 461, and the signal smoothing. The unit 462 and this embodiment may be combined. However, the silent section extraction unit 461 and the signal smoothing unit 462 use the power of the reverberation signal instead of the power of the voice digital signal in which the main speaker external sound is emphasized.

<Sixth embodiment>
A description will be given centering on differences from the fifth embodiment.
FIG. 13 is a functional block diagram of a speech segment detection apparatus 600 according to the sixth embodiment, and FIG. 14 shows the processing flow thereof. The voice section detection apparatus 600 further includes a voice signal power calculation unit 350 (see the third embodiment), and includes a main speaker outer sound section detection unit 660 instead of the main speaker outer sound section detection unit 560.

<Main speaker outside sound section detection unit 660>
Input: power of voice digital signal, power of reverberation signal output: section information The main speaker outside sound section detection unit 660 receives the power of the voice digital signal and the power of the reverberation signal, and normalizes it by the power of the voice digital signal. From the power of the reverberant signal, at least one of the main speaker voice section and the main speaker outer sound section is detected (s660), and section information indicating at least one of the main speaker voice section and the main speaker outer sound section is output. . For example, the power of the reverberation signal normalized by the power of the audio digital signal is compared with a threshold value. A process similar to the process in the main speaker outside sound section detection unit 560 except that the “reverberation signal power” normalized by the power of the audio digital signal is used in place of the simple “reverberation signal power”. I do.

<Effect>
With such a configuration, the direct sound component erroneously estimated as the reverberation component by the reverberation estimation unit can be attenuated by the power of the audio digital signal, and the same effect as in the fifth embodiment can be obtained. Note that the modification of the first embodiment may be combined with the present embodiment, the noise suppression unit 210 of the second embodiment, the audio signal power calculation unit 350 of the fourth embodiment, the silence interval extraction unit 461, and the signal smoothing. The unit 462 and this embodiment may be combined. However, the silent section extraction unit 461 and the signal smoothing unit 462 use the power of the reverberation signal instead of the power of the voice digital signal in which the main speaker external sound is emphasized.

<Seventh embodiment>
A description will be given centering on differences from the first embodiment.
<Points of the seventh embodiment>
In this embodiment, the reverberation component contained in the audio digital signal is estimated, and the main speaker sound is emphasized from the reverberation component. In other words, the main speaker sound section included in the input signal (voice analog signal) to the microphone is extracted from the reverberation component included in the input signal.

  In a mobile environment, reverberation components included in the speech of other speakers recorded with a single microphone may be difficult to estimate because the direct sound is unclear. Using such a property, the main speaker sound section detection unit 760 described later obtains section information.

  In the fifth embodiment, it has been described that the power of the reverberation signal is small in the voice of the main speaker and the power of the reverberation signal is large in the sound outside the main speaker. This is limited to the case where the speaker's voice is clear and the reverberation component can be accurately estimated. When the speech of another speaker included in the speech digital signal is small and the reverberation component is small, the reverberation component cannot be accurately estimated by the conventional reverberation component estimation method (see Reference 1). In such a case, conversely, it has been found that the reverberation component contained in the speech of the other speaker is smaller than the reverberation component contained in the speech of the main speaker. FIG. 15A and FIG. 15B respectively represent the power of the reverberation signal of the main speaker's voice and the main speaker's external sound when the other speaker's voice included in the voice digital signal is small and its reverberation component is small. Represents the image. As can be seen from FIGS. 15A and 15B, the power of the reverberation signal is large in the voice of the main speaker, and the power of the reverberation signal is small in the sound outside the main speaker. In the present embodiment, this discovery can be used to emphasize the main speaker sound with high accuracy. However, the present embodiment is not limited to a mobile environment and a single microphone, but can also be applied to audio analog signals obtained from other environments and a plurality of microphones.

<Audio section detection device 700 according to the seventh embodiment>
FIG. 16 is a functional block diagram of the speech section detection apparatus 700 according to the seventh embodiment, and FIG. 17 shows the processing flow. The speech segment detection device 700 includes a speech signal acquisition unit 110, a reverberation estimation unit 120, a main speaker sound segment detection unit 760, and a main speaker speech extraction unit 170. The speech segment detection device 700 does not need to include the gain adjustment unit 130 and the main speaker external sound enhancement unit 140, and includes a main speaker sound segment detection unit 760 instead of the main speaker external sound segment detection unit 160.

<Main speaker sound section detector 760>
Input: Reverberation signal output: Section information The main speaker sound section detector 760 receives the reverberation signal, detects at least one of the main speaker voice section and the main speaker outside sound section from the power of the reverberation signal (s760), Section information indicating at least one of the main speaker voice section and the main speaker outside sound section is output. For example, the power of the reverberation signal is compared with a threshold value. (1) If it is larger than the threshold, it is determined that it is the main speaker's voice section. The threshold is determined in advance using learning data with correct labels of the main speaker voice section and the main speaker outside sound section.

<Effect>
With such a configuration, in a real environment including reverberation, especially when the speech of another speaker included in the speech digital signal is small and the reverberation component is small, the main speaker and the other speaker to the single microphone The voice of the main speaker can be detected with high accuracy even for a voice digital signal mixed with the voices of a plurality of speakers including. As a result, the number of microphones used can be reduced, and the hardware configuration can be reduced.

<Modification>
This embodiment may be combined with other embodiments.
For example, when combining the second to fourth embodiments and the present embodiment, each of the speech section detection devices 200 to 400 may not include the gain adjustment unit 130 and the main speaker outside sound enhancement unit 140, respectively. Instead of the main speaker sound section detectors 160, 360, and 460, a main speaker sound section detector is included. The main speaker sound section detection unit performs the same processing as the main speaker outer sound section detection units 160, 360, and 460 using a reverberation signal instead of the voice digital signal in which the main speaker external sound is emphasized.

  Further, for example, when the fifth and sixth embodiments are combined with the present embodiment, a main speaker sound section detection unit is included instead of the main speaker outside sound section detection units 560 and 660, respectively. In this case, in the main speaker sound section detection unit, the determination result when the power of the reverberation signal or the power of the normalized reverberation signal is compared with the threshold value is the main speaker outside sound section detection unit 560. And 660 are opposite to the determination result. For example, when the fifth embodiment and the present embodiment are combined, the main speaker sound section detection unit performs processing as follows.

<Main speaker sound section detector>
Input: Reverberation signal power output: Section information The main speaker outer sound section detector receives the power of the reverberation signal and detects at least one of the main speaker voice section or the main speaker outer sound section from the power of the reverberation signal. Then, section information indicating at least one of the main speaker voice section and the main speaker outside sound section is output. For example, the power of the reverberation signal is compared with a threshold value. (1) If it is larger than the threshold, it is determined that it is a “main speaker voice section”, and (2) if it is smaller than the threshold, it is determined that it is a “main speaker outside sound section”. The threshold is determined in advance using learning data with correct labels of the main speaker voice section and the main speaker outside sound section.

  By combining this embodiment with other embodiments in this way, the effects described in each embodiment can be obtained even when the voice of another speaker included in the voice digital signal is small and the reverberation component is small. be able to.

  Moreover, the speech section detection device according to the first to sixth embodiments, and the speech section detection device according to this embodiment (or a speech section detection device combining the second to sixth embodiments and this embodiment), May be switched according to the volume of the voice of the other speaker included in the voice digital signal. When the loudness of the other speaker included in the speech digital signal is large, the loudness of the other speaker included in the speech digital signal is obtained using the speech section detection device according to the first to sixth embodiments. Is small, by using the speech section detection device according to the present embodiment (or a combination of the second to sixth embodiments and the present embodiment), the main speaker's accuracy is high in any situation. Voice can be detected.

  The “speaker-specific section detector” is a concept including the above-mentioned “main speaker outside sound section detector” and “main speaker sound section detector”.

<Eighth embodiment>
FIG. 18 is a diagram for explaining an arrangement of any of the speech section detection devices 100 to 700 and the speech recognition device 800. Any one of the speech section detection devices 100 to 700 is disposed in front of the speech recognition device 800.

  The speech recognition apparatus 800 performs speech recognition on the speech signal using a signal obtained by any of the speech section detection devices 100 to 700 described above using the speech signal as an input. The audio signal is a concept including an audio digital signal and an audio analog signal.

  For example, a voice digital signal corresponding to the voice of the main speaker obtained by any of the voice section detection devices 100 to 700 described above is received and the voice recognition result is output.

  Further, for example, section information is output as an output value of the voice section detection device 100 (see the modification of the first embodiment), and the start time of at least one of the main speaker voice section or the main speaker outside sound section is used as the section information. When the end time or the like is used, speech recognition is performed on the speech signal corresponding to the section information, and the speech recognition result is output.

  In addition, for example, section information is output as an output value of the voice section detection device 100 (see the modification of the first embodiment), and at least a main speaker voice section or a main speaker outside sound section is added to the voice digital signal as section information. When a signal or the like with one flag is used as the section information, voice recognition is performed on the voice digital signal with the flag of the main speaker voice section, and the voice recognition result is output.

  As described above, by using the voice digital signal corresponding to the voice of the main speaker obtained by any of the voice section detection devices 100 to 700 and the section information, the main voice can be obtained from the input voice (voice signal) used for the voice recognition process. It is possible to remove the external sound / silence / noise of the speaker and perform the speech recognition process only on the voice of the main speaker, thereby improving the accuracy. Normally, the main speaker's external sound and noise are recognized as speech without being determined as non-speech, and the speech recognition result is generated as misrecognition. By detecting only the speech section with high accuracy, it is possible to reduce adverse effects on the speech recognition system due to speech or noise that is not the recognition target.

<Other variations>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
In addition, various processing functions in each device described in the above embodiments and modifications may be realized by a computer. In that case, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its storage unit. When executing the process, this computer reads the program stored in its own storage unit and executes the process according to the read program. As another embodiment of this program, a computer may read a program directly from a portable recording medium and execute processing according to the program. Further, each time a program is transferred from the server computer to the computer, processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program includes information provided for processing by the electronic computer and equivalent to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In addition, although each device is configured by executing a predetermined program on a computer, at least a part of these processing contents may be realized by hardware.

100-700 Voice section detection device 110 Voice signal acquisition section 120 Reverberation estimation section 130 Gain adjustment section 140 Main speaker outside sound enhancement section 160, 360, 460, 560, 660 Main speaker outer sound section detection section 760 Main speaker sound Section detection section 170 Main speaker voice extraction section 210 Noise suppression section 350 Voice signal power calculation section 351 Signal smoothing section 461 Silent section extraction section 462 Signal smoothing section 540 Reverberation signal power calculation section 700 Speech recognition device

Claims (10)

A reverberation estimation unit that estimates a reverberation component included in the audio digital signal and obtains the reverberation signal;
Look including the speaker-specific section detection unit for detecting at least one of Shuhanashi's speech section or main speaker outside sound section based on the reverberation signal,
The section detection unit for each speaker detects at least one of the main speaker voice section or the main speaker outside sound section by comparing the acquired reverberation signal with a predetermined value.
Voice segment detection device. A reverberation estimation unit that estimates a reverberation component included in the audio digital signal and obtains the reverberation signal;
A section detection unit for each speaker that detects at least one of a main speaker voice section or a main speaker outside sound section based on the reverberation signal;
A reverberation signal power calculation unit for calculating the power of the reverberation signal;
The section detection unit for each speaker detects the power of the reverberation signal and a predetermined value to detect at least one of a main speaker voice section or a main speaker outside sound section,
Voice segment detection device. It is the audio | voice area detection apparatus of Claim 1, Comprising:
further,
An audio signal power calculation unit for calculating the power of the audio digital signal;
The section-by-speaker detection unit detects at least one of a main speaker voice section and a main speaker outside sound section by comparing a value obtained by normalizing a reverberation signal with the power of the voice digital signal and a predetermined value. ,
Voice segment detection device. The speech section detection device according to claim 2 ,
further,
An audio signal power calculation unit for calculating the power of the audio digital signal;
The section-by-speaker detection unit compares the power of the reverberation signal normalized by the power of the voice digital signal and a predetermined value to compare at least one of the main speaker voice section and the main speaker outside sound section. Detect
Voice segment detection device. It is the audio | voice area detection apparatus of Claim 1 or 3 ,
further,
The speaker-specific section detection unit
A signal smoothing unit that smoothes the sound digital signal, or the power of the reverberation signal, or a signal obtained by normalizing the power of the reverberation signal by the power of the sound digital signal;
Detecting at least one of a main speaker voice section and a main speaker outer sound section from the smoothed signal;
Voice segment detection device. The speech section detection device according to claim 2 or 4 ,
further,
The speaker-specific section detection unit
The power of the difference signal between the voice digital signal or the voice digital signal and the reverberation signal or the power of the difference signal between the voice digital signal and the reverberation signal is normalized by the power of the voice digital signal. Including a signal smoothing unit for smoothing the processed signal,
Detecting at least one of a main speaker voice section and a main speaker outer sound section from the smoothed signal;
Voice segment detection device. The speech section detection device according to any one of claims 2, 4, and 6 ,
further,
Including a gain adjustment unit that multiplies the reverberation signal by gain,
Voice segment detection device. A reverberation estimation step of estimating a reverberation component included in the audio digital signal and obtaining a reverberation signal;
It looks including the speaker-specific section detecting step of detecting at least one of Shuhanashi's speech section or main speaker outside sound section based on the reverberation signal,
The section-by-speaker detection step detects at least one of the main speaker voice section or the main speaker outside sound section by comparing the acquired reverberation signal with a predetermined value.
Voice segment detection method.   A reverberation estimation step of estimating a reverberation component included in the audio digital signal and obtaining a reverberation signal;
  A section-by-speaker detection step for detecting at least one of a main speaker voice section or a main speaker outside sound section based on the reverberation signal;
  A reverberation signal power calculating step of calculating the power of the reverberation signal;
  The section-by-speaker detection step compares the power of the reverberation signal with a predetermined value to detect at least one of a main speaker voice section or a main speaker outside sound section,
  Voice segment detection method. And a speech segment detection equipment according to any one of claims 1 to 7, a program for causing a computer to function.

Images