JP6794064B2 - Model learning device, speech interval detector, their methods and programs - Google Patents

Description

The present invention relates to a voice section detection technique.

One of the voice section detection techniques is a method called VAD (voice activity detection) (see, for example, Non-Patent Document 1 and the like). In VAD, a voice section is performed using sound intensity, vibration intensity (zero crossing number), acoustic features, and the like. However, VAD has a problem that a consonant, a short pause between words, or the like is determined as a non-voice section, and a fragmented voice section is detected. In order to deal with this, a method called hangover is used (see, for example, Non-Patent Document 1 and the like). This is a method of regarding these two voice sections as a continuous voice section when the number of frames of the non-voice section between the two voice sections obtained by VAD is shorter than the threshold value.

ITU, “A silence compression scheme for G.729 optimized for terminals conforming to recommendation V.70,” ITUT / Recommendation G.729-Annex B. 1996.

However, the conventional method using a hangover is less flexible. That is, the length of the non-voice section between the two voice sections obtained by VAD differs depending on the situation such as the speech task and the domain. Therefore, it is necessary to manually set the optimum threshold value for each assumed scene. Also, even in the same situation, ideally, an appropriate threshold value should be given for each utterance. However, conventional methods do not have such flexibility.

The present invention has been made in view of these points, and an object of the present invention is to improve flexibility in voice section detection.

A state transition model that obtains the estimation result of the state transition in each time interval for the audio state and non-audio state of the input acoustic signal by inputting the input audio likelihood series corresponding to the audio likelihood in each time interval of the input acoustic signal. , The input audio likelihood series corresponding to the audio likelihood of each time interval of the input acoustic signal is applied, and the estimation result of the state transition in each time interval for the audio state and non-audio state of the input acoustic signal is obtained and output. To do.

In the present invention, the flexibility in voice section detection can be improved.

FIG. 1A is a block diagram showing a functional configuration of the model learning device of the embodiment. FIG. 1B is a block diagram showing a functional configuration of the voice detection device of the embodiment. FIG. 2 is a state transition diagram of the state transition model of the embodiment. 3A and 3B are diagrams illustrating acoustic signals. FIG. 3C is an example of audio and non-audio sections detected by VAD. FIG. 3D is an example of audio and non-audio sections detected by the method of the embodiment. FIG. 4A is a diagram illustrating an acoustic signal having a plurality of utterance sections. FIG. 4B is an example of a voice section and a non-voice section detected by the method of the embodiment.

Hereinafter, embodiments of the present invention will be described.
[Overview]
First, the outline of each embodiment will be described. In each embodiment, an input voice likelihood sequence corresponding to the voice likelihood of each time interval of the input acoustic signal is applied to the state transition model, and the input acoustic signal has a voice state and a non-voice state in each time interval. Obtain the estimation result of the state transition and output it. The "state transition model" takes an input audio likelihood series corresponding to the audio likelihood of each time interval of the input acoustic signal as an input, and the state transition of the input acoustic signal regarding the audio state and the non-audio state in each time interval. It is a model to obtain the estimation result of. This state transition model models the transition of the actual voice section and the non-speech section (that is, the appearance of the actual voice section and the non-speech section) with respect to the input voice likelihood series. Therefore, by applying the input speech likelihood sequence to this state transition model, even if the input speech likelihood sequence represents a time interval such as a consonant or a short pause, it is not a non-voice interval but a speech interval. Can be properly estimated to be part of. This solves the problem of determining a consonant, a short pause, or the like as a non-voice section and detecting a fragmented voice section. Moreover, since the generation of the state transition model can be applied to various input speech likelihood series, it is more flexible than the hangover. In particular, the state transition model of this embodiment estimates the transition of the voice section and the non-voice section by using the input voice likelihood series as an input. The input voice likelihood series is a collection of input acoustic signals that fluctuate greatly depending on the environment into a series corresponding to voice likelihood with less fluctuation. Therefore, the state transition model of this embodiment flexibly responds to various environments and enables highly accurate estimation. As described above, the method of each embodiment can improve the flexibility in voice section detection as compared with the conventional method. The state transition model can be generated by machine learning using training data, and the tuning cost is lower than the hangover in which the threshold value is set manually.

The "input acoustic signal" is a time-series digital acoustic signal divided into a plurality of predetermined time intervals (for example, a frame, a subframe, etc.). The "speech likelihood" of a time interval represents the likelihood (likelihood) that the time interval is a speech interval. The "speech likelihood" may indicate the likelihood that the time interval is a speech interval as it is, or indirectly the likelihood that the time interval is a speech interval by indicating the likelihood that the time interval is a non-speech interval. It may be represented. The "input voice likelihood series" is a time series of values corresponding to the "voice likelihood" of each time interval. The "input voice likelihood series" may be a time series of "voice likelihood" of each time interval, or may be a time series of values representing "voice likelihood" of each time interval. The value representing "speech likelihood" may be a continuous value or a binary value. For example, the value representing "speech likelihood" may be a function value of "speech likelihood" or a binary value determined by a threshold value using "speech likelihood". However, from the viewpoint of estimation accuracy, it is desirable that the "input voice likelihood series" is a time series of "voice likelihood" of each time interval or a time series of values representing "voice likelihood" which is a continuous value. .. The "state transition" in each time interval for the voice state and the non-voice state is, for example, the state transition from the voice state to the voice state, the state transition from the non-voice state to the non-voice state, and the state transition from the voice state to the non-voice state. It is either a state transition of or a state transition from a non-voice state to a voice state. However, the state transition from the voice state to the voice state means that the voice state is sustained, that is, the "voice state". Similarly, a state transition from a non-voice state to a non-voice state means that the non-voice state is sustained, that is, a "non-voice state". The estimation result of the state transition obtained by applying the input voice likelihood series to the state transition model may be the likelihood of each state transition or a function value of the likelihood of each state transition. , It may be a value obtained by comparing the likelihoods of each state transition (for example, the state transition having the highest likelihood). The learning of the state transition model is a sequence of the voice likelihood series corresponding to the voice likelihood of each time interval of the acoustic signal and the sequence of the correct answer values of the state transitions in each time interval for the voice state and the non-voice state of the sound signal. It is performed using the training data including the set of and. In this learning data, the elements of each time interval of the speech likelihood series (for example, speech likelihood) and the correct answer value of the state transition in each time interval are associated with each other. For example, the element of each time interval of the voice likelihood series and the correct answer value of the state transition in each time interval are associated with the identifier representing the time interval. Usually, the training data includes a set of a voice likelihood series for a plurality of acoustic signals and a series of correct answer values of state transitions. That is, the learning data includes a plurality of sets of a speech likelihood series and a series of correct answer values of state transitions. However, the learning data may include only a set of a voice likelihood series for one acoustic signal and a series of correct answer values of state transitions. The "acoustic signal" is a time-series digital acoustic signal divided into a plurality of predetermined time intervals (for example, frames, subframes, etc.). The voice likelihood series of the training data does not have to be the same as the assumed input voice likelihood series, but must be of the same type. For example, if the assumed input voice likelihood series is a time series of voice likelihood in each time interval, the voice likelihood series of the training data also needs to be a time series of voice likelihood in each time interval.

[First Embodiment]
The first embodiment will be described.
<Model learning device 11>
First, the model learning device 11 of this embodiment will be described. As illustrated in FIG. 1A, the model learning device 11 of the present embodiment has a storage unit 111 for storing the learning data 111a and a learning unit 112 for learning the state transition model 123a.

The learning data 111a includes a voice likelihood sequence corresponding to the voice likelihood of each time interval of the acoustic signal, a sequence of correct answer values of state transitions in each time interval for the voice state and non-voice state of the sound signal, and Includes a set of. The learning unit 112 uses the learning data 111a read from the storage unit 111, and uses the input audio likelihood series corresponding to the audio likelihood of each time interval of the input acoustic signal as an input for the audio state and non-audio state of the input acoustic signal. The state transition model 123a for obtaining the estimation result of the state transition in each time interval of is obtained and output. The state transitions of this embodiment are state transitions from a voice state to a voice state, a state transition from a non-voice state to a non-voice state, a state transition from a voice state to a non-voice state, and a state from a non-voice state to a voice state. It is one of the transitions.

The voice likelihood series for each time interval included in the training data 111a can be obtained by extracting acoustic information and analysis information from the acoustic signal for the training data and applying a known VAD to them (for example, Non-Patent Document 1). Etc.). Examples of acoustic information and analysis information used for VAD include changes in sound power, the number of zero crossings of a waveform per fixed time period, changes in the characteristics of acoustic features, and combinations thereof. The acoustic signal for the training data is a time-series digital acoustic signal divided for each predetermined time interval. The acoustic signal for the training data may be an analog acoustic signal observed by a microphone or the like, AD-converted at a predetermined sampling frequency, or an arbitrary digital acoustic signal created in advance. An example of the voice likelihood series is as described above, and is a time series of values obtained for each time interval of the acoustic signal for learning data. When the voice likelihood series is a series of continuous values representing the voice likelihood or the voice likelihood of each time interval, the voice likelihood series is, for example, 0.1, 0.5, ..., 0.8. When the voice likelihood series is a series of binary values representing the voice likelihood of each time interval, the voice likelihood series is, for example, 0,0,0,1 ..., 1,0.

The series of correct answer values of the state transitions included in the training data 111a is included in each time interval of the voice likelihood series (hereinafter referred to as "sound likelihood series for training data") obtained from the above-mentioned acoustic signal for training data. It is obtained by giving the correct answer value of the state transition for the voice state and the non-voice state. The correct answer value of this state transition will be described in detail. For example, suppose you hear "Today is a nice day". At this time, even if the main part (today) and the predicate (good weather) are read out in succession as shown in FIG. 3A, the main part and the predicate are separated from each other as shown in FIG. 3B. Even if it is read aloud, we aim to estimate the entire "Today is good weather" as one true voice section from the input voice likelihood series corresponding to the input acoustic signal. That is, when the main part and the predicate are read aloud at intervals as shown in FIG. 3B, the corresponding input voice likelihood series has a short non-voice interval between "today" and "good weather". It becomes a series that represents that. For example, if the input voice likelihood series is a binary series representing a voice interval and a non-voice interval, this input voice likelihood series is the time interval of the non-voice interval between "today" and "good weather". (Fig. 3C). In this embodiment, the true voice section that absorbs the vibration of the voice likelihood due to the short non-voice section sandwiched between the voice sections is appropriately estimated. On the contrary, even if sudden noise or the like is mixed between the non-voice sections, the true non-voice section that absorbs the vibration of the voice likelihood due to the noise is appropriately estimated. The "true voice section" means a time section from the first observation of voice in one utterance section to the time when the last voice is no longer observed. "True non-speech interval" means a time interval other than the true audio interval. The "utterance section" means a time section in which a cohesive utterance is made, such as "Today is a good weather". Further, "start of utterance" means the start of "speech section", and "end of utterance" means the end of "speech section".

In order to perform such estimation, the state transition model 123a for obtaining the estimation result of the state transition in each time interval for the voice state and the non-voice state of the input acoustic signal is learned by using the input voice likelihood series as an input. In this embodiment, in order to emphasize the state transition from the non-voice state to the voice state (starting transition state) and the state transition from the voice state to the non-voice state (end transition state), only the voice state and the non-voice state are used. Instead, a special label is assigned to each of the start transition state and the end transition state. For example, the following labels are assigned to each state transition.
Label "0": State transition from non-voice state to non-voice state (non-voice state)
Label "1": State transition from voice state to voice state (voice state)
Label "2": State transition from non-voice state to voice state (starting transition state)
Label "3": State transition from voice state to non-voice state (termination transition state)
The voice state is a state corresponding to a true voice section, and the non-voice state is a state corresponding to a true non-voice section.

FIG. 2 shows a state transition diagram of the state transition model 123a when such a label is given. The state transition model 123a takes the input voice likelihood series as an input, loops the non-voice state and the voice state, and has labels corresponding to the non-voice state, the voice state, the start transition state, and the end transition state, respectively. Outputs the likelihood of. In this case, the training data pattern has a time interval corresponding to the speech likelihood series for the training data.
(1) Whether to include the true audio section
(2) Whether to include the true audio section and the true non-audio section immediately before it
(3) Whether to include the true audio section and the true non-audio section immediately after it
Can be subdivided by.

Pattern 1: When the time interval corresponding to the speech likelihood sequence for training data includes the true speech interval and the true non-speech interval immediately before and after it, this is the most common pattern. In this case, the following labels are given to each time interval corresponding to the speech likelihood series for training data.
-Time interval from the start of utterance to the end of the non-voice state: non-voice state "0"
-Time interval for transition from the end of the non-voice state to the voice state: Start transition state "2"
-Time interval from the beginning to the end of the voice state: Voice section "1"
-Time interval for transition from the end of the voice state to the non-voice state: End transition state "3"
-Time interval from the beginning (end of utterance) to the end (start of the next utterance) of the non-voice state: Non-voice state "0"
That is, the series of correct answer values of the state transitions in each time interval corresponding to the voice likelihood series is 0, ..., 0,2,1, ..., 1,3,0, ..., 0. For example, even if the main part and the predicate are read out in succession as shown in FIG. 3A, or even if the main part and the predicate are read out at intervals as shown in FIG. 3B, each The series of correct answer values (label series) of the state transitions in the time interval are 0, ..., 0,2,1, ..., 1,3,0, ..., 0, as illustrated in FIG. 3D. Normally, two or more time intervals in the start transition state do not continue, and two or more time intervals in the end transition state do not continue. However, two or more of these time intervals may be continuous.

Pattern 2: When the time interval corresponding to the speech likelihood series for training data includes the true speech section and the true non-speech section immediately after it, but does not include the non-speech section immediately before the speech section. The time interval of is regarded as the time interval of the start transition state. That is, the following labels are given to each time interval corresponding to the speech likelihood series for training data.
・ First time interval: Start transition state “2”
-Time interval from the next time interval to the end of the voice state: Voice section "1"
-Time interval for transition from the end of the voice state to the non-voice state: End transition state "3"
-Time interval from the beginning (end of utterance) to the end (start of the next utterance) of the non-voice state: Non-voice state "0"
That is, the series of correct answer values of the state transitions in each time interval corresponding to the voice likelihood series are 2,1, ..., 1,3,0, ..., 0.

Pattern 3: When the time interval corresponding to the speech likelihood series for training data includes the true speech section and the true non-speech section immediately before it, but does not include the non-speech section immediately after the speech section. In this case, the final The time interval of is regarded as the time interval of the terminal transition state. That is, the following labels are given to each time interval corresponding to the speech likelihood series for training data.
-Time interval from the start of utterance to the end of the non-voice state: non-voice state "0"
-Time interval for transition from the end of the non-voice state to the voice state: Start transition state "2"
-Time interval from the beginning of the voice state to just before the last time section: Voice section "1"
-Final time interval: Termination transition state "3"
That is, the sequence of correct answer values of the state transition in each time interval corresponding to the speech likelihood sequence is 0, ..., 0, 2, 1, ..., 1, 3.

Pattern 4: When the time interval corresponding to the speech likelihood sequence for training data includes the true speech interval but does not include the non-speech interval before and after it All the time intervals corresponding to the speech likelihood sequence for training data This is the case for the true audio section. In this case, the following labels are given to each time interval corresponding to the voice likelihood series.
-Time interval from the beginning to the end: Audio interval "1"
That is, the series of correct answer values of the state transitions in each time interval corresponding to the voice likelihood series are 1, ..., 1.

Pattern 5: When the time interval corresponding to the voice likelihood series does not include the true voice interval In this case, the following label is given to each time interval corresponding to the voice likelihood series.
-Time interval from the beginning to the end: Audio interval "0"
That is, the series of correct answer values of the state transition in each time interval corresponding to the voice likelihood series is 0, ..., 0.

The learning data 111a may include a series of correct answer values of all the state transitions of the above patterns 1 to 5, or may include only a series of correct answer values of some of the state transitions. .. In addition, the time interval corresponding to the speech likelihood series may include a plurality of true speech intervals. The learning data 111a in this case is composed of a combination of a series of correct answer values of the state transitions of the above patterns 1 to 5. For example, as illustrated in FIGS. 4A and 4B, when a true non-voice section is included before and after the first true voice section I and there is no non-voice section after the second voice section II. The training data 111a is a sequence of correct answer values of the speech likelihood sequence of the first true speech interval I and the corresponding state transition of the pattern 1, 0, ..., 0, 2, 1, ..., 1, 3, 0. , ..., 0, and the speech likelihood sequence of the second true speech interval I and the corresponding sequence of correct values of the state transition of pattern 1 0, ..., 0, 2, 1, ..., Includes pairs with 1 and 3.

The above-mentioned labeling is given not only whether each time section of the acoustic signal for training data is a voice section or a non-voice section, but also a voice section and a non-voice section as the utterance section of the sound signal for training data. It is necessary to perform based on (voice section and non-voice section associated with each utterance section). Various methods can be considered for assigning such a label. In the first method, a human being listens to each utterance included in an acoustic signal for learning data, observes a waveform, and assigns an accurate label. The second method is a method of automatically assigning the above label by using a known recognition decoder. However, in the second method, some training data may be given an erroneous label. Even in such a case, there is no big problem as long as the frequency of correctly labeling the training data as a whole is high. The third method is a compromise in which the above-mentioned decoder first labels the label and then manually checks for errors.

The learning unit 112 learns the state transition model 123a represented by the state transition diagram of FIG. 2 using the learning data 111a generated as described above. This state transition model 123a is a known method (for example, Reference 1: Hochreiter, S., & Schmidhuber, J., “Long Short-Term Memory,” Neural Computation, 9 (8), 1735-1780, 1997). Can be expressed and learned using. The method of Reference 1 can hold a very long series of information, and can construct a state transition model 123a in consideration of a state at a position farther in time and voice likelihood, which is very very. It is useful. In addition to the method of Reference 1, it is also possible to use a model (RNN, HMM, etc.) that can handle time series. However, in the case of HMM, the future state cannot be considered due to its structure.

<Voice section detection device 12>
Next, the voice section detection device 12 of this embodiment will be described. As illustrated in FIG. 1B, the voice section detection device 12 of the present embodiment includes an input unit 121, a voice section detection unit 122, a storage unit 123, an estimation unit 124, and an output unit 126.

<Memory unit 123>
The state transition model 123a output from the model learning device 11 is stored in the storage unit 123 as described above. The state transition model 123a may be stored in the storage unit 123 before the voice section detection by the voice section detection device 12 is started, or each time a new state transition model 123a is output from the model learning device 11. May be stored in the storage unit 123.

<Input unit 121>
The input acoustic signal of the voice section detection target is input to the input unit 121. The input acoustic signal to be detected in the audio section is a time-series digital acoustic signal divided for each predetermined time interval. The input acoustic signal may be an analog acoustic signal observed by a microphone or the like and AD-converted at a predetermined sampling frequency, or may be an arbitrary digital acoustic signal created in advance. The length of the time interval of the input acoustic signal is preferably the same as or close to the length of the time interval of the acoustic signal for the training data described above. The input acoustic signal is sent to the voice section detection unit 122.

<Voice section detection unit 122>
The audio section detection unit 122 extracts acoustic information and analysis information from the input acoustic signal, and applies a known VAD to them (see, for example, Non-Patent Document 1 and the like) to apply each time interval of the input acoustic signal. The input audio likelihood series corresponding to the audio likelihood of is obtained and output. Examples of acoustic information and analysis information used for VAD are as described above. The example of the input voice likelihood series is also as described above, but the type of the input voice likelihood series is the same as the type of the voice likelihood series for learning data. For example, when the voice likelihood sequence for training data is a series of voice likelihood in each time interval, the input voice likelihood series is also a series of voice likelihood in each time interval. The input voice likelihood series is sent to the estimation unit 124.

<Estimating unit 124>
The estimation unit 124 applies the sent input voice likelihood series to the state transition model 123a read from the storage unit 123, and estimates the state transitions of the voice state and non-voice state of the input acoustic signal in each time interval. Obtain the result and output it (post-filter processing). Examples of state transition estimation results are the likelihood of a non-voice state (label "0"), the likelihood of a voice state (label "1"), and the likelihood of a start transition state (label "2") in each time interval. It is a series of four likelihoods consisting of degrees and likelihoods of the terminal transition state (label “3”). Alternatively, only the series of likelihoods of the non-voice state (label "0") in each time interval may be used as the estimation result of the state transition, or only the series of likelihoods of the voice states (label "1") in each time interval. May be the estimation result of the state transition. Alternatively, a sequence of the likelihood of the non-voice state (label “0”) and the likelihood of the voice state (label “1”) in each time interval may be used as the estimation result of the state transition. In addition, a series of values representing the state having the highest likelihood in each time interval may be used as the estimation result of the state transition.

<Output unit 126>
The state transition estimation result output from the estimation unit 124 is sent to the output unit 126. The output unit 126 outputs the estimation result of the state transition as the voice interval estimation result.

[Second Embodiment]
The second embodiment is a modification of the first embodiment. In the present embodiment, the voice interval estimation result is obtained by performing post-processing on the state transition estimation result output from the estimation unit 124. In the following, the differences from the first embodiment will be mainly described, and the same reference numbers will be cited for the matters already described to simplify the explanation.

<Model learning device 11>
It is the same as the first embodiment.

<Voice section detection device 22>
As illustrated in FIG. 1B, the voice section detection device 22 of the present embodiment includes an input unit 121, a voice section detection unit 122, a storage unit 123, an estimation unit 124, a post-processing unit 225, and an output unit 226. The details of the post-processing unit 225 and the output unit 226, which are the differences from the first embodiment, will be described below.

<Post-processing unit 225>
Unlike the first embodiment, the state transition estimation result output from the estimation unit 124 is sent to the post-processing unit 225. The post-processing unit 225 performs predetermined post-processing on the sent state transition estimation result, obtains the voice interval estimation result, and outputs the result. For example, as the estimation result of the state transition, the likelihood of the non-voice state (label “0”), the likelihood of the voice state (label “1”), and the likelihood of the start transition state (label “2”) in each time interval. , And when a series of likelihoods of the terminal transition state (label “3”) is sent, the post-processing unit 225 uses these four series of likelihoods and is a representative state (non-voice state, of each time interval). A voice state, a start transition state, or a end transition state) may be selected, and a series of representative states of each time interval obtained thereby may be output as a voice interval estimation result. For example, the post-processing unit 225 may select and output the state having the highest likelihood among the four sent states for each time interval. That is, the post-processing unit 225 may convert the sent likelihood series of the four states into the maximum likelihood state series and output it. For example, n = 0, ..., N-1 is an identifier corresponding to a time interval, N is a positive integer, the likelihood of a non-voice state in the time interval n is P _{n, 0,} and the likelihood of a voice state is _Let P _{n, 1} , the likelihood of the start transition state be P _{n, 2} , and the likelihood of the end transition state be P _{n, 3} . In this case, the post-processing unit 225 sets the maximum likelihood states (P _{n, 0} , P _{n, 1} , P _{n, 2} , P _{n, 3} ) of the four states sent for each time interval n. it may be output by converting the sequence s _n.

ただし、α_ｉは尤度Ｐ_ｎ，ｉに与えられる重みである。例えば、α_ｉは０よりも大きな正値である。あるいは、特定の尤度Ｐ_ｎ，ｉに与えられる重みを０にしてもよい。例えば、α_２＝α_３＝０とすれば、始端遷移状態や終端遷移状態が音声区間推定結果として選択されることを避けることができる。 Alternatively, the post-processing unit 225 emphasizes or weakens the likelihood of a specific state of the likelihood series of four states (P _{n, 0} , P _{n, 1} , P _{n, 2} , P _{n, 3} ). the likelihood sequence obtained by may be converted and output to the maximum likelihood state sequence s _n. For example, the post-processing unit 225, the likelihood-series _{(P n, 0, P n} , 1, P n, 2, P n, 3) may be converted and output to the maximum likelihood state sequence _{s n} below ..

However, α _i is a weight given to the likelihood P _{n, i} . For example, α _i is a positive value greater than 0. Alternatively, the weight given to a specific likelihood P _{n, i} may be set to 0. For example, if α ₂ = α ₃ = 0, it is possible to avoid selecting the start end transition state and the end transition state as the voice interval estimation result.

In addition, even if the post-processing unit 225 applies a known correction method generally performed in VAD to the sent state transition estimation result, and outputs the result obtained by the correction method as a voice interval estimation result. Good. For example, when only the series of likelihood of the voice state (label "1") of each time interval is sent as the estimation result of the state transition, the post-processing unit 225 sets the likelihood of the voice state of each time interval and a predetermined threshold. May be compared and the voice interval estimation result corresponding to the series of the comparison results may be output. For example, the post-processing unit 225 may output a voice section estimation result in which a time interval in which the likelihood of the voice state is equal to or greater than a threshold value is a voice section and the other time section is a non-voice section. Even when only the series of likelihood of the non-voice state (label “0”) of each time interval is sent as the estimation result of the state transition, the post-processing unit 225 determines the likelihood of the non-voice state of each time interval and a predetermined value. The voice interval estimation result corresponding to the series of comparison results may be output by comparing with the threshold value. In addition, the post-processing unit 225 is known to be generally performed in VAD for the likelihood series obtained by emphasizing or weakening the likelihood of a specific state in the estimation result of the sent state transition. A correction method may be applied and the result obtained thereby may be output as a voice interval estimation result. The post-processing unit 225 may use other known techniques for improving the accuracy of voice section detection.

<Output unit 226>
The audio interval estimation result output from the estimation unit 224 is sent to the output unit 226. The output unit 226 outputs the voice section estimation result.

[Third Embodiment]
The third embodiment is a modification of the first and second embodiments. In this embodiment, the model learning device generates a plurality of state transition models, the voice interval detection device applies the input voice likelihood series to the plurality of state transition models, and the input sound is applied to each of the plurality of state transition models. The estimation result of the state transition in each time interval for the voice state and the non-voice state of the signal is obtained, and the most probable estimation result in each time interval is selected from the obtained state transition estimation results. In the following, the differences from the first and second embodiments will be mainly described, and the same reference numbers will be cited for the matters already described to simplify the explanation.

<Model learning device 31>
As illustrated in FIG. 1A, the model learning device 31 of the present embodiment has a storage unit 311 for storing learning data 311a and a learning unit 312 for learning the state transition model 323a.

The learning data 311a of the present embodiment includes a voice likelihood series corresponding to the voice likelihood of each time interval of the acoustic signals for a plurality of types of training data, and each time interval regarding the voice state and the non-voice state of the sound signal. Includes a series of correct values for state transitions and multiple pairs of. These sets are classified according to the type of acoustic signal for training data. For example, the learning data 311a includes a plurality of sets of a voice likelihood series corresponding to acoustic signals for learning data obtained in a plurality of types of environments and a series of correct answer values of state transitions, and these sets correspond to each other. It is classified according to the environment. Let this class be c = 0, ..., C-1 (where C is an integer of 2 or more), and a set of the voice likelihood series belonging to the class c included in the training data 311a and the series of correct answer values of the state transitions. The set of is expressed as D (c). That is, the training data 311a includes the sets D (0), ..., D (C-1).

The learning unit 312 uses the learning data 311a read from the storage unit 311, and inputs the input voice likelihood series corresponding to the voice likelihood of each time interval of the input acoustic signal as input, and describes the audio state and the non-audio state of the input acoustic signal. A plurality of state transition models 323a for obtaining the estimation results of the state transitions in each time interval are obtained and output. That is, the learning unit 312 uses the set D (c) included in the learning data 311a to generate a state transition model M (c) corresponding to the class c by the same method as in the first embodiment, and a plurality of state transitions. The models M (0), ..., M (C-1) are output as the state transition model 323a. Others are the same as in the first embodiment.

<Voice section detection device 32>
As illustrated in FIG. 1B, the voice section detection device 32 of this embodiment has an input unit 121, a voice section detection unit 122, a storage unit 323, an estimation unit 324, a post-processing unit 325, and an output unit 226. The details of the storage unit 323, the estimation unit 324, and the post-processing unit 325, which are the differences from the first and second embodiments, will be described below.

<Memory unit 323>
A plurality of state transition models 323a output from the learning unit 312 are stored in the storage unit 323. The state transition model 323a may be stored in the storage unit 323 before the voice section detection by the voice section detection device 32 is started, or each time a new state transition model 323a is output from the model learning device 31. It may be stored in the storage unit 323.

<Estimator 324>
The input voice likelihood series output from the voice section detection unit 122 is input to the estimation unit 324. The estimation unit 324 applies this input voice likelihood series to the plurality of state transition models 323a read from the storage unit 323, and for each of the plurality of state transition models 323a, regarding the voice state and the non-voice state of the input acoustic signal. The estimation result of the state transition in each time interval is obtained and output. That is, the estimation unit 324 applies the input speech likelihood series to each state transition model M (c) (where c = 0, ..., C-1), and for each class c, the state transition in each time interval. The estimation result R (c) is obtained and output. The estimation result R (c) for each class c is sent to the post-processing unit 325. Others are the same as those of the first and second embodiments.

<Post-processing unit 325>
The post-processing unit 325 selects the most probable result in each time interval from the results corresponding to the estimation result R (c) sent for each class c output from the estimation unit 324. A known technique for improving the accuracy of voice section voice detection can be used for this selection. For example, when the voice section detection unit 122 performs VAD on input acoustic signals obtained in a plurality of environments in which they are very different, the post-processing unit 325 uses reference 2 (RA Jacobs, MI Jordan, SJ Nowlan, and GE). The most probable estimation results may be selected using the method described in Hinton. “Adaptive interpolation of local experts.” Neural Computation, 3: 79-87, 1991.). That is, the post-processing unit 325 has the estimation result R (c') corresponding to the environment in which the input acoustic signal is obtained among the results corresponding to the sent estimation results R (0), ..., R (C-1). ) Corresponds to the most probable result. The result corresponding to the estimation result R (c) may be the estimation result R (c) itself, or the estimation result R (c) is processed by the post-processing unit 225 of the second embodiment. It may be obtained by. The post-processing unit 325 may output the selected most probable result as a voice interval estimation result, or the voice obtained by performing the processing of the post-processing unit 225 of the second embodiment on the selected most probable result. The interval estimation result may be output. The voice section estimation result is sent to the output unit 226. Subsequent processing is the same as in the second embodiment.

[Characteristics of the method of the embodiment]
As described above, in each embodiment, the accuracy of VAD can be improved without manual tuning in various environments, and both cost reduction and accuracy improvement can be realized. In addition, even in a single environment, it is possible to absorb speech blur and perform voice section detection with higher accuracy than existing methods. Furthermore, since the configuration of the estimation unit using the state transition model is a post-filter that is connected longitudinally to the VAD, it can be used in combination with other VAD accuracy improvement methods. For example, further performance improvement can be expected by performing this post-filter process after applying the conventional hangover method.

Conventionally, there has been a VAD using a state transition model that outputs a voice state / non-voice state by inputting an acoustic feature amount. However, the acoustic features fluctuate greatly depending on the environment, the model becomes enormous, and a large amount of learning data is required. On the other hand, in the state transition model of each embodiment, the estimation result of the state transition is obtained by using the voice likelihood series after the VAD processing as the input feature amount. The voice likelihood series has less variation due to the environment than the acoustic features. Therefore, the state transition model of each embodiment can be created with a small amount of training data, and also operates robustly as a post-filter of various VAD techniques. Further, in each embodiment, special labels are given to the start end (start end transition state) and end (end transition state) of the voice section. As a result, the information as a "section" can be learned more strongly, and it is less affected by the appearance of a temporary voice state frame due to sudden noise and the appearance of a temporary non-voice state due to consonants, breathing, etc. ..

[Other variants]
The present invention is not limited to the above-described embodiment. For example, in the first, second, and third embodiments, the input acoustic signal is input to the audio section detection devices 12, 22, and 32, and the audio section detection unit 122 corresponds to the audio likelihood of each time interval of the input acoustic signal. An input audio likelihood series was obtained. However, an input voice likelihood series may be generated from the input acoustic signal outside the voice section detection devices 12, 22, and 32, and the input voice likelihood series may be input to the voice section detection devices 12, 22, 32. In this case, the voice section detection unit 122 may be omitted from the voice section detection devices 12, 22, and 32.

The various processes described above may not only be executed in chronological order according to the description, but may also be executed in parallel or individually as required by the processing capacity of the device that executes the processes. In addition, it goes without saying that changes can be made as appropriate without departing from the gist of the present invention.

Each of the above devices is, for example, a general-purpose or dedicated computer including a processor (hardware processor) such as a CPU (central processing unit) and a memory such as a RAM (random-access memory) and a ROM (read-only memory). Is composed of executing a predetermined program. This computer may have one processor and memory, or may have a plurality of processors and memory. This program may be installed in a computer or may be recorded in a ROM or the like in advance. Further, a part or all of the processing units are configured by using an electronic circuit that realizes a processing function without using a program, instead of an electronic circuit (circuitry) that realizes a function configuration by reading a program like a CPU. You may. The electronic circuits constituting one device may include a plurality of CPUs.

When the above configuration is realized by a computer, the processing contents of the functions that each device should have are described by a program. By executing this program on a computer, the above processing function is realized on the computer. The program describing the processing content can be recorded on a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such recording media are magnetic recording devices, optical disks, opto-magnetic recording media, semiconductor memories, and the like.

The distribution of this program is carried out, for example, by selling, transferring, renting, or the like a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Further, the program may be stored in the storage device of the server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads the program stored in its own storage device and executes the process according to the read program. Another form of execution of this program may be for the computer to read the program directly from a portable recording medium and perform processing according to the program, and each time the program is transferred from the server computer to this computer. , Sequentially, the processing according to the received program may be executed. Even if the above processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition without transferring the program from the server computer to this computer. Good.

The processing functions of the present device may not be realized by executing a predetermined program on a computer, but at least a part of these processing functions may be realized by hardware.

The present invention can be used, for example, for detecting a voice section in a stage prior to voice recognition processing or voice dialogue processing. When the present invention is applied to these, the detection accuracy of the voice section can be improved, and the voice recognition and the voice dialogue in the subsequent stage can be performed with higher accuracy.

11,31 Model learning device 12,22,32 Voice section detection device

Claims (8)

Learning including a set of a voice likelihood series corresponding to the voice likelihood of each time interval of the acoustic signal and a series of correct answer values of state transitions in each time interval for the voice state and non-voice state of the acoustic signal. Using the data, the input audio likelihood series corresponding to the audio likelihood of each time interval of the input acoustic signal is input, and the estimation result of the state transition in each time interval for the audio state and non-audio state of the input acoustic signal is obtained. A model learning device that learns the state transition model to be obtained. The model learning device of claim 1.
The state transitions are a state transition from a voice state to a voice state, a state transition from a non-voice state to a non-voice state, a state transition from a voice state to a non-voice state, and a state transition from a non-voice state to a voice state. A model learning device that is either. A state transition model that obtains the estimation result of the state transition in each time interval for the audio state and non-audio state of the input acoustic signal by inputting the input audio likelihood series corresponding to the audio likelihood in each time interval of the input acoustic signal. A storage unit that memorizes
The input voice likelihood series corresponding to the voice likelihood of each time interval of the input acoustic signal is applied to the state transition model, and the state transition of the input acoustic signal regarding the voice state and the non-voice state in each time interval is applied. An estimation unit that obtains and outputs estimation results,
A voice section detection device having. A plurality of states for obtaining the estimation result of the state transition in each time interval for the audio state and the non-audio state of the input acoustic signal by inputting the input audio likelihood series corresponding to the audio likelihood in each time interval of the input acoustic signal. A storage unit that stores the transition model and
An input voice likelihood series corresponding to the voice likelihood of each time interval of the input acoustic signal is applied to the plurality of state transition models, and for each of the plurality of state transition models, the voice state and non-voice state of the input acoustic signal. An estimation unit that obtains and outputs the estimation result of the state transition in each time interval for the audio state,
Among the results corresponding to the state transition estimation results output from the estimation unit, the post-processing unit that selects the most probable result in each time interval, and the post-processing unit.
A voice section detection device having. Learning including a set of a voice likelihood series corresponding to the voice likelihood of each time interval of the acoustic signal and a series of correct answer values of state transitions in each time interval for the voice state and non-voice state of the acoustic signal. Using the data, the input audio likelihood series corresponding to the audio likelihood of each time interval of the input acoustic signal is input, and the estimation result of the state transition in each time interval for the audio state and non-audio state of the input acoustic signal is obtained. A model learning method of a model learning device that learns the state transition model to be obtained. A state transition model that obtains the estimation result of the state transition in each time interval for the audio state and non-audio state of the input acoustic signal by inputting the input audio likelihood series corresponding to the audio likelihood in each time interval of the input acoustic signal. The input audio likelihood series corresponding to the audio likelihood of each time interval of the input acoustic signal is applied to, and the estimation result of the state transition in each time interval for the audio state and non-audio state of the input acoustic signal is obtained. A method for detecting a voice section of a voice section detection device that obtains and outputs a sound section . A plurality of states for obtaining the estimation result of the state transition in each time interval for the audio state and the non-audio state of the input acoustic signal by inputting the input audio likelihood series corresponding to the audio likelihood in each time interval of the input acoustic signal. An input audio likelihood series corresponding to the audio likelihood of each time interval of the input acoustic signal is applied to the transition model, and the audio state and non-audio state of the input acoustic signal are obtained for each of the plurality of state transition models. A method for detecting a voice section of a voice section detection device, which obtains an estimation result of a state transition in each time section of the above and selects the most probable result in each time section from the results corresponding to the estimation result of the state transition. A program for operating a computer as the model learning device of claim 1 or 2 or the voice section detection device of claim 3 or 4.

Images