JP4748605B2 - Voice recognition method and apparatus, voice recognition program and recording medium therefor - Google Patents

Description

  The present invention relates to a speech recognition method and apparatus, a speech recognition program, and a recording medium thereof, and more particularly, a speech recognition method and apparatus for performing speech recognition using a state transition probability model typified by an HMM, a speech recognition program, and a recording thereof. It relates to the medium.

  In speech recognition, a word string closest to an input speech signal is determined based on a similarity (probability) with a word expressed by connecting acoustic models generated in advance. HMM (Hidden Marcov Model: Hidden Marcov Model) is one of acoustic models modeled based on probability and statistical theory, and is defined by a state transition probability and an output probability density function in each state. Hereinafter, a conventional speech recognition method will be described using an example in which the HMM is used as an acoustic model.

  In the speech recognition apparatus, recognition processing proceeds in accordance with a grammar in which a set of recognizable sentences is described as a word sequence and a word dictionary in which readings of words constituting the sentence (phoneme strings) are described. FIG. 6 is a diagram showing an example of a grammar, and here, a case where four voices “Ito Ito”, “Itoi Itoi”, “Imai Ito” and “Ito Doi” are identified will be described as an example. .

  The grammar shown in FIG. 6 is a state transition diagram in which the state “1” indicated by the circled number 1 is the beginning (start of sentence) and the state “5” is the end (end of sentence), and the words associated with the arrows Is output and the state transitions. Each word constituting the grammar is expressed as an HMM state sequence according to its reading (phoneme string), and a set of words included in the word dictionary is expanded as a tree structure dictionary as shown in FIG.

  In the tree structure dictionary, each word is decomposed into phoneme strings, and if it is the word “Itoi”, it is expanded into four phoneme “i”, “t”, “0”, and “i” strings. Each phoneme is usually composed of about three states (HMM states). The tree structure dictionary is a state transition diagram in which a branch expands toward the right by merging partial state sequences that are common from the beginning among words expressed as HMM state sequences. In the tree structure dictionary of FIG. 7, the HMM state series corresponding to “I” at the beginning of the word is merged with the three words “Ito”, “Itoi”, and “Imai”, and “Ito” and “Itoi” are further merged. The HMM state series corresponding to “Ito” is merged. In addition, the HMM state sequence corresponding to “d” at the beginning of the word is merged between “Doi” and “Is”. “Sil” in the figure represents a silent period (silence).

  In the speech recognition processing, a token called a state hypothesis transitions from the left to the right in the tree structure dictionary from the HMM state at the beginning of the word in the tree structure dictionary shown in FIG. 7 in accordance with the grammatical restrictions shown in FIG. When the state hypothesis reaches the HMM state at the end of the word, a history called a word hypothesis is left and the state transitions to the transition destination state of the corresponding word in the grammar of FIG. If the transition destination state is not the end (end of sentence), the tree structure dictionary is searched from the next time according to the grammatical constraints.

  While the state hypothesis transitions from the left to the right in the HMM state sequence in the tree structure dictionary, a word-likeness score (log likelihood) is calculated for the input speech. Each HMM state constituting the tree structure dictionary has a probability distribution (output probability density function) that outputs likelihood with respect to the input of acoustic feature parameters. Also, transition probabilities (state transition probabilities) are defined for transitions between HMM states. By accumulating these probabilities in the time direction, a wordiness score (log likelihood) is calculated.

  Each state hypothesis that has transitioned to each HMM state with a period of a feature vector frame obtained by analyzing the speech signal at regular time intervals (for example, 10 ms) is further transferred to its own HMM state (self-transition) and right The transition to the next HMM state (LR transition) is repeated simultaneously. At this time, if the log likelihood in which the state j exists in the t-th frame is αj (t), the log likelihood αj (t) is expressed by the following equation (1). Here, αij is a transition probability from the state i to the state j, and bj (ot) is a probability that the state j outputs the acoustic feature quantity ot.

  When searching for a word sequence consisting of N HMM states for a speech signal composed of T frames, that is, when the state hypothesis transits an HMM state sequence, the space between the self transition and the LR transition ( The trellis is shown in FIG. The trellis space is a grid graph showing possible state sequences with the horizontal axis representing the frame and the vertical axis representing the state, and each state sequence is a line that connects points (circles) representing the state at each time with line segments. It is represented by

  As shown in FIG. 8, there are many paths that reach the state j at the timing of the t-th frame, but since speech recognition is aimed at finding the most likely path (maximum likelihood path), In each HMM state, Viterbi processing that leaves a high score is performed according to the following equation (2).

  Since the speech recognition process needs to search all word chains allowed by the grammar, many state hypotheses change to their own HMM state at the same time (in FIG. 8, self-transition to the right side). And a transition to another adjacent HMM state (LR transition to the lower right neighbor in FIG. 8), the amount of calculation is enormous. In order to suppress this increase in the amount of calculation, pruning is normally performed during the Viterbi search to exclude state hypotheses with a low probability from the search space.

  In pruning, from the maximum likelihood at the time being processed, the state hypothesis whose likelihood is within a certain range is left as a search space for the next time, and the state hypothesis whose likelihood is lower than a certain range is Excluded from search space.

Patent Documents 1, 2, and 3 disclose a technique for adaptively changing the threshold value of the pruning condition in order to efficiently prun the number of state hypotheses and reduce the amount of calculation.
JP 2001-75596 A JP 2003-15683 A JP 2004-198832 A

  In the speech recognition apparatus, a search process is started from a silent section that is several hundred ms later than the timing at which speech is first detected, according to a grammar having a silent "sil" at the beginning as described above with reference to FIG. Is done.

  However, a phenomenon occurs in which the search space (a set of state hypotheses) unnecessarily spreads to the previous word in the silent section before utterance. In the example of FIG. 6, the search space expands beyond “sil” to “Ito”, “Itoi”, “Imai”, “Doi”, and “I”, so the number of state hypotheses increases. This delays the recognition process.

  In the silent period before the utterance, the likelihood difference between the state hypotheses does not become sufficiently large, so pruning is not effectively performed. Therefore, cost is consumed for useless likelihood calculation. On the other hand, if the voice detection threshold is raised, a situation may occur in which the utterance is missed and the search is not started.

  As described above, in the silent section before the utterance, the likelihood difference between the state hypotheses is not sufficiently large, and thus effective pruning is difficult. Since the pruning condition is set regardless of whether or not there is, there is a technical problem that the cost is consumed for useless likelihood calculation, particularly in the voiceless section before the utterance.

  The object of the present invention is to solve the above-described problems of the prior art and limit the forward expansion of the search space in a non-voice interval, thereby reducing the number of state hypotheses and improving the speed of recognition processing. A speech recognition method and apparatus, a speech recognition program, and a recording medium thereof.

  In order to achieve the above-described object, the present invention collates an acoustic parameter extracted from a speech signal with an acoustic model that is a probabilistic state transition model, and calculates the likelihood between the acoustic parameter and the acoustic model. In the speech recognition method in which the state hypothesis is changed and the most likely state transition path is recognized as the recognition result, the acoustic analysis means for extracting the acoustic parameters from the speech signal and temporarily storing them, and the beginning of the speech, that is, the utterance based on the acoustic parameters A transition search unit that transitions a state hypothesis based on acoustic parameters after a time that is a predetermined time earlier than the timing of speech detection, calculates a likelihood of the transition, and a transition destination of the state hypothesis A transition restriction unit that restricts the transition destination until the utterance is detected, and releases the restriction after the utterance is detected. It is characterized in.

  According to the present invention, the transition range of the state hypothesis can be set even in a silent section before the utterance is detected and the likelihood difference between the state hypotheses is not sufficiently large so that pruning cannot be effectively performed. By limiting, it is possible to reduce the amount of calculation by suppressing an increase in the state hypothesis, so that it is possible to reduce the recognition processing time by suppressing a decrease in voice recognition accuracy.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the best embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the main part of a speech recognition apparatus 1 according to the present invention.

  The audio signal input unit 11 converts the input audio signal into a digital signal. The acoustic analysis unit 12 converts the audio signal into an acoustic feature parameter and temporarily stores it. The acoustic feature parameter is a feature vector obtained by analyzing the input speech at regular time intervals (for example, 10 ms: hereinafter referred to as a frame). Therefore, the audio signal is converted into a sequence of feature vectors X = x1, x2,.

  The utterance detection unit 13 refers to the power included in the acoustic parameters of each frame, and determines the first frame whose power exceeds a predetermined threshold Wref as the voice start end frame, that is, the utterance frame. The HMM database 14 stores in advance an HMM as a stochastic state transition network of the acoustic model.

  When the utterance is detected by the utterance detection unit 13, the transition search unit 15 sequentially takes in the acoustic feature parameters after the time several hundred ms back from the voice detection timing from the acoustic analysis unit 12, and restricts the grammar. The acoustic log likelihood is calculated by comparing the HMM with the time-series data of the acoustic feature parameters. When the state series branches into a plurality of branches due to grammatical constraints, the transition search unit 15 duplicates the state hypotheses by the number of branches and advances the state hypothesis for each branch to calculate the likelihood.

  The transition restricting unit 16 detects the transition in the state direction on the trellis of the state hypothesis in the transition search unit 15, that is, the LR transition to the other HMM state except the self transition detected by the speech detecting unit 13. Until the start timing, it is restricted within a predetermined restricted area, and after the start timing of the utterance, the restriction on the state transition is canceled and the same state transition as the conventional one is allowed. The recognition result determination unit 17 sets the state transition path with the highest likelihood among the state hypotheses that have reached the grammatical sentence end as the speech recognition result.

  2 and 3 are flowcharts showing the likelihood calculation procedure in the likelihood calculation unit 15, and FIG. 4 compares the state transition on the trellis of the HMM in this embodiment with that of the prior art. FIG. In FIG. 4, (a) → (b) → (c) → (d) shows the state transition process in this embodiment, and (a) → (b) → (e) → (f) shows the state in the prior art. The transition process is shown.

  In FIG. 2, in step S <b> 1, it is determined whether or not an utterance has been detected by the utterance detection unit 13. When an utterance is detected, the process proceeds to step S2, and a variable n for counting the number of transitions of the HMM state after the utterance is detected is reset. In step S3, a frame that is several hundred ms earlier than the utterance detection timing is set as an acoustic feature parameter capturing start frame. In step S4, the acoustic feature parameters temporarily stored in the acoustic analysis unit 12 are taken into the transition search unit 15 in order from the reading start frame.

  In step S5, the transition search unit 15 selects one of the valid state hypotheses as the current calculation target. In step S6, self-transition is performed, and the likelihood is calculated and updated. In step S7, it is determined whether or not self-transition and likelihood calculation have been completed for all the state hypotheses. If not completed, the process returns to step S5, and the other state hypotheses to be transitioned at this time are also described above. Each process is repeated.

  When the self-transition and the likelihood calculation are completed for all state hypotheses to be transitioned at this timing, the process proceeds to step S8, and one of the valid state hypotheses is selected as the current calculation target. In step S9, it is determined whether or not the current acoustic feature parameter captured in step S4 is the acoustic feature parameter before utterance. Since it is initially determined that the acoustic feature parameter is before utterance, the process proceeds to the search before utterance in step S10.

  FIG. 3 is a flowchart showing the pre-utterance search procedure. In step S61, it is determined based on the number of transitions n whether or not the HMM state of the LR transition destination is within the restricted area. Here, as shown in FIGS. 4A and 4B, as long as the number of transitions n is still small and the HMM state of the LR transition destination is outside the restricted area, the process proceeds to step S62 and the LR transition likelihood calculation is performed. Is executed. In step S63, the Viterbi process is executed, and only the state hypothesis having the highest score is left based on the equation (2). In step S64, the number of transitions n is incremented.

  On the other hand, if it is the HMM state [1] shown in FIG. 4 (c), it is determined in step S61 that the HMM state [2] of the LR transition destination is within the restricted area, and the process jumps to step S64. . That is, LR transition is prohibited.

  Returning to FIG. 2, in step S <b> 13, it is determined whether or not the above-described LR transition and Viterbi processing have been completed for all state hypotheses to be transitioned at the current timing. If not completed, the process returns to step S8, and the above-described processes are repeated for other state hypotheses that should transition at the current timing.

  On the other hand, if it is determined in step S9 that the acoustic feature parameter captured in step S4 is the acoustic feature parameter after utterance, the process proceeds to step S11. In step S11, as shown in FIGS. 4E and 4F, each state hypothesis is LR-transitioned without limiting the transition destination as in the prior art, and the likelihood is calculated. In step S12, Viterbi processing is executed.

  Thereafter, when the above-described processes are completed for all the state hypotheses to be transitioned at this timing, the process proceeds to step S14, and the remaining state hypotheses are left with only the state hypothesis having the highest score among all the current state hypotheses. Pruning is excluded from the search. In this embodiment, the likelihood αj (t) at time t and state j is compared with the maximum likelihood αmax (t) among all state hypotheses at the same time, and the state hypothesis satisfying the following equation (3) is determined. The state hypothesis that satisfies the following formula (4) is left in the search space at the next time, and is excluded from the search space at the next time. Θpruning is a positive real number indicating a pruning threshold.

  In step S15, it is determined whether or not there is a next frame. If there is a next frame, the process returns to step S4, the acoustic feature parameter of the next frame is taken in, and the above-described processes are repeated. When each process described above is completed for all frames and the state hypothesis reaches the last grammatical HMM state, in step S16, in the recognition result determination unit 17, among the state hypotheses that have reached the grammatical sentence end, The state transition path with the maximum likelihood is determined as speech recognition.

  In the above-described embodiment, the transition restriction range is described as being fixed in advance. However, the present invention is not limited to this, and the utterance detection unit 13 detects the utterance. The transition limit range may be variably set based on the power level of the background noise measured immediately before.

  For example, when the background noise power measured before the utterance detection is large, the probability of delaying the detection of the utterance without capturing the start of the speech is higher than when the background noise power is small. In such a case, it is desirable to expand the transition restriction range.

  FIG. 5 is a diagram showing the relationship between the number of transitions n and the power of the audio signal when determining whether or not the transition is to the limit range based on the number of transitions n. As the power immediately before the utterance is detected exceeding Wref, that is, the power of the background noise is larger, the number of transitions n is larger, and the transitionable range is expanded.

  Furthermore, although omitted in the above description, the above-described restriction of the transition destination and pruning based on the conventional score may be performed in parallel.

It is the block diagram which showed the structure of the principal part of the speech recognition apparatus 1 which concerns on this invention. It is the flowchart which showed the likelihood calculation procedure. It is the flowchart which showed the procedure of the likelihood calculation before speech. It is the figure which expressed typically the mode of the state transition on the trellis of HMM. It is the figure which showed the relationship between the frequency | count n of a transition which prescribes | regulates a transition restriction | limiting range, and audio | voice signal power. It is the figure which showed an example of grammar. It is the figure which showed an example of the tree structure dictionary. It is the figure which showed an example of the trellis space.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Voice recognition apparatus, 11 ... Voice signal input part, 12 ... Acoustic analysis part, 13 ... Speech detection part, 14 ... HMM database, 15 ... Likelihood calculation part, 16 ... Transition restriction part, 17 ... Recognition result determination part

Claims (7)

The acoustic parameter extracted from the speech signal is compared with the acoustic model that is a probabilistic state transition model, the state hypothesis is transitioned while calculating the likelihood between the acoustic parameter and the acoustic model, and the maximum likelihood state transition path Is a speech recognition apparatus that uses the speech recognition result as
Acoustic analysis means for extracting and temporarily storing acoustic parameters from the audio signal;
Speech detection means for detecting speech based on the acoustic parameters;
A transition search unit that performs likelihood calculation and state transition based on acoustic parameters after a time that is a predetermined time later than the timing of speech detection;
Transition limiting means for limiting the transition destination of the state hypothesis,
The transition search unit causes a valid state hypothesis to repeat self-transition and LR transition, and the transition restriction unit narrows the transition permission range of the LR transition until after the utterance is detected until the utterance is detected. A speech recognition apparatus characterized by that. The transition permission range up speech is detected, the speech recognition apparatus according to claim 1, characterized in that is determined based on the power of the background noise immediately before the utterance is detected. The transition destination allowed until speech is detected, the speech recognition apparatus according to claim 1 or 2 wherein the utterance is characterized in that it is narrower smaller the power of the background noise immediately before the detection. The acoustic model speech recognition apparatus according to any one of 3 claims 1, characterized in that a HMM. The acoustic parameter extracted from the speech signal is compared with the acoustic model that is a probabilistic state transition model, the state hypothesis is transitioned while calculating the likelihood between the acoustic parameter and the acoustic model, and the maximum likelihood state transition path In the speech recognition method with the speech recognition result as
Storing the acoustic parameters;
Detecting speech based on the acoustic parameters;
A procedure for transitioning the state hypothesis based on acoustic parameters after a time that is a predetermined time earlier than the timing of speech detection, and calculating the likelihood thereof,
A step of restricting the transition destination of each state hypothesis in the process of transitioning the state hypothesis,
In the procedure of transitioning the state hypothesis and calculating the likelihood, self transition and LR transition are repeated as valid state hypotheses, and in the procedure of limiting the transition destination, the transition permission range of the LR transition is the utterance A voice recognition method characterized by being narrowed until it is detected until it is detected . The acoustic parameter extracted from the speech signal is compared with the acoustic model that is a probabilistic state transition model, the state hypothesis is transitioned while calculating the likelihood between the acoustic parameter and the acoustic model, and the maximum likelihood state transition path In the speech recognition program that
Storing the acoustic parameters;
Detecting speech based on the acoustic parameters;
A procedure for transitioning the state hypothesis based on acoustic parameters after a timing that is a predetermined time later than the timing of the utterance detection, and calculating the likelihood thereof,
A step of restricting the transition destination of each state hypothesis in the process of transitioning the state hypothesis,
In the procedure of transitioning the state hypothesis and calculating the likelihood, self transition and LR transition are repeated as valid state hypotheses, and in the procedure of limiting the transition destination, the transition permission range of the LR transition is the utterance A speech recognition program for causing a computer to execute a speech recognition method that is narrower than after detection until it is detected . A recording medium storing the voice recognition program according to claim 6 so as to be readable by a computer.

Images