JP2019219598A - Voice recognition apparatus, voice recognition method, and program - Google Patents

Abstract

To provide a voice recognition apparatus, a voice recognition method, and a program, capable of improving the accuracy in voice recognition.SOLUTION: In a voice recognition apparatus 10, a score-calculating part 152 calculates, for each of a plurality of candidates of corresponding relations between a plurality of frames and a plurality of phonemes, a score showing a likelihood of the corresponding relation; a specifying part 153 specifies one of the plurality of candidates as the corresponding relations, on the basis of the score calculated for each of the plurality of candidates by the score-calculating part 152; a determining part 160 determines whether an object word is output in a voice signal, on the basis of the corresponding relation specified by the specifying part 153; and a score-calculating part 152 calculates the score by normalizing a value based on output probability of the phoneme corresponding to each frame, in each of the plurality of candidates, and then accumulating the values over the plurality of frames.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a speech recognition device, a speech recognition method, and a program.

  There are known voice recognition techniques. For example, Patent Literature 1 discloses, as a technology related to voice recognition, a voice search device that searches a voice signal for a portion in which voice corresponding to a search word (query) is emitted. More specifically, the speech search device disclosed in Patent Document 1 searches for a correspondence between a frame in a speech signal to be searched and a phoneme corresponding to a search term using a dynamic programming (DP matching), On the basis of the search result, a section in which it is estimated that a voice corresponding to the search word is emitted is specified from among the voice signals to be searched. At that time, the voice search device disclosed in Patent Document 1 sets a value based on the output probability of the phoneme associated with each frame in order to suppress erroneous detection of the search word from the voice signal. The normalized likelihood is calculated by normalizing with the number of frames associated with the phonemes.

JP 2015-169698 A

  In the method of recognizing speech by searching the correspondence between frames and phonemes as described above, the accuracy of speech recognition is improved by improving the search accuracy when searching for the correspondence between frames and phonemes. It is hoped that it will.

  An object of the present invention is to solve the above-described problems, and an object of the present invention is to provide a speech recognition device, a speech recognition method, and a program capable of improving the accuracy of speech recognition.

In order to achieve the above object, a speech recognition device according to the present invention comprises:
The output probability output from each of the plurality of phonemes corresponding to the target word to be determined whether or not the feature amount of the audio signal is issued in the audio signal, for each of the plurality of frames in the audio signal Output probability obtaining means for obtaining;
A score for calculating a likelihood of the correspondence between the plurality of frames and the plurality of phonemes based on the output probability acquired by the output probability acquisition unit, for each of the plurality of candidates of the correspondence. Calculating means;
Identification means for identifying any of the plurality of candidates as the correspondence relationship based on the score calculated for each of the plurality of candidates by the score calculation means,
Determining means for determining whether or not the target word is issued in the audio signal, based on the correspondence specified by the specifying means;
With
The score calculation unit normalizes a value based on the output probability of a phoneme corresponding to each frame in each of the plurality of candidates by the number of frames corresponding to each phoneme, and accumulates the value over the plurality of frames. By calculating the score,
It is characterized by the following.

  According to the present invention, the accuracy of speech recognition can be improved.

FIG. 1 is a diagram illustrating a hardware configuration of a speech recognition device according to an embodiment of the present invention. It is a figure showing the functional composition of the speech recognition device concerning the embodiment of the present invention. FIG. 5 is a diagram illustrating an example of a frame set in a speech signal to be recognized in the embodiment of the present invention. It is a figure showing the example of the distance table in an embodiment of the present invention. FIG. 5 is a diagram illustrating an example in which the correspondence between frames and phonemes is specified over the entire section of the speech signal to be recognized in the distance table illustrated in FIG. 4. FIG. 5 is a diagram illustrating an example in which one frame of a plurality of frames and one phoneme state of a plurality of phoneme states are specified in the distance table illustrated in FIG. 4. In the distance table shown in FIG. 6, two examples of the correspondence between each frame from the first frame to the specified frame and each phoneme state from the first phoneme state to the specified phoneme state are shown. FIG. It is a figure showing an example of calculation of a state stop score in an embodiment of the present invention. It is a figure showing an example of calculation of a state transition score in an embodiment of the present invention. (A) and (b) are diagrams illustrating an example for comparing a case where a score is calculated by normalizing with the number of frames for each phoneme and a case where a score is calculated with normalizing with the total number of frames. . 5 is a flowchart illustrating a flow of a voice recognition process performed by the voice recognition device according to the embodiment of the present invention. 12 is a flowchart showing a flow of a search process in the voice recognition process shown in FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding portions are denoted by the same reference numerals.

  FIG. 1 shows a hardware configuration of a speech recognition device 10 according to the present embodiment. As shown in FIG. 1, the speech recognition device 10 includes a control unit 11, a storage unit 12, an input unit 13, an output unit 14, and a communication unit 15.

  The control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU is, for example, a microprocessor or the like, and is a central processing unit that executes various processes and calculations. In the control unit 11, the CPU reads the control program stored in the ROM and controls the operation of the entire speech recognition device 10 while using the RAM as a work memory. The control unit 11 functions as control means.

  The storage unit 12 is a nonvolatile memory such as a flash memory and a hard disk. The storage unit 12 stores programs and data used by the control unit 11 to perform various processes, including an OS (Operating System) and application programs. The storage unit 12 stores data generated or obtained by the control unit 11 performing various processes.

  The input unit 13 includes input devices such as input keys, buttons, switches, touch pads, and touch panels. The input unit 13 receives an operation instruction input from a user, and transmits the received operation instruction to the control unit 11. The input unit 13 includes a voice input unit such as a microphone, and receives an input of a voice signal generated outside the voice recognition device 10. The audio signal received by the audio input unit is converted into a digital signal at a specified sampling frequency by an analog-to-digital converter (not shown), and transmitted to the control unit 11.

  The output unit 14 includes a display unit such as a liquid crystal display and an organic EL (Electro Luminescence) display, and an audio output unit such as a speaker. The display unit is driven by a display drive circuit (not shown) and displays various images according to the situation. The display unit may be arranged so as to overlap with the input unit 13, and the display unit and the input unit 13 may constitute a so-called touch panel (touch screen). The audio output unit includes a speaker and an audio output interface, converts audio data generated by the control unit 11 into audio, and outputs the audio to the outside.

  The communication unit 15 is a communication module for the voice recognition device 10 to communicate with an external device. The communication unit 15 communicates with external devices according to communication standards such as Wi-Fi (Wireless Fidelity), USB (Universal Serial Bus), Bluetooth (registered trademark), and NFC (Near Field Communication). The communication unit 15 transmits and receives various data and information to and from an external device through such wired or wireless communication under the control of the control unit 11.

  Next, a functional configuration of the voice recognition device 10 will be described with reference to FIG. As shown in FIG. 2, the speech recognition device 10 functionally includes a speech signal acquisition unit 110, a feature amount calculation unit 120, an output probability acquisition unit 130, a conversion unit 140, a search unit 150, a determination unit 160. In the control unit 11, the CPU functions as these units by reading the program stored in the ROM into the RAM and executing it.

  Further, the speech recognition device 10 includes an acoustic model storage unit 21 and a candidate word storage unit 22. The acoustic model storage unit 21 and the candidate word storage unit 22 are constructed in appropriate storage areas of the storage unit 12, and function as an acoustic model storage unit and a candidate word storage unit, respectively.

  The audio signal acquisition unit 110 acquires an audio signal to be recognized. The voice signal to be recognized is a digital signal indicating a voice to be recognized by the voice recognition device 10. The audio signal acquisition unit 110 acquires, as the audio signal to be recognized, a signal indicating, for example, a sound emitted from a user or a sound emitted from a conference, a television, a movie, or the like via the microphone of the input unit 13. Alternatively, the audio signal acquisition unit 110 may acquire an audio signal to be recognized from the outside via the communication unit 15 or may acquire the audio signal by reading an audio signal stored in the storage unit 12 in advance. . The audio signal acquisition unit 110 is realized by the control unit 11 cooperating with the storage unit 12, the input unit 13, or the communication unit 15. The audio signal acquiring unit 110 functions as an audio signal acquiring unit.

  The feature amount calculation unit 120 calculates a feature amount (acoustic feature amount) of the audio signal acquired by the audio signal acquisition unit 110 for each frame. The feature amount is a parameter serving as an index indicating the feature of the audio signal. More specifically, the feature amount is obtained by calculating a frequency axis characteristic parameter obtained by converting an audio signal called a cepstrum or a mel cepstrum on a frequency axis, and calculating a sum of energy squares of the audio signal or a logarithm thereof. It is obtained by combining the obtained power system characteristic parameters.

  For example, the feature amount is obtained by taking the difference between the 12 components (12-dimensional) of the frequency axis system parameter and one component (1D) of the power system parameter and the components of the immediately preceding frame. (12 dimensions) and △ power system feature parameter 1 component (1 dimensional) and difference of the difference between each component of the immediately preceding frame and △△ frequency axis feature parameter 12 component (12 dimensional) It is represented as a 38-dimensional vector quantity having components. Alternatively, a 40-dimensional mel filter bank applied to the input layer of the neural network may be used as the feature value.

  A frame is a time window having a predetermined time length in an audio signal. Specifically, the upper part of FIG. 3 shows an example of the waveform of a speech signal to be recognized having a time length L from the beginning to the end. In this waveform diagram, the vertical axis represents the amplitude (energy) of the waveform, and the horizontal axis represents time. The lower part of FIG. 3 shows a frame set in the audio signal shown on the upper side. From the first frame to the last frame of the audio signal, T frames each having a frame length F are set by shifting by a specified shift length S. The frame length F and the shift length S are set in accordance with the time length determined when the acoustic model was created, for example, F = 25 msec, S = 10 msec. Since the frame length F is longer than the shift length S, each frame overlaps with an adjacent frame by a time length (FS).

  The feature amount calculation unit 120 converts the speech signal to be recognized into a frequency spectrum by performing a Fourier transform on the frame unit thus set. Then, the characteristic amount calculation unit 120 extracts a characteristic amount of the audio signal for each frame by applying a mel filter bank to the obtained frequency spectrum. The feature amount calculation unit 120 is realized by the control unit 11. The feature amount calculation unit 120 functions as a feature amount calculation unit.

  Returning to FIG. 2, the output probability acquisition unit 130 acquires an output probability that the feature amount calculated by the feature amount calculation unit 120 is output from each phoneme of the acoustic model stored in the acoustic model storage unit 21. The acoustic model is obtained by modeling the frequency characteristics of each of a plurality of phonemes constituting words recognizable by the speech recognition device 10. A phoneme is a basic unit that serves as a speech delimiter in a language to be subjected to speech recognition, such as a vowel or a consonant.

  The acoustic model storage unit 21 stores, for example, an acoustic model (monophone model) using a monophone (one phoneme), an acoustic model (biphone model) using a biphone (two phonemes), and an acoustic model (triphone model) using a triphone (three phonemes). Etc. are stored. The monophone model is an acoustic model generated for each phoneme, and does not depend on adjacent phonemes, that is, an acoustic model in which a state transition between previous and subsequent phoneme states is fixed. The biphone model and the triphone model are acoustic models generated for every two phonemes and every three phonemes, respectively, and are acoustic models dependent on adjacent phonemes. The biphone model is an acoustic model that takes into account the state transition between the front and rear phoneme states. The triphone model is an acoustic model that takes into account state transitions with both front and rear phoneme states.

  In the following, for ease of understanding, a case where a monophone model is used as an acoustic model will be described as an example. However, a case where a biphone model or a triphone model is used can be similarly described. The speech recognition device 10 learns the acoustic model by a general method, and stores the acoustic model in the acoustic model storage unit 21 in advance.

  As the acoustic model, for example, a Hidden Markov Model (HMM), which is an acoustic model used in general speech recognition, can be used. The HMM is a model for probabilistically estimating a word from which the voice signal is output from the voice signal by a statistical method. The HMM uses a standard pattern in which transition probabilities indicating temporal state fluctuations and probabilities of outputting feature values input from each state (output probabilities) are used as parameters.

  The output probability is represented by a mixed Gaussian distribution obtained by adding a plurality of Gaussian (normal) distributions weighted by a predetermined weighting factor. The output probability acquisition unit 130 compares each phoneme of the acoustic model stored in the acoustic model storage unit 21 with the feature amount in each frame calculated by the feature amount calculation unit 115, and determines that the feature amount in each frame is each phoneme. Calculate the output probability output from. The output probability acquisition unit 130 is realized by the control unit 11 cooperating with the storage unit 12. The output probability obtaining unit 130 functions as an output probability obtaining unit.

  Alternatively, the output probability obtaining unit 130 may obtain the output probability using a neural network. When a neural network is used, the output probability acquisition unit 130 inputs the feature amount of the audio signal obtained as a 40-dimensional mel filter bank to the input layer of the neural network. Then, the output probability obtaining unit 130 obtains a value output from the output layer of the neural network as an output probability of each phoneme output from the feature amount.

  More specifically, the output probability obtaining unit 130 obtains the output probability for each phoneme state (phoneme state). The phoneme state is a unit obtained by subdividing the phoneme in the time direction, and corresponds to a minimum unit of the acoustic model. Each phoneme of the acoustic model has a predetermined number of states.

  Hereinafter, a case where the number of states determined for each phoneme is “3” will be described as an example. For example, the phoneme “a” includes a first state “a1” including the start of the utterance of the phoneme, a second state “a2” that is an intermediate state, and a third state “a3” including the end of the utterance. It can be divided into three states. When the number of all phonemes used in the acoustic model is represented by K, there are (3 × K) phoneme states. The output probability acquisition unit 130 outputs (3 × K × T) output signals for each of the (3 × K) phoneme states and for each of T frames from the beginning to the end of the speech signal to be recognized. Calculate the probability.

  The candidate word storage unit 22 stores a plurality of candidate words. The candidate word is a candidate word that is uttered in the speech signal to be recognized, and is a target word to be determined as to whether or not it is uttered in the speech signal to be recognized. The candidate word storage unit 22 stores, as candidate words, a plurality of words such as typical words, phrases, and phrases in a language to be subjected to speech recognition. For example, when the target language for speech recognition is Japanese, the candidate word storage unit 22 stores many Japanese phrases as candidate words. Alternatively, when the target language for speech recognition is English, the candidate word storage unit 22 stores many English words and phrases as candidate words. The speech recognition device 10 stores a plurality of candidate words assumed as recognized words in the candidate word storage unit 22 in advance.

  The conversion unit 140 converts each of the plurality of candidate words stored in the candidate word storage unit 22 into a phoneme string. The phoneme sequence is a sequence in which phonemes corresponding to at least one character constituting the character string are arranged in the same order as the characters in the character string. The conversion unit 140 arranges, for each of the plurality of candidate words, phonemes corresponding to at least one character forming the candidate word, among a plurality of phonemes of the acoustic model stored in the acoustic model storage unit 21. Convert candidate words to phoneme strings.

  For example, if the candidate word is Japanese "Ramen", "Ramen" includes five phonemes (monophone) of "r", "a:", "m", "e", and "N", The conversion unit 140 converts the candidate word “ramen” into a phoneme string “r, a :, m, e, N”. Alternatively, when the candidate word is English “cake”, since “cake” includes four phonemes (monophone) of “k”, “e”, “i”, and “k”, the conversion unit 140 The word “cake” is converted into a phoneme string “k, e, i, k”. The conversion unit 140 is realized by the control unit 11 cooperating with the storage unit 12. The conversion unit 140 functions as a conversion unit.

  The search unit 150 determines the correspondence between each frame in the speech signal to be recognized and each phoneme included in the phoneme string converted by the conversion unit 140, based on the output probability obtained by the output probability obtaining unit 130. Explore. The correspondence between a frame and a phoneme is that each frame in the speech signal corresponds to any phoneme in the phoneme sequence corresponding to the candidate word, assuming that the candidate word is issued in the speech signal to be recognized. This is the information that determines

  The search unit 150 solves the HMM for each of the plurality of candidate words stored in the candidate word storage unit 22 by a dynamic programming (DP) matching method, thereby obtaining the correspondence between the frame and the phoneme. To explore. The search unit 150 is realized by the control unit 11. The search unit 150 functions as a search unit.

  To search for the correspondence between the frame and the phoneme, the search unit 150 generates a distance table. Here, the distance is an index indicating the degree of difference between the acoustic model of each phoneme and the feature amount of the audio signal in each frame. The distance is obtained by reversing the sign of the logarithmic value of the output probability. The smaller the distance of a certain phoneme in a certain frame, the greater the probability that the feature of the audio signal in the frame is output from the phoneme. In other words, the smaller the distance of a certain phoneme in a certain frame, the closer the acoustic model of the phoneme and the feature amount of the audio signal in the frame are. The search unit 150 calculates a distance from the output probabilities obtained by the output probability obtaining unit 130 for each phoneme of the phoneme string converted from the candidate word, and generates, for example, the distance table 30 shown in FIG.

  The distance table 30 is a data table in which the distances obtained for each phoneme of the phoneme string converted from the candidate words are arranged for each frame. FIG. 4 shows, as an example, the distance table 30 generated for the phoneme string “r, a :, m, e, N” converted from the candidate word “ramen”. The search unit 150 takes T frames from the first frame to the last frame in the speech signal to be recognized as rows, and calculates a total of 15 frames in the five phonemes of the phoneme string “r, a :, m, e, N”. A matrix is prepared by taking the phoneme states in columns. Then, the search unit 150 arranges the distance calculated from the output probability in each element of the matrix.

  Specifically, in the m-th frame from the beginning (the m-th frame; m is a natural number satisfying 1 ≦ m ≦ T), the i-th phoneme state from the beginning (i-th phoneme state; i is 1 ≦ i ≦ 15) When the output probability of a natural number that satisfies is expressed as S (m, i), the search unit 150 determines that the distance D (m, i) = − logS The value of (m, i) is arranged. Thereby, the distance table 30 shown in FIG. 4 is obtained. In the distance table 30, the smaller the distance D (m, i), the higher the probability that the frame corresponds to the phoneme state. In FIG. 4, the value indicating the distance described at each position in the distance table 30 is an example for facilitating understanding, and is not necessarily an accurate value.

  When the distance table 30 is generated, the search unit 150 searches the correspondence between the frames and the phonemes in the distance table 30 by the dynamic programming. FIG. 5 shows a hatched path (series) as an example of the correspondence between the frames and phonemes specified in the distance table 30 shown in FIG. In FIG. 5, the first frame corresponds to the first state of the phoneme “r”, the second and third frames correspond to the second state of the phoneme “r”, and the fourth frame corresponds to the third state of the phoneme “r”. The fifth frame corresponds to the first state of the phoneme “a:” corresponding to the state, and one phoneme state is associated with each of the frames from the beginning to the end, such as. As described above, the search unit 150 determines the path (sequence) connecting the position (1, 1) corresponding to the first phoneme state in the first frame to the position (T, 15) corresponding to the last phoneme state in the last frame. Explore).

  The search by the search unit 150 for specifying such a correspondence will be described in more detail. As illustrated in FIG. 2, the search unit 150 has functions of a designation unit 151, a score calculation unit 152, a specification unit 153, and a repetition unit 154. These units are realized by the control unit 11 and function as a designating unit, a score calculating unit, a specifying unit, and a repeating unit, respectively.

  The specifying unit 151 specifies one frame of the plurality of frames in the generated distance table 30, and specifies one phoneme state of the plurality of phoneme states. Here, to designate a phoneme state means that when there are a plurality of states constituting each phoneme, one phoneme of the plurality of phonemes is designated, and one state of the plurality of states is further designated. Means that. On the other hand, when there is only one state that constitutes each phoneme, designating a phoneme state is equivalent to designating a phoneme.

  The search unit 150 is specified by the specifying unit 151 from each of the frames from the first frame of the plurality of frames to the one frame specified by the specifying unit 151 and the first phoneme state of the plurality of phoneme states. The correspondence between each phoneme state and one phoneme state is searched for by a dynamic programming method. For example, when the specifying unit 151 specifies the m-th frame from the top (m-th frame) and the i-th phoneme state from the top (i-th phoneme state), the search unit 150 sets the m-th frame to the m-th frame from the top frame. A search is made for a correspondence R (m, i) between the m frames up to and the i phoneme states from the head phoneme state to the i-th phoneme state. The correspondence R (m, i) is the correspondence between the m-th frame from the beginning and the i-phoneme states from the beginning, assuming that the m-th frame and the i-th phoneme state correspond to each other. .

  Specifically, FIG. 6 shows an example in which the designation unit 151 designates the fifteenth frame (m = 15) and the eighth phoneme state (i = 8) in the distance table 30, that is, the element shaded in FIG. Is shown. Here, the eighth phoneme state corresponds to the second state of the third phoneme “m” from the top. In this case, the search unit 150 determines the correspondence R (15,8) between the first 15 frames and the first 8 phoneme states from the position (1,1) to the position (15,8). The search is performed based on the values of 120 (= 15 × 8) distances D (m, i) in FIG. 6, that is, the values of the distances D (m, i) in the portion surrounded by the thick line in FIG.

  More specifically, the search unit 150 searches for the correspondence in the specified frame by using the correspondence specified in the frame immediately before the frame specified by the specifying unit 151. More specifically, the search unit 150 determines the correspondence R (m, i) when the m-th frame and the i-th phoneme state are specified by the specifying unit 151 from the (m-1) th frame to the m-th frame. The phoneme state is specified from two types of candidates, that is, a case where the phoneme state changes over the frame and a case where the phoneme state does not change.

  Here, the case where the phoneme state transitions from the (m-1) th frame to the mth frame means that the (m-1) th frame becomes the (i-1) th phoneme state in the correspondence R (m, i). In this case, the m-th frame corresponds to the i-th phoneme state. On the other hand, the case where the phoneme state does not transition from the (m−1) th frame to the mth frame means that both the (m−1) th frame and the mth frame in the correspondence R (m, i) This is a case corresponding to the i-phoneme state.

  FIG. 7 shows two examples of the correspondence R (15, 8) in a state where the fifteenth frame and the eighth phoneme state shown in FIG. 6 are designated. In FIG. 7, a path (series) indicated by a solid line represents a candidate C1 of the correspondence R (15, 8) when the phoneme state does not transition from the 14th frame to the 15th frame. On the other hand, the path (series) indicated by the broken line represents the candidate C2 of the correspondence R (15, 8) when the phoneme state transitions from the 14th frame to the 15th frame.

  Each of the two candidates C1 and C2 is represented by a path connecting the designated position (15, 8) to the designated position (15, 8) in the distance table 30 shown in FIG. Here, the path of the first candidate C1 passes through the position (14, 8) surrounded by a thick solid line in the fourteenth frame immediately before the designated fifteenth frame. In other words, the first candidate C1 corresponds to the correspondence (15, 8) when it is assumed that the fourteenth frame and the fifteenth frame both correspond to the eighth phoneme state. On the other hand, the route of the second candidate C2 passes through the position (14, 7) surrounded by a thick broken line in the fourteenth frame immediately before the specified fifteenth frame. In other words, the second candidate C2 corresponds to the correspondence relationship (15, 15) when it is assumed that the fourteenth frame corresponds to the seventh phoneme state and the fifteenth frame corresponds to the eighth phoneme state. 8).

  The search unit 150 specifies the fourteenth frame and the eighth phoneme state by the specifying unit 151 as the path from the position (1, 1) to the position (14, 8) in the path of the first candidate C1. The route of the correspondence R (14, 8) specified at this time is used. In addition, the search unit 150 specifies the fourteenth frame and the seventh phoneme state by the specifying unit 151 as the path from the position (1, 1) to the position (14, 7) in the path of the second candidate C2. The route of the correspondence R (14, 7) specified at the time of the identification is used. The search unit 150 specifies the correspondence R (15, 8) from the two candidates C1 and C2 based on the score calculated by the score calculation unit 152.

  The score calculation unit 152 calculates a score for searching for the correspondence R (m, i) based on the output probability acquired by the output probability acquisition unit 130. The score is a measure of the likelihood of the correspondence between a frame and a phoneme, and is also called likelihood. The score calculation unit 152 normalizes the value based on the output probability of the phoneme corresponding to each frame with respect to each of the two candidates C1 and C2 by the number of frames corresponding to each phoneme, and over a plurality of frames. By accumulating, a score is calculated. Here, the value based on the output probability specifically corresponds to a distance obtained by taking the logarithm of the output probability.

  More specifically, the score calculation unit 152 determines the score of a path connecting the head position (1, 1) to the specified position (m, i) in the distance table 30 as P in the following equation (1). (M, n) is calculated. P1 (k) and P2 (m, n) in the following equation (1) are calculated using the distance D (t) or the output probability S (t) in the t-th frame on the path for which the score is to be calculated, as follows: It is expressed by the expression 2) and the following expression (3).

  In the above expression (1), n is a value indicating the number of the phoneme including the i-th phoneme state designated by the designation unit 151 from the head phoneme. For example, since the eighth phoneme state specified in FIG. 7 is the second state of the third phoneme “m” from the top, i = 8 corresponds to n = 3. Generally, when the number of states of each phoneme is 3, the i-th phoneme state and the n-th phoneme including the i-th phoneme state are “3 × (n−1) <i ≦ 3 × n”. Satisfy the relationship.

  The first term on the right side of the above equation (1) is a score for each phoneme from the first phoneme to the (n−1) th phoneme in the distance table 30. The score calculation unit 152 calculates the score P1 (k) for the k-th phoneme (k-th phoneme) from the head according to the above equation (2). In the above equation (2), T (k) represents the number of frames corresponding to the k-th phoneme (k-th phoneme) from the head in the correspondence R (m, i) candidate for which the score is to be calculated. . That is, the value “T (1) + T (2) +...” Obtained by integrating T (k) from the first phoneme to the last phoneme in the distance table 30 matches the total number T of frames.

  In the above equation (2), a (k) is a value indicating the number of the T (k) frames corresponding to the k-th phoneme starting from the top in the distance table 30. In other words, in the candidates for the correspondence R (m, i), the k-th phoneme is associated with T (k) frames starting from the a (k) -th frame. Here, a (k) and T (k) satisfy the relationship of “a (k) = T (1) + T (2) +... T (k−1) +1”.

  The score calculation unit 152 calculates the distance D (t) or the logarithm logS (t) of the output probability, which is a value based on the output probability of the k-th phoneme, according to the above equation (2), using the a (k) corresponding to the k-th phoneme. ) Accumulate over T (k) frames from the frame. Then, the score calculation unit 152 normalizes the accumulated value by the number of frames T (k) corresponding to the k-th phoneme, that is, divides the accumulated value by the value of T (k), thereby obtaining the score P1 ( k) is calculated.

  The second term on the right side of the above equation (1) is a score relating to the n-th phoneme in the distance table 30, that is, the phoneme including the i-th phoneme state specified by the specifying unit 151. The score calculation unit 152 calculates the score P2 (m, n) for the n-th phoneme according to the above equation (3). The score calculation unit 152 calculates the distance D (t) or the logarithm of the output probability over “ma (n) +1” frames from the a (n) frame to the mth frame corresponding to the nth phoneme. Accumulate lgS (t). Then, the score calculation unit 152 calculates the score P2 (m, n) by dividing the accumulated value by the number of frames “ma (n) +1”.

  As described above, the score calculation unit 152 calculates the score P2 (m, n) of the n-th phoneme based on an equation different from the score P1 (k) from the first phoneme to the (n-1) -th phoneme. . This is because the number T (n) of frames corresponding to the n-th phoneme has not yet been determined when the search unit 150 searches for the correspondence R (m, i). In other words, it is not determined whether the phoneme corresponding to the (m + 1) th and subsequent frames is the nth phoneme or the (n + 1) th or later phoneme. For example, in the distance table 30 shown in FIG. 7, it is not determined whether or not the phonemes corresponding to the 16th frame and subsequent frames are still “m”.

  Therefore, the score calculation unit 152 determines the remaining frames excluding the frames corresponding to the (n-1) th phoneme from the first phoneme of the n phonemes of the m frames, to the n-th phoneme. The score P2 (m, n) of the n-th phoneme is calculated using at least one frame corresponding to the phoneme. That is, when calculating the score P2 (m, n) of the n-th phoneme, the score calculation unit 152 replaces the value T (n) finally determined as the number of at least one frame corresponding to the n-th phoneme. A value obtained by accumulating the distance D (t) using the number of remaining frames “ma (n) +1” obtained by excluding the frame corresponding to the (n−1) phonemes from the head from the m frames. Is normalized.

  The score calculation unit 152 calculates a score P (m, n) for each of the plurality of candidates of the correspondence R (m, i). More specifically, the score calculation unit 152 determines that the (m−1) th frame and the i-th phoneme state correspond to each other as a plurality of candidates for the correspondence R (m, i). For each of the first candidate C1 and the second candidate C2 when it is assumed that the (m-1) th frame and the (i-1) th phoneme state correspond to each other, the score P (m , N).

  The score P (m, n) calculated for the first candidate C1 is a score when the phoneme state does not transition (stops) from the (m-1) th frame to the mth frame, and thus the “state stop score” ". On the other hand, the score P (m, n) calculated for the second candidate C2 is a score when the phoneme state transitions from the (m−1) th frame to the mth frame, and thus “state transition” The score is called.

  With reference to FIG. 8 and FIG. 9, an example of calculating the state stop score and the state transition score will be described. FIG. 8 shows an example of calculating the state stationary score for the first candidate C1 of the correspondence R (15, 8) shown in FIG. On the other hand, FIG. 9 shows an example in which the state transition score is calculated for the second candidate C2 of the correspondence R (15, 8) shown in FIG. 8 and 9 show only a part of the distance table 30 shown in FIG. 7 for easy understanding, and omit the value of the distance D (t) of a part that is not involved. I have.

  (A1) In the first candidate C1 shown in FIG. 8, three states of the first phoneme “r” are associated with four frames from the first to fourth frames. Therefore, the score calculation unit 152 calculates the sum “D (1) + D (2) + D (3) + D (4)” of the distances along the first candidate C1 in the first to fourth frames according to the above equation (2). = 3 + 4 + 3 + 2 = 12 "is divided by the number of corresponding frames" T (1) = 4 ". Thereby, the score calculation unit 152 obtains the score “P1 (1) = 12/4 = 3” for the first phoneme “r”.

  (A2) Next, the three states of the second phoneme “a:” are associated with seven frames from the fifth to eleventh frames. Therefore, the score calculation unit 152 calculates the sum “D (5) + D (6) +... + D (11) = 4 + 5 + 5 + 6 + 5 + 2 + 1” of the distances along the first candidate C1 in the fifth to eleventh frames according to the above equation (2). = 28 "by the number of corresponding frames" T (2) = 7 ". Thereby, the score calculation unit 152 obtains the score “P1 (2) = 28/7 = 4” for the second phoneme “a:”.

  (A3) The third phoneme “m” is a phoneme that includes the eighth phoneme state specified by the specifying unit 151. Therefore, the score calculation unit 152 determines from the twelfth frame, which is the first frame corresponding to the third phoneme “m”, the first candidate C1 in the fifteenth frame specified by the specification unit 151 according to the above equation (3). The sum of the distances along the line “D (12) + D (13) + D (14) + D (15) = 5 + 3 + 2 + 4 = 14” is divided by the number of frames from the twelfth to fifteenth frames “15−12 + 1 = 4”. . Thereby, the score calculation unit 152 obtains the score “P2 (15,3) = 14/4 = 3.5” for the third phoneme “m”.

  When the scores P1 (1), P1 (2), and P2 (15, 3) from the first frame and the phoneme state to the designated frame and the phoneme state are calculated in this manner, the score calculation unit 152 calculates ), The state stationary score “P (15,3) = P1 (1) + P1 (2) + P2 (15,3) = 3 + 4 + 3.5 = 10.5” of the correspondence R (15,8) is obtained.

  (B1) In the second candidate C2 shown in FIG. 9, the three states of the first phoneme “r” are associated with four frames from the first to ninth frames. Therefore, the score calculation unit 152 calculates the sum of the distances along the second candidate C2 in the first to ninth frames “D (1) + D (2) +... + D (9) = 3 + 4 + 3 + 4 + 6 + 4 + 7 + 3 + 2 according to the above equation (2). = 36 "by the number of corresponding frames" T (1) = 9 ". Thereby, the score calculation unit 152 obtains the score “P1 (1) = 36/9 = 4” for the first phoneme “r”.

  (B2) Next, the three states of the second phoneme “a:” are associated with four frames from the tenth to thirteenth frames. Therefore, the score calculation unit 152 calculates the sum “D (10) + D (11) + D (12) + D (13) of the distances along the second candidate C2 in the tenth to thirteenth frames according to the above equation (2). = 4 + 5 + 5 + 1 = 15 "is divided by the number of corresponding frames" T (2) = 4 ". Thereby, the score calculation unit 152 obtains the score “P1 (2) = 15/4 = 3.75” for the second phoneme “a:”.

  (B3) The third phoneme “m” is a phoneme that includes the eighth phoneme state specified by the specifying unit 151. Therefore, according to the above equation (3), the score calculation unit 152 converts the fourteenth frame, which is the first frame corresponding to the third phoneme “m”, into the second candidate C2 in the fifteenth frame specified by the specification unit 151. The sum of the distances along the line “D (14) + D (15) = 6 + 4 = 10” is divided by the number of frames from the 14th to the 15th frame, “15−14 + 1 = 2”. Thereby, the score calculation unit 152 obtains the score “P2 (15,3) = 10/2 = 5” for the third phoneme “m”.

  When the scores P1 (1), P1 (2), and P2 (15, 3) from the first frame and the phoneme state to the designated frame and the phoneme state are calculated in this manner, the score calculation unit 152 calculates ), The state transition score “P (15,3) = P1 (1) + P1 (2) + P2 (15,3) = 4 + 3.75 + 5 = 12.75” of the correspondence R (15,8) is obtained.

  As described above, the score calculation unit 152 calculates the score at the k-th phoneme by normalizing the cumulative value of the distance D (t) by the number of frames for each phoneme. Then, the score calculation unit 152 calculates the scores P1 (1), P1 (2),..., P1 (n) of the n phonemes calculated when the value of k in the k-th phoneme is 1 to n, respectively. -1), by accumulating P2 (m, n), calculate the respective scores P (m, n) of the two candidates C1, C2 of the correspondence R (m, i).

  Here, when the score calculation unit 152 calculates the score, the process of normalizing the accumulated value of the distance D (t) by the number of frames is performed separately for each phoneme. This is for suppressing erroneous recognition of the audio signal due to the influence of only the phoneme. Specifically, the score calculation unit 152 normalizes the accumulated value of the distance D (t) by the number of all frames from the first frame to the m-th frame specified by the specifying unit 151, not by phoneme. Consider a case where the score of the correspondence R (m, i) is calculated.

  When the cumulative value of the distance D (t) is normalized by the number of all frames, the score calculation unit 152 calculates the score of the correspondence R (m, i) as P ′ (m) shown in the following equation (4), for example. Is calculated. That is, the score calculation unit 152 calculates the distance D (t) or the cumulative value of the logarithm lgS (t) of the output probability as the number “m” of all frames from the first frame to the m-th frame specified by the specification unit 151. , The score P ′ (m) of the correspondence R (m, i) is calculated.

  Specifically, referring to FIGS. 10A and 10B, a case where normalization is performed by the number of frames for each phoneme according to the above equation (1), and a case where normalization is performed using the total number of frames according to the above equation (4) , Will be described. Note that the distance tables 31 and 32 shown in FIGS. 10A and 10B are examples different from the distance tables 30 shown in FIGS. 7 to 9 and cover 11 frames and 5 phonemes. The correspondence R (11, 5) is shown. FIGS. 10A and 10B show an example in which the number of states of each phoneme is one for easy understanding.

  When the score is calculated by normalizing the number of frames for each phoneme according to the above equation (1), the score calculation unit 152 sets the score of the correspondence R (11, 5) shown in FIG. 11,5) = 6/1 + (2 + 2 + 2 + 3 + 3 + 1 + 2) /7+7/1+4/1+6/1=25.1 ". Similarly, the score calculation unit 152 converts the score of the correspondence R (11,5) shown in FIG. 10B into “P (11,5) = (4 + 4) / 2 + (3 + 3 + 3 + 3) / 4 + 4/1 + ( 3 + 3) / 2 + (4 + 5) /2=18.5 ". Thus, a score smaller in FIG. 10B than in FIG. 10A is obtained.

  On the other hand, when the score is calculated by normalizing with the total number of frames according to the above equation (4), the score calculation unit 152 calculates the score of the correspondence R (11, 5) shown in FIG. “P ′ (11) = (6 + 2 + 2 + 2 + 3 + 3 + 1 + 2 + 7 + 4 + 6) /11=3.45” is calculated. Similarly, the score calculation unit 152 calculates the score of the correspondence R (11, 5) shown in FIG. 10B as “P ′ (11) = (4 + 4 + 3 + 3 + 3 + 3 + 4 + 3 + 3 + 4 + 5) /11=3.54”. In this way, unlike the case where the number of frames is normalized for each phoneme, the score in FIG. 10A is smaller than that in FIG. 10B.

  The reason why the scores differ depending on whether or not to normalize by the number of frames for each phoneme is that the path of only some phonemes in the phoneme sequence occupies a long part in the entire path. caused by. For example, in FIG. 10A, the phoneme “a:” of the phoneme string “r, a :, m, e, N” is the 7th from the second to eighth frames of the 11 frames in total. Corresponding to each frame. As described above, when only some phonemes correspond to many frames in all frames, the distance D (t) calculated for some phonemes tends to greatly affect the score of the entire route. As a result, even when only some of the phonemes are similar to the speech signal to be recognized, it is likely to be erroneously determined that the similarity of the entire phoneme sequence is high.

  In order to avoid such erroneous determination, the score calculation unit 152 calculates a score by normalizing the number of frames for each phoneme according to the above equation (1). As a result, the weight of each phoneme is made uniform, so that only a part of the phonemes hardly greatly affects the score of the entire route, and the accuracy of DP matching can be improved.

  Returning to the description of the function of the voice recognition device 10 shown in FIG. Based on the score calculated by the score calculation unit 152, the identification unit 153 determines each frame from the first frame to the frame specified by the specification unit 151 and the phoneme state specified by the specification unit 151 from the first phoneme state. The correspondence between each phoneme state up to and is specified. For example, when the m-th frame and the i-th phoneme state are specified by the specifying unit 151, the specifying unit 153 sets the state calculated by the score calculation unit 152 for the first candidate C1 of the correspondence R (m, i). The stop score is compared with the state transition score calculated for the second candidate C2. Then, the specifying unit 153 specifies, as the maximum likelihood sequence of the correspondence R (m, i), the path having the better score among the two candidates C1 and C2.

  Here, a good score corresponds to a small value of the score P (m, n) expressed by the above equation (1). In the example of FIG. 7, the state stop score “10.5” calculated for the first candidate C1 is smaller than the state transition score “12.75” calculated for the second candidate C2. Therefore, the identifying unit 153 selects the first candidate C1 as the maximum likelihood sequence of the correspondence R (m, i), and excludes the second candidate C2 from the maximum likelihood sequence of the correspondence R (m, i). . As described above, the specifying unit 153 specifies the correspondence between the frame and the phoneme by selecting the candidate having the better score from the two candidates C1 and C2 having different routes.

  The repetition unit 154 repeats the processes of the score calculation unit 152 and the specification unit 153 while changing at least one of the frame specified by the specification unit 151 and the phoneme state. In other words, the repetition unit 154 increases at least one of the value of m in the m frames from the beginning and the value of n or i in the n phonemes or the i phoneme states from the beginning, The processing of the score calculation unit 152 and the specification unit 153 is repeatedly executed.

  In the repetition process by the repetition unit 154, when the designation unit 151 designates the m-th frame and the i-th phoneme state, the score calculation unit 152 designates the (m-1) -th frame and the i-th phoneme state. Using the correspondence R (m-1, i) specified at this time, the score (state stationary score) of the first candidate C1 of the correspondence R (m, i) is calculated. Further, the score calculation unit 152 uses the correspondence R (m−1, i−1) specified when the (m−1) th frame and the (i−1) th phoneme state are specified, The score (state transition score) of the second candidate C2 of the correspondence R (m, i) is calculated. Then, the specifying unit 153 specifies, as the correspondence R (m, i), the path of the better score between the state stop score and the state transition score calculated by the score calculator 152.

  In this way, the search unit 150 shifts the frame and the phoneme state designated by the designation unit 151 in order from the beginning to the end, and adjusts the correspondence between the T frames and the 15 phoneme states in the entire distance table 30. The relation R (T, 15) is gradually built.

  Further, the repetition unit 154 repeats the processing of the specification unit 151, the score calculation unit 152, and the identification unit 153 for each of the plurality of candidate words stored in the candidate word storage unit 22. Thereby, the score calculation unit 152 calculates a score for each of the plurality of candidate words by the above-described method. Then, the specifying unit 153 specifies, for each of the plurality of candidate words, the correspondence between the frame and the phoneme state based on the score calculated by the score calculating unit 152.

  The determination unit 160 determines whether each of the plurality of candidate words stored in the candidate word storage unit 22 is a word uttered in the speech signal to be recognized based on the correspondence specified by the specifying unit 153. Is determined. More specifically, the determination unit 160 compares any one of the plurality of candidate words with the score of the correspondence relationship specified for each of the plurality of candidate words by the specifying unit 153, and determines one of the plurality of candidate words as a speech to be recognized. It is determined as a word uttered in the signal.

  The correspondence specified for each of the plurality of candidate words is finally determined as the correspondence between each frame from the beginning to the end and each phoneme state from the beginning to the end in each distance table 30 of the plurality of candidate words. Is the correspondence relationship specified in. The determination unit 160 compares the magnitude relation of the scores calculated by the score calculation unit 152 in the correspondence specified for each of the plurality of candidate words. Then, the determination unit 160 determines the candidate word having the highest score among the plurality of candidate words, that is, the candidate word having the lowest score, as the word uttered in the speech signal to be recognized. The determination unit 160 is realized by the control unit 11. The determination unit 160 functions as a determination unit.

  The flow of the voice recognition processing executed by the voice recognition device 10 configured as described above will be described with reference to the flowcharts shown in FIGS.

  The speech recognition process illustrated in FIG. 11 starts when an instruction to start speech recognition is received from the user via the input unit 13 or the like. When the voice recognition process is started, the control unit 11 acquires a voice signal to be recognized (step S1). For example, the control unit 11 acquires, via the input unit 13 or the communication unit 15, a signal indicating a sound emitted from a user or a sound emitted from a conference, a television, a movie, or the like. It should be noted that if the voice signal to be recognized has already been acquired when the instruction to start the voice recognition process is received, the process of step S1 is omitted.

  After acquiring the audio signal, the control unit 11 specifies the first frame of the acquired audio signal (Step S2). When the first frame is specified, the control unit 11 functions as the feature amount calculation unit 120, and calculates the feature amount of the audio signal in the specified frame (Step S3). More specifically, the control unit 11 performs Fourier transform on the audio data in the first frame to convert the audio data into a frequency spectrum, and applies a mel filter bank to extract the feature amount from the audio signal.

  After calculating the feature amount, the control unit 11 functions as the output probability obtaining unit 130, and obtains the output probabilities for all phonemes of the acoustic model stored in the acoustic model storage unit 21 (Step S4). For example, the control unit 11 calculates an output probability that the calculated feature amount is output for all phonemes of the monophone model using a normal mixture continuous distribution or a neural network technique. Thus, the control unit 11 calculates an index indicating which phoneme in the monophone model corresponds to the audio signal in the specified frame with a high probability.

  After acquiring the output probability, the control unit 11 determines whether or not the specified frame has reached the last frame of the audio signal (Step S5). If the specified frame has not reached the last frame (step S5; NO), the control unit 11 specifies the next frame (step S6). For example, when the first frame is currently specified, the control unit 11 specifies the second frame from the top as the next frame. When the t-th frame from the head is currently specified, the control unit 11 specifies the (t + 1) -th frame from the head as the next frame.

  When the next frame is designated, the control unit 11 returns the process to step S3. Then, for the newly specified frame, the control unit 11 executes a feature amount calculation process in step S3 and an output probability acquisition process in step S4. As described above, the control unit 11 calculates the feature amount for each frame from the beginning to the end of the audio signal to be recognized. Then, the control unit 11 acquires the output probabilities of the calculated feature amounts for all phonemes of the acoustic model.

  Finally, when the specified frame reaches the last frame (Step S5; YES), the control unit 11 functions as the conversion unit 140, reads a plurality of candidate words stored in the candidate word storage unit 22, and Each is converted into a phoneme string (step S7). For example, when the read candidate word is “ramen”, the control unit 11 converts the candidate word “ramen” into a phoneme string “r, a :, m, e, N”. Alternatively, when the read candidate word is English “cake”, the control unit 11 converts the candidate word “cake” into a phoneme string “k, e, i, k”.

  When each of the plurality of candidate words is converted into a phoneme string, the control unit 11 functions as the search unit 150, and searches for a correspondence between a frame and a phoneme state for each of the plurality of candidate words (step S8). Details of the search processing in step S8 will be described with reference to the flowchart shown in FIG.

  When the search process illustrated in FIG. 12 starts, the control unit 11 functions as the specifying unit 151 and specifies one candidate word from a plurality of candidate words (step S801). When one candidate word is specified, the control unit 11 generates the distance table 30 for the specified candidate word (Step S802). For example, when the designated candidate word is “ramen”, the control unit 11 takes 15 phoneme states constituting the phoneme string “r, a :, m, e, N” obtained in step S7 into a row. , A matrix in which T frames of a speech signal to be recognized are arranged in rows is prepared. Then, the control unit 11 calculates the distance from the output probability obtained in step S4, and arranges the calculated distance in each element of the matrix to generate the distance table 30 illustrated in FIG.

  When the distance table 30 is generated, the control unit 11 functions as the specifying unit 151, and specifies one frame in the generated distance table 30 (Step S803). Further, the control unit 11 specifies one phoneme state in the generated distance table 30 (step S804). Then, the control unit 11 functions as the score calculation unit 152, and calculates, for the specified frame and the phoneme state, the state stationary score and the state transition score from the immediately preceding frame (step S805).

  More specifically, when the m-th frame from the beginning and the i-th phoneme state from the beginning are specified, the control unit 11 determines the above-mentioned equation (1) for the two candidates of the correspondence R (m, i). ) Is calculated according to the score P (m, n). For example, when the fifteenth frame from the beginning and the eighth phoneme state from the beginning (the second state of the third phoneme from the beginning) are specified as shown in FIG. The score P (15,3) of the first candidate C1 in (15,8) is calculated as the state stop score, and the score P (15,3) of the second candidate C2 is calculated as the state transition score.

  After calculating the state stop score and the state transition score, the control unit 11 functions as the specifying unit 153, compares the calculated state stop score with the state transition score, and leaves a route with a better score (step S806). . To be more specific, the control unit 11 sets the path corresponding to the smaller one of the state stationary score and the state transition score to the maximum likelihood sequence from the first frame and the phoneme state to the specified frame and the phoneme state. To be determined. At this time, the control unit 11 removes the route with the larger score from the candidates.

  When the route is selected in this way, the control unit 11 determines whether or not the specified phoneme state has reached the last phoneme state (step S807). If the specified phoneme state has not reached the last phoneme state (step S807; NO), the control unit 11 specifies the next phoneme state (step S808). For example, if the i-th phoneme state from the beginning is currently designated, the control unit 11 designates the (i + 1) -th phoneme state from the beginning as the next phoneme state.

  When the next phoneme state is designated, the control unit 11 returns the process to step S805. Then, the control unit 11 functions as the repetition unit 154, and executes the process of calculating the state transition score and the state stationary score in step S805 and the process of selecting a route in step S806 for the newly specified phoneme state. As described above, the control unit 11 specifies phoneme states one by one in order from the top in the distance table 30 for the frame specified in step S803. Then, the control unit 11 specifies the correspondence between each frame from the first frame to the specified frame and each phoneme state from the first phoneme state to the specified phoneme state.

  When the first phoneme state is specified in step S804, there is no previous phoneme state, and thus the control unit 11 does not calculate the state transition score in step S805. In this case, the control unit 11 determines in step S806 that the path corresponding to the state stationary score, that is, all frames from the first frame to the specified frame correspond to the first phoneme state. To identify.

  When the specified phoneme state reaches the last phoneme state (step S807; YES), the control unit 11 determines whether the specified frame has reached the last frame in the speech signal recognition section (step S809). . If the specified frame has not reached the last frame (step S809; NO), the control unit 11 specifies the next frame (step S810). For example, if the m-th frame from the head is currently specified, the control unit 11 specifies the (m + 1) -th frame from the head as the next frame.

  When the next frame is designated, the control unit 11 returns the process to step S804. Then, the control unit 11 functions as the repetition unit 154, and executes the processing from step S804 to step S808 for the newly designated frame. When the first frame is specified in step S803, the control unit 11 skips the processing from step S804 to step S808 because there is no previous frame. In this way, the control unit 11 specifies each frame in the speech signal to be recognized and each phoneme state in the designated candidate word while designating one frame at a time from the second frame of the speech signal to be recognized. , The correspondence relationship is specified.

  When the specified frame reaches the last frame (step S809; YES), the control unit 11 determines whether all candidate words have been specified (step S811). If all candidate words have not been specified (step S811; NO), the control unit 11 specifies an unspecified candidate word among the plurality of candidate words stored in the candidate word storage unit 22 as the next candidate word. (Step S812).

  When the next candidate word is specified, the control unit 11 returns the process to step S802. Then, the control unit 11 functions as the repetition unit 154, and executes the processing from step S802 to step S811 for the newly specified candidate word. As described above, the control unit 11 searches for the correspondence between each frame of the speech signal to be recognized and each phoneme state for each of the plurality of candidate words stored in the candidate word storage unit 22.

  Finally, when all candidate words have been designated (step S811; YES), the control unit 11 terminates the search processing shown in FIG.

  The process returns to the speech recognition processing shown in FIG. When the correspondence between the frame and the phoneme state is searched for each of the plurality of candidate words, the control unit 11 functions as the determination unit 160 and determines the candidate word having the best score among the plurality of candidate words as the recognition result. (Step S9). More specifically, when the specified frame reaches the end frame and the specified phoneme state reaches the end phoneme state, the control unit 11 determines the state stop score and the state stop score calculated in step S805. Among the transition scores, the score of the route left in step S806, that is, the better score, is compared among a plurality of candidate words. Then, the control unit 11 determines the candidate word having the best score among the plurality of candidate words as the word uttered in the audio signal.

  When determining the words uttered in the speech signal to be recognized, the control unit 11 outputs a recognition result (step S10). For example, the control unit 11 displays on the display unit of the output unit 14 a word determined to be uttered in the audio signal. Alternatively, the control unit 11 outputs a word determined to be uttered in the audio signal as a sound from the speaker. Thereby, the user can confirm the recognition result of the audio signal. Thus, the speech recognition processing shown in FIG. 11 ends.

  As described above, the speech recognition device 10 according to the present embodiment is a device that searches for a correspondence between a frame and a phoneme by a dynamic programming method and determines a word uttered in a speech signal to be recognized. In the process of searching for the correspondence between a frame and a phoneme, a value based on the output probability of the phoneme corresponding to each frame is normalized by the number of frames corresponding to each phoneme, and accumulated over a plurality of frames. Then, the correspondence is specified based on the calculated score.

  By using the score normalized by the number of frames corresponding to each phoneme, the bias of the weight of each phoneme can be reduced, and the contribution of each phoneme to the entire score is made uniform. Thereby, when some phonemes correspond to many frames in the phoneme sequence, it is possible to suppress that an accurate score cannot be obtained due to the influence of only some phonemes. As a result, the accuracy of the search when searching for the correspondence between the frame and the phoneme can be improved, which leads to an improvement in the accuracy of speech recognition.

  Further, according to the speech recognition device 10 according to the present embodiment, by calculating an accurate score, it is possible to improve the accuracy of the pruning of candidates in the process of searching for the correspondence between the frame and the phoneme. The number of candidates to be left can be reduced. Therefore, the calculation cost at the time of searching for the correspondence can be suppressed, and the memory can be saved. In particular, when judging a word uttered in a speech signal to be recognized from among many candidate words, efficient speech recognition can be performed with a small calculation cost.

(Modification)
Although the embodiment of the present invention has been described above, the above embodiment is an example, and the scope of the present invention is not limited to this. That is, the embodiments of the present invention can be applied in various ways, and all embodiments are included in the scope of the present invention.

  For example, in the above-described embodiment, the speech recognition device 10 acquires a speech signal to be recognized, and emits one of a plurality of candidate words stored in the candidate word storage unit 22 in the acquired speech signal. Was determined to be a word. However, in the present invention, the speech recognition device 10 may acquire a search word as a target word, and may search a speech signal to be searched for a section in which the acquired search word is issued. That is, the voice recognition device 10 may function as a so-called voice search device.

  When the voice recognition device 10 functions as a voice search device, the search unit 150 performs the search process of the correspondence between the frame and the phoneme, which is performed for each of the plurality of candidate words in the above embodiment, in a plurality of different sections in the voice signal. For each of More specifically, the score calculation unit 152 determines, based on the output probabilities obtained by the output probability obtaining unit 130, a plurality of candidates for the correspondence between the frame and the phoneme for each of a plurality of different sections in the audio signal. The score is calculated according to the above equation (1). The specifying unit 153 specifies a correspondence between a frame and a phoneme based on the score calculated by the score calculating unit 152 for each of the plurality of sections. Then, the determination unit 160 determines the section in which the target word is emitted from the plurality of sections by comparing the scores of the correspondence relationships specified for each of the plurality of sections by the specifying unit 153.

  In the above embodiment, the output probability acquisition unit 130 acquires the output probabilities for all phonemes of the acoustic model stored in the acoustic model storage unit 21. However, in the present invention, the output probability obtaining unit 130 only needs to obtain the output probabilities for at least each of the plurality of phonemes corresponding to the plurality of candidate words stored in the candidate word storage unit 22. In other words, the output probability obtaining unit 130 may obtain the output probabilities for at least some of the phonemes used in speech recognition among all phonemes of the acoustic model, and obtain the output probabilities for the phonemes not used in speech recognition. You don't have to.

  In the present invention, the voice recognition device 10 does not have to include all the components shown in FIG. For example, the acoustic model storage unit 21 or the candidate word storage unit 22 may be provided in a device external to the speech recognition device 10. In this case, the speech recognition device 10 communicates with an external device as necessary, thereby obtaining information on the acoustic model stored in the acoustic model storage unit 21 or a plurality of candidate words stored in the candidate word storage unit 22. Get the information of.

  Further, in the present invention, the voice recognition device 10 may not have the function of the feature amount calculation unit 120, the output probability acquisition unit 130, or the conversion unit 140. For example, the external device has a function of converting a plurality of candidate words stored in the candidate word storage unit 22 into a corresponding phoneme sequence, and the speech recognition device 10 outputs a phoneme corresponding to each candidate word from the external device. Column information may be obtained. Alternatively, an external device executes a process of calculating the feature amount of the audio signal by the feature amount calculation unit 120 or a process of acquiring the output probability by the output probability acquisition unit 130, and the speech recognition device 10 outputs information indicating the result. May be obtained from an external device.

  In the above embodiment, the control unit 11 functions as each unit shown in FIG. 2 by the CPU executing the program stored in the ROM. However, in the present invention, the control unit 11 includes dedicated hardware such as an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), and various control circuits instead of the CPU. However, it may function as each unit shown in FIG. In this case, each function of each unit may be realized by individual hardware, or the function of each unit may be realized by a single piece of hardware. In addition, a part of the function of each unit may be realized by dedicated hardware, and the other part may be realized by software or firmware.

  It should be noted that the present invention can be provided not only as a speech recognition device having a configuration for realizing the function according to the present invention but also as an existing information processing device or the like as a speech recognition device according to the present invention by applying a program. You can also. That is, by applying a program for realizing each functional configuration by the voice recognition device 10 exemplified in the above embodiment so that a CPU or the like that controls an existing information processing device or the like can execute the voice according to the present invention. It can function as a recognition device.

  The method of applying such a program is arbitrary. The program can be applied by being stored in a computer-readable storage medium such as a flexible disk, a CD (Compact Disc) -ROM, a DVD (Digital Versatile Disc) -ROM, and a memory card. Furthermore, the program can be superimposed on a carrier wave and applied via a communication medium such as the Internet. For example, the program may be posted on a bulletin board (BBS: Bulletin Board System) on a communication network and distributed. Then, this program may be activated and executed in the same manner as other application programs under the control of an OS (Operating System), so that the above-described processing can be executed.

As described above, the preferred embodiments of the present invention have been described, but the present invention is not limited to the specific embodiments, and the present invention includes the invention described in the claims and the equivalents thereof. included. Hereinafter, the invention described in the claims of the present application is additionally described.
(Appendix 1)
The output probability output from each of the plurality of phonemes corresponding to the target word to be determined whether or not the feature amount of the audio signal is issued in the audio signal, for each of the plurality of frames in the audio signal Output probability obtaining means for obtaining;
A score for calculating a likelihood of the correspondence between the plurality of frames and the plurality of phonemes based on the output probability acquired by the output probability acquisition unit, for each of the plurality of candidates of the correspondence. Calculating means;
Identification means for identifying any of the plurality of candidates as the correspondence relationship based on the score calculated for each of the plurality of candidates by the score calculation means,
Determining means for determining whether or not the target word is issued in the audio signal, based on the correspondence specified by the specifying means;
With
The score calculation unit normalizes a value based on the output probability of a phoneme corresponding to each frame in each of the plurality of candidates by the number of frames corresponding to each phoneme, and accumulates the value over the plurality of frames. By calculating the score,
A speech recognition device characterized by the above-mentioned.
(Appendix 2)
The score calculation means calculates the score for each of a plurality of candidates having a correspondence relationship between m frames from the beginning of the plurality of frames and n phonemes from the beginning of the plurality of phonemes. Calculate,
The specifying means, based on the score calculated by the score calculation means for each of the plurality of candidates, any one of the plurality of candidates, the m frames and the n phonemes Specified as the correspondence,
A repetition unit that repeats the processes of the score calculation unit and the identification unit while changing at least one of the value of m in the m frames and the value of n in the n phonemes,
3. The speech recognition device according to claim 1, wherein:
(Appendix 3)
The score calculation means may include, as the plurality of candidates for the correspondence relationship between the m frames and the n phonemes, the (m-1) th frame to the mth frame from the top of the plurality of frames. The score is calculated for each of a first candidate in the case where the state stops over the frame and a second candidate in the case where the state transitions from the (m-1) th frame to the mth frame. ,
The specifying unit is configured to determine which of the first candidate and the second candidate based on the score calculated for each of the first candidate and the second candidate by the score calculating unit. One of them is specified as the correspondence between the m frames and the n phonemes,
3. The speech recognition device according to claim 2, wherein:
(Appendix 4)
The score calculation means,
A value based on the output probability of the k-th phoneme from the head of the n phonemes is accumulated over at least one frame corresponding to the k-th phoneme, and is calculated by the number of the at least one frame. By normalizing, the score of the k-th phoneme is calculated,
By accumulating the scores of the n phonemes calculated when the value of k in the k-th phoneme is 1 to n, the correspondence between the m frames and the n phonemes is calculated. Calculating a score for each of the plurality of candidates in the relationship;
The speech recognition device according to claim 2 or 3, wherein:
(Appendix 5)
The score calculation means calculates the remaining frames excluding the frames corresponding to the (n-1) th phoneme from the first phoneme of the n phonemes of the m frames. Using at least one frame corresponding to the n-th phoneme from the beginning of the n phonemes, calculating the score of the n-th phoneme;
The speech recognition device according to claim 4, wherein
(Appendix 6)
A feature value calculating unit configured to calculate the feature value of the audio signal for each frame,
The output probability obtaining unit obtains the output probability based on the feature amount calculated by the feature amount calculating unit,
6. The speech recognition device according to any one of supplementary notes 1 to 5, wherein:
(Appendix 7)
Audio signal acquisition means for acquiring the audio signal to be recognized,
Candidate word storage means in which a plurality of candidate words that are candidates for words emitted in the audio signal acquired by the audio signal acquisition means are stored,
The output probability obtaining means obtains the output probability, each of the plurality of candidate words stored in the candidate word storage means as the target word,
The score calculating means calculates the score based on the output probability obtained by the output probability obtaining means, for each of the plurality of candidate words,
The specifying unit specifies, for each of the plurality of candidate words, the correspondence based on the score calculated by the score calculating unit,
The determination unit is configured to compare any of the scores of the correspondence specified for each of the plurality of candidate words by the specifying unit, thereby causing any of the plurality of candidate words to be emitted in the audio signal. Judge as the said word,
7. The speech recognition device according to any one of supplementary notes 1 to 6, wherein:
(Appendix 8)
The score calculation unit calculates the score based on the output probability acquired by the output probability acquisition unit in each of a plurality of different sections in the audio signal to be searched,
The specifying unit specifies, for each of the plurality of sections, the correspondence based on the score calculated by the score calculating unit,
The determining unit determines a section in which the target word is emitted from the plurality of sections by comparing the scores of the correspondence relationships specified for each of the plurality of sections by the specifying unit. ,
7. The speech recognition device according to any one of supplementary notes 1 to 6, wherein:
(Appendix 9)
The output probability output from each of the plurality of phonemes corresponding to the target word to be determined whether or not the feature amount of the audio signal is issued in the audio signal, for each of the plurality of frames in the audio signal An output probability obtaining step to obtain;
A score for calculating a likelihood of the correspondence between the plurality of frames and the plurality of phonemes based on the output probability acquired in the output probability acquisition step, for each of the plurality of candidates of the correspondence. A calculating step;
A specifying step of specifying any of the plurality of candidates as the correspondence relationship based on the score calculated for each of the plurality of candidates in the score calculating step;
A determining step of determining whether or not the target word is issued in the audio signal based on the correspondence relationship identified in the identifying step;
Including
In the score calculating step, in each of the plurality of candidates, a value based on the output probability of a phoneme corresponding to each frame is normalized by the number of frames corresponding to each phoneme, and accumulated over the plurality of frames. By calculating the score,
A speech recognition method characterized in that:
(Appendix 10)
Computer
The output probability output from each of the plurality of phonemes corresponding to the target word to be determined whether or not the feature amount of the audio signal is issued in the audio signal, for each of the plurality of frames in the audio signal Output probability obtaining means to obtain,
A score for calculating a likelihood of the correspondence between the plurality of frames and the plurality of phonemes based on the output probability acquired by the output probability acquisition unit, for each of the plurality of candidates of the correspondence. Calculation means,
Specifying means for specifying any of the plurality of candidates as the correspondence, based on the score calculated for each of the plurality of candidates by the score calculating means,
Determining means for determining whether or not the target word is issued in the audio signal, based on the correspondence specified by the specifying means;
Function as
The score calculation unit normalizes a value based on the output probability of a phoneme corresponding to each frame in each of the plurality of candidates by the number of frames corresponding to each phoneme, and accumulates the value over the plurality of frames. By calculating the score,
A program characterized by the following.

DESCRIPTION OF SYMBOLS 10 ... Voice recognition apparatus, 11 ... Control part, 12 ... Storage part, 13 ... Input part, 14 ... Output part, 15 ... Communication part, 21 ... Acoustic model storage part, 22 ... Candidate word storage part, 30, 31, 32 ... Distance table, 110 ... Sound signal acquisition unit, 120 ... Feature amount calculation unit, 130 ... Output probability acquisition unit, 140 ... Conversion unit, 150 ... Search unit, 151 ... Designation unit, 152 ... Score calculation unit, 153 ... Specification unit 154: Repeating unit 160: Judging unit

Claims (10)

The output probability output from each of the plurality of phonemes corresponding to the target word to be determined whether or not the feature amount of the audio signal is issued in the audio signal, for each of the plurality of frames in the audio signal Output probability obtaining means for obtaining;
A score for calculating a likelihood of the correspondence between the plurality of frames and the plurality of phonemes based on the output probability acquired by the output probability acquisition unit, for each of the plurality of candidates of the correspondence. Calculating means;
Identification means for identifying any of the plurality of candidates as the correspondence relationship based on the score calculated for each of the plurality of candidates by the score calculation means,
Determining means for determining whether or not the target word is issued in the audio signal, based on the correspondence specified by the specifying means;
With
The score calculation unit normalizes a value based on the output probability of a phoneme corresponding to each frame in each of the plurality of candidates by the number of frames corresponding to each phoneme, and accumulates the value over the plurality of frames. By calculating the score,
A speech recognition device characterized by the above-mentioned. The score calculation means calculates the score for each of a plurality of candidates having a correspondence relationship between m frames from the beginning of the plurality of frames and n phonemes from the beginning of the plurality of phonemes. Calculate,
The specifying means, based on the score calculated by the score calculation means for each of the plurality of candidates, any one of the plurality of candidates, the m frames and the n phonemes Specified as the correspondence,
A repetition unit that repeats the processes of the score calculation unit and the identification unit while changing at least one of the value of m in the m frames and the value of n in the n phonemes,
The speech recognition device according to claim 1, wherein: The score calculation means may include, as the plurality of candidates for the correspondence relationship between the m frames and the n phonemes, the (m-1) th frame to the mth frame from the top of the plurality of frames. The score is calculated for each of a first candidate in the case where the state stops over the frame and a second candidate in the case where the state transitions from the (m-1) th frame to the mth frame. ,
The specifying unit is configured to determine which of the first candidate and the second candidate based on the score calculated for each of the first candidate and the second candidate by the score calculating unit. One of them is specified as the correspondence between the m frames and the n phonemes,
The speech recognition device according to claim 2, wherein: The score calculation means,
A value based on the output probability of the k-th phoneme from the head of the n phonemes is accumulated over at least one frame corresponding to the k-th phoneme, and is calculated by the number of the at least one frame. By normalizing, the score of the k-th phoneme is calculated,
By accumulating the scores of the n phonemes calculated when the value of k in the k-th phoneme is 1 to n, the correspondence between the m frames and the n phonemes is calculated. Calculating a score for each of the plurality of candidates in the relationship;
The speech recognition device according to claim 2 or 3, wherein: The score calculation means calculates the remaining frames excluding the frames corresponding to the (n-1) th phoneme from the first phoneme of the n phonemes of the m frames. Using at least one frame corresponding to the n-th phoneme from the beginning of the n phonemes, calculating the score of the n-th phoneme;
The speech recognition device according to claim 4, wherein: A feature value calculating unit configured to calculate the feature value of the audio signal for each frame,
The output probability obtaining unit obtains the output probability based on the feature amount calculated by the feature amount calculating unit,
The speech recognition device according to any one of claims 1 to 5, wherein: Audio signal acquisition means for acquiring the audio signal to be recognized,
Candidate word storage means in which a plurality of candidate words that are candidates for words emitted in the audio signal acquired by the audio signal acquisition means are stored,
The output probability obtaining means obtains the output probability, each of the plurality of candidate words stored in the candidate word storage means as the target word,
The score calculating means calculates the score based on the output probability obtained by the output probability obtaining means, for each of the plurality of candidate words,
The specifying unit specifies, for each of the plurality of candidate words, the correspondence based on the score calculated by the score calculating unit,
The determination unit is configured to compare any of the scores of the correspondence specified for each of the plurality of candidate words by the specifying unit, thereby causing any of the plurality of candidate words to be emitted in the audio signal. Judge as the said word,
The speech recognition device according to any one of claims 1 to 6, wherein: The score calculation unit calculates the score based on the output probability acquired by the output probability acquisition unit in each of a plurality of different sections in the audio signal to be searched,
The specifying unit specifies, for each of the plurality of sections, the correspondence based on the score calculated by the score calculating unit,
The determining unit determines a section in which the target word is emitted from the plurality of sections by comparing the scores of the correspondence relationships specified for each of the plurality of sections by the specifying unit. ,
The speech recognition device according to any one of claims 1 to 6, wherein: The output probability output from each of the plurality of phonemes corresponding to the target word to be determined whether or not the feature amount of the audio signal is issued in the audio signal, for each of the plurality of frames in the audio signal An output probability obtaining step to obtain;
A score for calculating a likelihood of the correspondence between the plurality of frames and the plurality of phonemes based on the output probability acquired in the output probability acquisition step, for each of the plurality of candidates of the correspondence. A calculating step;
A specifying step of specifying any of the plurality of candidates as the correspondence relationship based on the score calculated for each of the plurality of candidates in the score calculating step;
A determining step of determining whether or not the target word is issued in the audio signal based on the correspondence relationship identified in the identifying step;
Including
In the score calculating step, in each of the plurality of candidates, a value based on the output probability of a phoneme corresponding to each frame is normalized by the number of frames corresponding to each phoneme, and accumulated over the plurality of frames. By calculating the score,
A speech recognition method characterized in that: Computer
The output probability output from each of the plurality of phonemes corresponding to the target word to be determined whether or not the feature amount of the audio signal is issued in the audio signal, for each of the plurality of frames in the audio signal Output probability obtaining means to obtain,
A score for calculating a likelihood of the correspondence between the plurality of frames and the plurality of phonemes based on the output probability acquired by the output probability acquisition unit, for each of the plurality of candidates of the correspondence. Calculation means,
Specifying means for specifying any of the plurality of candidates as the correspondence, based on the score calculated for each of the plurality of candidates by the score calculating means,
Determining means for determining whether or not the target word is issued in the audio signal, based on the correspondence specified by the specifying means;
Function as
The score calculation unit normalizes a value based on the output probability of a phoneme corresponding to each frame in each of the plurality of candidates by the number of frames corresponding to each phoneme, and accumulates the value over the plurality of frames. By calculating the score,
A program characterized by the following.

Images