JP2008052178A - Voice recognition device and voice recognition method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide technology for suppressing incorrect recognition by appropriately asking back, when recognizing voice of a speaker. <P>SOLUTION: A voice recognition device is equipped with: a voice input means for inputting voice and converting it to voice data; an utterance section extracting means for extracting an utterance section; a voice analysis means for calculating a time sequence of a feature quantities of the voice; a word likelihood calculating means for calculating likelihood of each candidate word group; a sentence likelihood calculating means for calculating likelihood of each candidate sentence group; a reliability calculating means for calculating reliability of each candidate word group; a sentence specifying means for specifying a sentence which the speaker speaks, based on the reliability of the word included in the sentence; a first asking-back judgment means for judging whether the asking-back is required or not, based on the reliability of the word included in the specified sentence; and an asking-back means for asking back to the speaker, when it is judged that the asking-back to the speaker is required. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an apparatus and method for recognizing speech spoken by a dialog person.

  When a human controls the operation of the device, there is a technique for controlling the device by emitting a voice without operating an interface such as a keyboard or a lever. In such a technique, the content of words expressed by speech is recognized from speech input from speech input means such as a microphone, and control according to the recognized content of words is performed. When performing control using such voice recognition, it is important to suppress erroneous recognition as much as possible. If the device is controlled based on the erroneously recognized result, the device malfunctions.

  In order to prevent misrecognition in speech recognition, a technology has been developed that allows a conversation person to be heard back when speech is not successfully recognized. For example, Patent Document 1 discloses a voice recognition system that listens to a conversation person. In this system, an input speech is compared with an acoustic model of a word prepared in advance, and a word expressed by an acoustic model having a high coincidence rate with speech is recognized as a word spoken by a conversation person. In this system, when the coincidence rate between the voice and the acoustic model is lower than a predetermined threshold value, the conversation person is returned.

JP 2003-44756 A

  Even when the matching rate between the speech and the acoustic model is high, erroneous recognition may occur, and even when the matching rate is low, erroneous recognition may not occur. In general, the coincidence rate between a speech and an acoustic model tends to be a high value if the conversation person's utterance is clear, and a low value if the utterance is unclear. For example, when the voice of a clear talker such as an announcer is input, the match rate with the acoustic model of the word actually spoken by the talker is naturally highly evaluated, but the talker is speaking. There is a tendency that the coincidence rate with acoustic models of other similar words that are not high is also highly evaluated. In such a case, there are multiple candidate words with a high match rate, and the ambiguity about which words are actually spoken by the interlocutor is not resolved, which may lead to misrecognition. is there. Contrary to the above, when the voice of a conversation person whose utterance is unclear is input, the matching rate with the acoustic model is evaluated low for any candidate word. However, if the matching rate for one word is relatively large compared to the matching rate for other words, the candidate words are narrowed down, so that even if the matching rate is low, the recognition error Does not cause. In this case, you don't have to ask the talker.

  As described above, it is not possible to appropriately determine whether or not it is necessary to hear back in order to suppress misrecognition from the matching rate between the voice and the acoustic model. There is a need for a technique that can more appropriately determine whether it is necessary to hear back.

  The present invention solves the above problems. The present invention provides a technique capable of suppressing misrecognition by performing appropriate listening when recognizing a voice spoken by a conversation person.

  The present invention is embodied as an apparatus for recognizing voice spoken by a dialog person. The speech recognition apparatus according to the present invention includes a speech input unit that inputs speech and converts it into speech data, a speech segment extraction unit that extracts speech segments from the speech data, and a time series of speech feature values in the speech segments from speech data Speech analysis means for calculating the likelihood, word likelihood calculation means for calculating the likelihood for each of the candidate word groups from the time series of the feature amount of the speech in the utterance section, and the likelihood for each of the candidate word groups The sentence likelihood calculation means for calculating the likelihood for each candidate sentence group from the degree, and the likelihood for each candidate sentence group and the likelihood for each candidate word group become candidates. Candidates for calculating confidence for each word group and sentences spoken by the interlocutor in the utterance interval based on the certainty of words contained in the sentence are candidates. A sentence specifying means for specifying from among a sentence group, a first answer judging means for judging whether or not to ask a conversation person based on a certainty factor of a word included in the identified sentence, and a reply to the conversation person. It is provided with a means for listening back to the interlocutor when it is judged necessary.

  In the speech recognition apparatus of the present invention, the likelihood for each candidate word group and the likelihood for each candidate sentence group are calculated from the time series of speech feature values in the utterance section. As the audio feature quantity, for example, the frequency spectrum itself may be handled, or a mel frequency cepstrum coefficient (MFCC) may be handled. Then, the certainty factor for each word group is calculated from the likelihood for each word group and the likelihood for each sentence group. The certainty of a word is an index indicating how reliable the word is with respect to other competing word candidates. For details on the certainty of words, see, for example, Lee Shin-nobu, Tatsuya Kawahara, Kiyohiro Shikano, “High-speed word posterior probability calculation method in 2-pass search algorithm”, IEICE Technical Report, The Institute of Electronics, Information and Communication Engineers December 2003, SP 2003-160, p. 35-40 etc. In the speech recognition apparatus of the present invention, a sentence spoken by a dialogue person is specified from a group of candidate sentences based on the certainty of words included in the sentence. According to the speech recognition apparatus of the present invention, it is possible to accurately recognize a sentence spoken by a conversation person by specifying a sentence including many words with high certainty as a sentence spoken by the conversation person.

  In the speech recognition apparatus of the present invention, it is determined whether or not it is necessary to hear back to the conversation person based on the certainty factor of the word included in the sentence specified as the sentence spoken by the conversation person. When the certainty of the words included in the sentence is low, it is considered that there is not so much difference in likelihood between the specified sentence and the other sentences, so the conversation person is asked. On the other hand, when the certainty factor of the word included in the sentence is high, the identified sentence is considered to have a relatively high likelihood compared to the other sentences, and therefore no replay is performed. In this way, the speech recognition apparatus of the present invention determines whether or not it is necessary to listen back by paying attention to the certainty of the words included in the sentence. Misrecognition in speech recognition can be suppressed.

  The voice recognition apparatus further includes a volume detection unit that detects a volume in the utterance section from the voice data, and a second listen determination unit that determines whether or not a conversation person needs to be answered based on the volume in the utterance section. It is preferable.

  In general, a range of loudness that can be detected is defined in the voice input means, and a voice that does not fall within this range cannot be accurately detected. Therefore, if the conversation person's voice is too loud or too small, accurate speech recognition cannot be performed. In the above speech recognition device, it is possible to suppress misrecognition caused by the fact that speech cannot be accurately detected by the speech input means by determining whether or not to listen back based on the detected volume.

  In the above speech recognition apparatus, it is preferable that the second answer determination unit determines that the answer to the talker is required when the volume in the utterance section exceeds the upper limit value.

  The voice input means has an upper limit of the loudness of detectable voice. Sound that exceeds the upper limit cannot be detected accurately. Therefore, if the voice of the interlocutor is too loud, the speech waveform cannot be accurately detected, and accurate speech recognition cannot be performed. According to the above speech recognition apparatus, when the volume is too high, the speaker is asked to talk back and speak with a lower voice. Thereby, it is possible to suppress misrecognition caused by the voice of the conversation person being too loud.

  In the above speech recognition apparatus, it is preferable that the second answer determination unit determines that the answer to the talker is necessary when the volume in the utterance section is less than the lower limit.

  The voice recognition device has a lower limit of the loudness of a detectable voice. Sound that is below the lower limit cannot be detected accurately. Therefore, when the voice of the interlocutor is too low, the voice waveform cannot be accurately detected, and accurate voice recognition cannot be performed. According to the above speech recognition apparatus, when the volume is too low, the speaker is asked to listen back and speak to a louder voice. Thereby, the misrecognition caused by the voice of the conversation person being too low can be suppressed.

  The above speech recognition apparatus further includes a conversation person identifying means for identifying a person who is a conversation person from a group of candidate persons, and a threshold value setting means for setting a threshold value according to the identified person. When the average value of the certainty level of words that are independent words among the words included in the specified sentence is less than the threshold value, It is desirable to judge that it is necessary.

  According to the above speech recognition apparatus, the threshold value used when determining whether or not to hear back can be set individually according to the interlocutor. By adopting such a configuration, it is possible to appropriately determine whether it is necessary to listen back to a wide variety of interlocutors.

  The present invention is also embodied as a speech recognition method. The method of the present invention includes a speech input step of inputting speech and converting it into speech data, a speech segment extraction step of extracting speech segments from the speech data, and calculating a time series of speech feature values in the speech segments from speech data From the speech analysis step, the word likelihood calculation step of calculating the likelihood for each candidate word group from the time series of the speech feature amount in the utterance section, and the likelihood for each of the candidate word groups A word likelihood candidate step based on a sentence likelihood calculating step for calculating a likelihood for each candidate sentence group, and a likelihood for each candidate sentence group and a likelihood for each candidate word group. A confidence level calculation step for calculating the confidence level for each of the above, and a sentence group in which the sentences spoken by the talker in the utterance interval based on the confidence level of the words included in the sentence are candidates It was determined that a sentence identification process to identify from the inside, a rehearsal determination process to determine whether or not to ask the conversation person to answer based on the certainty of the words contained in the identified sentence, and a conversation person need to be heard back In this case, there is provided a process of replaying back to the interlocutor.

  According to the voice recognition device and the voice recognition method of the present invention, it is possible to suppress misrecognition by performing an appropriate response when recognizing the voice spoken by the conversation person.

The best mode for carrying out the invention is listed below.
(Mode 1) The word likelihood calculating means calculates a likelihood for each candidate word group from a time series of speech feature values in an utterance interval using a hidden Markov model (HMM). calculate.

  In this embodiment, an example will be described in which the voice recognition apparatus 100 illustrated in FIG. The voice recognition device 100 is a guidance robot arranged in, for example, a showroom or an event venue, and recognizes a voice spoken by a visitor (interactive person) V who talks for guidance.

  The speech recognition apparatus 100 includes a head 102, a body part 108, and an arm part 116. The voice recognition device 100 includes a right camera 104 and a left camera 106 arranged side by side in front of the head 102, a speaker 118 arranged in front of the head 102, and the head 102 and arms with respect to the body portion 108. An actuator group 110 for driving the unit 116; a microphone 112 provided in front of the body unit 108; and a controller 114 for controlling the operations of the right camera 104, the left camera 106, the speaker 118, the actuator group 110, and the microphone 112. Yes.

  The right camera 104 and the left camera 106 are general CCD cameras. The right camera 104 and the left camera 106 repeatedly shoot at a predetermined time interval, and output the shot image data to the controller 114.

  The microphone 112 detects the sound pressure applied to the membrane surface by the input sound, and outputs a voltage value corresponding to the detected sound pressure to the controller 114.

  The speaker 118 amplifies the signal transmitted from the controller 114 by an amplifier, vibrates the diaphragm according to the fluctuation of the amplified current, and outputs sound.

  Actuator group 110 drives head 102 and arm 116 based on a control signal transmitted from controller 114.

  FIG. 2 is a block diagram showing the configuration of the controller 114. The controller 114 is a computer device that includes a processing device (CPU), a storage device (an optical storage medium, a magnetic storage medium, or a semiconductor memory such as a RAM or a ROM), an input / output device, an arithmetic device, and the like.

  The image A / D conversion unit 202 A / D converts the image data input from the right camera 104 to generate digital image data. Hereinafter, the digital image data generated by the image A / D conversion unit 202 is referred to as right digital image data. The image A / D conversion unit 202 transmits the generated right digital image data to the image recognition unit 206. The image A / D conversion unit 204 A / D converts image data input from the left camera 106 to generate digital image data. Hereinafter, the digital image data generated by the image A / D conversion unit 204 is referred to as left digital image data. The image A / D conversion unit 204 transmits the generated left digital image data to the image recognition unit 206.

  Based on the right digital image data input from the image A / D conversion unit 202 and the left digital image data input from the image A / D conversion unit 204, the image recognition unit 206 is a person database (hereinafter referred to as a database DB). The conversation person V is identified using 208. In the person DB 208, an identification code indicating a person who is a candidate for the conversation person V and a positional relationship between the feature points of the face of the person are registered in association with each other. The image recognition unit 206 extracts the feature points of the face of the captured conversation person V from the right digital image data and the left digital image data, calculates the positional relationship of the extracted feature points, and is registered in the person DB 208. The person with the most similar positional relationship of the facial feature points is searched from the selected persons.

  Each time the right camera 104 and the left camera 106 photograph, the image recognition unit 206 identifies the person identified as the conversation person V based on the image data photographed at the same time by the right camera 104 and the left camera 106. Is identified. The image recognizing unit 206 associates the identification code of the conversation person V with the shooting time, and outputs it to the first listen determination unit 234.

  The audio A / D converter 210 A / D converts the temporal change in the sound pressure input from the microphone 112 to generate digital audio data. The voice A / D conversion unit 210 outputs the generated digital voice data to the utterance section extraction unit 212, the volume detection unit 214, and the voice analysis unit 216.

  The utterance section extraction unit 212 detects the utterance start time and the utterance end time from the digital voice data input from the voice A / D conversion unit 210. FIG. 3 shows an example of the speech waveform 302 indicated by the digital speech data input to the speech segment extraction unit 212. In a situation where the start of utterance is not detected, the utterance section extraction unit 212 monitors whether or not the sound pressure in the speech waveform 302 exceeds a predetermined threshold value P1. More specifically, when the average sound pressure exceeds the threshold value P1 and the number of times that the speech waveform 302 intersects the zero sound pressure line 304 becomes equal to or greater than a predetermined number in the unit time T1, the speech segment extraction is performed. Unit 212 determines that the utterance has started. When the start of the utterance is detected, the utterance section extraction unit 212 specifies the utterance start time TS, and notifies the volume detection unit 214 and the voice analysis unit 216 of the utterance start time TS.

  The utterance section extraction unit 212 counts the number of times the speech waveform 302 intersects the zero sound pressure line 304 in the unit time T2, and monitors whether the counted number reaches a predetermined threshold value. Specifically, in the unit time T2, when the number of times counted does not reach the predetermined threshold value and the average sound pressure does not reach the predetermined threshold value P2, the utterance section extraction unit 212 ends the utterance. Judge that When the end of the utterance is detected, the utterance section extracting unit 212 specifies the utterance end time TE, and notifies the volume detecting unit 214 and the voice analyzing unit 216 of the utterance end time TE.

  Based on the digital audio data input from the audio A / D converter 210, the sound volume detector 214 integrates the square value of the sound pressure between the utterance start time TS and the utterance end time TE. When the utterance start time TS is notified from the utterance section extraction unit 212, the sound volume detection unit 214 starts to accumulate the square value of the sound pressure. When the utterance end time TE is notified from the utterance section extraction unit 212, the sound volume detection unit 214 ends the integration of the square value of the sound pressure. Then, the integrated value of the square value of the sound pressure is divided by the length TE-TS of the utterance section, and the average sound volume in the utterance section is specified. Thereafter, the sound volume detection unit 214 outputs the average sound volume in the utterance section to the second listen-back determination unit 218.

  Based on the average sound volume in the utterance interval input from the sound volume detection unit 214, the second rehearsal determination unit 218 determines whether or not the conversation person V needs to be replayed. First, the second listening determination unit 218 compares the average sound volume in the utterance section with a predetermined upper limit value. When the average sound volume exceeds the upper limit value, the second answer determination unit 218 determines that the voice of the conversation person V in the utterance section is too loud and accurate voice recognition cannot be performed. In this case, the second response determination unit 218 instructs the response determination unit 240 to perform a response prompting the conversation person V to speak with a smaller voice. Next, the second listening determination unit 218 compares the average volume in the utterance section with a predetermined lower limit value. When the average volume is lower than the lower limit value, the second hearing determination unit 218 determines that the voice of the conversation person V in the utterance section is too low to perform accurate voice recognition. In this case, the second answer determination unit 218 instructs the response determination unit 240 to ask the conversation person V to speak more loudly. The above processing by the second listening determination unit 218 is performed every time the utterance end time TE is detected by the utterance section extraction unit 212 and the average volume in the utterance section is input from the volume detection unit 214.

  The voice analysis unit 216 calculates a time series of voice feature values between the utterance start time TS and the utterance end time TE. The voice analysis unit 216 according to the present embodiment performs framing processing on the input digital voice data, and specifies the frequency spectrum of the sound data corresponding to each frame. FIG. 4 shows how voice data is framed and how the frequency spectrum of the voice data of each frame is specified. In this embodiment, the frame length is 20 ms, and the frame interval is 10 ms. As shown in FIG. 4, frames F1, F2, F3,. The voice analysis unit 216 identifies the frequency spectrums f1, f2, f3,... Of the voice data 402 in each of the frames F1, F2, F3,. The frequency spectrum is given as a distribution of amplitude with respect to frequency. The specification of the frequency spectrum can be performed using, for example, a fast Fourier transform. When the utterance start time TS is notified from the utterance section extraction unit 212, the voice analysis unit 216 starts the framing process and the frequency spectrum specifying process. The voice analysis unit 216 sequentially executes the above processes until the utterance end time TE is notified, and sequentially outputs the frequency spectrum of each frame to the phoneme likelihood calculation unit 220. When the utterance end time TE is notified from the utterance section extraction unit 212, the voice analysis unit 216 ends the framing process and the frequency spectrum specifying process.

  The phoneme likelihood calculating unit 220, the word likelihood calculating unit 224, the sentence likelihood calculating unit 228, the certainty factor calculating unit 232, and the sentence specifying unit 238 are hidden from the frequency spectrum for each frame input from the speech analyzing unit 216. A sentence as a time series of phonemes is specified using a Markov model (HMM). Here, the phoneme means an element that constitutes a voice uttered when a human speaks a word. For example, a voice uttered when a person speaks the word “grape” is composed of four phonemes “b”, “u”, “d”, and “o:”. When specifying a phoneme time series using the HMM, it is assumed that one phoneme is composed of a plurality of states, transition probabilities of transitioning each state to the next state, and transition to the next state are not made. Characterized by the probability of stopping at In the following, the phoneme state is described as a phoneme state. In this embodiment, an example in which one phoneme is composed of three phoneme states will be described. For example, the phoneme “b” is composed of phoneme states b1, b2, and b3. The phoneme state “b” is realized by making a transition from a phoneme state to the phoneme state b1, transitioning from the phoneme state b1 to the phoneme state b2, and transitioning from the phoneme state b2 to the phoneme state b3. The phoneme state b1 may transit to the phoneme state b2 that is the next phoneme state, or may remain in the phoneme state b1. The same applies to the phoneme states b2 and b3. In this embodiment, a phoneme is specified as a phoneme state time series, a word is specified as a phoneme time series, and a sentence is specified as a word time series. In this embodiment, the likelihood of words and sentences as a time series of phoneme states is calculated, the certainty of words included in the sentences is calculated based on the likelihood of words and sentences, and the certainty of words Based on the above, the sentence spoken by the conversation person V is specified.

  The phoneme likelihood calculating unit 220 evaluates the likelihood of each phoneme state for the frame from the frequency spectrum specified for each frame. Each phoneme state has a probability distribution for the frequency spectrum observed as speech when that phoneme state is realized. This probability distribution can be acquired in advance by experiments or the like. From this probability distribution and the frequency spectrum specified for the frame, the likelihood of the phoneme state for the frame can be calculated. In the present embodiment, a function for calculating likelihood from a frequency spectrum is stored in advance in the phoneme DB 222 for each phoneme state of each phoneme to be subjected to likelihood evaluation. The phoneme likelihood calculation unit 220 calculates the likelihood for each phoneme state of each phoneme for each of the frequency spectra f1, f2, f3,. For example, the likelihoods of the phoneme states b1, b2, and b3 of the phoneme “b” for the frame F1 are calculated from the frequency spectrum f1 of the frame F1. Similarly for the phoneme states of other phonemes, the likelihood for the frame F1 is calculated. Similarly for the subsequent frames F2, F3,..., The likelihood of each phoneme state of each phoneme for that frame is calculated.

  When the likelihood of each phoneme state for each frame is calculated, the word likelihood calculation unit 224 evaluates the likelihood of each phoneme and the likelihood of each word. The evaluation of the likelihood of each phoneme and the evaluation of the likelihood of each word will be described with reference to FIG. FIG. 5 illustrates an example in which the likelihood of the word “grape” is evaluated as an example. In the left column of FIG. 5, the word “grape” is configured as a sequence of phonemes “b”, “u”, “d”, “o:”, and the phoneme “b” is the phoneme state b1, b2, b3. Phoneme “u” is configured as a sequence of phoneme states u1, u2, u3, phoneme “d” is configured as a sequence of phoneme states d1, d2, d3, and phoneme “ It is shown that “o:” is configured as a sequence of phoneme states o: 1, o: 2, and o: 3. In FIG. 5, the state where the phoneme state b1 is realized in the frame F1 is expressed by a point 502, and the phoneme states b1, b2, b3,... Are realized in the subsequent frames F2, F3,. Are expressed by points 504, 506, 508, 510, 512. Further, from each of the points 502, 504, 506,..., A path for transitioning to the next phoneme state in the next frame and a path for stopping without transitioning to the next phoneme state extend. For example, from a point 502 indicating a state in which the phoneme state b1 is realized in the frame F1, a branch 514 that makes a transition to the next phoneme state b2 in the next frame F2, and a phoneme state b1 without making a transition to the next phoneme state b2 A branch 516 that stops at is extended. The branch 514 extends to a point 504 indicating a state where the phoneme state b2 is realized in the frame F2. The branch 516 extends to a point 506 indicating a state in which the phoneme state b1 is realized in the frame F2.

  The likelihood of each point 502, 504, 506,... In FIG. 5 can be calculated as the likelihood of each phoneme state for each frame. The likelihood of each branch 514, 516,... Can be calculated from the transition probability and stationary probability of each phoneme state. For example, the likelihood of the branch 514 can be calculated from the transition probability from the phoneme state b1 to the phoneme state b2. The likelihood of the branch 516 can be calculated from the retention probability of the phoneme state b1. The transition probability and the retention probability of each phoneme state of each phoneme constituting the word are acquired in advance by an experiment or the like and stored in the word DB 226.

The word likelihood calculation unit 224 calculates the likelihood of the points 502, 504, 506,... Calculated by the likelihood of each phoneme state for each frame, and the branches 514, 516,. ... Based on the likelihoods, the likelihoods are calculated for all possible routes at that time, and the route with the highest likelihood is specified. Here, the likelihood for a route refers to the likelihood that an event has progressed along the route. The likelihood that an event has progressed along a route can be calculated from the likelihood of points included in the route and the likelihood of branches. When the route with the highest likelihood in the word is specified, the word likelihood calculating unit 224 specifies the likelihood that the event has progressed along the route as the likelihood of the word.
In the example shown in FIG. 5, the route 518 is specified as the route with the highest likelihood in the word “grape” at the time when the processing is progressing to the frames F1, F2,. In such a case, the likelihood that the event has progressed along the path 518 is specified as the likelihood of the word “grape”. The likelihood that the event has progressed along the path 518 is calculated from the likelihood of the points 502, 504, 510,... Included in the path 518 and the likelihood of the branches 514,.

  The word DB 226 stores a transition probability and a retention probability of each phoneme state of each phoneme constituting the word for each word group assumed as a word spoken by the conversation person V. Although FIG. 5 illustrates an example in which the likelihood of the word “grape” is evaluated, the word likelihood calculating unit 224 performs the above-described likelihood evaluation for all the words stored in the word DB 226. Thereby, the likelihood about all the words memorize | stored in word DB226 is evaluated.

In parallel with the likelihood evaluation for each word, the sentence likelihood calculating unit 228 evaluates the likelihood of each sentence. The sentence likelihood calculation unit 228 evaluates the likelihood for all sentences stored in the sentence DB 230. In the sentence DB 230, for each sentence group assumed as a sentence spoken by the conversation person V, a series of words constituting the sentence is stored in association with each other.
FIG. 6 shows how the likelihood of a sentence is evaluated. In the example shown in FIG. 6, a series of words “Prius” (registered trademark) — “no” — “fuel consumption” — “ha” — “how much” — “what” constitutes one sentence. In addition, a series of words “Prius”-“No”-“Fuel consumption”-“O”-“Teach me”-“Please” make up one sentence. These sentences and a series of words constituting the sentences are stored in the sentence DB 230 in advance.

  The sentence likelihood calculating unit 228 calculates the likelihood of the sentence from the likelihood of the word included in the sentence and the connection probability from the word to the word in the sentence. The connection probability from word to word is specified using the word connection table 700 shown in FIG. The word connection table 700 has a probability (denoted as an appearance rate in the figure) that a connection from a certain word (denoted as the previous word in the figure) to the next word (denoted as the subsequent word in the figure) appears. Is described). The probability that such a word-to-word connection appears can be obtained by experiments or the like. The word connection table 700 is stored in the sentence DB 230 in advance, and the sentence likelihood calculation unit 228 reads the word connection table 700 from the sentence DB 230 as necessary. The sentence likelihood calculating unit 228 evaluates the likelihood for all sentences stored in the sentence DB 230.

  When the likelihood is evaluated for each sentence, the certainty factor calculation unit 232 calculates the certainty factor for each word based on the likelihood for each sentence and the likelihood of each word included in each sentence. The certainty of a word is an index indicating how reliable the word is with respect to other competing word candidates. For the time series X of speech, the certainty factor C that the period from time τ to t is the word w is calculated by the following equation.

  Here, W indicates a sentence, and W [w; τ, t] indicates a set of sentences including the word w in the period from time τ to t. g (W) represents the likelihood of the sentence W in logarithm. α is a positive number less than 1 called a smoothing coefficient. p (X) indicates the likelihood that the time series of the speech is X, and is given here as the sum of the likelihoods of all sentences.

  The certainty factor calculation unit 232 calculates the certainty factor for each of the words included in the sentence for each sentence. The certainty factor calculation unit 232 calculates an average value of the certainty factor of the independent word for each sentence. The certainty factor calculation unit 232 associates each sentence with the average value of the certainty factor of the independent word of the sentence and outputs the associated value to the sentence specifying unit 238.

  The sentence specifying unit 238 specifies the sentence having the highest average confidence level of the independent word as the sentence spoken by the conversation person V. The sentence specifying unit 238 outputs the specified sentence to the correspondence determining unit 240. In addition, the sentence specifying unit 238 outputs the specified sentence and the average value of the certainty level of the independent word of the sentence to the first listening determination unit 234.

  The first answer determination unit 234 determines whether it is necessary to answer the conversation person V from the sentence input from the sentence specifying unit 238 and the average value of the confidence level of the independent word of the sentence. The sentence input from the sentence specifying unit 238 has the highest average confidence level of independent words in the candidate sentence group. When the confidence level of the independent word of the sentence is high, the likelihood of the sentence is much higher than the likelihood of other sentences, and it can be said that there is not much ambiguity in the recognition result. Therefore, in such a case, it is considered that the content of the sentence spoken by the conversation person V can be accurately recognized without having to ask the conversation person V back. On the contrary, when the average value of the confidence level of the independent word of the sentence input from the sentence specifying unit 238 is low, the likelihood of the other sentence and the likelihood of the sentence is not so large, and the recognition result It can be said that there is ambiguity. Therefore, in such a case, it is necessary to listen to the conversation person V and recognize the sentence spoken by the conversation person V more accurately.

  The first answer determination unit 234 determines whether or not the answer is necessary by comparing the average value of the certainty factor input from the sentence specifying unit 238 with a threshold value. When the average value of the certainty factor is equal to or greater than the threshold value, the first answer determination unit 234 determines that the answer is unnecessary. When the average value of the certainty level is less than the threshold value, the first answer determination unit 234 determines that the answer is necessary, and instructs the correspondence determination unit 240 to ask the conversation person V to speak more clearly. To do.

  The first listening determination unit 234 determines the threshold value used for the above determination based on the identification code of the conversation person V input from the image recognition unit 206. The first hearing determination unit 234 searches the certainty factor DB 236 using the identification code of the conversation person V as a key, and reads an appropriate threshold value for the person indicated by the identification code. An appropriate threshold value for a person is acquired in advance by an experiment or the like. The certainty factor DB 236 stores a person identification code and a threshold value appropriate for the person in association with each other. By adopting such a configuration, it is possible to appropriately determine whether or not it is necessary to listen back to any person who is the conversation person V.

  The speech analysis unit 216, phoneme likelihood calculation unit 220, word likelihood calculation unit 224, sentence likelihood calculation unit 228, certainty factor calculation unit 232, and sentence specification unit 238 perform the above-described framing processing to sentence data estimation. A series of processing is repeatedly performed until the utterance end time TE is notified from the utterance section extraction unit 212. When the utterance end time TE is notified from the utterance section extraction unit 212, the sentence specifying unit 238 outputs the sentence specified from the speech in the utterance section to the correspondence determining unit 240 as a character string.

  The correspondence determining unit 240 determines the correspondence to the conversation person V based on the presence / absence of the instruction to hear back from the second listening determination unit 218 and the first listening determination unit 234 and the character string input from the sentence specifying unit 238. To do.

  If the second answer determination unit 218 instructs the conversation person V to speak back with a smaller voice, the response determination section 240 “speak a little less loudly” as a reply to the conversation person V. Is output to the speech synthesizer 242. In addition, the correspondence determination unit 240 outputs an operation pattern indicating a gesture for pressing the arm unit 116 downward to the operation generation unit 244.

  When the second answer determination unit 218 instructs the conversation person V to speak back to speak in a louder voice, the response determination section 240 "speak a little louder" as a reply to the conversation person V. Is output to the speech synthesizer 242. In addition, the correspondence determination unit 240 outputs an operation pattern indicating a gesture that covers the periphery of the speaker 118 with the tip of the arm unit 116 to the operation generation unit 244.

  When the first answer determination unit 234 has instructed the conversation person V to speak more clearly, the response determination part 240 sends the character “Please speak a little more clearly” as a reply to the conversation person V. The sequence is output to the speech synthesizer 242. In addition, the correspondence determination unit 240 outputs to the motion generation unit 244 a motion pattern that indicates a gesture that spreads the arm unit 116 left and right and swings the head 102 left and right.

  When neither of the second answer determination unit 218 and the first answer determination unit 234 is instructed to answer, the correspondence determination unit 240 determines the conversation person V based on the character string of the sentence input from the sentence specifying unit 238. Determine the response to. In the correspondence DB 246, a character string of a sentence spoken by the conversation person V, a character string indicating an appropriate response voice for the sentence, and an operation pattern indicating an appropriate response action are stored in association with each other. The correspondence determining unit 240 searches the correspondence DB 246 using the character string of the text input from the text specifying unit 238 as a key, and determines a character string indicating an appropriate response voice and an operation pattern indicating an appropriate response action. The correspondence determination unit 240 outputs the determined character string to the speech synthesis unit 242 and outputs the determined operation pattern to the operation generation unit 244.

  The voice synthesizer 242 generates a response voice to the conversation person V as digital voice data based on the character string input from the correspondence determination unit 240. The voice synthesizer 242 outputs the generated digital voice data to the voice D / A converter 248.

  The voice D / A converter 248 performs D / A conversion on the digital voice data input from the voice synthesizer 242 and outputs it to the speaker 118. As a result, an appropriate response corresponding to the content of the sentence spoken by the conversation person V or a reply to the conversation person V is output from the speaker 118 by voice.

  The motion generation unit 244 drives the actuator group 110 based on the motion pattern input from the correspondence determination unit 240 to operate the head 102 and the arm unit 116.

  Processing performed by the controller 114 will be described with reference to the flowchart of FIG. In step S802, the process waits until the speech segment extraction unit 212 detects the start of speech. If the start of the utterance is detected in step S802, the controller 114 executes the processing shown in steps S804 to S814, the processing shown in step S816, and the processing shown in step S818 in parallel.

  First, the processing shown in steps S804 to S814 will be described. In step S804, the speech analysis unit 216 executes speech framing processing. In step S806, the voice analysis unit 216 specifies a frequency spectrum for each frame. In step S808, the phoneme likelihood calculation unit 220 calculates the likelihood for each phoneme state for each frame. In step S810, the word likelihood calculating unit 224 calculates the likelihood for each word from the likelihood for each phoneme state for each frame. In step S812, the sentence likelihood calculating unit 228 calculates the likelihood for each sentence from the likelihood for each word. In step S814, the certainty factor calculation unit 232 calculates the certainty factor for each word from the likelihood for each word and the likelihood for each sentence. Such processing from step S804 to step S814 is repeatedly executed until the end of the utterance is detected in step S822.

  In parallel with the above processing, in step S816, the sound volume detection unit 214 performs a process of integrating the square value of the sound pressure in the speech section. The sound pressure square value integration process is repeatedly executed until the end of the utterance is detected in step S822.

  Further, in parallel with the above processing, in step S818, the image recognition unit 206 identifies the conversation person V. The process of step S818 is repeatedly executed until the end of the utterance is detected in step S822.

  In step S822, it is determined whether or not the utterance section extraction unit 212 detects the end of the utterance. When the end of the utterance is detected in step S822, the processing in step S824 and subsequent steps is executed.

  In step S824, the second hearing determination unit 218 determines whether or not the voice of the conversation person V is too loud. If the voice of the conversation person V is too loud (YES in step S824), the process proceeds to step S830, and a rehearsal is performed to urge the conversation person V to speak with a smaller voice. If the conversation person V is not too loud (NO in step S824), the process proceeds to step S826.

  In step S826, the second hearing determination unit 218 determines whether or not the voice of the conversation person V is too low. When the voice of the conversation person V is too low (in the case of YES at step S826), the process proceeds to step S832, and a response is made to prompt the conversation person V to speak with a louder voice. If the conversation person V's voice is not too low (NO in step S826), the process proceeds to step S828.

  In step S828, the first hearing determination unit 234 determines whether the average value of the confidence level of the independent word in the sentence specified by the sentence specifying unit 238 is less than a threshold value. The threshold value at this time is determined according to the identification result of the conversation person V in step S818. When the average value of the confidence level of the independent words is less than the threshold value (in the case of YES in step S828), the process proceeds to step S834, and a response is made to prompt the conversation person V to speak more clearly. . If the average value of the confidence level of the independent words is equal to or greater than the threshold value (NO in step S828), the process proceeds to step S836, and an appropriate response corresponding to the sentence specified by the sentence specifying unit 238 is executed. .

Specific examples of the present invention have been described in detail above, but these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples illustrated above.
In addition, the technical elements described in the present specification or the drawings exhibit technical usefulness alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. In addition, the technology illustrated in the present specification or the drawings achieves a plurality of objects at the same time, and has technical utility by achieving one of the objects.

FIG. 1 is a view showing the appearance of the speech recognition apparatus 100. FIG. 2 is a diagram schematically showing the configuration of the controller 114. FIG. 3 is a diagram for explaining the detection of the utterance start time TS and the end time TE. FIG. 4 is a diagram for explaining framing processing of audio data 302 and identification of a frequency spectrum. FIG. 5 is a diagram for explaining the likelihood evaluation of the word “grape”. FIG. 6 is a diagram for explaining sentence likelihood evaluation. FIG. 7 is a diagram illustrating a word connection table 700. FIG. 8 is a flowchart for explaining processing performed by the controller 114.

Explanation of symbols

100: Voice recognition device 102: Head 104: Right camera 106: Left camera 108: Body 110: Actuator 112: Microphone 114: Controller 116: Arm 118: Speaker 202, 204: Image A / D converter 206: Image Recognition unit 208: person DB
210: Speech A / D conversion unit 212: Speech segment extraction unit 214: Volume detection unit 216: Speech analysis unit 218: Second answer determination unit 220: Phoneme likelihood calculation unit 222: Phoneme DB
224: Word likelihood calculation unit 226: Word DB
228: sentence likelihood calculating unit 230: sentence DB
232: Certainty factor calculation unit 234: First listening determination unit 236: Certainty factor DB
238: sentence specifying unit 240: correspondence determining unit 242: speech synthesizing unit 244: action generating unit 246: correspondence DB
248: D / A converter 302: voice waveform 304: zero sound pressure lines 502, 504, 506, 508, 510, 512: point 514, 516: branch 518: path 700: word connection table

Claims (6)

A device that recognizes the voice spoken by the interlocutor,
Voice input means for inputting voice and converting it into voice data;
Speech segment extraction means for extracting a speech segment from voice data;
Speech analysis means for calculating a time series of speech feature values in the utterance section from speech data;
Word likelihood calculating means for calculating a likelihood for each candidate word group from a time series of voice feature values in the utterance section;
Sentence likelihood calculating means for calculating the likelihood for each of the candidate sentence groups from the likelihood for each of the candidate word groups;
A certainty factor calculating means for calculating a certainty factor for each candidate word group from the likelihood for each candidate sentence group and the likelihood for each candidate word group;
A sentence identifying means for identifying a sentence spoken by a conversation person in an utterance section from candidate sentence groups based on the certainty of words included in the sentence;
Based on the certainty factor of the word included in the identified sentence, a first answer determination unit for determining whether or not to ask the dialogue person is necessary;
A speech recognition apparatus comprising a means for performing a replay to a conversation person when it is determined that a replay to the conversation person is necessary. Volume detection means for detecting the volume in the utterance section from the voice data;
The speech recognition apparatus according to claim 1, further comprising second listening / return determining means for determining whether or not to hear back to the conversation person based on the volume in the utterance section.   The speech recognition apparatus according to claim 2, wherein the second answer determination unit determines that the answer to the talker is required when the volume in the utterance section exceeds an upper limit value.   The speech recognition apparatus according to claim 3, wherein the second answer determination unit determines that the answer to the talker is required when the volume in the utterance section is less than the lower limit. A conversation person identifying means for identifying a person who is a conversation person from among a group of candidates,
A threshold setting means for setting a threshold according to the identified person;
If the average value of certainty of words that are independent words among the words included in the specified sentence is less than the threshold value, the first response determination unit determines that a response to the conversation person is necessary. The speech recognition apparatus according to claim 1. A method of recognizing the voice spoken by the interlocutor,
A voice input process for inputting voice and converting it into voice data;
An utterance interval extraction step of extracting an utterance interval from voice data;
A voice analysis step of calculating a time series of voice feature values in the utterance section from the voice data;
A word likelihood calculating step of calculating a likelihood for each candidate word group from a time series of voice feature values in an utterance section;
A sentence likelihood calculating step for calculating the likelihood for each of the candidate sentence groups from the likelihood for each of the candidate word groups;
A certainty factor calculation step for calculating a certainty factor for each candidate word group from the likelihood for each candidate word group and the likelihood for each candidate word group;
A sentence identifying step for identifying a sentence spoken by a dialogue person in an utterance section from candidate sentence groups based on the certainty of words included in the sentence;
Based on the certainty factor of the word included in the identified sentence, a replay determination step of determining whether or not a replay to the talker is necessary,
A speech recognition method comprising a replaying step of performing a replay to a conversation person when it is determined that a replay to the conversation person is necessary.

Images