JP3277579B2 - Voice recognition method and apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition method and apparatus.

[0002]

2. Description of the Related Art Conventional speech recognition techniques are classified into two types, a continuous speech recognition method and a discrete speech recognition method, according to the accepted speech form.

[0003]

Both of the above two recognition methods have a drawback that an unnatural restriction is imposed on the utterance form of the user. Specifically, in the discrete utterance recognition method, the user has to utter the words registered in advance in short breaks, and in the continuous utterance recognition method, the user is forced to utter the sentences continuously.

[0004] Natural human natural utterances include phenomena such as unnecessary words that are not related to the meaning of the utterance, pauses, paraphrases, and utterances being stopped even during the utterance. In the above-described conventional method, since the degree of freedom of the spoken language is excessively restricted, psychological pressure is applied to the user, and the speech recognition device is difficult to use.

[0005] Even in the conventional method, it is possible to accept a halfway utterance for a given grammar.
It is realized by connecting a jump-arc from a state where the utterance may stop for a given grammar to an end state (see FIG. 6).

[0006] However, this method can recognize a halfway utterance, but when the user makes a longer utterance, it is necessary to perform the utterance without cutting off the utterance at all (for example, in the utterance of "record on video"). There should be no pause between video, video, recording, and recording). If I do not,
Although the user intends to speak to the end, the analysis result in the middle is output as a recognition result, which has the disadvantage that psychological pressure during speech is large for the user.

The present invention has been made in view of such a situation, and an object of the present invention is to provide a voice recognition method and apparatus which enable a user to pause speech for an arbitrary time.

[0008]

According to the speech recognition method of the present invention, a duration is set independently in a part where the speech may be interrupted in the information defining the word order of the speech, and the speech recognition is performed. During this time, the pause of the utterance continued for the set duration or longer (for example, Y in step S6 in FIG. 4).
ES), detecting utterance completion, outputting an utterance analysis result up to that point, and generating a response using speech synthesis based on the utterance analysis result.

The speech recognition apparatus according to the present invention includes storage means for storing a duration independently set in a part of the information defining the word order of the utterance where the utterance may pause in the middle. When the pause of the utterance continues for the duration or more, the utterance analysis result up to that point is output.

[0010]

According to the speech recognition method and apparatus of the present invention,
When the utterance in the information that defines the word order of the utterance may be paused in the middle, the duration is set independently, and if the pause of the utterance continues for the set duration, the utterance analysis result up to that point will be Is output. Therefore, the user can pause the utterance for an arbitrary time.

[0011]

FIG. 1 shows the configuration of an embodiment of the speech recognition apparatus according to the present invention. The audio signal processing unit 1 analyzes the audio signal input in the time domain and the frequency domain, and outputs a VQ code string. The voice recognition processing unit 2 receives the VQ code string from the voice signal processing unit 1 and refers to the phoneme model storage unit 3, the word dictionary storage unit 4, and the utterance grammar storage unit 5 to recognize a continuously uttered voice. This is a module that performs search processing. That is, the speech recognition processing unit 2 obtains, by trellis calculation, a word string that most surely describes the VQ code string from among the following constraint information (A), (B), and (C) prepared as data. Then, the recognized word string and the meaning structure of the word string are output.

(A) Phoneme model represented by the HMM stored in the storage unit 3 (B) Content stored in the word dictionary storage unit 4: Phonetic notation of a group of words in the system is represented by a sequence of phonemes or a network. (C) Stored contents of the utterance grammar storage unit 5: The one in which the restriction of the user's utterance word string is expressed by a context-free grammar or a framework of a network method.

The dialog management unit 6 interprets a user's request from the recognition result from the recognition processing unit 2 and generates a command for the device 7, and in some cases, transmits insufficient information to the sentence generation unit 8. And the user speaks through the voice synthesizing unit 9 to urge the user to provide information.

The sentence generation unit 8 converts the corresponding utterance from the response meaning expression, that is, the utterance meaning information output from the dialog management unit 6 into a phonetic symbol string, and outputs it.
Receives the phonetic symbol string from the sentence generator and actually utters a voice.

In the configuration of FIG. 1, the parts relating to the present invention are the speech recognition processing unit 2 and the dialog management unit 6. Particularly important points are (i) setting of the duration, (ii) utterance completion detection using the duration, and (iii) generation of a response from the dialog management unit 6 to the user based on the utterance analysis result.
The following describes how to specify the grammar, the silent model, how to capture the duration setting, and
An utterance end detection method using the duration will be described.

FIG. 2 shows the utterance grammar in the embodiment of FIG.
3 shows the relationship between a word dictionary, a phonemic model, and a silent model. The utterance grammar defines the order of words (of sentences) accepted as utterances. In this embodiment, a network grammar is used. However, as will be described later, this description method is not necessarily required, and a free context grammar (refer to the reference document Shiever 86 at the end) may be used. As shown in FIG. 2, in the network grammar, a state (state) and an arc are expressed, and each arc is assigned a word or a vocabulary category, thereby indicating a variation of an accepted utterance sentence. In the present invention, the following types (type), destination (destination) and duration fields are provided in each state as shown in FIG.

Type: Indicates the type of state, and determines whether a silent model described later is inserted or whether it is a state on a mere transition. Destination: It is a pair of (word (word), destination-state), and is a table describing the transition destination state when the input is a word in this state. Duration: Holds the timeout value set when the type is the silent model. This value can be set independently for each state.
It is also possible to dynamically change the setting value while analyzing the utterance.

Normally, a continuation time length is set in a state in which speech may be interrupted halfway.

The non-speech model is a model of a sound in a portion where the user does not speak. In the conventional technology,
There is a method of inserting this silence model into the group of states on the network to cope with the stagnation between words (see KFLee88 at the end). In the embodiment of the present invention, the insertion of the silent model into the state is the same, but the duration control is introduced in this part as in the algorithm described later.

The recognition search is performed on a data structure called a trellis of a two-dimensional array (one axis is time (normal number of frames), and the other axis is each state of the phoneme model) (see FIG. 3). ).

The time axis corresponds to the time axis of the input voice. Assuming that a certain time is t, the next time is t + 1. Usually, the unit change time is 10 msec, and the state group at each time is called a frame. The transition of each state on the trellis in the time axis direction is based on phonological models, word dictionaries,
Determined from grammar.

Each state on the trellis has a recognition score,
It consists of a back pointer (indicating from which state a transition has been made in the previous frame). In the state constituting the silent model, an area for storing the duration is additionally provided.

In the beam search method used in the embodiment of the present invention, all states are not arranged, and only states within a certain range are searched based on the state having the highest probability score. . (In other words, only states with some future potential are considered).

The method of generating a trellis from the above phoneme model, word dictionary and grammar is described in the reference (Bridl) at the end.
e82), and the beam search technique is described at the end of the reference (Kai-Fu Lee 91).
Is described in detail.

FIG. 4 shows the speech recognition algorithm of the embodiment of FIG. Hereinafter, the recognition algorithm according to the embodiment of the present invention will be described with reference to FIG.

The processing generally proceeds according to the following procedure. For an outline of the processing, refer to the reference at the end (Kai-Fu Lee 91).
Is described in detail.

Step 1. Initialization (Step S1 in FIG. 4)
And S2) Prepare a trellis B for holding the state of each frame. B has a size corresponding to N frames from t = 0 to the speech end frame. Hereinafter, B [t] represents the heap of the t-th frame, and usually includes states that fall within the range of the beam (thus, not all states at time t are included). In step S2, time t = 0
In the grammar, the probability score of the initial state of the utterance is set to 1, 0 and registered in B [0].

Step 2. Processing for Each Frame (Steps S3 to S8 in FIG. 4) All states are processed in frame synchronization (after all processing at time t is completed, processing at time t + 1 is performed). For processing of each frame, a Viterbi beam search method of ordinary syntax information control is used (see the reference document Kai-Fu Lee 91 at the end).

Step 3. Recognition end and back trace processing (Step S9 in FIG. 4) The processing of each frame is repeated up to the utterance frame length N. Then, in the last frame (N-1),
The back pointer is traced from the state having the highest probability score (back trace processing), and a word sequence of the recognition result is obtained.

The difference between the embodiment of the present invention and the above-described ordinary search method is as follows. Measuring the duration of a transition in a silent model. II. If the duration exceeds a certain limit, the recognition is stopped and the recognition result up to that point is output (timeout process). 2 points.

Hereinafter, I and II will be described with reference to FIG.

I. To measure the duration of the silent model,
Step S5 in FIG. 4 is performed in the update / registration step of the transition destination state. There are the following three cases depending on the state. (1) Case 1: In this ordinary step (in the case of transition between states other than the silent model), (i). The state of the transition destination (time t + 1) is B [t +
1], this state is represented by a probability score, B [t +
1), (ii). In other cases (when the state of the transition destination is already in B [t + 1]), a process of registering a better probability score in B [t + 1] is performed. (2) Case 2: Transition from the other model to the silent model. In the steps (i) and (ii), when registering the transition destination state to B [t + 1], an area for recording the duration is provided, and the initial value is registered. This initial value can be set according to the grammatical position of the silent model as shown in FIG. 2, or can be set dynamically according to the context of the conversation. (3) Case 3: A transition in the silent model. When registering the transition destination state in B [t + 1], the duration time information is decremented by -1.

II. The timeout process for the duration is
(1) The state having the maximum value of the probability score for each frame is the silent model, and its duration is 0.
Stop the Viterbi search (step S
6), and (2) the back tracing process (step S9) of the operation of outputting the normal recognition result is performed starting from the above state.

Other procedures are based on the conventional Viterbi beam search method.

Next, a dialog management method according to the embodiment of the present invention will be described. The recognition result of the recognition processing unit 2 in FIG. 1 is passed to the dialog management unit 6, and the dialog management unit 6 transmits the utterance result (and the current situation), a control command to the device 7, and an utterance to the user. decide.

By using the above-described recognition method, a more user-friendly man-machine interface can be realized in the dialogue manager 6. This will be described below using a simple utterance grammar as an example.

FIG. 5 shows an utterance grammar. This grammar represents an utterance for controlling a video, an LD, and a cassette in a network expression, and the state of ◎ indicates an end state.

In the embodiment of the present invention, by associating a timeout value with each state, the recognition result up to that state can be passed to the dialog management unit 6.

For example, when the utterance “to video” is made, 1) the recognition processing unit 2 detects the timeout after the particle “ni”, and sends the utterance analysis result (@video゛) is output. 2) The dialogue management unit 6 receives the recognition result, and utters the user (question, etc.) in consideration of other information such as the state of the conversation.
Can be generated.

For example, a user utterance such as "What should be recorded in a video?" By using an utterance syntax and using only "recording" after "@" for "video". Can be generated based on the question.

The important point of the present invention is that the end of the utterance is detected by setting the duration time to a state where the utterance can be broken.

Although the above example has been described using the network grammar, a context-free grammar and other methods can also be used.

In the above example, the silent model is inserted between words (words). However, the silent model can be inserted between phonemes.

Further, an interruption of the timer processing in the multiprocessor configuration can be used.

The number of non-voice models is not limited to one, but may be plural.

References are listed as follows.

[Bridle 82] Bridle. J.
S. et al, @ An Algorithmism for
Connected Word Recogniti
on @, Proc. ICASPSP82, pp899-9
02, Paris, May 1982

[Kai-Fu Lee 88] Lee.
K. F.゛ Large-Vocabulary Spe
aker-independentcontinuou
s Speech Recognition: The
SPHINX System @, CMU-CS-88-
148, ComputerScience Dept.
Carnegie-Mellon Univ.

[Kai-Fu Lee91] Lee.
K. F. and Alleva. F. , @ Contin
uous Speech Recognition ゛,
in Advances in Speech Sign
al Processing, pp623-650, 1
991 Marel Dekker Inc.

[Shieber 86] Shiebe
r. S.゛ An Introduction to U
nifation-based Approach
es to Grammar ゛, Lecture No
tes of CSLI, Stanford Univ
erity, 1986

[0051]

According to the speech recognition method and apparatus of the present invention, the duration is independently set at a portion where the utterance in the information defining the word order of the utterance may be interrupted in the middle, and the set continuation is set. When the pause of the utterance continues for more than the time, the utterance analysis result up to that point is output, so that the user can pause the utterance for an arbitrary time. Further, it is possible to reduce the psychological pressure of the user and realize a more user-friendly human interface.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of a speech recognition device of the present invention.

FIG. 2 is a diagram showing a relationship among an utterance grammar, a word dictionary, a phoneme model, and a non-voice model in the embodiment of FIG.

FIG. 3 is a diagram showing a trellis of the embodiment of FIG. 1;

FIG. 4 is a diagram showing a speech recognition algorithm of the embodiment of FIG. 1;

FIG. 5 is a diagram showing a sample grammar.

FIG. 6 is a diagram illustrating an example of a conventional grammar for accepting a halfway utterance.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Speech signal processing part 2 Speech recognition processing part 3 Phoneme model storage part 4 Word dictionary storage part 5 Utterance grammar storage part 6 Dialogue management part 7 Equipment 8 Sentence generation part 9 Speech synthesis part

──────────────────────────────────────────────────続 き Continued on front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G10L 15/00-15/28

Claims (6)

(57) [Claims] 1. A speech recognition method for recognizing an input speech in accordance with information defining a word order of an utterance, wherein the information defining the word order of the utterance includes a part where the utterance is likely to be interrupted halfway. A time is set, during the speech recognition, the pause of the utterance continues for the set duration or more, the completion of the utterance is detected, and the utterance analysis result up to that point is output, based on the utterance analysis result. A speech recognition method comprising generating a response using speech synthesis. 2. The speech recognition method according to claim 1, wherein a position of a portion where the utterance is likely to pause is stored. 3. The speech recognition method according to claim 1, wherein the set duration is stored. 4. A speech recognition apparatus for recognizing an input speech in accordance with information defining the word order of an utterance, wherein the speech in the information defining the word order of the utterance is independently set to a portion where the utterance may pause in the middle. A speech recognition apparatus, comprising: storage means for storing a duration time, and outputting an utterance analysis result up to the time point when the pause of the utterance continues for the set duration time or more. 5. The speech recognition device according to claim 4, further comprising a dialogue management unit that presents missing information to a voice entrant according to the utterance analysis result. 6. The voice according to claim 4, further comprising a dialogue management unit that controls the device in accordance with the result of the utterance analysis, and presents lacking information to a voice entrant in response to a response from the device. Recognition device.