JP2905686B2 - Voice recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition apparatus, and more particularly, to a speech recognition apparatus that detects a pause (silent section) or a silent section such as a redundant word in a uttered speech and continuously executes speech recognition. . In this specification, a section including a pause, a redundant word, and a delimiter based on prosodic information or the like is referred to as a silent section or the like.

[0002]

2. Description of the Related Art In recent years, continuous speech recognition has been actively studied, and some research institutions have constructed sentence speech recognition systems. Many of these systems target carefully uttered speech. However, in human-to-human communication, redundant words such as “Ah” and “Ehto” and pauses and silences and restatements that are pauses in states such as silence periods where there is no utterance are temporarily present. Appears frequently.

FIG. 2 is a diagram showing a speech recognition operation of a conventional continuous speech recognition device in a stack format.
When you say "Apply for the meeting", continuous speech recognition will output four candidates: "Apply for the meeting", "Apply for the meeting", "Apply for the member", and "Apply for the member". The process is depicted.

First, the sound "ka" is recognized and stacked as characters. Next, the sound "I" is recognized by speech,
Stacked as characters. Then, the sound of "gi" and
Since the sounds "I" and "N" are recognized, the character stacking device is separated into two, and both characters are stacked. Since "kaigi" and "kaiin" are listed in the speech recognition dictionary, both are converted to the character "n" representing a noun. Next, "Ni" is recognized, and as a result of dictionary lookup, it is converted to the letter "p" representing a particle. Then, the particle is connected to the noun and is converted into a character "NP" representing a noun phrase. Here, since “to the meeting” and “to the member” are both noun phrases “NP”, the same candidates “apply” and “apply” can be connected thereafter.

[0005]

However, in the conventional continuous speech recognition apparatus, when a long utterance including a break such as a silent section in a speech is handled, the same function is syntactically performed as the speech recognition progresses. , It is necessary to individually process identical candidates that can lead to the candidates. That is,
In the conventional continuous speech recognition device, the same candidates must be individually processed, and the processing amount is unnecessarily increased.

An object of the present invention is to solve the above problems and to provide a speech recognition apparatus capable of greatly reducing the processing amount as compared with the conventional example and increasing the processing speed of speech recognition. It is in.

[0007]

According to a first aspect of the present invention, there is provided a speech recognition apparatus comprising: a speech recognition apparatus having a speech recognition unit for recognizing an uttered speech sentence composed of an input character string. Detecting means for detecting at least one of a pause, a redundant word, a phrase or a clause boundary based on the uttered voice sentence, and outputting a detection signal, wherein the voice recognition means uses a hidden Markov model. A speech recognition process is performed using the LR method, and when the detection signal is input, a cell used for the LR method using the hidden Markov model is placed at the top of a state stack indicating a speech recognition result candidate. By combining and merging cells having the same contents, a plurality of speech recognition candidates having the same function syntactically can be identified as one.
The speech recognition process is performed by compressing the speech recognition candidates into one speech recognition candidate.

[0008]

[0009] Further, the speech recognition apparatus according to the second aspect,
2. The speech recognition device according to claim 1, wherein said detecting means includes: a first condition that the power of the uttered voice sentence is equal to or less than a predetermined threshold for a predetermined time range; During a predetermined time period,
The pause is detected by detecting that at least one of a second condition that is equal to or greater than a predetermined threshold is satisfied. Further, in the speech recognition apparatus according to the third aspect, in the speech recognition apparatus according to the first aspect, the detection unit determines whether or not the speech model matches a language model of a plurality of redundant words stored in advance. It is characterized by detecting redundant words. Claim 4
2. The voice recognition device according to claim 1, wherein the detection unit detects that the fundamental frequency of the uttered voice sentence suddenly rises or falls above a predetermined gradient. By doing so, the boundaries of the above phrases or clauses are detected.

[0010]

In the speech recognition apparatus according to the first aspect, the detection means detects and detects at least one of a pause, a redundant word, and a phrase or clause boundary based on the input uttered speech sentence. Output a signal. The speech recognition means executes a speech recognition process using an LR method using a hidden Markov model, and, when the detection signal is input, a cell used in the LR method using the hidden Markov model. Then, a plurality of speech recognition candidates having the same function syntactically are compressed into one speech recognition candidate by connecting and merging the cells having the same contents at the top of the state stack indicating the speech recognition result candidate. To execute voice recognition processing.

Further, in the voice recognition device according to the second aspect, the detecting means preferably includes a first threshold value which is lower than a predetermined threshold value for a predetermined time range. The pause is detected by detecting that at least one of a condition and a second condition in which the number of zero crossings of the uttered voice sentence is equal to or more than a predetermined threshold value during a predetermined time is satisfied. Is detected. Further, in the voice recognition device according to the third aspect, the detecting means preferably detects the redundant word by determining whether or not the word model matches a language model of a plurality of redundant words stored in advance. Further, in the voice recognition device according to the fourth aspect, preferably, the detecting means detects that the fundamental frequency of the uttered voice sentence suddenly rises or falls above a predetermined inclination degree and changes. Thus, the boundary of the phrase or clause is detected.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a continuous speech recognition apparatus according to one embodiment of the present invention. The continuous speech recognition apparatus according to the present embodiment has an SSS (Successive State Splittin).
g: sequential state division method)-LR (left-to-right rightmos)
t derivation type (that is, rightmost derived type) speaker-independent continuous speech recognizer, which is a hidden Markov network (hereinafter HM)
It is called a net. ) Using a hidden Markov model (hereinafter, referred to as HMM) stored in the memory 11, phoneme matching is executed by the phoneme matching unit 4, and the resulting speech recognition score is used as a phoneme context-dependent LR parser (hereinafter, referred to as LR parser). .), And in response, the LR parser 5 performs continuous speech recognition and sends phoneme prediction data to the phoneme matching unit 4 to perform speech recognition processing. In the present embodiment, in particular, based on the time series of the characteristic parameters output from the buffer memory 3, a pause, a redundant word, a silent section including a delimiter based on prosodic information, and the like are detected, and the detection signal is detected by LR.
In response to this, the LR parser 5 includes a plurality of speech recognition result candidates having the same function syntactically as one speech detection result. The speech recognition process is performed while compressing the speech recognition result candidates. Here, in the SSS, a likelihood model expressing a temporal transition of a speech parameter by a stochastic transition between probabilistic stationary signal sources (states) allocated on a feature space of a phoneme is used. The refinement of the model is performed sequentially by repeating the operation of dividing each state in the context direction or the time direction based on the criterion of degree maximization.

In FIG. 1, a uttered voice of a speaker is input to a microphone 1 and converted into a voice signal, and then input to a feature extracting unit 2. After performing A / D conversion on the input audio signal, the feature extraction unit 2 performs, for example, LPC analysis, and performs 34-dimensional feature parameters including logarithmic power, 16th-order cepstrum coefficient, Δlogarithmic power, and 16th-order Δcepstrum coefficient. Is extracted. The time series of the extracted feature parameters is input to the phoneme matching unit 4 via the buffer memory 3.

HM network memory 1 connected to phoneme matching unit 4
The HM network in 1 is represented as a plurality of networks having each state as a node, and each state has the following information. (A) State number (b) Acceptable context class (c) List of preceding and succeeding states (d) Parameters of output probability density distribution (e) Self transition probability and transition probability to succeeding state

In this embodiment, since the HM network needs to specify which speaker each distribution originates from,
A predetermined speaker mixed HM network is converted and created. Here, the output probability density function is a Gaussian mixture distribution having a 34-dimensional diagonal covariance matrix, and each distribution is learned using a specific speaker sample.

The phoneme matching section 4 executes a phoneme matching process in response to a phoneme matching request from a phoneme context-dependent LR parser (hereinafter, referred to as an LR parser) 5. At this time, the LR parser 5 passes phoneme context information including a phoneme matching section, a phoneme to be matched, and phonemes before and after the phoneme. The phoneme matching unit 4 connects the states on the HM network capable of accepting such a context based on the received phoneme context information within the constraints of the preceding state list and the following state list, thereby forming one model. Is selected. Then, the likelihood for the data in the phoneme matching section is calculated using this model, and the value of the likelihood is returned to the LR parser 5 as a phoneme matching score. Since the model used at this time is equivalent to HMM,
The forward path algorithm used in the normal HMM is used as it is for the calculation of the likelihood.

On the other hand, the silent section etc. detecting section 30 detects a silent section including a pause based on a pause, a redundant word and prosodic information based on the time series of the characteristic parameters output from the buffer memory 3. Then, the detection signal is output to the LR parser 5. Here, the detection unit 30
Indicates that a redundant word is recognized as a redundant word by comparing and comparing with a phoneme model of a redundant word (for example, a redundant word shown in Tables 1 to 3 below) stored in an internal memory in advance, while a pause which is a silent section Is detected as a pause when one of the following two conditions is satisfied. (First Detection Condition) When the time t0 when the power is equal to or lower than the predetermined threshold level is, for example, a value in the following range. Preferably, 50 milliseconds ≦ t0 ≦ 3 seconds. More preferably, 50
Ms ≦ t0 ≦ 500 ms. (Second detection condition) When the time t1 during which the number of zero crosses at which the input audio signal crosses the zero potential is equal to or greater than a predetermined threshold value is, for example, in the following range. Preferably, 50
Milliseconds ≦ t1 ≦ 3 seconds. More preferably, 50 ms ≦ t
1 ≦ 500 milliseconds. Furthermore, a break based on prosodic information etc.
Specifically, when the intonation rapidly rises or falls, it is presumed that it is a phrase or clause boundary. In this regard, the boundary or the boundary is determined by detecting that the fundamental frequency of the input characteristic parameters suddenly rises or falls at or above a predetermined inclination degree and changes.

The LR parser 5 is provided with the detecting unit 30
Each time a detection signal is input from the CPU, the speech recognition processing is executed while compressing a plurality of speech recognition result candidates having the same function syntactically into one speech recognition result candidate. The redundant words include, for example, redundant words as shown in Tables 1 to 3 below.

[0019]

[Table 1] ──────────────── Redundant words ──────────────── “A” “Ah” “Ahto” “ "Ah" "Ah" "Ah" "Ah" "Ah" "Ah" "Ahhh" "Ah" "Ah" "Noh" "U" "Wh" "Whh" "Hhmm" "Hhmm" "Hhmm" 」「 え 」「 え っ ー 「え っ っ ──────────────── ────────────────

[0020]

[Table 2] ──────────────── Redundant words ──────────────── “Em, it is” “Em,” “Em,” "Well," "Well," "Well," "Well," "Well," "Well," "Well," "Well," "Well," "Well," "Well," "O," "O," "O," "This" "this" "this" "ja" "su" "su" ────────────────

[0021]

[Table 3] 冗長 Redundant words ──────────────── “Sus” “Sou” “Usu” “Usu” "A little" "one" "de" "de" "to" "to" "ha" "ha" "hoo" "ma" "ma" "ma" "ma" "ma" "ma" "mo" 」ん ん ー「 ん ん ん ────────────────

As is well known, a predetermined context-free grammar (CFG) in the context-free grammar database memory 20 is automatically converted to create an LR table, and an LR table memory 13 is created.
Is stored in The LR parser 5 refers to the speaker model memory 12 including, for example, a phoneme duration model and the LR table, and processes the input phoneme prediction data from left to right without backtracking. If there is syntactic ambiguity, the stack is split and the analysis of all candidates is processed in parallel. The LR parser 5 predicts the next phoneme from the LR table in the LR table memory 13 and outputs phoneme prediction data to the phoneme matching unit 4. In response, the phoneme matching unit 4 performs matching by referring to information in the HM network memory 11 corresponding to the phoneme, returns the likelihood as a speech recognition score to the LR parser 5, and sequentially connects the phonemes. As a result, continuous speech recognition is performed. Here, the LR parser 5 detects a silent section or the like including a pause or a redundant word based on the time series of the characteristic parameter output from the buffer memory 3 and outputs the detected signal to the LR parser 5. Output to parser 5. In response, every time a detection signal is input, the LR parser 5 executes a speech recognition process while compressing a plurality of speech recognition result candidates having the same function syntactically into one speech recognition result candidate. . That is, as shown in FIG. 3, for example, after a plurality of subtrees of speech recognition that have been processed before the input of the detection signal are connected and merged, after the input of the detection signal, starting from the connected one node is performed. To perform voice recognition processing. Then, after processing to the end of the input speaker's voice, the one with the highest overall likelihood or a plurality of predetermined higher-order ones is output as recognition result data or result candidate data.

FIG. 3 is a diagram showing the speech recognition operation of the continuous speech recognition apparatus of the present embodiment of FIG. 1 in the form of a stack. In the case where there is a paused silent section as a break in the input uttered speech. Is shown. The processing up to the point at which “in a meeting” and “in a member” are recognized is the same as the conventional method. If a silence section is detected by the silence section etc. detection unit 30 immediately after the processing of “meeting”, a detection signal is input from the detection unit 30 to the LR parser 5, and in the processing after the timing, syntactically A plurality of candidates having the same function are compressed into one. In this example, since both “to the meeting” and “to the member” are converted into the noun phrase “NP”, the two partial trees of the two speech recognition result candidates are compressed into one partial tree of the speech recognition result candidate. Is done. Therefore, the processing which has been duplicated in the conventional apparatus can be avoided by the method according to the embodiment of the present invention.

In the present embodiment, the point at which the operation of compressing a plurality of speech recognition result candidates having the same function syntactically is started is detected in the input uttered speech by detecting a break such as a silent section. When it is limited. The reason is as follows. Since this device is a speech recognition device,
The compression operation must be started in time. On the other hand, in reality, the number of characters corresponding to the same voice section is often different. Also in this example, "Kaini" has four characters, but "Kaini" has five characters. Even if the number of characters is aligned, it does not correspond to the start time of the compression operation at all. Therefore, the compression operation is activated only when a break such as a silent section can be detected in the voice.

Further, the processing of the LR parser 5 for processing a detection signal from the silent section etc. detecting section 30 will be described in detail. FIG. 5 is a diagram showing a data structure of a cell used in the continuous speech recognition apparatus of FIG. As shown in FIG. 5, a data structure holding information necessary for analysis of speech recognition by the conventional HMM-LR method, that is, a top cell representative pointer, a phoneme sequence located in a layer below the top, and its state An LR work area consisting of a stack and a layer located therebelow.
In the cell in the data structure including the two voice recognition scores and the HMM work area including the probability table, a cell having the same contents at the top of the state stack indicating the voice recognition result candidate, that is, the same stack top as the last content is stored. A merge pointer is added for merging the cells. In the example of FIG. 3, the plurality of cells are two cells before the detection of a silent section. Further, a cell list for pause section processing (hereinafter referred to as a pause cell list) is newly prepared.

FIG. 6 is a flowchart showing a speech recognition process executed in the continuous speech recognition apparatus of FIG. The main points of the cell merge processing and the pause synchronization processing in this processing will be described below. In the following description, a pause includes a redundant word. (1) A pause is detected at a branch of a subtree indicating a certain voice recognition result candidate, that is, it is detected that the top of the symbol stack becomes a pause, and the input frame of voice reaches the end of the voice section of the pause unit. If so, the cell is registered in the pause cell list. (2) If the branch cannot be extended due to pruning by beam search or rejected syntactically, a compression operation (reduce operation is performed on the branch registered in the pause cell list. A set of syntactic categories "
Prunes all branches that are not reduced to elements belonging to. (3) Further, the cells having the same contents at the top of the state stack in the remaining branches are merged. The voice recognition score of the plurality of subtrees is represented by the best one. Note that an arbitrary syntactic category can be defined in “a certain syntactic category set”. If it is desired to accept an utterance in which a pause occurs at a word boundary, the syntactic category set may be changed to all word breaks.

Next, the point of the cell split processing will be described. (4) When processing needs to be performed retroactively from the merged position, the pointer is changed and the cell is split or divided into a plurality of subtrees. Here, the speech recognition score after the split processing is returned to the original value.

Hereinafter, the speech recognition processing will be described with reference to FIG. First, in step S1, initialization of the HMM work area and initialization of the LR parser 5 are executed. Specifically, one cell of the state stack 0 is created. Then, in step S2, it is determined whether or not the end of the last pause unit (pause unit) in the pause section composed of a plurality of pause units has been reached. If the end has been reached, the voice recognition processing ends. On the other hand, when the end of the last pause unit has not been reached in step S2 (NO in step S2), step S2 is executed.
In step 3, the data of the analyzed speech section for each pause is read. Further, in step S4, it is determined whether or not the pause unit is the first pause unit in the pause section. If it is the first pause unit (Y in step S4)
ES) Proceed to step S7. On the other hand, if it is not the first pause unit (NO in step S4), step S5
At the pre-likelihood, the point of the best preceding speech recognition score (pr
e-bestpoint), where the number of cells is equal to the maximum beam width. And the HM of the pause cell list
Perform initialization of the M work area. Next, in step S6, a plurality of cells having the same content such as part of speech are merged with the stack top located at the top of the state stack.
At this time, the representative cell of the merged cell list is selected with the best speech recognition score. Further, in step S7, the HMM-
A pause unit speech recognition module process using the LR method is executed (see FIGS. 7 and 8).

FIGS. 7 and 8 are flowcharts showing the pause unit speech recognition module processing of FIG. In the voice recognition module process, the processes in steps S12 to S21 are repeated until the analyzed phoneme string length reaches the end condition (end). As shown in FIG. 7, in step S11, it is determined whether or not the phoneme string length has reached the end. If it has reached (YES), the process returns to the main routine. On the other hand, if it has not reached (step S
(NO at 11) The process proceeds to step S12.

The following processing is executed in the processing from step S12 to step S14. That is, a plurality of cells having a compression operation for all representative cells, that is, a plurality of cells having the same stack top as described above, perform a compression operation after creating one new cell with them, and Connect to cell list.

Next, from step S15 to step S1
In the processes up to 7, the following processes are executed. That is, for all the merged cells, a split process in which a cell whose stack top is not the same in the state stack is divided into a plurality of cells is executed. At this time, H of each cell
The MM probability table is returned to the original value.

Further, in step S18 in FIG. 8, if the last pause unit is not used, for all representative cells, the top of the symbol stack is silence or pause and the point of the best speech recognition score is the speech section. If it exceeds the length, it is copied to a pause cell list and used as an initial cell candidate for the next voice section. Next, in steps S19 to S22, the following processing is performed on all representative cells. (A) When the next operation is a shift operation, a phoneme matching process is performed on the representative cell to match a predicted phoneme. (B) When the next operation is an accept (accept) operation, if the input voice check is OK, the merge cell is registered in the accepted cell list. (C) Otherwise, the above processing of the cell is not executed. Then, in step S23, the representative cells are pruned to the number of beam widths by a known method. The score at the time of pruning uses the score of the representative cell. Further, step S11
Return to Then, in step S11, if it is the last pause unit in the voice section (YES), the process ends.

FIG. 4A shows a speech section and a phoneme section in the conventional continuous speech recognition apparatus, and FIG. 4B shows a speech section and a phoneme in the continuous speech recognition apparatus of this embodiment shown in FIG. It is a figure showing a section. As shown in FIG. 4A, the conventional HMM-LR method is based on a horizontal search synchronized with a phoneme. Therefore, as time progresses, the possible range of the matching phoneme gradually increases. According to the apparatus of this embodiment, as shown in FIG. 4B, the compression processing by merging is performed using the detected pose information, so that the possible range of the verification phoneme can be narrowed. It is. As a result, the processing amount of the computer to be processed can be greatly reduced, and therefore, there is also an effect that the processing time of continuous speech recognition can be shortened and high-speed speech recognition can be performed.

As described above, according to the embodiment of the present invention, for each of a plurality of candidates generated in the speech recognition process, a pause or a segment such as a silent section including a redundant word is detected in the speech. Furthermore, since a plurality of candidates having the same function syntactically can be compressed, continuous speech recognition that can avoid redundant processing can be realized. Therefore, the processing amount can be significantly reduced as compared with the conventional example, and the processing speed of voice recognition can be increased.

In the above embodiments, the speech recognition apparatus using the HMM-LR method has been described. However, the present invention is not limited to this, and other types of speech recognition apparatuses such as a speech recognition apparatus using a neural network. Can be applied to In the above embodiment, the silent section etc. detecting unit 30 detects redundant words and pauses and boundaries between phrases or clauses. However, the present invention is not limited to this, and among redundant words, pauses and boundaries between phrases or clauses. You may comprise so that at least one is detected.

[0036]

As described above in detail, according to the present invention, in a speech recognition apparatus provided with speech recognition means for recognizing an uttered speech sentence composed of an inputted character string, the speech recognition apparatus is provided based on the inputted uttered speech sentence. Detection means for detecting at least one of a pause, a redundant word, a phrase or a clause boundary, and outputting a detection signal, wherein the voice recognition means uses a LR method using a Hidden Markov Model to generate a voice. When the recognition process is performed and the detection signal is input, a cell having the same top-level content of a state stack indicating a speech recognition result candidate is set as a cell used for the LR method using the hidden Markov model. By linking and merging, a plurality of speech recognition candidates having the same function syntactically are compressed into one speech recognition candidate, and the speech recognition process is executed. Therefore, for a plurality of candidates generated in the speech recognition process, each time a break such as a silence section including a pause or a redundant word is detected in the speech, a plurality of candidates having the same function syntactically can be compressed. Therefore, continuous speech recognition that can avoid duplicate processing can be realized.
Therefore, the processing amount can be significantly reduced as compared with the conventional example, and the processing speed of voice recognition can be increased.

[Brief description of the drawings]

FIG. 1 is a block diagram of a continuous speech recognition apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing a speech recognition operation of a conventional continuous speech recognition device in a stack format.

FIG. 3 is a diagram showing a speech recognition operation of the continuous speech recognition apparatus of the present embodiment in FIG.

4A is a diagram showing a speech section and a phoneme section in a conventional continuous speech recognition apparatus, and FIG. 4B is a view showing a speech section and a phoneme section in the continuous speech recognition apparatus of this embodiment in FIG. FIG.

FIG. 5 is a diagram showing a data structure of a cell used in the continuous speech recognition device of FIG. 1;

FIG. 6 is a flowchart showing a speech recognition process executed in the continuous speech recognition device of FIG. 1;

FIG. 7 is a flowchart showing a first part of the pause unit speech recognition module processing of FIG. 6;

FIG. 8 is a flowchart showing a second part of the pause-based speech recognition module processing of FIG. 6;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Microphone, 2 ... Feature extraction part, 3 ... Buffer memory, 4 ... Phoneme collation part, 5 ... LR parser, 11 ... Hidden Markov network memory, 12 ... Speaker model memory, 13 ... LR table memory, 20 ... Context free Grammar database memory, 30 ... silence section detection unit.

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-5-19785 (JP, A) JP-A-4-84197 (JP, A) JP-A-4-86946 (JP, A) JP-A-1- 321498 (JP, A) (58) Field surveyed (Int. Cl. ⁶ , DB name) G10L 3/00-9/18 JICST file (JOIS)

Claims (4)

(57) [Claims] 1. A speech recognition apparatus comprising speech recognition means for recognizing an uttered speech sentence consisting of an input character string, comprising: a pause, a redundant word, a boundary between phrases or clauses, based on the inputted uttered speech sentence. And detecting means for detecting at least one of the above, wherein the voice recognition means comprises an LR using a hidden Markov model.
When the speech recognition process is performed using the method and the detection signal is input, the cell used for the LR method using the hidden Markov model indicates the speech recognition result candidate. Is characterized by compressing a plurality of speech recognition candidates having the same function syntactically into one speech recognition candidate by executing the speech recognition process by connecting and merging the same cells. . 2. The method according to claim 1, wherein the first condition that the power of the uttered voice sentence is equal to or less than a predetermined threshold value for a predetermined time period, and the number of zero crossings of the uttered voice sentence are
2. The pause according to claim 1, wherein the pause is detected by detecting that at least one of a second condition that is equal to or greater than a predetermined threshold is satisfied during a predetermined time. Voice recognition device. 3. The speech recognition according to claim 1, wherein said detecting means detects the redundant word by determining whether or not the word model matches a language model of a plurality of redundant words stored in advance. apparatus. 4. The detecting means detects the boundary of the phrase or clause by detecting that the fundamental frequency of the uttered voice sentence suddenly rises or falls at a predetermined gradient or more. The speech recognition device according to claim 1, wherein: