JPS6283798A - Continuous voice recognition equipment - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of Industrial Application The present invention relates to a speech recognition device that identifies input speech by comparing an input pattern with a plurality of standard patterns represented by a series of feature vectors. In particular, the present invention relates to a speech recognition device that can be applied to recognition of continuously uttered word sounds.

2. Description of the Related Art Conventionally, speech recognition devices based on a specific speaker registration method have been put into practical use. That is, a speaker who intends to use a recognition device converts all the words to be recognized into a series of feature vectors using his/her own voice and registers them as standard patterns in a word dictionary, and then Similarly, speech is converted into a series of feature vectors, which word in the word dictionary is closest is calculated according to predetermined rules, and the most similar word is taken as the recognition result.

However, this method is good when the number of recognized words is small, but as the number of words increases to hundreds or thousands of words, the following three problems become impossible to ignore.

(1) The burden on the speaker during registration increases significantly. (2) The time required to calculate the similarity or distance between the uttered voice and the standard pattern during recognition increases significantly, which increases the burden on the recognition/recognition device. (3) The memory required for the word dictionary becomes very large.

As a method to avoid the above drawbacks, the unit of recognition is a monosyllable of consonant + vowel (hereinafter referred to as CV).

OC represented by ■ means a consonant, and ■ means a vowel. ). That is, monosyllables are registered as a series of feature vectors as standard patterns, and input speech converted into a series of feature vectors during recognition is converted into a series of monosyllables by matching with the standard pattern of monosyllables. It is something to do. In Japanese, there are at most 101 monosyllables.
Since monosyllables correspond to kana characters, according to this method, any Japanese word or sentence can be converted (recognized) into a monosyllable string, and (1) to
All problems in (3) will be solved0However,
One of the problems in this case is segmentation. In other words, segmentation is the process of dividing continuously uttered speech into monosyllabic units, but a definitive method to do this reliably has not yet been found.To solve this problem, there are currently Each monosyllable is uttered separately, and some devices are in practical use.

Problems to be Solved by the Invention However, it is stressful for the speaker to input Japanese sentences using discrete utterances of monosyllables, and it is desirable to be able to input them using continuous utterances.

An object of the present invention is to provide a speech recognition device that solves the problem of segmentation of speech input by continuous utterance.

Means for Solving the Problems The present invention converts the input signal into feature vectors a1, a2,
. . . , al, . . . , aI. bn
□, ......, ,bnj,n, a standard pattern storage means for storing a standard pattern Rn (where n=1 to N), and a signal (hereinafter referred to as segment・Segment marker input means for inputting ° (referred to as marker) °, and in a grid graph with the input frame l as the horizontal axis and the standard pattern frame j as the vertical axis, corresponding to each segment marker of the input signal. DP matching is performed with the standard pattern Rn and the partial pattern of the input pattern whose starting point is near the position and the ending point is a frame included in a predetermined range from there, with the standard pattern side as the basic axis. DP matching means; and syllable recognition means for further determining the optimum value for the n from among the optimum values for the standard pattern Rn obtained as a result of this DP matching, and for determining the optimum value for the n as the syllable recognition result of the partial pattern of the input pattern. It is a continuous speech recognition device including.

Operation With the above configuration, the present invention converts an input signal into feature vectors a 1 , a 2 , . . . by the feature extraction means.
・r ai +..., aI, and input a signal synchronized with each syllable (hereinafter referred to as a segment marker) using the segment marker input means, and In a grid graph with frame i as the horizontal axis and frame j of the standard pattern as the vertical axis, the starting point is near the position corresponding to each segment marker of the input signal, and frames included in a predetermined range from there are A sequence of partial patterns and feature vectors of the input pattern to be used as terminal points bn1. bn2.・・・・・・＋bnjp・・
... Standard pattern Rn (
However, DP matching is performed with free starting and ending points for n=1 to N) with the standard pattern side as the basic axis, and among the optimal values for the standard pattern in obtained as a result of this DP matching, the optimal value for n is further determined, Let n be the syllable recognition result of the partial pattern of the input pattern.

Embodiment One method of speech recognition is called word spotting. That is, the present invention is a method of finding possible existence candidate intervals of each of the recognition basic units for continuous speech inputted by continuously uttering basic recognition units such as words and syllables. It is based on a method called word spotting, and first, the principle of word spotting will be explained.

FIG. 1 shows an example of path constraint conditions in DP matching used in word spotting. D
P-matching refers to matching an input pattern with a series of feature vectors &1. a2,...pai+...,
Let the standard pattern Rn be the series of feature vectors bn
1. bn3. ,・,,,・, ,bnj, ,,,,
・ , , , bnj,n, the correspondence between the feature vectors of both series is calculated using dynamic programming (DP), and the weighted sum of the similarity (or distance) between the feature vectors according to the correspondence is maximized (or The similarity (or distance) at that time is set as the cumulative similarity (still cumulative distance) between both series. To explain this graphically, if we consider a grid graph in which the horizontal axis is the feature vector of the input pattern and the vertical axis is the feature vector of the standard pattern,
Since the correspondence between the feature vectors of both patterns is represented by each grid point, the maximization (minimization) problem is to maximize (or minimize) the weighted sum of the similarity (or distance) between the series in this grid graph. The problem is to find a grid point sequence, that is, a path. In this case, constraints are set on how to select routes to avoid extreme correspondence. Figure 1 is an example of this, and what it means is that in the grid graph, when the coordinates of the input pattern are i and the coordinates of the standard pattern are j, the path to point (i, H) is point (5). -2s
5-1L point (L -1゜j-1), point (i-1,j-
This means that the route is selected so that it always passes through one of 2). In addition, the numerical values attached to each route in the same figure indicate the weighting coefficient when that route is selected, and this example is an example of how to determine the weighting coefficient used when performing word spotting. be.

・・・・・・・・・・・・・・・・・・(1) Here,
d! 1l(i,j) is the distance between the feature vector ai and the feature vector, bnj, and can be expressed most simply as d''(
i, +) = l ai - bn5 1. When using similarity, the trick may be m&X. Formula (1)
The initial value Dn (i, 1) = dn (1, 1) is calculated as follows, Dn (ilg) is the partial pattern A (m
, i) and the partial pattern "('yj) from the first frame to the first frame of the standard pattern Rn". In (1), if D" (i, j) is calculated and stored each time D" (i, j) is determined, the optimal m=B"
(i, j) By determining the weighting coefficients in this way, the sum of the weighting coefficients in matching Rn(1°j) and any section of the input pattern becomes constant,
For example, paths 301 and 302 as shown in FIG. 6 have the same sum of weighting coefficients, so which of the partial patterns 303 and 304 of the input pattern is more similar to the standard pattern Rn is determined by the length of the input pattern. It is only necessary to compare the cumulative distances on each route, regardless of the distance. Also, as is clear from Fig. 1 b1 or equation (1), the cumulative distance to point ('11) is calculated by point (i - 2
, j -1), point (i -1, i -1), point ('
1. Since it is sufficient to know the cumulative distance in )-2), in reality, calculation of equation (1) is performed sequentially for j = 1 to ■0 in each frame of i = 1 to H,
Dn(i + In) (D i K related minimum value D
If it becomes n(i*, 1″), the standard pattern R
The part of the input pattern closest to n is A(B″
(i*lJ”), L*).

The present invention aims to improve the accuracy of recognition by limiting candidate sections of partial patterns of an input pattern to be matched with the basic unit of recognition using certain information, and the basic unit of recognition is In this embodiment, it is a syllable, and the candidate interval for the syllable is limited based on a segment marker input in synchronization with the pronunciation of the syllable.

FIG. 3 is a time chart showing the change in the power of the voice when uttering "Yokohama" and the time relationship between the segment marker b input manually using a key or the like. In actual cases, it has been observed that the starting point of a continuously uttered syllable and the position of a manually inputted segment marker match fairly well. The principle of the present invention is that the starting point of each syllable is segment·
If the syllable standard pattern is from the beginning of each syllable to the stationary part of the vowel, then the end point of each syllable is considered to be between this segment marker and the following segment marker. Therefore, the word spotting range is limited to this range, and DP matching is performed freely at the beginning and end. Here, the syllable standard pattern is from the beginning of each syllable to the fixed part of the vowel.The reason why the syllable standard pattern is from the beginning of each syllable to the fixed part of the vowel is because the information as a syllable is divided into three parts: the consonant part, the transition part from the consonant to the vowel part, and the vowel part. However, the vowel part has the longest duration, so if it were adopted as it is as a standard pattern, the weight of the vowel part would be too large compared to the other three, which would be unfavorable for recognition.

FIG. 1a is a block diagram showing an embodiment of the present invention based on the above principle.

Reference numeral 1 denotes a feature extraction unit composed of a filter bank, etc., which converts an input audio signal into a series A of feature vectors ai.

Reference numeral 4 denotes a standard pattern storage unit, which stores the above-mentioned feature vector series R'' (n=1~
N) is registered in advance.

5 is an inter-vector distance calculation unit which calculates the distance d(' +
+).

Reference numeral 3 denotes a standard pattern counter, which sequentially sets standard patterns Rn to be matched as n=1 to N.

Reference numeral 2 denotes a standard pattern frame counter, which sequentially points to standard pattern RnO frames i=1 to J''Kff.

Reference numeral 8 denotes a segment information input section, in which the segment information is manually input in synchronization with the input syllable by pressing down a key or the like.

Reference numeral 9 denotes a voice section detecting section, which detects a voice section according to a well-known method based on the magnitude of the input signal, etc.0.1Q is an input frame counter, and the voice section detecting section 9 detects a voice section according to a well-known method. When the input frame counter 1o detects that the input frame has been input, the input frame counter 1o starts counting for each frame.

Reference numeral 7 denotes a matching start/end frame determining unit, which determines a partial section of the input pattern to be matched with the standard pattern Rn according to the above explanation based on the output of the input frame counter 1o and the output of the segment information input unit 8. .

Reference numeral 6 denotes an intra-segment counter, which indicates the matching start frame v-mukara? determined by the matching start and end frame determination unit 7 in each of the segments. Count the frames up to the end frame of the link and point to each frame of the input pattern within the segment.

Reference numeral 5 denotes an inter-vector distance calculation unit which calculates the distance between the frame of the input pattern specified by the intra-segment counter 6 and the frame of the standard pattern specified by the standard pattern counter 3 and the standard pattern frame counter 2. Calculate the distance between the vectors.

Reference numeral 11 denotes an inter-vector distance storage unit which temporarily stores the inter-vector distance calculated by the inter-vector distance calculation unit 6 until it is no longer needed.

Reference numeral 13 denotes a cumulative distance calculation unit, which calculates the cumulative distance according to the above equation (1).

12 is a cumulative distance storage unit, and a cumulative distance calculation unit 13
The cumulative distance calculated in is temporarily stored until it is no longer needed.

14 is a minimum value determination unit, and the cumulative distance storage unit 12
In each segment, the minimum value of the cumulative distance with respect to the standard pattern Rn for the frame in that segment is determined from the cumulative distances stored in the segment.

15 is a syllable determining unit which further determines a minimum value regarding n from the minimum value of the cumulative distance determined for the standard pattern Rn determined by the minimum value determining unit 14;
Letting n t-n* at that time, let n* be the syllable recognition result of the segment.

FIG. 2 is a processing procedure diagram for explaining the detailed processing procedure of the embodiment. Of course, this can also be followed when implementing it using software.

In the same figure, the notation NDDO indicates that LS executes Y while X is satisfied.
The E NDIF notation means to execute Y if the condition X is satisfied, and to execute Z otherwise.

Furthermore, by changing the notation regarding cumulative distance, distance between vectors, etc. as follows, the memory for storing intermediate results can be saved. That is, when using the route shown in Figure 1, 1
．． The calculation of the cumulative distance of the i-th frame of the input pattern is
It is only necessary to store the cumulative distances of the -1st stage and the i-2nd stage, and the distance between the vectors of the i-1st stage.In the end, the required memory is Dn (iF)), which is the 3 frames of input. minute, d
Regarding n(irt), it is possible to calculate the recurrence formula if two frames of input are memorized, but
Furthermore, the cumulative distance calculation can be performed in the order of t = 1n ~ 1, and in this case, D'' (i - 2, j - 1) used in the calculation of ``(' p ])'' is used again. Therefore, it is possible to use the storage location of the newly found Dn(i, j) where D"(i -2, 5) was stored, and the required memory 1f-Dn('
l 3 ) * ”(' F ] ) This means that it is two frames of input. Also, 1 for each word n.
Performing calculations for the segment Nislever, the Dn(
t + s ) + d” (t p j) n at Hatabe
Since these can be used in common, they can be set to D (=? You can save money.

In step 101, k represents a number according to the input order of input segment markers, and k=o represents the beginning of a word.

Step 102 sets the starting frame S for matching in the second segment. ik is the input frame number of the second segment marker, and rl defines the free range of the starting point when performing the matching calculation of the second segment, and is set in advance.

Step 103 sets the matching end frame t for the second segment. r2 defines the free range of the starting point when performing the matching calculation for the second segment, and is set in advance. The smaller of 1k-1+r2 and ik is selected as t.

In step 104, a memory Dn that stores the minimum value for n=1 to n among the distances obtained in the range of 8 to t for the partial pattern of the input pattern that best matches the standard pattern Rn is initialized to -. .

Step 105 is to perform a matching calculation between the input pattern and the standard pattern Rn in the section 3 to t.

Step 106 initializes the memory D, which stores the minimum distance of the segment to the frame, to φ in order to find the partial pattern of the input pattern that best matches the standard pattern Rn in the range of 3 to t. ing.

In step 107, the cumulative distance is initialized to - before performing the matching calculation for the segment.

Step 108 is a process performed to determine the first memory to be used in each segment when cyclically using memories to store cumulative distances, back pointers, and inter-vector distances.

In step 109, a partial pattern of the input pattern that best matches the standard pattern Rn is found in the range s-t, and its value is stored in the memory Di.

Step 11o is a process performed to cyclically use the memory for storing the cumulative distance and inter-vector distance, and means the negation of 4m.

In step 111, the vector distance between frame a1 of the input pattern and frame b''j of the standard pattern Rn is determined by j.
Calculations are made for = 1 to ■0. This example is a so-called urban area distance.

Step 112 performs calculations corresponding to equation (1) for j=In~3.

Step 113 is the calculation of cumulative distance for the case of j=2. In this case, the route will be as shown in FIG.

Step 114 is the calculation of the cumulative distance for the case j=1.

Step 115 stores the minimum value of the cumulative distance to the standard pattern Rn in frame ts-t as Di in the memory.

Step 116 converts the minimum value Di into a standard pattern an
is normalized by the number of frames, and the value is stored as Dt.

Step 118 calculates the minimum value of n from the value of Dt, and stores n at that time as N (k). That is, N(II is the standard pattern closest to the partial pattern of the input segment.

From N(k) obtained in the above manner, the input is the syllable string N(1), N(2),...+++, N(
6) is obtained.

Effects of the Invention As described above, in the present invention, when syllables are uttered continuously, a segment marker is manually input for each syllable, and word spotting is performed based on the position of this marker. By introducing this idea and performing DP matching with free start and end points, it is possible to not only eliminate segmentation errors but also to perform highly accurate segmentation.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention, FIG. 2 is a processing procedure diagram for explaining in detail the processing procedure in the embodiment, and FIG. 3 is an example of the present invention for which detailed explanation is not necessary. FIG. 4 is an explanatory diagram for explaining the exception handling of DP matching in this embodiment, and FIG. 6 is an explanatory diagram for explaining the nature of DP matching used in the present invention. . 1...Feature extraction unit, 2...Standard pattern frame counter, 3...Standard pattern e counter, 4...Standard pattern storage unit, 6・
. . . Inter-vector distance calculation unit, 6 . . . Intra-segment counter, 7 . . . Matching start and end frame determination unit, 8 . . . . Voice section detection section, 10 . . . Input frame/color inter, 11 .
... Minimum value determining section, 14... Syllable string determining section.

Claims (1)

[Claims] The input signal is converted into feature vectors a_1, a_2,...
, a_i, ..., a_ I, and a feature vector sequence b^n_1, b^n
＿２、・・・・・・、b＾n＿j、・・・・・・、b＾
Standard pattern R^n consisting of n_J^n (where n=
1 to N), a segment marker input means that inputs a signal synchronized with each syllable (hereinafter referred to as a segment marker), and input frame i on the horizontal axis. , in a lattice graph with frame j of the standard pattern as the vertical axis, an input pattern in which the starting point is near the position corresponding to each segment marker of the input signal, and the ending point is a frame included in a predetermined range from there. The partial pattern and the standard pattern R^
n and the standard pattern side is the basic axis, and the starting and ending points are free D.
DP matching means for performing P matching, and this DP
further determining the optimal value for the standard pattern R^n among the optimal values for the standard pattern R^n obtained as a result of matching;
syllable recognition means for determining the n as a syllable recognition result of a partial pattern of the input pattern.