JPS634198B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a speech recognition device that takes into account expansion and contraction of speaking speed, and is a speaker-independent speech recognition device that has a learning function of standard speech information and has speaker adaptability. Regarding.

The present invention relates to a speech recognition device that recognizes input speech by measuring the degree of similarity between a feature vector series of input speech and a feature vector series of standard speech stored in advance.

Note that in this specification, a feature vector refers to a plurality of audio features in one time window, and a feature vector series refers to a time series of feature vectors over a certain time length, and refers to the order of feature vectors with respect to the time axis. We use the word sample position as . As a specific representative example, the audio is
When analyzing with a BPF filter (band pass filter) and extracting features by sampling at a certain time interval, the analysis output of L BPF filters at a certain sample position is called a feature vector, and the continuous A time series of feature vectors is called a feature vector series.

One of the factors that deteriorates the recognition rate of speech recognition is variation in speaking speed, and some kind of time axis normalization is generally required in similarity measurement. Dynamic programming is known as a typical example of time axis normalization. However, although this method restricts the computational route by a constant recurrence formula, it essentially reduces the variations in speaking rate to each of the sample positions of the standard voice over a relatively large number of input voices ( Since the similarity is measured by associating sample positions (for example, 30%), it requires an extremely large amount of calculation processing time. In addition, in this method, the time axis normalization regarding the speaking rate is performed based on the similarity between feature vectors, so nonlinear expansion and contraction of the time axis may occur beyond normal speaking rate fluctuations. As a result, there are cases where the degree of similarity with other standard speech that is not the target is increased, which may even worsen the recognition rate.

The present invention is based on the recognition that non-linear fluctuations in speaking rate are caused by a mixture of parts where the speaking rate is extremely small, such as in articulatory combination parts, and parts where the speaking rate fluctuates greatly, such as in steady vowel parts. In the former part, the similarity is measured based on the same time scale, and in the latter part, the similarity is measured based on linear expansion/contraction, and the overall similarity is measured in response to nonlinear variations in speech rate. It is something. To this end, in the present invention, sample positions indicating a subsequence (specific subsequence) with a low speaking rate among the standard speech feature vector sequence are stored. In addition, the sample positions of the standard speech and the input speech are linearly correlated, and feature vector sequence candidates of the input speech that correspond in a fixed format to a specific subsequence of the standard speech are determined, and the feature vector sequence candidates of the input speech and the specific subsequence of each candidate The method includes the steps of measuring the degree of similarity, determining a candidate giving the maximum degree of similarity as a corresponding subsequence, and detecting the maximum degree of similarity corresponding thereto as the degree of similarity of a specific subsequence. Furthermore, the degree of similarity between the residual sequence obtained by removing the specific subsequence from the feature vector sequence of the standard speech and the remaining sequence obtained by removing the corresponding subsequence from the feature vector sequence of the input speech is measured using linear expansion/contraction correspondence. The degree of similarity between the standard voice and the input voice is the total similarity of the specific partial sequence and the remaining sequence, and the input voice is recognized from this total similarity.

Speech recognition devices based on such partially linear combinations are suitable for speaker-independent recognition devices that tend to impose various restrictions on the memory pattern of standard speech. In the case of a speaker-independent recognition device whose purpose is to improve recognition rate by learning and adapting, it is necessary to devise a learning method for establishing correspondence between standard speech and input speech.

Conventionally, to solve the learning problem when the number of feature vectors of the standard speech and the number of feature vectors of the input speech are different, there has been a method of converting the number of feature vectors to match the number of feature vectors of either the standard speech or the input speech. However, the length of the standard voice may deviate from the natural utterance of each person, the quality of the feature vector may deteriorate, and the stability of the feature vector of the standard voice cannot be achieved.

The purpose of the present invention is to solve these drawbacks by using linear interpolation to associate the remaining sequences of standard speech and input speech, and changing the number of feature vectors of standard speech and the values of the elements of the feature vectors. This will be explained in detail below.

For example, a standard speech expressed by information including the following feature vector series and sample positions of a specific subsequence is prepared in advance. Feature vector series: x ₁ ,
x ₂ ,..., x _i , x _i+1 ,..., x _i+k ,..., x _n , each feature vector sample position: T ₁ , T ₂ ,..., T _i , T _i+1 ,
..., T _i+k , ..., T _n , number of feature vectors in feature vector sequence X: m, specific subsequence x _i ~ _{x i+k}
First sample position: T _i , specific subsequence x _i ~ x _i+k
Number of consecutive feature vectors in: k+1,
If there are multiple specific subsequences, prepare one for each. The specific subsequence is a part where the variation in speaking rate is small.

Generally, when a certain word is uttered, articulatory connections occur between adjacent phonemes, and important information for speech recognition exists in these articulatory connections, and variations in speech rate are extremely small in these articulatory connections.

This articulatory connection part can be easily recognized,
For example, this can be determined by visually observing a sonogram or by detecting a formant transient. Alternatively, a plurality of harmonized parts are designated for each standard voice as a specific partial sequence, but the designation method is not necessarily uniform.

For example, if a relatively large number of feature vectors are detected in an articulatory connection part, it is sufficient to make this part a specific subsequence. For example, in the case of the word "nana", if we consider the case where the feature vectors are extracted by sampling at 10 msec intervals, only 1 to 2 feature vectors can be extracted in the first articulatory combination part, and as described later, similar Since the stability of the frequency measurement is lacking, it is better to designate a specific partial series that includes part of the stationary part. Also, as a matter of course, for standard speech for which it is not possible to know parts with small variations in speech rate, a specific subsequence is not specified, and the degree of similarity is measured by another method not directly related to the present invention.

In order to determine corresponding subsequences in the feature vector series of input speech, multiple corresponding subsequence candidates are set by correlating sample positions by linear expansion and contraction, and the similarity between these candidates and a specific subsequence is measured on the same time scale. Determine the corresponding subsequence by measuring . The corresponding subsequence consists of the same number of consecutive feature vectors as that of the specific subsequence. Even if the corresponding subsequence corresponds to an articulatory combination part, its position cannot generally be specified because of the time expansion and contraction of the vowel stationary parts and stop parts before and after it.

However, as long as the duration of the word speech is less than several seconds, estimation can be made within a relatively narrow range. For example, the duration of words such as "ichi", "nii", "san", "Tokyo", "Yokohama", etc. is at most 400.
ms to 500ms long, and such short words
When sampling at a period of 10 ms, the sample position variation of the leading feature vector of the articulatory connection part is about 4 to 10 samples.

Therefore, the sample positions T ₁ to _{T n} of the standard voice are made to correspond linearly to the sample positions T ₁ to _{T o} of the input voice, and the first sample position T _i of the specific subsequence x _i to _{x i+k} is
A position T _je corresponding to the position T je and multiple sample positions T _ja , ..., T _je , including the position T _je and surrounding it.
…, T _jj is the sample position of the input audio in a certain format
The corresponding subsequence can be well estimated by finding a candidate between T ₁ and _{T o} and using this as the leading sample position. The number of corresponding subsequence candidates is required to be about 4 to 10, and this number can be determined for each standard voice, but since it is about 10 at most, it may be uniform.

Input speech feature vector sequence Y: y ₁ , y ₂ , ...,
y _je , y _o , input audio sample positions T ₁ , T ₂ ,...,
T _je , ..., T _o , the sample position of the following equation (1)
T _ja , . . . , T _{je ,} .

Here, T _n is the final sample position in the standard speech feature vector series X, T _o is the final sample position in the input speech feature vector series Y, and T _i is the sample position of the first feature vector X _i of the specific subsequence.

Equation (1) associates the range of corresponding subsequence candidates by linear expansion/contraction, but it includes multiple sample positions T _je −5 to T _je − including the position T _je and surrounding it.
4 as the corresponding subsequence candidate in the input audio (2)
The corresponding subsequence can also be well estimated by the second form of the equation.

To determine the corresponding subsequences, T _ja ,…,
The degree of similarity between the specific subsequence and 10 corresponding subsequence candidates consisting of the same number of consecutive feature vectors as that of the specific subsequence with T _je , ..., T _jj as the first sample position is measured on the same time axis, and the maximum The corresponding subsequence is determined as the corresponding subsequence. Similarity can be measured by feature distances such as absolute value distances and square distances, and when the feature distances are expressed as d( ), for example, a candidate whose first sample position is the sample position T _je and a specific subsequence. The distance D _Je is as follows.

D _je = d (x _i , y _je ) + d (x _i+1 , y _je+1 ) + d (x _i+k , y _je+k ) ...(3) However, y _je ~ _{y je+k} is a sample This is the feature vector of the corresponding subsequence candidate with the position T _je as the leading position.

Let the distance to each candidate be D _ja ~ D _jj , and select the minimum one among them, that is, the candidate with the maximum similarity.
Assuming y _j , y _j+1 , ..., y _i+k , these are determined as corresponding subsequences, and the distance D _j at that time is used as a measure of the similarity of the specific subsequences. Note that the number of feature vector sequences (k
The normalized distance _j obtained by dividing the distance D _j by +1) may be calculated in advance.

_j = 1/k + 1D _j ...(4) Also, if the number of feature vectors in a specific subsequence is extremely small, the determination of the corresponding subsequence becomes unstable, so feature vectors such as vowel stationary parts are used as the specific subsequence. It is better to make it a certain length, including
Calculate all corresponding subsequences for multiple specific subsequences. After determining all corresponding subsequences, the similarity of the remaining feature vector sequences is measured.

FIG. 1 shows the linear correspondence of the residual feature vector series. The similarity measurement for the specific subsequence was performed on the same time scale, but for the remaining sequences, the similarity was calculated by linearly expanding and contracting the sample positions of each remaining sequence of the standard speech excluding the specific subsequence.

For example, in Figure 1, in the correspondence between the feature vectors x _i+k+1 ~ x _n of the last residual sequence of the standard speech and the final residual sequence y _i+k+1 ~ y _o of the input voice, the sample Positions T _u and T _v correspond to feature vectors x _u and x _v in accordance with the following relationship.

T _v =T _o −T _j+k+1 ／T _n −T _i+k+1 (T _u −T _i+k+1 )＋T _j+k+1 …
...(4) However, T _u = T _ik+1 , T _{i+k+2 ,} ..., _{T n} The distance d ( x _u , y _v ) and synthesize them to calculate the distance D ₂ of the final residual series.
In other words, it is the degree of similarity.

D ₂ = _n 〓 ^u=1+k+1 d(x _u , y _vIn addition, in order to normalize the similarity with respect to speech length, the normalized distance is calculated by dividing the distance D ₂ by the number of remaining feature vectors m - (i + k) The sum ₂ may be calculated in advance.

₂ = 1/m−i+kD ₂ ………(6) If normalization by the number of samples is not performed in advance, the similarity between the standard speech and the unknown speech is determined by the similarity in the specific subsequence and the similarity in the remaining sequence. It is calculated by dividing the sum of the degrees and the number of samples of standard speech, m. Furthermore, when the degree of similarity based on the normalized distance is determined for each specific partial sequence and the remaining sequence, the degree of similarity between the standard speech and the input voice is determined by the sum of the respective degrees of similarity. In the case of recognition, the degree of similarity between the required standard speech and the input speech is calculated, and the standard speech with the highest similarity (the smallest distance) among them is detected.

In such a speech recognition device, in order to perform learning, the degree of similarity of input speech to a specific standard speech is calculated, and if the distance is less than a certain value, it is allowed to be registered, and if the distance is within a linear range between the standard speech and input speech, Calculate the registered voice by inserting.

In Figure 1, the registered speech feature vector series is
Let Z ₁ ,..., Z _h ,..., Z _h+k ,..., Z _s ,... Z _l and the corresponding registered voice sample positions are T ₁ ,..., T _h ,
..., T _h+k , ..., T _s , ...T _l . The number l of feature vectors in the registered speech feature vector series Z is calculated by interpolating the number of feature vectors of the standard speech and the input speech.

l=(1−α)m+α _o , 0<α<1 (8) where α is a time interpolation coefficient.

Registered speech feature vector Z _h ~ _{Z h+k} of specific subsequence
To find the corresponding sample position T _h ~ _{T h+k}
is calculated using equations (9) and (10).

T _h = (1-α)T _i +αT _j , 0<α<1 ...(9) T _h+p =T _h+p , p=0, 1, ..., k ...(10) Also, registration The value Z _h+p of the audio feature vector is given by the following formula (11)
Calculate by.

Z _{h + p} = (1 - β) <1.

The remaining series of registered voices are different from the specific series,
Calculate all sample positions of standard speech and input speech by linear interpolation. For example, the last residual sequence feature vector Z _h+k+1 ~Z _l of the registered speech in Figure 1
To calculate the sample position T _h+k+1 and T _l as follows
It is calculated from the first and last sample positions of the last remaining sequence of the standard speech and input speech using equations (12) and (13).

T _h+k+1 = (1−α)T _i+k+1 +αT _j+k+1 …(12) T _l =(1−α)T _n +αT _o …(13) In this way, Although the last remaining series will be explained as an example, the same applies to the remaining series sandwiched between the beginning or a specific series.

An arbitrary feature vector Z _s (where s = h + k + 1 to e) of the last remaining sequence of registered speech can be calculated using the following equations (14) and (15), where the sample position T _u of the standard speech and the sample position T _v of the unknown speech are Calculate using equation (16).

T _u = (T _n −T _i+k+1 )T _s −T _h+k+1 ／T _l −T _h+k+1 T _i+k+1 (14
) T _v (T _o −T _i+k+1 )T _s −T _h+k+1 ／T _l −T _h+k+1 +T _j+k+1 (15
) Z _s = (1-β) x _u + βy _v ... (16) In this way, the registered speech feature vector Z ₁ ,
..., Z _l are all calculated and stored as the result of changes made by learning the standard voice. Time interpolation coefficient α
and the amplitude interpolation coefficient β are determined experimentally by considering the reliability of the feature vectors of the standard speech and input speech, and the adaptation speed by learning, so it is appropriate that both α and β be about 0.1 to 0.25.

As is clear from the above explanation, the size of the feature vector of the registered speech is made to correspond linearly to the feature vector of the standard speech and the feature vector of the known input speech with the weight of the amplitude interpolation system, and the size of the feature vector of the registered speech is The sample position of the first feature vector in the series and the sample positions of all feature vectors in the remaining series are
Since the standard speech and the known input speech are made to correspond linearly using the weight of the time interpolation system, there is an advantage that the method can be adapted to the speaker and reliable speech recognition can be performed.

FIG. 2 shows a first embodiment of the present invention, in which 1
is a mode switching terminal, 2 is an address control section, 3 is an input audio input terminal, 4 is an input audio feature vector series storage section, 5 is an input audio sample number storage section, 6 is a standard audio input terminal, 7 is a standard audio 1 time Memory, 8 is a standard speech feature vector series storage section, 9 is a registered word number input terminal, 10 is a standard speech sample information storage section, 11 is a specific subsequence corresponding position calculation section, 12 is a distance calculation section, 13 is an addition section, 14 is a minimum value calculation unit, 15 is a minimum value signal line, 16 is an optimum matching position signal line, 17 is an optimum matching position storage unit,
18 is an adder, 19 is an input signal line of the adder 18,
20 is a residual sequence corresponding position calculation unit, 21 is an addition unit 1
8 is an input signal line, 22 is a normalization unit, 23 is a registration determination unit, 24 is a registered voice corresponding position calculation circuit, 25 is a registered voice interpolation value calculation circuit, 26 is an optimal pattern detection unit, and 27 is a result output terminal. be.

To operate this, first, a learning instruction signal is input through the mode switching terminal 1, and the address control section 2 is placed in the registration mode. Next, the input speech feature vector sequence y ₁ to y _o is stored in the storage unit 4 through the input terminal 3, and the number of samples n thereof is stored in the storage unit 5.
Store. Also, standard audio 1 is input through input terminal 6.
The standard voice information of all necessary standard voices is stored in the time memory 7. Standard audio 1: Storage section 8 from memory 7
The word number registered in , the standard speech feature vector sequence of the word number specified by the input signal of input terminal 9 x ₁ ~
x _n , and the number m of samples, specific subsequence sample position T _i , and number k in the storage unit 10.
Store +1.

The specific subsequence corresponding position calculation unit 11 calculates the equation (1) when a specific subsequence exists, calculates the corresponding position, and sends the corresponding position information to the address control unit 2.
Send T _ja ~ T _jj . The feature vectors y _ja to y _ja+k of the first corresponding subsequence candidate and the feature vectors x _i to x _i+k of the specific subsequence are extracted from the storage location specified by the address control unit 2, and the distance calculation unit 12
The feature distance is calculated, and the result is sent to the adding unit 13. The addition unit 13 executes equation (3) and sends the distance D _ja to the minimum value calculation unit 14 . First sample position T _i of specific subsequence x _i ~ _{x i+k}
Similar calculations are performed for multiple corresponding subsequence candidates including the corresponding position T _je corresponding to T je and several sample positions T _ja ~ T _jj before and after it as the leading sample positions, and each time the distance D _ja ~ _{D jj} is Minimum value calculation section 1
Sent on 4th. If the newly input distance is smaller, the minimum value calculation unit 14 updates the minimum value with the sum of the distances, and also issues an instruction to the address control unit 2 via the signal line 15. The address control section 2 sends the position information of the corresponding partial sequence of the input audio at that time to the optimum matching position storage section 17 through the signal line 16 to update the sample position information. When all operations are completed for one specific subsequence x _i ~ _{x i + k} , the minimum value detector detects the minimum value at that time.
The adder of D _j is sent to the adder 18 through the signal line 19. If there is one specific subsequence, the calculation for the specific subsequence ends. If there are more, repeat the above process. Furthermore, if there are no specific subsequences, the above process is completely omitted.

Similarity calculations are then performed on the remaining sequences. This process is almost the same as the specific partial sequence, but in the remaining sequence corresponding position calculation unit 20, the sample position information stored in the storage unit 5 and the storage unit 10, and the sample position information stored in the storage unit 17 are calculated. Then, using equations (4) and (5) to determine the corresponding position by linear expansion and contraction, the remaining sequence of standard speech x ₁ ~
x _i-1 , x _i+k+1 ~x _o , and the remaining sequence of input audio y ₁ ~
The feature vectors y _i-1 , y _i+k+1 to y _o are taken out, and the output of the adder 13 is sent directly to the adder 18 through the signal line 21, and the minimum value calculator 14 does not operate. When the calculations of all remaining sequences are completed, the normalization unit 22 receives the data of the number m of samples from the storage unit 10, executes division by the number m of samples, and sends the result to the registration determination unit 23.

The above calculation of the degree of similarity for one standard voice is common to the case of recognition as well as the case of registration, as will be described later.

The registration determination unit 23 does not register the input voice when the sum of distances from the normalization unit 22 is greater than a certain value, but enters the registration audio signal creation process when it is less than a certain value. Using the contents of the optimal matching position storage section 17, the number of input audio samples storage section 5, and the standard audio sample information storage section 10, the corresponding sample of the feature vector is determined using equations 8, (9), and (10) for specific series. The position is calculated by the registered voice corresponding position calculation circuit 24, the corresponding registered voice vector value is calculated by the registered voice interpolation value calculation circuit 25 using equation (11), and the sample position of the feature vector is calculated by ( 12),
The registered voice corresponding position calculation circuit 24 calculates the registered voice corresponding position calculation circuit 24 using formulas (13), (14), and (15), and the registered voice interpolation value calculation circuit 25 calculates the registered voice vector value using formula (16). The sample position and vector value of the registered speech are stored in the standard speech temporary memory 7 at word number addresses corresponding to each specific series and each remaining series, and the learning of one standard speech is completed.

At the recognition stage, a recognition instruction signal is input from the mode switching terminal 1 to set the operation of the address control section 2 to the recognition mode. In the recognition mode, the storage of the input speech feature vector sequence and the storage of all necessary standard speech are the same as in the registration case. Next, the process of calculating the degree of similarity for one standard voice and obtaining a normalized output is the same as in the case of registration. In the case of recognition, the normalization result is sent to the optimal pattern detection section 26. The similarity calculation is executed for all standard voices, the sum of distances from the input voice is calculated for each standard voice, and the optimal pattern detection unit 26 detects the standard voice with the minimum distance among them and outputs the result. The signal is sent to a control unit (not shown) through the terminal 27.

As is clear from the above description, since the present invention has speaker adaptability, it can be used as a high-performance speaker-independent recognition device.

[Brief explanation of the drawing]

FIG. 1 is a diagram illustrating nonlinear correspondence and linear interpolation of feature vectors according to the present invention, and FIG. 2 is a block diagram showing an example of a speech recognition device according to the present invention. 1...Mode switching terminal, 2...Address control section, 3...Input audio input terminal, 4...Input audio feature vector series storage section, 5...Input audio sample number storage section, 6...Standard audio input terminal , 7...Standard speech temporary memory, 8...Standard speech feature vector series storage unit, 9...Registered word number input terminal, 10...
...Standard audio sample information storage unit, 11...Specific subsequence corresponding position calculation unit, 12...Distance calculation unit, 1
3, 18...addition section, 14...minimum value calculation section, 1
5... Minimum value signal line, 16... Optimum matching position signal line, 17... Optimum matching position storage section,
19, 21... Input signal line of addition section 18, 20...
... Residual series corresponding position calculation unit, 22... Normalization unit,
23...Registration determination unit, 24...Registered voice corresponding position calculation circuit, 25...Registered voice interpolation value calculation circuit, 2
6...Optimal pattern detection unit, 27...Result output terminal.

Claims (1)

[Scope of Claims] 1. A feature vector sequence of a standard voice and a sample position of a specific subsequence in that sequence are used as standard voice information, and a specific vector sequence of a known input voice is used as known input voice information, and both voice information are combined. Based on the information, registered voice information is created using a specific vector sequence and the sample position of a specific subsequence in that sequence, and the unknown input voice information is measured by measuring the similarity between the unknown input voice information and the registered voice information. In a speech recognition device that recognizes input speech, by measuring the degree of similarity according to a predetermined format, the number of feature vectors (k+1) of a specific subsequence (x _i to _{x i + k} ) of standard speech is comprising means for determining a corresponding subsequence (y _j ~y _j+k ) consisting of continuous feature vectors in the known input speech information, the number (m) of feature vectors of standard speech and the number of feature vectors of known speech; (n) and means for determining the number (l) of feature vectors of the registered speech according to a predetermined time interpolation coefficient (α), the means for creating a specific subsequence in the registered speech, the feature vector ( Z _h , Z _h+1 ,…, Z _h+k )
The size of is the relative weight of the amplitude interpolation coefficient (β), and the feature vectors of standard speech (x _i , x _i+1 , ..., x _i+k ) and the feature vectors of known input speech (y _j , y _j+1 , ..., y _j+k ), and the sample position (T _h ) of the first feature vector is known to be the sample position (T _i ) of the standard speech with the weight of the time interpolation coefficient (α). A means for creating a partial sequence linearly proportional to the sample position (T _j ) of the input audio, and creating a remaining sequence (Z _h+k+1 to Z _l ) of the registered audio excluding the specific partial sequence. , and the sample position (T _s ) of the feature vector (Z _s ) is the sample position (T _u ) of the standard speech residual sequence and the sample position (T _v ), and the size of the feature vector Z _s is linearly proportional to the amplitude interpolation coefficient (β), and the feature vector (x _u ) of the standard speech residual sequence and the feature vector (y _(v )) and one for creating a residual sequence linearly proportional to.