JPH02254500A - Vocalization speed estimating device - Google Patents

Abstract

PURPOSE:To obtain a stable estimated value even if plural stationary parts exist such as a word, a sentence clause and a continuous voice, etc. by calculating an estimated value of an average generation speed, on the basis of a signal related to the average generation speed outputted from a voice speed estimation use neural network. CONSTITUTION:An input sound signal is brought to A/D conversion in a voice analyzing part 11, and a 16-th cepstrum coefficient is derived at every frame. The cepstrum coefficient is inputted to a stationary part discrimination use neural network 13 through a delaying part 12. The network 13 discriminates whether the sound signal of a prescribed number of frames is a stationary part or a non-stationary part. Subsequently, on the basis of a discriminating signal of the number corresponding to a prescribed time from the network 13, an estimated value of an average vocalization speed is calculated at every prescribed time through an average vocalization speed calculating part 16 of an input voice by a vocalization speed estimation use neural network 15. In such a way, even if plural stationary parts exist, a stable estimated value is obtained without being influenced by each stationary part.

Description

DETAILED DESCRIPTION OF THE INVENTION <Industrial Application Field> The present invention relates to an improvement in a speech rate estimation device used in a speech recognition device or the like.

<Prior Art> In a speech recognition device, a plurality of word candidates obtained as a result of word recognition are narrowed down to word candidates that are likely to be correct, as follows. That is, when a word (or phrase) is uttered, the length of the speech interval can be relatively easily determined using parameters such as power and spectral change. Therefore, if the average speech rate is known, the number of syllables included in the speech segment can be found by dividing the speech segment length by the average syllable length, which is the reciprocal of the average speech velocity.

Once the number of syllables is known in this way, word candidates can be narrowed down to word candidates that are likely to be correct by selecting word candidates with the same number of syllables as the estimated number of syllables from among the multiple word candidates obtained. In this way, it is important to estimate the average speaking rate in speech recognition.

Previously, the present inventor proposed a method for estimating the average speaking rate as described below (Japanese Patent Laid-Open No. 59-61900). FIG. 5 is a schematic block diagram of an average speaking rate estimating device in this average speaking rate estimation method. The audio signal input to the audio analysis section 1 is A/D converted, and characteristic parameters such as power and cepstral coefficients are determined at a constant Freno length.

The stationary part detection unit 2 detects consecutive similar frames (i.e., the minimum value) from among several frame sections before and after a frame in which the spectrum change (difference in cepstrum coefficient values between frames several frames apart) takes a minimum value. The number of frames in which the difference in cepstrum coefficient value from the captured frame is less than or equal to a threshold value is determined, and the number of consecutive similar frames is determined as the stationary section length. Then, the utterance rate estimator 3 refers to a pre-stored correspondence table between steady-state section lengths and utterance speeds, and calculates the average utterance speed based on the above-determined steady-state section lengths.

<Problems to be Solved by the Invention> As mentioned above, in the above average speaking rate estimating device,
The speech rate estimator 3 outputs an estimated average speech rate for each stationary portion.

However, since a word is made up of multiple syllables, and there is no constant part in the vowels, nasals, etc. that form each syllable, the estimated average speaking rate of adjacent stationary parts varies, and the average speaking rate of the whole word is estimated. There is a time when it is difficult to do.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a speech rate estimating device that can stably estimate the average speech rate even if there are multiple stationary parts such as a single word phrase or continuous speech. .

<Means for Solving the Problems> In order to achieve the above object, the speech rate estimation device of the present invention inputs a signal representing a characteristic parameter of a predetermined number of frames of an input audio signal, and calculates the speech rate of the predetermined number of frames. A neural network for steady region identification that identifies whether a signal is a steady region or an unsteady region and outputs an identification signal, and a neural network for identifying a steady region that outputs an identification signal at a predetermined time. Based on the neural network for estimating speaking rate which inputs the corresponding number and outputs a signal related to the average speaking rate, and the signal related to the average speaking rate output from the neural network for estimating speaking rate, the average The present invention is characterized in that it includes a speech rate calculation section that calculates an estimated value of the speech rate.

Effect> When a signal representing a characteristic parameter of a predetermined number of frames of an audio signal is input to the neural network for identifying a stationary part, it is possible to determine whether the audio signal of the predetermined number of frames is a stationary part or an unsteady part. It is identified and an identification signal is output.

When the identification signals outputted from the steady-state region identification neural network are input into the vocalization rate estimation neural network in a number corresponding to a predetermined time, a signal related to the average vocalization rate is output. .

Then, based on the signal related to the average speaking rate outputted from the speaking rate estimation neural network, the speaking rate calculator calculates and outputs an estimated value of the average speaking rate. Therefore, the average speech rate is estimated every predetermined time period.

<Example> Hereinafter, the present invention will be explained in detail with reference to illustrated embodiments.

The present invention uses a neural network as a method for identifying the stationary portion of an input speech signal and as a method for estimating the average speaking rate. The above-described identification using a neural network is a method in which a rule for identifying the category to which input data belongs is determined by learning, and the category to which input data belongs is identified according to the determined rule.

Therefore, by using a neural network that has been correctly trained using appropriate training data in advance, it is possible to correctly identify the category to which input data belongs with simple processing.

FIG. 1 is a block diagram of an embodiment of the speech rate estimating device of the present invention. The input audio signal is A/D converted at a sampling period of 12 KHz in the audio analysis section 11,
A 16th-order cepstral coefficient is obtained for each frame (I frame is about 8 ms).

The 16th-order cepstral coefficient outputted from the speech analysis section 11 is inputted to the steady-state identification neural network 13 via the delay section 12, which will be described in detail later. Then, as will be described in detail later, the steady-state identification neural network 13 identifies whether the T-frame audio signal is a steady part or an unsteady part, and outputs identification data. Then, this stationary region identification neural
The identification data from the network 13 is input to the speech rate estimation neural network 15 via a delay unit I4, which will be described in detail later. The speaking rate estimation neural network 15 estimates the average speaking rate as will be described in detail later, and outputs data representing the estimated average speaking rate. Then, the speech rate calculation unit 16 calculates and outputs an estimated value of the average speech rate based on the data representing the estimated average speech rate from the speech rate estimation neural network 15.

Figure 2 shows the above-mentioned neural network 1 for identifying the stationary region.
FIG. 3 is a schematic diagram of the structure of No. 3. This neural network is a three-layer perceptron type neural network having a three-layer structure consisting of an input layer 21, an intermediate layer 22, and an output layer 23 in order from the bottom in the figure. The input layer 21 has 16XT units, the intermediate layer 22 has 8 units, and the output layer 23 has 2 units 24.2.
5 is placed. Above human horn layer 2! Data of one-order cepstrum coefficient of one frame among the 16-order cepstral coefficients of one frame of the input audio signal is input into one of the 16×T units. Further, the category "stationary part" is assigned to the unit 24 of the out-horn layer 23, and the category "unsteady part" is assigned to the unit 25. Input layer 2! Each unit is connected to all units of the intermediate layer 22, respectively. Further, each unit of the intermediate layer 22 is connected to all units 24 and 25 of the output layer 23, respectively.

However, units within each layer are not connected.

Here, the method of inputting the 16th order cepstral coefficients for T frames to the 16×T units of the input 521 is performed as follows. Figure 4 shows the delay section 12.14.
FIG. 2 is a detailed block diagram of FIG.

The number of tex'r units of the input layer 21 in the neural network 13 for stationary region identification is ! It is divided into 16 blocks consisting of Tpl units sequentially from the end unit, and the i-th cepstral coefficient is input to the first unit of the first block (l≦i≦16). Furthermore, a cepstrum coefficient obtained by delaying the i-th order cepstrum coefficient by one frame by the delay element 121 is input to the next unit. Furthermore, a cepstrum coefficient obtained by delaying the i-th order cepstrum coefficient by two frames by two delay elements 121 is input to the next unit. Thereafter, similarly, the cepstrum coefficient obtained by delaying the i-th order cepstrum coefficient by (T-1) frames by (T-1) delay elements 121 is input to the last unit. In this way, the 16th order cepstral coefficients are sequentially distributed from the θ frame (T
-1) frames, and 16th-order cepstral coefficients for T frames are input to each unit of the input layer 21.

Learning of the above-mentioned neural network 13 for identifying a stationary region is performed by the error backpropagation method as follows.

That is, it is determined by inspection whether the T frame portion of the voice signal of many speakers is a stationary portion or an unsteady portion, and the 16th order of the T frame portion of the voice signal, which is known as the stationary portion or the unsteady portion, is determined by inspection. Find the cepstral coefficients of and use them as learning data.

Then, the learning data consisting of 16th order cepstral coefficients for T frames, which is the stationary part, is transferred to the 16×T
, the input value to the unit 24 to which the category "stationary part" is assigned is "B", and the input value to the unit 25 to which the category "unsteady part" is assigned is input to the output layer 23. Input training data whose input value is "0".On the other hand, 16 T frames of non-stationary part are input.
The training data consisting of the following cepstral coefficients is input to the input layer 2I.
When the input value is input to each unit in the output layer 23, the input value to the unit 24 to which the category "stationary part" is assigned is "0", and the input value to the unit 25 to which the category "unsteady part" is assigned is "0". Input training data in which the input value is "Bi. Then, the neural network 13 for stationary region identification uses the unit 24.25 of the output Ji 123.
The network structure is determined by resetting the weights of the network so that the output value is the same as the training data.

Identification of the category to which the input audio signal for T frames belongs is performed as follows by the neural network 13 for stationary part identification which has been trained in two series. That is, 1 'r frames from the speech analysis section 11 are added to the input horn layer 21 of the neural network I3 for stationary region identification.
Sixth-order cepstral coefficients are input. As a result, identification data is output such that the output point value from the unit of the output layer 23 to which the category to which the 16th-order cepstrum coefficients for the 'r frames belong is assigned is maximized.

Therefore, if the input audio signal is a stationary part, “
Identification data that maximizes the output value from the unit 24 assigned to the "stationary part" is output.In addition, when the input audio signal is an unsteady part, the identification data that is assigned to the "unsteady part" or the assigned Identification data such that the output value from the unit 25 is the maximum is output.

FIG. 3 is a schematic diagram of the structure of the neural network 15 for estimating speech rate. This neural network is a three-layer persebutron type neural network similar to the neural network 13 for stationary region identification, and is based on the discrimination from the output layer 23 of the neural network 13 for stationary region identification learned as described above. Operates using signals as input data.

The input layer 31 of the neural network 15 for estimating speech rate has 2×25=50 units, the intermediate layer 32 has 20 units, and the output layer 33 has 1 unit.
It has one unit. The 50 units of the input layer 31 are divided into 25 groups on two sides.

Then, one unit 34, 35 .・・・
, 36 are input with the output signal from the unit 24 of the output layer 23 of the neural network 13 for stationary section identification (ie, the signal corresponding to the category "stationary section"). Also, the other units 37, 38 of each group...
, 39 includes the neural network 1 for stationary region identification.
The output signal from the unit 25 of the output layer 23 of No. 3 (that is, the signal corresponding to the category "unsteady part") is input. At that time, 34 units out of the above 25 groups
．． The group consisting of 37 has a neural network for stationary region identification.
The first identification data from the network 13 is input, and the group consisting of units 35 and 38 is input with the second identification data from the neural network I3 for stationary region identification, and so on, and the group consisting of units 36 and 39. Neural network I for stationary region identification is included in the group.
Enter the 25th identification data from 3. Zunawara,
Input F of neural network 15 for estimating speech rate
J31 contains 16 frames of 25 x T frames of the input audio signal.
Based on the following cepstral coefficients, 25 consecutive identification data output from the neural network 13 for stationary region identification are input.

Here, the reason why 25 continuous pieces of identification data outputted from the neural network 13 for stationary region identification are input to the input layer 31 is as follows.

That is, since one frame in this embodiment is 8 ms, there are 125 frames in one second. Therefore, in this embodiment, if the number T of frames used to identify a stationary area by the stationary area identification neural network 13 is 5 frames, then 25×T frames will be 125 frames, or 1 second. Therefore, by inputting 25 continuous pieces of identification data to the neural network 15 for stationary part identification, identification data corresponding to one second of input audio signal is inputted. In short, the input layer 31 of the speech rate estimation neural network 15 only needs to be input with identification data from the steady-state region identification neural network 13 in a number corresponding to a predetermined amount of time.

In this case, neural network 1 for estimating speech rate
As a method of inputting 25 continuous pieces of identification data from the neural network I3 for stationary region identification to the input layer 31 of No. 5, for example, there is the following method. Here, the fourth
As shown in the figure, the delay section 14 includes two delay sections 141,
Consists of 142. 3 and 4, the output signal from the unit 24 assigned to the "stationary section" in the output layer 23 of the neural network 13 for stationary section identification is directly input to the unit 34 of the input layer 3I,
The unit 37 directly receives the output signal from the unit 25 assigned to "non-stationary" in the neural network 13 for identifying a stationary region.

Further, the output signal from the unit 24 of the neural network 13 for stationary region identification is input to the unit 35 after being delayed by 'r frames by the delay element 143 of the delay section 141. The output signal from the unit 25 of the section identification neural network 13 is input after being delayed by a time of T frames by the delay element 143 of the delay section 142. Similarly, the output signal from the unit 24 of the stationary region identification section neural network is input to the unit 36 after being delayed by 24X'l' frames by 124 delay elements 1/13. , a unit 39 receives the output signal from the unit 25 of the neural network 13 for stationary region identification by a delay element 143 of 24 pl to 24×T.
All you have to do is delay the time by one frame and shift the input.

Furthermore, among the 11 units of the output layer 33, the category "silence" is assigned to the unit 41, the category "1" (1 mora/second) is assigned to the unit 42, and the category "2" is assigned to the unit 43. ” (2 mora/sec), and in the same way, the unit 46 is assigned the category “10” (
10 mora/sec). Each unit of the input layer 31 is connected to all units of the intermediate layer 32, respectively. Furthermore, each unit of the intermediate layer 32 is connected to all units of the output layer 33, respectively. However, units within each layer are not connected.

The learning of the neural network 15 for estimating the speaking rate is performed by the error backpropagation method as described below while connected to the neural network 13 for identifying the stationary part 13 as described above. In other words, one second of audio signals from multiple speakers (
The time series of 16th-order cepstral coefficients for each frame in 25XT frames is used as learning data. Further, the average speaking speed in the learning data is calculated through observation, and the data representing the calculated average speaking speed is used as teacher data. Then, for example, the learning data of the silent section is used as the neural network I for stationary region identification, which has already been trained.
When inputting to each unit of the input layer 21 in 3,
In the neural network 15 for estimating speech rate, the input value to the unit 41 to which "silence" is assigned in the output layer 33 is "bi", and the input value to other units is "0".
Input the teacher data that is ”.Also, in the sound section, the average speaking rate calculated from the inspection is Nmo97.
If the learning data for seconds is input to the input layer 21 of the neural network 13 for stationary part identification, the input value to the unit 45 of the output layer 33 of the neural network 15 for estimating speech rate is "Bi" and other units Input teacher data whose input value is "0".In that case, the calculated average speaking rate is, for example, 2.5 mo/
If it is seconds, the input value to the unit 43 to which "2" of the output layer 33 is assigned is "0.5", and the input value to the unit 44 to which "3" is assigned is "0.5". ” and the input values to other units are “0”. That is, in this learning, when the 16th order cepstral coefficient of the audio signal for 1 second (that is, 25×'r frames) is input to the input layer 21 of the neural network 13 for stationary region identification, It learns to identify the number of moras contained in an input audio signal of minutes. Then, the neural network 15 for estimating the speaking rate is connected to each unit 41 . ..., the network structure is determined by resetting the weights of the network so that the output value from 46 becomes the same as the teacher data. In this learning, we will use a neural network for stationary region identification! Leave the weight of network 3 unchanged.

The number of moras (that is, the estimated speaking rate) included in the input audio signal for one second is calculated as follows by the neural network 13 for identifying the stationary part and the neural network 15 for estimating the speaking rate that have been trained as described above. It is identified by

First, the 16th order cepstral coefficients for the first T frames of the input audio signal from the audio analysis unit 2 are input to the input layer 21 of the neural network 13 for stationary part identification via the delay unit 12 as described above. Ru. As a result, identification data representing the category to which the 16th-order cepstral coefficients for T frames belong is output from the output layer 23, and input layer 3!
unit 34.37.

Subsequently, the 16th order cepstral coefficients for the second T frame of the input audio signal are input to the input layer 21 of the neural network 13 for stationary region identification.

Then, identification data representing the category to which the 16th-order cepstral coefficients for the second T frame belong is output from the output layer 23, and the unit 34.3 of the input layer 31 in the neural network I5 for estimating speech rate is outputted from the output layer 23.
7 is input. At the same time, the identification data for the 16th-order cepstral coefficients for the first T frames delayed by the delay element 143 of the delay unit I4 by the time corresponding to T frames is transmitted to the input layer 3I in the neural network 15 for estimating speech rate. unit) 35.3
8 is input.

Similarly, when the 16th order cepstral coefficients for the 25th T frame are input to the input layer 21 of the neural network 13 for stationary region identification, this 25th T
Identification data for the 16th order cepstral coefficients for frames are output from the output layer 23 and input to the neural network 15 for estimating speech rate/1iff31.
unit 3437. At the same time, the identification data for the 16th-order cepstral coefficient for the 24th T frame, delayed by the time corresponding to T frames by the delay element 143 of the delay unit 14, is transmitted to the input layer 31 in the neural network 15 for estimating speech rate. unit 35, 38, and so on, 24
The identification data for the 16th-order cepstral coefficients for the first T frames delayed by the time corresponding to 24×T frames by the delay elements 143 are sent to the units 36. 39. In this way, the neural system for stationary region identification based on the 16th order cepstral coefficients of the input audio signal for 1 second (i.e., 25×T frames)
The 25 pieces of identification data from the network 13 are input to the input layer 3I of the neural network 15 for estimating speech rate.

Then, each unit 41, . . . , 4 of the output layer 33 in the neural network 15 for estimating speech rate
From 6 onwards, the number ranges from 0 to 1 depending on the degree of identification of the category to which the 16th order cepstral coefficient of the input audio signal for 1 second belongs.
The output value that takes the value up to is output.

As described above, in this embodiment, the neural network 13 for stationary part identification and the neural network 15 for estimating speech rate calculate the input signal for one second based on the 16th-order cepstral coefficients for one second of the input audio signal. Identify the number of moras included in the audio signal (ie, estimate the average speaking rate). Therefore, even if a plurality of steady parts exist in one second of the input audio signal, it is possible to stably estimate the average speaking rate without being bound by individual steady parts.

'' Output data representing the estimated average speaking speed outputted from the output layer 33 of the neural network for estimating speaking speed 15 is input to the speaking speed calculating section 16 in a double series. Then, based on the output data representing the estimated average speaking speed, an estimated value of the average speaking speed is calculated as follows.

That is, neural network 1 for estimating speech rate
Out of the output values from all the units 41..., 46 of the output layer 33 in 5, the unit outputting the maximum value (
This unit is designated as Ul) and the unit outputting the second largest value (this unit is designated as U2) is selected. If the unit UI is the unit 41 assigned to the category "silence", it is determined that the corresponding input audio signal for one second is a silent section, and an estimated average speech rate value "0" is output.

In other cases, the average speech rate estimate is calculated according to the following equation.

Average speaking rate estimate - (Ulxv(Ul)+U2xV(tJ2)l/(v(U
l)+v(U2)1 However, V(Un): The output value of the unit Un, that is, the estimated speech rate is calculated by the output value from each unit weighted by the number of moras assigned to that unit. It is.

Thus, a neural network for estimating speech rate!
An estimated value of the average speaking speed is calculated and outputted based on the output data representing the estimated speaking speed outputted from step 5.

As described above, in the average speech rate estimating device of this embodiment, when the 16th order cepstral coefficients for one second of the input audio signal are input to the input layer 21 of the neural network 13 for stationary part identification, The neural network I3 identifies whether the input audio signal is a stationary portion or an unsteady portion for each T-foam, and sequentially outputs identification data. Then, the speaking rate estimation neural network 15 identifies the number of moras included in the input audio signal for one second (that is, estimates the average speaking rate) based on the identification data for one second of the input audio signal. ), and outputs output data representing the estimated average speaking rate. And the speaking speed calculator! 6, the estimated average speaking speed is calculated based on the output data representing the estimated average speaking speed. In other words, the average speaking rate is estimated based on the 16th order cepstral coefficients for one second of the input audio signal.

Therefore, according to this embodiment, a neural network that generates by itself a rule for identifying whether an input audio signal for T-foams is a stationary part or an unsteady part by learning, and It is possible to estimate the average speaking rate of an input audio signal by using a neural network that automatically generates rules for identifying the number of moras contained in an input audio signal of 1 second duration. Even if there are multiple steady parts, the average speaking rate can be stably estimated without being bound by individual steady parts.

In the embodiment described above, a number of pieces of identification data corresponding to one second of the input audio signal are input to the neural network 15 for estimating the speaking rate. However, the present invention is not limited to this, and the number of identification data may be equivalent to, for example, 0.5 seconds.

In the above embodiment, the input layer 3 of the neural network 15 for estimating speech rate! The number of units is set at 50, but the present invention is not limited to this, and may be optimally determined depending on the frame time and the number of frames used to identify a stationary area by the neural network 13 for identifying a stationary area. Just set it.

In the above embodiment, 16th order cepstral coefficients are used as feature parameters. However, the present invention is not limited to this, and a spectrum, an autocorrelation coefficient, a bandpass filter group output, etc. may be used.

Further, in the above embodiment, a three-layer percept (non-type neural network) is used, but a perceptron-type neural network with four or more layers may be used.

<Effects of the Invention> As is clear from the above, the speech rate estimating device of the present invention includes a neural network for identifying a stationary region, a neural network for estimating a speech rate, and a speech rate calculation unit, Based on the characteristic parameter of the number of frames, the above-mentioned neural network for identifying a stationary section identifies whether the audio signal of the predetermined number of frames is a stationary section or an unsteady section, and from this neural network for identifying a stationary section, Based on the number of identification signals corresponding to a predetermined time period, the neural network for estimating speaking rate outputs a signal related to the average speaking rate of the input audio signal, and outputs a signal related to the average speaking rate of the input audio signal.
Based on No. 3, the estimated value of the average speaking rate is calculated by the speaking rate calculation unit every predetermined time period, so that even if there are a plurality of steady parts in the input audio signal for the predetermined time period, It is possible to stably estimate the average speaking rate without being bound to individual stationary parts.

[Brief explanation of drawings]

FIG. 1 is a block diagram of an embodiment of the speech rate estimating device of the present invention, FIG. 2 is a schematic configuration diagram of a neural network for identifying a stationary region in FIG. 1, and FIG. FIG. 4 is a detailed block diagram of the delay section in FIG. 1, and FIG. 5 is a block diagram of a conventional speech rate estimation device. 11...Speech analysis unit, 12.14...Delay unit, 13...Neural network for stationary part identification,
15... Neural network for estimating speech rate,
16... Speech rate calculation unit, 21.31... Input layer, 22.32... Intermediate layer, 23.33... Output layer, 143... Delay element.

Claims (1)

[Claims] (1) Input a signal representing the characteristic parameters of a predetermined number of frames of the input audio signal, identify whether the audio signal of the predetermined number of frames is a stationary part or an unsteady part, and output an identification signal. inputting a neural network for identifying a stationary part and the identification signal outputted from the neural network for identifying a stationary part in a number corresponding to a predetermined time;
A neural network for estimating speaking speed that outputs a signal related to the average speaking speed, and an estimated value of the average speaking speed is calculated based on the signal related to the average speaking speed output from the neural network for estimating speaking speed. A speech rate estimating device comprising a speech rate calculation unit that calculates a speech rate.