JPH0222960B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION FIELD OF INDUSTRIAL APPLICATION The present invention relates to a device for detecting the start and end of speech used in a speech recognition device.

Configuration of Conventional Example and Its Problems As a conventional example of a method for detecting the start and end of speech, a method using signal energy and the number of zero crossings is known. This is Yasunaga Niimi: Speech Recognition, Kyoritsu Shuppan (1979), or LRRabiner
and MRSambur: An algovithm for
determining the endpoint of isolation
utterances, Bell Syst.Tech.J., (1975).

The number of zero crossings means leaving only the sign of the signal,
This is the average number of zero-crossings in a certain time period of a zero-crossing wave whose amplitude is quantized to 1 bit. The number of zero crossings of a signal with a spectral structure, such as voice, corresponds well to the dominant frequency components in the spectrum. Figures 1a to 1c show the distribution of the number of zero crossings of the audio signal, where a is the distribution of silent sounds, b is the distribution of unvoiced sounds, and c is the distribution of voiced sounds. As can be seen from the figure, the number of zero-crossings in the audio signal is dominated by low frequency components like voiced sounds, small values as shown in Figure 1c in voiced sounds, and high frequency components like unvoiced sounds. Speech with a dominant component shows a large value as shown in FIG. 1b. The conventional method for detecting the beginning and end of a voice uses the number of zero crossings to improve the detection accuracy of unvoiced consonants, which have a small signal energy but a large number of zero crossings.

Referring to the drawings below, we will explain the starting point and
The termination detection method will be explained.

FIG. 2 shows the configuration of a conventional example, and FIG. 3 shows an example for explaining the operation of the voice start/end detection method in the conventional example.
A signal including audio is divided into frames (e.g.
(10 msec length), the signal energy E(n) (n is the frame number) and the number of zero crossings Nz(n) are converted into two characteristic parameters. Reference numeral 3 denotes a start/end candidate determination unit that detects a portion that is definitely a voice section based on the energy level of the signal, and has two thresholds E ₁ and E ₂ for the signal energy E(n).
Applying (E ₁ > E ₂ ), start candidate n ₁ and end candidate of audio
Find n ₂ . This is shown in the example in Figure 3a,
When the energy value exceeds E ₂ and then exceeds E ₁ without falling below E ₂ , it is considered that the voice section has entered, and the point where it exceeds E ₂ is set as the starting point candidate n ₁ . Termination candidate n ₂ reverses the time axis,
Determine in a similar manner. 4 in FIG. 2 is a voice start/end determining section. Here, using the number of zero crossings Nz(n) of the signal calculated by the number of zero crossings calculation unit 2 and the threshold value No. but,
It is checked whether there is a voice start/end candidate (n ₁ , n ₂ ) determined by the start/end candidate determining unit 3. As shown in the example in Figure 3b, the number of zero crossings Nz in the section of several frames before the starting edge candidate n ₁
Count the number of frames in which (n) is greater than the threshold No. If the number is greater than a certain value (for example, 3), it is assumed that there is an unvoiced sound before the starting point candidate n ₁ , and the frame n that exceeds the threshold No. ′ Move the starting end to ₁ . The same applies to the termination. However, FIG. 3b shows the case where the terminal end _n2 remains as it was. In this way, the final beginning and end of the voice (n′ ₁ , n ₂ )
is determined.

However, the method using the number of zero crossings as described above cannot reduce the dropout of voiced consonants (for example, /b/, /d/), which have small energy and a small number of zero crossings. In addition, noise caused by opening the lips or breathing sounds is likely to be added to the beginning and end of the voice. Figures 4a and 4b show the energy changes of the voice added with the above noise, and a
shows the power change of the strange voice (/ijoo/) as an example when noise due to lip movements is added to the beginning, and b shows the power change of the voice (/ideju/) as an example when noise due to breathing sounds is added to the beginning. This shows the change in the power of the voice. In a case like the example shown in the figure, in the conventional example, the starting end becomes a noise part. As described above, the conventional method has the disadvantage that the beginning and end positions may be incorrect and phonemes may be dropped or phonemes may be added due to noise.

OBJECTS OF THE INVENTION In view of the above-mentioned drawbacks, the present invention provides a voice start/end detection device that is less likely to drop out of voice, add less noise, and has high positional accuracy.

Structure of the Invention In order to achieve the above object, it is necessary to
10msec), a voice/non-speech determination section that determines whether there is voice or no voice, a section that detects voice start/end candidates based on the persistence of voice/silence determination results for each frame, and a section that detects voice start/end candidates based on the persistence of voice/silence determination results for each frame; A part that determines the starting and ending positions based on the dynamic characteristics of the change in the energy of the signal and the magnitude of the change in the spectrum when the signal changes from sound to silence,
The position of the start and end of the voice is detected from a signal including the input voice.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 5 shows a block diagram of a speech start/end detection device incorporated in a speech recognition device according to an embodiment of the present invention. In the figure, reference numeral 5 denotes an energy extraction section, which is composed of a rectifying and smoothing circuit and extracts the power of the signal for each frame. 6 is a spectrum shape extraction section, which is composed of three types of bandpass filter groups for low frequency (250 to 600 Hz), mid frequency (600 to 1500 Hz), and high frequency (1500 to 4000 Hz), and a rectification and smoothing circuit. The power per frame in the band is used as spectral information. The energy extractor 5 and the spectral shape extractor 6 constitute a feature extractor 13. Reference numeral 7 denotes a multiplexer for inputting the signal power from the energy extraction section 5 and the band filter power from the spectrum shape extraction section 6 to the sound/non-sound determination section 8 in a time-division manner. Reference numeral 8 denotes a voiced/non-sound determining section, which is used to determine whether there is no voice, unvoiced sound, or voiced sound. Reference numerals 9 and 10 are threshold memories and standard pattern memories, in which constant values used by the voice/silence determining section 8 are stored. The threshold memory 9 stores two power thresholds E ₁ ,
E ₂ (E ₁ > E ₂ ) is stored. Further, the standard pattern memory 10 stores coefficients of two types of linear discriminant functions: a linear discriminant function for discriminating silent/voiceless sounds, and a linear discriminant function for discriminating silent/voiced sounds. And these two threshold values E ₁ ,
E ₂ and the coefficients of the two linear discriminant functions are obtained in advance by statistical processing of voice data uttered under the environment to be used and are stored. 11 is a start/end candidate detection unit, and a voice/non-speech determination unit 8
Based on the duration of the sound/non-sound determination results for each frame sent from the system, candidates for the start and end of the audio are detected. Reference numeral 12 denotes a starting end/terminating end determining section, which determines the final starting end/end end. Note that FIGS. 8 to 12 are constructed with one microprocessor.

The operation of the audio start/end detection device configured as described above will be explained.

The signal containing the voice input from the microphone etc. is the fifth
Each frame is converted into power PW and three band powers P _i (i=1 to 3) by the energy extractor 5 and spectral shape extractor 6 shown in the figure. this
PW and P _i are input to the sound/non-sound determination unit 8 via the multiplexer 7 . The voice/silence determination unit 8 logarithmically transforms the input four parameters PW and P _i (i=1 to 3) to obtain logarithmic power LPW and logarithmic band power LP _i (i=1 to 3). And L.P.W.
and LP _i (i = 1 to 3), the thresholds E ₁ and E ₂ stored in the threshold memory 9 and the standard pattern memory 10, and the coefficients of the two linear discriminant functions. Determine whether the frame is speech or silent. This sound/non-sound judgment first uses two energy thresholds E ₁ and E ₂ (E ₁ > E ₂ ).
Judgment is made by comparing the logarithmic power LPW with the logarithmic power LPW. The two thresholds E ₁ and E ₂ are set to values such that if LPW>E ₁ , there is definitely a sound, and if LPW<E ₂ , there is definitely no sound, so the judgment result is given by formula (1)
It will look like the one shown below.

If LPW>E ₁ then sound LPW<E ₂ then no sound E ₂ If LPWE ₁ then Indefinite Equation (1) If the determination using the amount of energy called LPW yields an undetermined result, then we can further consider the existence by the spectral shape. Performs sound/silence judgment. This is the logarithmic power LP _i of the three bands: low, mid, and high.
(i=1 to 3) is a parameter representing the spectral shape, and the presence/absence of speech is determined by calculating the value of the discriminant function using the coefficients of two types of linear discriminant functions stored in the standard pattern memory 10. It is something. One of these two linear discriminant functions is for discriminating between voiced and unvoiced sounds, and the other is for discriminating between voiced and unvoiced sounds. The linear discriminant function FX is shown in equation (2),
The standard pattern memory 10 stores A _i (i=1~
3) and _i (i=1 to 3) are stored for each of two types of linear discriminant functions: silent/unvoiced and silent/voiced.

FX= ₃ 〓 ⁱ⁼¹ A _i (LP _i − _i ) ...Equation (2) (A _i is a coefficient, _i is an average value) A _i in Equation (2) is the optimal discrimination between two classes. Intraclass variance of two classes,
It is obtained from the condition for maximizing the Fisher ratio, which is the ratio of interclass variance. In this example, A _i and
LP _i is determined in advance by statistically processing the silence, unvoiced sounds, and voiced sounds of the audio data uttered under the usage environment. And the FX value is negative when the input is silent,
It is set to take a positive value when the input is an unvoiced sound or a voiced sound. Therefore, to determine voice presence or non-voice based on the spectral shape, calculate two linear discriminant functions: silent/unvoiced and silent/voiced, and if either one takes a positive value, there is a voice, and both have negative values. If so, it is determined that there is no sound. The sound/silence determination result for each frame thus obtained is sent to the start/end candidate detection section 11 shown in FIG. Starting point/
The end candidate detection unit 11 detects a start end candidate and an end end candidate of the audio based on the duration of the sound/silence determination result obtained for each frame. The start/end candidate detection section 11 is constructed using two registers of a microprocessor as a counter and further uses a comparison calculation function. Only one counter is used to detect the starting edge candidate, and both counters are used to detect the ending edge candidate. FIG. 6 shows the flow of processing for detecting a starting edge candidate. FIG. 6 shows that when five or more consecutive frames are determined to have sound, the first frame is selected as the starting edge candidate. Processing A in Fig. 6 is a counter of sound frames (COUNT in Fig. 6),
Starting edge candidate frame number storage area (Fig. 6
FRAMES) and a reset for initializing the processing frame position (FIG. 6I). 6th
Figure processing b is an update of the processing frame position. Process C is branching based on a comparison of whether the processing frame is sound or silent. If the frame being processed is a sound frame, 1 is added to the sound frame counter (COUNT) (processing d in Figure 6).
Further, if the starting edge candidate frame number storage area (FRAMES) remains reset to 0, the number (I) of the frame currently being processed is stored (processing E, F). In the processing step, it is determined whether the counter 5 of a sound frame has been reached. If the counter is less than or equal to 5, the process returns to step B. If the counter is greater than or equal to 5, it means that a starting edge candidate has been detected, and the starting edge candidate detection process ends. If a silent frame is found in process C until the process is completed, the sound frame counter and the starting edge candidate frame number storage area are reset in process C, and the process returns to B. Since the sound frame counter is reset by the processing unit when there is a silent frame, it becomes a counter for the number of consecutive frames with sound. Therefore, the determination as to whether or not to process is a determination as to whether there is a continuous sound for five or more frames. Therefore, even if there are two or three frames that are determined to be voiced due to noise caused by lip movement before the start of the voice, if there is even one frame that is determined to be silent after that, that frame is removed. Once the starting edge candidate is detected in this way, processing for detecting the ending edge candidate is then performed. FIG. 7 shows the flow of processing for detecting termination candidates.

Process A in Fig. 7 is for initializing the silent frame counter (COUNT1 in Fig. 7), the sound frame counter (COUNT2 in Fig. 7), and the end candidate frame number storage area (FRAMEE in Fig. 7). This is a reset. Process B in FIG. 7 is an update of the processing frame position (FIG. 7 I). Process C is branching based on a comparison of whether the processing frame is sound or silent. If the frame being processed is silent, the silent frame counter is updated and the sound frame counter is reset (processing D, E).
Furthermore, if the silent frame counter is 2 or more and the end frame number storage area has been reset, the number of the frame for which the silent frame counter becomes 1 is stored as the end candidate frame in the end frame storage area (processes B and T). . In processing Q, it is determined whether the silent frame counter has reached 30 or not. Then, if the silent frame counter is less than 30, the process returns to processing, and if it becomes 30 or more, it is assumed that the audio has ended and the process ends. Processing 3, which branches when there is sound in processing C, is a process that determines how many consecutive frames have had sound after the end candidate frame is stored, and if there are 5 or more consecutive frames, the sound is If it is not completed, return to processing A and redo the terminal candidate detection. If the number of sound frames is less than 5 frames, it is regarded as noise, and since that section is a silent section, the section length is added to the silent frame counter in process 2.

The end candidate is the possibility of the end of the audio when there are two consecutive silent frames, so the first silent frame is the end candidate for the audio, and there are 5 or more consecutive frames with sound within 29 frames from the end candidate frame. If there is no end candidate frame, the previous end candidate frame is set as the end candidate frame. If from the end candidate
If there are 5 or more consecutive frames with sound after 29 frames, it is assumed that the audio has not ended yet, the counter and end candidate frames are all reset, and the end detection process shown in Figure 7 is restarted from the next frame. . Through such processing, noise added to the end of 4 frames or less is removed.
The start/end determination unit 12 determines the final start/end based on the magnitude of change in the power LPW and spectrum LP _i in the vicinity of the start/end candidate frame detected by the start/end candidate detection unit 11 . As a parameter representing the magnitude of power change, the difference value LPWD of logarithmic power LPW obtained for each frame is used as shown in equation (3).

LPWD _j = LPW _j −LPW _j-1 ...Equation (3) (where j is the frame number) In addition, as a parameter representing the magnitude of the change in the spectrum, the Euclidean value of the band logarithmic power LP _i shown in Equation (4) is Use distance SPD.

SPD _j = ₃ 〓 ⁱ⁼¹ (LP _ij -LP _ij-1 ) ² ...(4) (However, i represents the band and j represents the frame number) The parameter LPWD is used when the power is increasing. It takes a positive value, and if the power is decreasing, it takes a negative value. Furthermore, SPD takes a large value where the shape of the spectrum changes significantly, such as when changing from silence to sound. To determine the starting edge, first, a frame in which LPWD takes a positive value is searched from the starting edge candidate toward the trailing edge. next
LPWD has a positive value in a total of 3 frames, from the first frame where LPWD becomes positive to the next two frames.
Find the frame with the maximum SPD and determine that frame as the starting frame.

To determine the end, first, a frame whose LPWD takes a negative value is searched from the end candidate frame toward the start end. Next, find the frame in which LPWD has a negative value and SPD is maximum among a total of three frames, two frames before the frame where LPWD first becomes negative, and determine the frame one before that frame as the terminal frame. do. The start and end points obtained in this way are used in a speech recognition device.

According to this embodiment, the sound/silence determination unit 8 uses a linear discriminant function for an input signal with a low energy level to determine whether or not there is sound based on the difference in spectral shape from silence. Therefore, dropout of voiceless consonants and voiced consonants with low energy can be reduced. Furthermore, since the start/end candidate detection unit 11 performs detection taking into consideration the continuity of the voice, short noises added before and after the start/end of the voice can be removed. Furthermore, the start/end determining unit 12 determines the start/end positions using the change in energy and the magnitude of the change in spectral shape from silent to active or vice versa, resulting in high positional accuracy. You can get the start and end of the audio. 8th
The figure shows an example in which the start/end detection according to an embodiment of the present invention is applied to a voice uttered as "base" (/dodai/), and FIG. 8a shows the logarithmic power LPW.
b is spectrum change SPD, c is power change
The solid line of LPWD, d shows the value of the linear discriminant function for discriminating silent/voiceless sound, and the broken line shows the value of the linear discriminant function for discriminating silent/voiced sound. In the example shown in FIG. 8, noise can be seen at both the start and end ends. In the silence/speech determination unit 8 for each frame,
If LPW is greater than or equal to E ₁ , or if LPW is between E ₁ and E ₂ , then by considering the sign of the two linear discriminant functions shown in d, we can calculate the difference between A to B and C to D shown in a. Determine the interval as having sound. This removes the noise at the beginning. The start/end candidate detection unit 11 sets the start end candidate frame as A and the end candidate frame as B depending on the persistence of the voiced and silent frames. At this time, since the sound intervals from C to D are less than 5 frames, they are determined to be noise. Then, the start/end determining section 12 determines the start/end A' and the end/B' based on the change c in the logarithmic power and the change b in the spectrum, thereby obtaining the correct start/end positions from which noise has been removed. As a result of conducting an evaluation experiment of an embodiment of the present invention using 212 words uttered by one male speaker whose starting and ending points have been visually labeled in advance, it was found that the difference from the label was 2.
93.4% are within the frame at the start end, and 93.4% at the end
We obtained results of 92.9%, 97.6% at the start end, and 97.2% at the end, where the difference from the label was within 3 frames. There were only two words with serious errors in which a phoneme was dropped at the beginning, and only two words in which a phoneme was dropped at the end, and there were no errors caused by the addition of noise.
Good results were obtained, and it was confirmed that the audio start/end detection device according to the present invention operates effectively.

In addition, in the above explanation, we use band logarithmic power as a parameter representing the spectrum shape.
We have explained the case where a linear discriminant function is used to determine silence, but the power spectrum obtained by Fourier transform of the signal or linear predictive analysis and the LPC-cepstral coefficient obtained by linear predictive analysis are used as parameters representing the spectral shape.
A statistical distance measure such as Bayesian judgment or Mahalanobis distance may be used to determine whether there is a sound or no sound.

Effects of the Invention As described above, the present invention includes a speech presence/non-speech determination unit for each frame that uses not only signal energy information but also spectral shape, a start/end candidate detection unit that takes into consideration the continuity of audio, The voice start/end position is composed of a determining section that determines the start/end position based on changes in energy and amount of change in spectral shape.
This device provides an end detection device that uses the difference in spectral shape using a statistical distance measure from standard spectral patterns of silence, unvoiced sounds, and voiced sounds to determine whether there is a sound or not. It is possible to reduce the dropout of unvoiced consonants and voiced consonants, and since starting and ending candidates are detected based on the persistence of voicing, there is less noise added, and the starting and ending positions can be determined based on the magnitude of changes in energy and spectrum. This has the advantage of providing a high level of location information for determining the location.

[Brief explanation of drawings]

Fig. 1 is a distribution diagram of the number of zero crossings used in the past, Fig. 2 is a block diagram of a conventional start/end detection device, and Fig. 3 is a diagram illustrating an example of the operation of the conventional start/end detection device. FIG. 4 is a diagram showing the energy change of a voice to which noise has been added in a conventional method. FIG. 5 is a block diagram of a voice start/end detection device according to an embodiment of the present invention. FIG. 6 is an example of an embodiment of the present invention. Flowchart diagram showing the start end candidate detection process in 7th
FIG. 8 is a flowchart showing termination candidate detection processing in an embodiment of the present invention, and FIG. 8 is a diagram illustrating an example of operation in the embodiment of the present invention. 5...Energy extractor, 6...Spectrum shape extractor, 7...Multiplexer, 8...Sound/
Silence determination unit, 9... Threshold memory, 10... Standard pattern memory, 11... Start/end candidate detection unit,
12... Start/end determining unit, 13... Feature extraction unit.

Claims (1)

[Claims] 1. A feature extraction unit that extracts feature quantities representing the energy and spectral shape of the signal from a signal containing audio for each section of a certain time length, and a signal input using the feature quantities. a sound/silence determination unit that determines whether there is sound or silence for each section of a certain length of time;
A start/end candidate detection unit that detects candidates for the start/end of speech based on the duration of the judgment results using the time series of the voice/silence judgment results, and a signal energy change and spectrum before and after the start/end candidates. 1. A voice start/end detection device comprising: a start/end determining section that determines the start/end positions using the magnitude of change in the voice. 2. As a feature representing the spectral shape of the signal, a power spectrum obtained by a group of band filters, Fourier transform, or linear predictive analysis, or obtained by linear predictive analysis.
2. The audio start/end detection device according to claim 1, which uses any of the LPC cepstrum coefficients. 3 The voiced/non-sounded determination unit compares the energy of the signal with two threshold values, and the first determination unit compares the energy of the signal with two threshold values, and the statistical distance measure between the spectrum of the input signal and the three standard patterns of silence, unvoiced sound, and voiced sound. and a second determination unit that performs determination based on the similarity of the used spectra,
2. The speech start/end detection device according to claim 1, wherein one of a linear discriminant function, Mahalanobis distance, and Bayesian judgment is used as the statistical distance measure. 4. The feature is that the Euclidean distance between the feature representing the spectrum in an interval of a certain time length and the feature representing the spectrum in the previous interval is used as the feature representing the magnitude of change in the spectrum of the start/end determining section. A voice start/end detection device according to claim 1.