JP2000029486A - Speech recognition system and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correctly distinguish speech from noise in an actual use environment. SOLUTION: To distinguish a speech block from a non-speech block over the whole block from the start to the end of a speech with respect to the inputted speech, this speech recognition system is provided with a speech block detecting part 103 for detecting the speech block by detecting information on the non-speech block. This detecting part 103 detects power levels of steady or unsteady noise and all other sound, and specifies the power level indicating the start and end of the speech in accordance with the power level fluctuating every moment. If the power level of an inputted speech exceeds the above specified power level, it is judges as the start of the speech, and if it is lower than the specified power level for a predetermined period of time or longer, it is judged as the end of the speech.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition system and method, and relates to a small information device represented by a car navigation system, an in-vehicle PC, a PDA, and a handheld PC, a portable speech translator, a game, and a home appliance. The present invention relates to a speech recognition system and method used. The present invention is particularly suitable for application to a speech recognition system and method under a time-varying noise environment such as a car navigation system and an in-vehicle PC.

[0002]

2. Description of the Related Art In recent years, small information systems using voice recognition technology have been around for a long time. In addition to car navigation systems, small information devices represented by PDAs, portable translators, and the like. As an example of such a speech recognition system, Japanese Patent Application Laid-Open No. 5-35776 discloses a "translation device with automatic language selection function" that recognizes and translates the voice of an operator input from a microphone and translates the translated language. A technology related to a portable translation device that outputs voice is disclosed.

[0003] An outline of such a speech translation apparatus according to the prior art will be described below with reference to FIG. FIG.
FIG. 11 is a block diagram illustrating a configuration of a speech translation device according to the related art. The control unit 701 includes a microprocessor or the like, and controls each unit of the device. Voice section extraction unit 702
Converts the voice input from the microphone 709 into a digital signal, cuts out the digital signal, and sends the digital signal to the voice recognition unit 703. The voice recognition unit 703 is connected to the line 71 by a keyboard or a switch.
The microphone 709 and the voice segment extraction unit 7 are instructed by the control unit 701 based on the operation signal input via the control unit 1.
After 02, the extracted voice is analyzed. Then, the speech recognition is performed by comparing the analysis result with a standard speech pattern stored in the speech recognition dictionary unit 707. The speech synthesis unit 705 reads a translated word corresponding to the speech recognized by the speech recognition unit 703 from the translated word data memory card 706, converts the translated word into an audio signal, and outputs the signal through the speaker amplifier 710 and the speaker 708.

[0004] The display unit 704 gives instructions to the user of the translation apparatus, displays translated characters, and the like. The translation data memory card 706 is composed of a ROM card or the like,
When a translated word is synthesized and output, audio data is stored. Further, a character code corresponding to the translated word is read from the translated word data memory card 706 and displayed on the display unit 704 as a character. And
By exchanging the translated word data memory card 706 with one for another language, translation into a plurality of languages becomes possible. The voice recognition dictionary unit 707 is composed of a RAM or the like, and stores a standard voice pattern according to the utterance of the operator. This standard voice pattern is stored in advance by the operator.

[0005]

SUMMARY OF THE INVENTION Such speech recognition,
In the field of speech synthesis technology, its development is expected from the demand that the system should provide a more human-like user interface with the improvement of semiconductor technology. In the above-mentioned small information systems using the conventional speech recognition technology, as car navigation systems, portable information devices represented by PDAs, portable translators, and information home appliances with voice interfaces, the future will continue. It is expected to become more and more popular. However, in speech recognition, the amount of information to be processed is enormous. In the conventional technology, it is necessary to set a limit on the number of words to be recognized in order not to lower the performance of the recognition rate and the recognition response time. To do this, for a word or sentence registered in advance, the statistical characteristics of the speaker's voice in the character string and the characteristics of the voice actually spoken by the speaker are compared, and the probability is calculated. The value closest to is used as the recognition result. It is expected that the performance of the recognition rate and the recognition response time will improve in the future due to technological innovation in speech recognition and the improvement of the software and hardware that realize it. However, from a practical point of view of the speech recognition system, recognition performance in an environment actually used becomes a problem. For example, in a voice recognition system of a car navigation system, the environment is different when the engine is stopped, when the vehicle is idling, when driving on a general road, an urban area, a highway, a tunnel, and the like. Changes the environment. In such an environment, in a speech recognition system on the premise of detecting a voice section, the voice section detection accuracy decreases.

An object of the present invention is to solve the above-mentioned drawbacks and to provide a speech recognition system in which the recognition performance does not deteriorate in an actual use environment.

[0007]

In order to achieve the object of the present invention, a dictionary in which words and sentences to be subjected to speech recognition are collected, and contents obtained from the dictionary as a speech recognition result are displayed or spoken. In the voice recognition system according to the present invention for outputting, a voice detection unit for detecting a voice section based on information of a detected non-voice section is provided, and performs voice recognition of the detected voice section.

In this speech recognition system, a noise threshold is obtained from the power in the non-speech section. Further, in this speech recognition system, the noise threshold is compared with the power of the speech section, and speech recognition processing is started near the time when the power of the speech section reaches the noise threshold. In this speech recognition system, comparing the noise threshold with the power of the speech section,
The voice recognition processing is performed going back a predetermined time from the time when the power of the voice section reaches the noise threshold. Further, in this speech recognition system, the noise threshold is obtained based on an average power of a predetermined number of aggregates of frames, which are units for analyzing the power of speech or noise. In this speech recognition system, a power threshold is obtained based on the noise threshold. Furthermore, in this voice recognition system, when the power of the voice section exceeds the noise threshold value and the power threshold value is reached, it is determined that the voice has started, and a predetermined time before this time point Perform voice recognition processing. In this speech recognition system,
A voice input button is provided, and when the power of the voice section reaches the noise threshold after the button is pressed, a frame, which is a unit of voice analysis at that time, is stored. Further, in this speech recognition system, when the power of the speech section reaches the power threshold, speech recognition processing is performed from at least the stored frames.

In the above-described speech recognition system, if the period during which the power of the speech section falls below the power threshold is less than or equal to a predetermined time, it is determined that the portion is unvoiced between speeches. Further, in the voice recognition system, after the power of the voice section falls below the power threshold, the power of the voice section is maintained at a value lower than the power threshold for a predetermined time. If it is, it is determined that the voice section has ended. In the voice recognition method according to the present invention for displaying or outputting as a voice the contents picked up from a dictionary in which words and sentences to be voice-recognized are collected as voice recognition results, the voice recognition is performed based on information of the detected non-voice section. Detecting a section; and performing voice recognition of the detected voice section. In this speech recognition method, a step of obtaining a noise threshold from the power of the non-speech section is provided.

In this speech recognition method, there is a step of comparing the noise threshold value with the power of the speech section, and starting a speech recognition process near the time when the power reaches the noise threshold value. This speech recognition method includes a step of obtaining a noise threshold from the power of the non-speech section, and a step of obtaining a power threshold based on the noise threshold. In this voice recognition method, when a period during which the power of the voice section falls below the power threshold is equal to or less than a predetermined time,
Determining a voiceless portion between the voices. In this voice recognition method, after the power of the voice section falls below the power threshold value, it is determined that the voice section has ended when a predetermined time has elapsed while maintaining this state.

[0011] Words and sentences to be subjected to speech recognition are collected and defined as a dictionary, and the words and sentences recognized by speech are picked up from the dictionary, and are displayed using character strings, images indicated by the words, and speech synthesis. In the speech recognition system according to the present invention, which outputs as speech, in order to distinguish the speech segment from the beginning to the end of the inputted speech from the non-speech segment containing no speech, information on the non-speech segment is used. A voice section detection unit that detects a voice section while detecting a voice, a voice analysis unit that performs voice analysis processing on the captured voice, an acoustic model having a voice pattern in phoneme units, and a voice analysis result. And a voice recognition unit that performs voice recognition processing by linking the acoustic model and the dictionary, and performs voice recognition on the voice detected in the voice section.

In the voice recognition system, the voice section detection unit detects a power level of a sound in a non-voice section, and determines a voice level indicating a start and an end of the voice in accordance with the power level that changes every moment. Is determined, and if the input voice exceeds the power level of the voice, it is determined that the voice has started. If the input voice falls below the power level of the voice for a predetermined time or more, it is determined that the voice has ended. In the voice recognition system, the voice section detection unit detects a power level for a sound in a non-voice section, and calculates a noise threshold and a power threshold from the power level, If the input voice exceeds the noise threshold and then exceeds the power threshold,
Judge as the beginning of voice. In this voice recognition system, when the power level of the voice falls below the threshold for a predetermined time or more, it is determined that the voice has ended. In this speech recognition system, the power level is an average power obtained over a plurality of frames by dividing the power of a frame divided by a predetermined time unit, and the noise threshold is set to N times the average power. I do. In the speech recognition system, the power threshold value PTH is set so as to satisfy a relationship of PTH> NTH compared to the noise threshold value NTH. In this speech recognition system, the relationship between the noise threshold NTH and the power threshold PTH is ΔPTH = PTH−N in a relatively quiet environment.
TH is set to be small, and conversely, in an environment with large noise, ΔPTH = PTH−NTH is set to be large.

[0013] Words and sentences to be subjected to speech recognition are collected and defined as a dictionary. The words and sentences recognized by speech are picked up from the dictionary, and are displayed using character strings, images indicated by the words, and speech synthesis. Sound Output as Speech In the voice recognition method according to the present invention, in order to distinguish an input speech from a speech section from the beginning to the end of the speech with a non-speech section containing no speech, Detecting a voice section while detecting information, performing a voice analysis process on the voice detected in the voice section, and connecting an acoustic model having a voice pattern in phoneme units and the dictionary. Performing a voice recognition process on the voice analysis result.

In this speech recognition method, a step of detecting a power level of a sound in a non-speech section, and determining a power level of the speech indicating the beginning and end of the speech in accordance with the power level which changes every moment. , When the power level of the input voice exceeds the power level of the voice, it is determined that the voice starts, and when the power level of the input voice falls below the power level for a predetermined time or more, it is determined that the voice ends. In this speech recognition method, a power level is detected for the sound in the non-speech section, and a noise threshold and a power threshold are calculated from the power level. If the level exceeds the noise threshold, and if the level exceeds the power threshold, determining that the sound has begun. This voice recognition method includes a step of determining the end of the voice when the power level of the input voice falls below the power threshold for a predetermined time or more. In this speech recognition method, a step of obtaining the power of a frame delimited by a predetermined time unit as an average power obtained over a plurality of frames as the power level; and setting the noise threshold to N times the average power. Setting. This speech recognition method includes a step of setting the power threshold value PTH so as to satisfy a relationship of PTH> NTH, as compared with the noise threshold value NTH. In the speech recognition method, a noise threshold NTH and a power threshold PT
In a relatively quiet environment, the relation of H is ΔPTH = PT
H-NTH is set to be small, and conversely, in an environment with large noise, ΔPTH = PTH-NTH is set to be large.

In order to achieve the above object, the speech recognition system of the present invention collects words and sentences to be subjected to speech recognition, defines them as a dictionary, and picks up those words and sentences as a speech recognition result. In a speech recognition system that outputs a recognition result as speech using character string display, an image indicated by a word, or speech synthesis, a speech section from the beginning to the end of the speech is input. A voice section detection unit that detects a voice section while always detecting information on a non-voice section, a voice analysis unit that performs a voice analysis process on the captured voice, and a voice An audio model having a pattern for each phoneme, and a voice recognition unit for performing voice recognition processing by connecting the voice model and the dictionary to the voice analysis result; It is obtained so as to speech recognition for.

More specifically, for the input voice,
In order to distinguish the voice section from the beginning to the end of the voice from the non-voice section, the voice section detection unit that detects the voice section while always detecting the information of the non-voice section,
Detects the power level of all non-voice stationary noise, non-stationary noise, and sound in quiet environments, and adjusts the power level of the voice to indicate the beginning and end of the voice according to the constantly changing power level. The level is determined, and when the input voice exceeds the power level of the voice, the start of the voice is determined, and when the power level of the voice falls below the power level for a certain time or more, the end of the voice is determined.

More specifically, the voice section detection section always detects the power level of all non-voice stationary noise, non-stationary noise, and sound in a quiet environment, and starts and ends the voice. Are calculated as two noise thresholds NTH and PTH as the power level of the voice indicating
Exceeds the power threshold value PTH, the start of voice is determined, and the power threshold value PT
If the time falls below H for a certain time or more, the end of the voice is determined.

More specifically, the noise threshold value NTH is
Calculates power for stationary noise that is not always input, non-stationary noise, and sound in a quiet environment, and calculates the average N of frame power PW divided in short time units.
The power threshold value PTH is set so as to satisfy the relationship of PTH> NTH as compared with the noise threshold value NTH. More specifically, the relationship between the noise threshold NTH and the power threshold PTH is
In a relatively quiet environment, ΔPTH = PTH-NTH is set to be small, and conversely, in an environment with large noise, ΔPTH = PTH-NTH is set to be large.

[0019]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of a speech recognition system and method according to the present invention; FIG. FIG. 1 is a block diagram showing the functions of the speech recognition system according to the present invention and the flow of the processing. Fig. 1
, Noise and voice corresponding to the environment are captured. The analog signal, which is the noise or voice captured, converts the analog signal into a digital signal.
The D converter 102 converts analog data into digital data at an arbitrarily determined sampling period. In the process of converting analog data into digital data, a high-pass filter (HPF) (not shown) or the like is used before or after the conversion, for example, to remove stationary noise.

The power of the converted non-stationary noise, the noise that cannot be removed by the high-pass filter, and the voice digital data is calculated by the voice section detection unit 103 in frame units. The frame unit is a unit in which the input noise or voice is divided for each short time (5 to 20 ms) in the order of the input time. The power is a zero-order autocorrelation of data sampled in the frame unit. Therefore, the sampled data is electrically a voltage value with respect to time. These values are normalized, and the value obtained by squaring the value of each sampling point is the power value of each frame. The value obtained by adding the value of the square of each sampling point is the sum of the power values of all the frames. The calculated power value is, for example, assuming that the total power value of 1 frame to i frame (i = 32) is PW, the average power value of 1 frame to i frame is PW
It is determined by W / i. Every time one frame elapses, the previous data is discarded, and data of the average power value is updated by adding new one frame of data. From the average power PW / i obtained in this way, parameters necessary for voice section detection, a noise threshold NTH, and a power threshold PTH are determined. Noise threshold NT
H and the power threshold value PTH will be described later in detail.

Further, the digital data of the converted speech is subjected to preprocessing such as noise processing, speech analysis, and speaker adaptation by a speech analysis unit 104, and speech recognition is performed by a speech recognition unit 105 using the speech analysis result. Is matched. Here, the speech recognition executes two processes. The first process is a voice section detection process in which the voice section detection unit 103 detects voices correctly in a noise environment. In the second process, the voice signal is analyzed by the voice analyzing unit 104 and the voice recognizing unit 105, and the voice signal is analyzed for a short time (5 to 20 m).
s) is analyzed as a phoneme, the pattern is analyzed, and a corresponding word or sentence is selected from a dictionary. From the above two processes, as a speech recognition system,
The voice recognition result 109 is output.

The speech recognition unit 105 checks the acoustic analysis result of the input speech analyzed by the speech analysis unit 104 against the acoustic model 106 and the word dictionary 107 for each phoneme connected by the model connection unit 108, Registered word dictionary 1
In 07, pick up the closest word. The acoustic model 106 is a model used for speech recognition. Specifically, the acoustic model 106 is a correspondence between a character and a phoneme used in the word dictionary 107, and a distribution of a probability that a feature of the phoneme appears, It stores the distribution of the probability that the feature will transition to a state where the next feature appears. Actually, a parameter indicating the distribution is stored, and the distribution is calculated from the parameter for each speech recognition process. Thereby, the memory capacity of the entire speech recognition system can be reduced. The acoustic model 106 is capable of recognizing the voice of any speaker without registering the voice in advance.
But it is becoming more common. As such an acoustic model, for example, a hidden Markov model (HMM: Hid:
den Markov Model) can be used.

The word dictionary 107 is a collection of words, words (nouns, verbs, etc.), and sentences. For example, in a car navigation system, it is a collection of words necessary for a minimum necessary conversation, such as street names, place names, building names, town names, street addresses, intersection names, private houses (personal names), telephone numbers, and the like. . This word dictionary 107 is composed of words of, for example, 10 to 5000 words per one dictionary according to the capability of the system. From the above, the speech recognition system used in the actual environment is a speech section detection that correctly identifies noise and speech, analyzes the speech signal, analyzes it as phonemes every short time, and analyzes the pattern Then, a corresponding word or sentence is selected from the dictionary. Note that each processing block illustrated in FIG. 1 may be a system configured with a plurality of LSIs and memories, or may be one or a plurality of system-on-chips configured on a semiconductor element. Also,
Each process may be hardware processed by a dedicated LSI or a dedicated IC, or may be middleware implemented by software such as a DSP or a RISC microcomputer.

FIG. 2A is a voice input waveform diagram, and FIG.
Is a characteristic diagram showing audio power. In FIG. 2, the horizontal axis represents time t, and the vertical axis in FIG.
The vertical axis in (b) shows the power P. FIG. 2 shows a voice uttered in a car, a voice input waveform of "Shibuya" and a voice power when the voice recognition system described in FIG. 1 is used in a car navigation system. The voice input waveform 201 in FIG. 2A shows a voice waveform when the speaker utters "reluctant" with normal voice. The audio signal is an unsteady signal that changes every moment.

The surrounding environment at this time is the inside of a passenger car traveling on a relatively quiet general road at 40 km / h. All windows are closed, the radio and car stereo are turned off, and the air conditioner output is set to a low value. If this audio signal is cut out and viewed in a short time of 20 ms, the same spectral audio analysis as that of a stationary signal can be performed.
When the autocorrelation function is calculated from the sampled values of the extracted audio signal in, for example, LPC analysis widely used in audio analysis, the power of the audio is obtained as one of the characteristic parameters of the audio. The curve 202 is the power calculated from the audio signal of the audio waveform 201, and the time t
Represents the change in power with respect to.

Here, in order to correctly perform voice recognition on the voice of "shibuya", it is necessary to detect a voice section. For this purpose, a threshold value is arbitrarily set for the power information, that is, set to a value calculated by a predetermined formula or a value obtained from an experiment, and the threshold is set for each input noise and voice. Observe whether the threshold has been exceeded. This observation is performed by the voice section detection unit 103.

The above observation may be performed by always inputting noise and voice. In a system executing a plurality of processes such as a car navigation system, the load on the CPU is reduced as much as possible. I want to reduce power consumption. Therefore, in order to capture noise or voice only when performing voice recognition, noise or voice is input from the point in time when the voice input button is pressed. In FIG. 2A, reference numeral 204 denotes a point in time when the voice input button is pressed. As the voice input button, the button may be kept pressed while taking in the voice, or the voice input button may be pressed first to take in the voice, and when the voice and noise fall below a predetermined value, this is detected and the voice is automatically detected. Alternatively, the input of the voice may be turned off.

In FIG. 2B, the level NTH indicates a first-stage threshold value for distinguishing noise from speech, and is referred to as a noise threshold value. In the vicinity of the beginning of the voice, the noise and the voice have the same power level, and it is difficult to distinguish between the voice and the noise. Therefore, when the noise threshold NTH is exceeded, it is determined that the possibility of the start of voice is high, and the frame position at the time (or intersection) 205 where the noise threshold NTH intersects with the curve 202 indicating the power value P is stored. Keep it.
Next, PTH is the second
This is a step threshold, and is referred to as a power threshold. Here, since a considerable power level is detected, it can be understood that the sound is sound. At that time, that is, at the time (or intersection) 206 at which the power threshold value PTH and the curve 202 indicating the power value P intersect, it is understood that the sound is sound. Therefore, at time point 206 when it is detected that the voice is voice, the recognition process is started from the frame position stored at time point 205. Alternatively, the recognition processing is started from a frame k frames before the stored frame. This allows
Correct recognition is possible by detecting the beginning of speech buried in noise.

The noise threshold NTH can be calculated such that noise and voice are input through the microphone 101 and written into the RAM, which will be described later, without the need to press the voice input button. Then, the value of NTH immediately before the voice input button is pressed is adopted. Thus, the time point 205 can be detected by calculating the noise threshold value NTH from the data for i frames before pressing the input button and comparing the noise threshold value NTH with the power value for each frame. However, the recognition process can be started before the time point 205. In addition, voices always have a non-voice portion between characters. For example, in "Shibuya", "shi" and "bu", "b
A silent part exists between "u" and "ya". At this time, the power level of the audio is lower than the power threshold value PTH. However, since the sound has not ended yet, it is erroneous to determine that the sound has ended. Therefore,
Even if the power level of the audio is lower than the power threshold value PTH, if the period is less than a set number of frames, the recognition processing is continued assuming that the audio has not been completed. Conversely, if the period is equal to or greater than the set number of frames, it is determined that the voice has ended, and the recognition process ends. In FIG. 2B, a time point 207 is a time point at which it is determined that the voice has ended, and the recognition processing ends at a time j frames after the time point. The reason why the audio power is compared with the power threshold from the time point when it is determined that the audio has ended to after j frames is to determine whether the audio has actually ended or whether it is a silent part between the audios. is there. Note that the value of the j frame is determined in advance by an experiment. As a result, the end of the voice buried in the noise can be detected and the correct recognition can be performed, similarly to the start of the voice. In this case, the noise threshold NTH is set so as not to exceed the power threshold PTH. As described above, the voice section 203, which is the voice section detection necessary for correct voice recognition, is detected.

FIG. 3 (a) is a voice input waveform diagram, and FIG. 3 (b)
Is a characteristic diagram showing audio power. FIG. 3 (a), (b)
In FIG. 3, the horizontal axis indicates time t, the vertical axis in FIG. 3A indicates voltage V, and the vertical axis in FIG. 3B indicates power P. FIG.
(A) shows the voice input waveform and the power of the voice of the voice "Shibuya shibuya" which the speaker uttered in the car in the voice recognition system applied to the car navigation system described in FIG. The surrounding environment at the time is the interior of a passenger car that is parked in a very quiet parking and idling, all windows are closed, radio and car stereo are turned off, and the output of air conditioner is also low. Is set. The voice input waveform 301 in FIG. 3A shows a waveform under such an environment. In the environment described in FIG. 2, that is, on a relatively quiet general road, driving at 40 km / h with all windows closed, the radio and car stereo turned off, and the air conditioner kept at a low value. The power level of the voice shown in FIG. 3A is lower than the power level of the voice in the environment of the passenger car. This phenomenon is because the power level of the noise around the speaker is low, and the speaker's own voice can be heard well even if the voice is small, so that the power level of the voice is low. Therefore, in an environment in which the power level of noise is low and the power level of voice is low, it is necessary to lower the noise threshold NTH and the power threshold PTH in order to perform correct voice section detection.
Further, the difference ΔPTH between PTH and NTH also becomes smaller.

When this voice signal is cut out in a short time of 20 ms and viewed, the same spectrum voice analysis as that of a stationary signal can be performed. When the autocorrelation function is calculated from the sampled values of the extracted audio signal in, for example, LPC analysis widely used in audio analysis, the power of the audio is obtained as one of the characteristic parameters of the audio.

In FIG. 3B, reference numeral 302 denotes a voice waveform 3
12 is a curve showing the power P calculated from the audio signal No. 01, and shows a change in power with respect to time t. Here, it is necessary to detect a voice section in order to correctly perform voice recognition on the voice of "shibuya". For this purpose, a threshold value is arbitrarily set for the power information, and it is observed whether the threshold value is exceeded for each of the input noise and voice. This observation is performed by the voice section detection unit 103. Therefore, the above observation may be performed by always inputting noise and voice. However, in a system that executes a plurality of processes such as a car navigation system, the load on the CPU is reduced as much as possible. I want to reduce power consumption. Therefore, in order to capture noise and voice only when recognizing voice, it is assumed that noise and voice are input when the voice input button is pressed.
Reference numeral 304 denotes a point in time when the voice input button is pressed.

NTH is a first-stage threshold value for distinguishing between noise and speech, and is referred to as a noise threshold value. In the vicinity of the beginning of the voice, the noise and the voice have the same power level, and it is difficult to distinguish between the voice and the noise. Therefore,
If the noise threshold value NTH is exceeded, it is determined that the possibility of the start of speech is high, and the frame position indicated by the time point 305 is stored. Next, PTH is a second-stage threshold value for determining that the voice is speech, and is referred to as a power threshold value. Here, since a considerable power level is detected, it can be understood that the sound is sound. 306
It is. Therefore, at the time 3 when the voice is detected,
At 06, the recognition process is started from the frame position stored at time 305. Alternatively, the recognition processing is started from a frame k frames before the stored frame. As a result, the beginning of the voice buried in the noise can be detected and correct recognition can be performed. In addition, voices always have a non-voice portion between characters. For example, in "Shibuya", "s
There is a silent portion between "hi" and "bu" and between "bu" and "ya". At this time, the power level of the audio is lower than the power threshold value PTH. However, since the sound has not ended yet, it is erroneous to determine that the sound has ended. Therefore, even if the power level of the audio is lower than the power threshold value PTH, if the period is less than the set number of frames, the recognition processing is continued assuming that the audio has not been completed. Conversely, if the period is equal to or greater than the set number of frames, it is determined that the voice has ended, and the recognition process ends. In FIG. 3B, it is assumed that the time point 307 is the time point at which it is determined that the voice has ended, and the recognition processing ends after j frames from that time point.

As described above, the end of the voice buried in the noise can be detected and the correct recognition can be performed in the same manner as the start of the voice. At this time, the noise threshold N
TH shall not exceed the power threshold value PTH.
As described above, the voice section 303 which is a voice section detection necessary for correct voice recognition is detected.

FIG. 3 (c) is a voice input waveform diagram, and FIG. 3 (d).
Is a characteristic diagram showing audio power. FIG. 3 (c), (d)
In FIG. 3, the horizontal axis indicates time t, the vertical axis in FIG. 3C indicates voltage V, and the vertical axis in FIG. 2D indicates power P. FIG.
(C) and (d) show the voice “Shibuya shibuya” uttered by the speaker in the car in the voice recognition system applied to the car navigation system described in FIG.
3 shows the voice input waveform and the power of the voice. In a waveform 311 shown in FIG. 3C, all windows are closed,
Even though the radio and car stereo are turned off and the air conditioner output is set to a low value,
Since the vehicle is traveling at km / h, there is considerable noise in the vehicle, and the voice input waveform when the vehicle is in a noisy environment is shown.

The audio power level of the audio input waveform 311 shown in FIG. 3C is equal to the audio input waveform 301 shown in FIG.
Is considerably higher than the audio power level.
This phenomenon occurs because the power level of the noise around the speaker in the case of FIG. 3 (c) is high and the speaker's own voice is not well heard and is generated in a loud voice, so that the power level of the voice increases. It is. In an environment where the power level of noise increases and the power level of voice increases, it is necessary to increase the noise threshold NTH and the power threshold PTH in order to perform correct voice section detection. In addition, PTH and NT
The difference ΔPTH in H also increases.

When this voice signal is cut out in a short time of 20 ms and viewed, the same spectrum voice analysis as that of a stationary signal can be performed. When the autocorrelation function is calculated from the sampled values of the extracted audio signal in, for example, LPC analysis widely used in audio analysis, the power of the audio is obtained as one of the characteristic parameters of the audio.

A curve 312 shown in FIG. 3D represents the power P calculated from the audio signal of the audio waveform 311 and represents a change in power at time t. Here, in order to correctly perform speech recognition for the speech of “shibuya”, it is necessary to detect a speech section. For this purpose, a threshold value is arbitrarily set for the power information, and it is observed whether the threshold value is exceeded for each of the input noise and voice. This observation is performed by the voice section detection unit 103. Therefore, the above observation may be performed by always inputting noise and voice. However, especially in a system executing a plurality of processes such as a car navigation system, the load on the CPU is reduced as much as possible. I want to reduce power consumption. Therefore, in order to capture noise and voice only when recognizing voice, it is assumed that noise and voice are input when the voice input button is pressed. Reference numeral 314 indicates a point in time when the voice input button is pressed.

In FIG. 4D, NTH is a first-stage threshold for distinguishing noise from speech, and is referred to as a noise threshold. In the vicinity of the beginning of the voice, since the noise and the voice are at the same power level, it is difficult to distinguish between the voice and the noise. Therefore, if the noise threshold value NTH is exceeded, it is determined that there is a high possibility of the start of speech, and the frame position indicated at time point 315 is stored. Next, PTH
Is a second-stage threshold for determining that the voice is voice, and is referred to as a power threshold. Here, since a considerable power level is detected, it can be understood that the sound is sound. That time is time point 316. Therefore, at the time point 316 when it is detected that the voice is voice, the recognition processing is started from the frame position stored at the time point 315. Alternatively, the recognition processing is started from a frame k frames before the stored frame. As a result, the beginning of the voice buried in the noise can be detected and correct recognition can be performed.

In addition, voices always have a non-voice portion between characters. For example, in "Shibuya", a non-voice portion exists between "shi" and "bu" and between "bu" and "ya". In this case, the power level of the audio is lower than the power threshold PTH. However, since the sound has not ended yet, it is erroneous to determine that the sound has ended. Therefore, the power level of the sound is equal to the power threshold PT
Even if it becomes lower than H, if the period is less than a certain set number of frames, the recognition processing is continued assuming that the sound has not ended yet. Conversely, if the period is equal to or greater than the set number of frames, it is determined that the voice has ended, and the recognition process ends.
In FIG. 3D, reference numeral 317 denotes a point in time when it is determined that the voice has ended, and it is assumed that the recognition processing ends after j frames from that point. As a result, the end of the voice buried in the noise can be detected and the correct recognition can be performed, similarly to the start of the voice. At this time, the noise threshold value NTH
Does not exceed the power threshold value PTH. From the above, the voice section 313, which is the voice section detection necessary for correct voice recognition, is detected.

However, suppose now that the noise threshold N
Assume that TH is set to the position of NTH1 in FIG. Normally, the correct voice section is the section indicated by 313, and the intersection 315 of the noise threshold NTH and the power level must be detected. However, there is no intersection between the noise threshold value NTH1 and the power level,
Since the power level of the noise has already exceeded NTH1 at 314 when the voice input button is pressed, it is determined that the voice starts from time 319 immediately after the button is pressed, and the voice section is 318. , And an incorrect voice section is detected, so that the recognition result is erroneously recognized. As is clear from the above, the noise threshold value NTH and the power threshold value PTH are set according to the environment in which they are actually used.
It is necessary to set and update the optimum value in a short cycle (for example, every three seconds). In particular, car navigation systems, car electronics products, PDAs,
In an environment where a handheld PC or the like is used, the noise level fluctuates considerably at short time intervals.

One example of the equations for calculating the noise threshold value NTH and the power threshold value PTH is shown in (Equation 1) to (Equation 5). In (Equation 1) to (Equation 5), PW indicates the total sum of the input noise powers from 1 to i frames immediately before the voice input button is pressed after the voice recognition mode is set, and PW / i is The average value of the input noise power from one frame to the i-th frame is shown. N1, N2, and N3 are safety factors and are positive integers determined by experiments. In the embodiment of the present invention, N1 and N2 are set to 5 and N6 is set to 1
It is set to 0. The values of N1 to N3 may be changed according to the average power value of the i-frame (for example, 32 frames) before pressing the voice input button. P1 is a value that changes depending on the state of the noise, and takes a predetermined constant value when the noise is constant. For example, looking at the average power value of the i frame (for example, 32 frames) before pressing the voice input button, if the noise value is large, P1 is set large, and if the noise value is small, P1 is set small. . Since NTH and PTH take normalized values, P1 = 100,000 in this embodiment. P2 depends on the noise value like P1,
Determined by experiment.

NTH = (PW / i) × N1 (Equation 1) PTH = NTH + P1 (Equation 2) Alternatively, NTH = (PW / i) × N2 (Equation 3) PTH = (PW / i) × N3 (Equation 4) However, if PTH <P2, then PTH = P2 (Equation 5) Next, the hardware configuration of the speech recognition system and method according to the present invention will be described with reference to FIG. FIG.
1 is a block diagram showing one embodiment of a speech recognition system according to the present invention. As the microphone 401 for capturing voice, a car navigation system, a portable information terminal,
In PDA, handheld PC, games, portable translators, home appliances such as air conditioners, etc., directional microphones having directivity are used in order not to capture ambient noise. An A / D converter 404 converts analog audio data captured by the microphone 401 into digital audio data. The voice input button 402 is a button for designating a section in which voice is being input. While the button is being pressed, or from the time the button is pressed, the system is notified that a voice has been input. 4
Reference numeral 05 denotes an interface for connecting the voice input button 402 to the system.

The key input device 409 is, for example, a digitizer for pen input in the case of a portable information terminal, and is a keyboard in the case of a handheld PC. In the case of a game console such as a NES, a keypad for operating a character or the like or a joystick is used. 4
Reference numeral 10 denotes an interface for connecting the key input device 409 to the system. The CPU 403 includes a car navigation system, a portable information terminal, a PDA,
It controls a main system such as a handheld PC, a game, a portable translator, and a home appliance, and performs voice recognition processing in a voice recognition system. A recent trend is to use a RISC microcomputer or DSP for the CPU 403. ROM 406 is a word dictionary for speech recognition,
This is a storage device that stores acoustic models and programs. A memory card may be used to store a plurality of dictionaries and acoustic models.

The RAM 407 stores some dictionaries, acoustic models, and programs transferred from the ROM 406, and is a minimum necessary work memory required for speech recognition processing. Is used. The bus 408 is used as a data bus, an address bus, and a control signal bus in the system. The display 412 for outputting and displaying the voice recognition result is an LC such as a TFT liquid crystal display.
D to display the speech recognition result. An interface 411 connects the display 412 to the system. A speaker 414 for outputting the speech recognition result as sound synthesizes and outputs the speech recognition result. An A / D converter 413 converts the speech recognition result from text to speech synthesis data, and then converts the digital speech synthesis data to an analog speech signal.

An embodiment of the speech recognition system and method according to the present invention will be described below with reference to FIGS.
In this embodiment, a case will be described in which the speech recognition system of the present invention is applied to a car navigation system, car multimedia, and car electronics products.

FIG. 5 is a flowchart for explaining one embodiment of the operation flow of voice section detection used in the voice recognition system and method according to the present invention. In the figure, step 501 is a start indicating that the car navigation system has been started. In step 501, when the car navigation system starts,
Moving to step 502, the voice recognition system of the car navigation system is activated. For example, a state in which the remote control is operated to switch to the voice recognition mode is shown.
In the state of the voice recognition mode, the process proceeds to step 503, and the calculation of noise and voice power input from the microphone is started for each frame. For example, when the calculation for 32 frames is completed, NTH and PTH are calculated according to the equations of the noise threshold NTH and the power threshold PTH shown in (Equation 1) to (Equation 5). From the following, for example, 1
The power values of the new frame and the oldest frame are changed for each frame, and the NTH and PTH are calculated and updated again. This frequency depends on the system, and is one per second.
Once every 2 seconds, once every 3 seconds, and so on.

Next, in step 504, the speaker presses the voice input button 402 to execute voice recognition. At this time, in step 505, the latest values of NTH and PTH calculated in step 503 are determined as NTH and PTH for detecting a speech section to be actually used.
In step 506, the NT determined in step 505
Voice section detection is performed using H and PTH. Step 507 indicates that the voice section detection and the voice recognition processing have been completed.

The voice section detection in step 506 will be described in detail with reference to FIG. FIG. 6 is a flowchart for explaining an embodiment of the operation flow of voice section detection described in FIG. In step 601, voice section detection starts. In step 602, the power PW of each frame of the input noise or voice is compared with a noise threshold NTH. If the result is NO with respect to NTH <PW, it is determined that the beginning of the speech is not attained, and the same processing is performed for the next frame to be input. In the case of YES, it is determined that the sound has begun, and the process proceeds to step 604.

At step 604, power PW
Is compared with the power threshold PTH. The result is P
For TH <PW, if NO, step 608
And the counter (CNT) is incremented by 1 (+
1), that is, (increment), and step 60
Proceed to 6. In the case of YES, in step 609, the value of the counter (CNT) is set to CNT = 0, and
Proceeding to the voice recognition process of 05, the recognition flag (RF) is set to RF = 1, and voice recognition processes such as voice analysis and voice verification are performed on the frame. In the case of NO in step 604 described above,
If the recognition processing in step 605 has never been executed, N is set for CNT <n & RF = 1 in step 606.
Since it is O, in step 603, RF = 0, CNT =
In step 607, the voice section detection for each frame is terminated. In step 606, n for CNT <n is a number between voices, for example, between "shi" and "bu",
The number indicates that the sound is interrupted as between “bu” and “ya”. Therefore, if CNT <n is NO, it indicates that the counter value is larger than n, that is, the voice has ended, and a predetermined frame, for example, 3
This means that no sound is heard over 0 frames. Recognition RF takes a value of 0 or 1, takes 1 when speech recognition processing is performed, and takes 0 in other cases.

In the case where the recognition processing in step 605 is being executed, and if the answer is NO in step 604, RF =
If the counter value is smaller than n, CNT
<N & RF = 1 is YES, and it is determined that the voice has not ended, and the process proceeds to the voice recognition process of step 605 to execute the recognition process. Further, when the recognition processing of step 605 is being executed,
, RF = 1, and if the counter value is greater than n, NO for CNT <n & RF = 1,
It is determined that the voice has ended, and in step 603, RF =
0, CNT = 0, and ends the voice section detection for each frame in step 607. With the above operation, voice section detection is performed.

[0052]

According to the present invention, in a car navigation system, a small information system, and a voice recognition system used in a game, a threshold for voice section detection is set in accordance with a noise level in an environment actually used. Can be realized, and a good speech recognition system can be realized in which speech section detection by automatic threshold setting and recognition performance do not deteriorate even in a real environment.

[Brief description of the drawings]

FIG. 1 is a block diagram showing functions of a speech recognition system according to the present invention and a flow of processing thereof.

FIG. 2 is a characteristic diagram showing an audio input waveform and audio power.

FIG. 3 is a characteristic diagram showing an audio input waveform and audio power.

FIG. 4 is a block diagram showing a hardware configuration of a speech recognition system according to the present invention.

FIG. 5 is a flowchart illustrating an embodiment of a voice section detection operation used in the voice recognition system and method according to the present invention;

FIG. 6 is a flowchart illustrating an example of an operation flow of voice section detection illustrated in FIG. 5;

FIG. 7 is a block diagram of a portable translation device using a conventional speech recognition system.

[Explanation of symbols]

101: voice input microphone, 102: A / D converter, 1
03: Voice section detection unit, 104: Voice analysis unit, 105:
Speech recognition unit, 106: acoustic model, 107: word dictionary,
108: a connection part between the acoustic model and the word dictionary; 201, a voice input waveform; 202, a voice power.

 ──────────────────────────────────────────────────続 き Continuing from the front page (72) Inventor Hiroaki Kokubo 1-280 Higashi-Koigakubo, Kokubunji-shi, Tokyo F-term in Hitachi Central Research Laboratory 5B091 CB12 CD01 5D015 CC14 DD05 KK01

Claims (33)

[Claims] 1. A speech recognition system for collecting a word or a sentence to be subjected to speech recognition and displaying or outputting as speech the contents obtained from the dictionary as a speech recognition result. A voice recognition system, comprising: a voice detection unit that detects a voice section based on information; and performing voice recognition of the detected voice section. 2. The speech recognition system according to claim 1, wherein a noise threshold is obtained from the power of the non-speech section. 3. The speech recognition system according to claim 2, wherein said noise threshold value is compared with the power of said speech section, and said speech threshold value is compared with the power of said speech section when said power level reaches said noise threshold value. A speech recognition system for starting a recognition process. 4. A speech recognition system according to claim 2, wherein said noise threshold value is compared with the power of said voice section, and said power is predetermined in advance when said power of said voice section reaches said noise threshold value. A speech recognition system characterized by performing speech recognition processing retroactively to a given time. 5. The speech recognition system according to claim 2, wherein said noise threshold value is based on an average power of a predetermined number of aggregates of frames, which is a unit for analyzing speech or noise power. A speech recognition system characterized by what is required. 6. A speech recognition system according to claim 2, wherein a power threshold is obtained based on said noise threshold. 7. The speech recognition system according to claim 6, wherein the power of the speech section exceeds the noise threshold, and when the power reaches the power threshold, it is determined that the speech has begun, and from this point in time the speech is determined in advance. A speech recognition system wherein speech recognition processing is performed from a predetermined time. 8. The speech recognition system according to claim 6, further comprising a voice input button, wherein when the power of the voice section reaches the noise threshold after the button is pressed, A speech recognition system characterized by storing a frame which is a unit of speech analysis. 9. The speech recognition system according to claim 8, wherein when the power of the speech section reaches the power threshold, speech recognition processing is performed at least from the stored frame. Recognition system. 10. The voice recognition system according to claim 6, wherein a period during which the power of the voice section falls below the power threshold is equal to or less than a predetermined time, and an unvoiced sound portion between voices is generated. A speech recognition system characterized by making a decision. 11. The speech recognition system according to claim 6, wherein said power in said speech section is said power threshold for a predetermined time after said power in said speech section falls below said power threshold. A speech recognition system characterized by determining that a speech section has ended when the value is kept lower than the value. 12. A voice recognition method for displaying or outputting as a voice a content picked up from a dictionary in which words and sentences to be voice-recognized are collected as voice recognition results, based on information of a detected non-voice section. Detecting a voice section by performing a voice recognition, and performing a voice recognition of the detected voice section. 13. The speech recognition method according to claim 12, further comprising the step of obtaining a noise threshold from the power of the non-speech section. 14. A speech recognition method according to claim 13, wherein said noise threshold value is compared with the power of said speech section, and a speech recognition process is started near the time when said power reaches said noise threshold value. Voice recognition method. 15. The speech recognition method according to claim 12, further comprising: a step of obtaining a noise threshold from the power of the non-speech section; and a step of obtaining a power threshold based on the noise threshold. A speech recognition method characterized by the following. 16. The voice recognition method according to claim 15, wherein when a period during which the power of the voice section falls below the power threshold is less than or equal to a predetermined time, it is determined that the voice section is a voiceless part between voices. Voice recognition method. 17. The voice recognition method according to claim 15, wherein after the power of said voice section falls below said power threshold value, when a predetermined time has elapsed while maintaining this state, said voice section ends. A voice recognition method, comprising the step of determining that the voice recognition has been performed. 18. Words and sentences to be subjected to speech recognition are collected and defined as a dictionary, and words and sentences recognized by speech are picked up from the dictionary, and character strings are displayed, images indicated by the words, and speech synthesis are performed. In a speech recognition system that outputs speech as speech, information on non-speech sections is detected to distinguish speech segments from the beginning to end of speech from non-speech segments that do not contain speech. A voice section detection unit that detects voice sections, a voice analysis unit that performs voice analysis processing on the captured voice, an acoustic model that has a voice pattern in phoneme units, and an audio model A voice recognition system comprising: a voice recognition unit that performs voice recognition processing by connecting a model and a dictionary; and performs voice recognition on voice detected in a voice section. 19. A speech recognition system according to claim 18, wherein said speech section detection section detects a power level of a sound in a non-speech section, and adjusts the start of the speech in accordance with the power level which changes every moment. When the input voice exceeds the power level of the voice, the voice is determined to be the start of the voice, and when the power level of the voice falls below the power level for a predetermined time or more, the voice is determined to be the end. A speech recognition system characterized by: 20. The speech recognition system according to claim 18, wherein said speech section detection unit detects a power level of a sound in a non-speech section, and outputs a noise threshold value and a power from the power level. Calculates the threshold value of the threshold, the input voice exceeds the noise threshold,
A speech recognition system, wherein when a power threshold is exceeded, it is determined that speech has begun. 21. The speech recognition system according to claim 20, wherein the end of the speech is determined when the power level of the speech falls below the threshold for a predetermined time or more. 22. The speech recognition system according to claim 20, wherein the power level is an average power obtained over a plurality of frames by dividing a power of a frame divided by a predetermined time unit, and the noise threshold is set. A speech recognition system, wherein the value is set to N times the average power. 23. The speech recognition system according to claim 22, wherein said power threshold value PTH is set so as to satisfy a relationship of PTH> NTH as compared with a noise threshold value NTH. Voice recognition system. 24. The speech recognition system according to claim 23, wherein the relationship between the noise threshold NTH and the power threshold PTH is ΔPTH = PTH− in a relatively quiet environment.
A speech recognition system, wherein NTH is set to be small, and conversely, ΔPTH = PTH-NTH is set to be large in an environment where noise is large. 25. Words and sentences to be subjected to speech recognition are collected and defined as a dictionary, and words and sentences recognized by speech are picked up from the dictionary to display character strings, images represented by the words, and speech synthesis. In the speech recognition method that outputs speech as speech, information on non-speech sections is detected for the input speech in order to distinguish speech sections from the beginning to end of speech from non-speech sections that do not contain speech. Detecting a voice section while performing
Performing voice analysis processing, and connecting an acoustic model having a voice pattern in phoneme units and the dictionary to perform voice recognition processing on a voice analysis result. Method. 26. A speech recognition method according to claim 25, wherein a power level is detected for a sound in a non-speech section, and a speech indicating a beginning and an end of the speech in accordance with the power level which changes every moment. Is determined, and when the power level of the input voice exceeds the power level of the voice, it is determined that the voice starts, and when the power level of the voice falls below the power level for a certain time or more, it is determined that the voice ends. A speech recognition method comprising steps. 27. The voice recognition method according to claim 25, wherein a power level is detected for the sound in the non-voice section, and a noise threshold and a power threshold are calculated from the power level. Step, the power level of the input voice exceeds the noise threshold,
Determining the start of voice if the power threshold is exceeded. 28. A speech recognition method according to claim 27, further comprising the step of judging the end of the speech when the power level of the inputted speech falls below the power threshold for a predetermined time or more. Voice recognition method. 29. A speech recognition method according to claim 27, further comprising: determining a power of a frame delimited by a predetermined time unit as the average power obtained over a plurality of frames; and the noise threshold. Setting the value to N times the average power. 30. A speech recognition method according to claim 29, wherein said power threshold value PTH is a noise threshold value NTH.
A speech recognition system comprising a step of setting so as to satisfy a relationship of PTH> NTH as compared with the above. 31. A speech recognition method according to claim 30, wherein the relationship between the noise threshold value NTH and the power threshold value PTH is ΔPTH = PTH−N in a relatively quiet environment.
Setting the TH to be small, and conversely, setting ΔPTH = PTH-NTH to be large in an environment with a large amount of noise. 32. The speech recognition system according to claim 22, wherein the noise threshold value is set so as to decrease the value of N in an environment where the noise power is relatively quiet and the fluctuation of the noise power in a short time is small. A speech recognition system characterized in that the value of N is set to be large in an environment in which noise power fluctuates greatly in a short time with large noise. 33. The speech recognition method according to claim 29, wherein the noise threshold value is set to be small in an environment where the noise power is relatively quiet and the fluctuation of the noise power in a short time is small; On the other hand, a speech recognition method characterized by comprising a step of setting the value of N to be large in an environment where noise power fluctuates greatly in a short time with large noise.