JP2002149191A - Voice input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an on-vehicle voice input device with a high recognition rate. SOLUTION: The voice input device is provided with a sensor 5 for detecting a speaker state or/and the traveling state of an automobile, and a mental state estimating device 10 for estimating the mental state of the speaker by the output. A voice recognition dictionary group 40 is also arranged and is provided with dictionaries 41, 42, 43, etc., which are generated based on uttering under various kinds of tensions in the automobile. A dictionary selecting unit 15 selects the optimum dictionary, e.g. the dictionary 41 from the voice recognition dictionary group 40 in accordance with the mental state estimation value (a value for estimating a composure degree, for example) when the voice is inputted. A voice recognizing device 25 recognizes the voice by using the optimum dictionary corresponding to the mental state of the speaker so that the recognition rate is improved. This configuration is utilized in the on-vehicle voice input device.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice input device for controlling a device by voice. In particular, the present invention relates to a voice input device that estimates a mental state of a person, selects a dictionary corresponding to the mental state, and improves a voice recognition rate. The present invention estimates the driver's mental state from the running state, for example,
The present invention can be applied to a voice input device that performs accurate voice recognition according to the mental state.

[0002]

2. Description of the Related Art Conventionally, various voice input devices using voice recognition have been proposed. In particular, when it is applied to automobiles, further improvement in recognition rate is required for improving safety. For example, there are a speech recognition device and a vehicle speech recognition device disclosed in Japanese Patent Application Laid-Open No. 11-126092 as examples for improving the recognition rate. This is an example of increasing the recognition rate by eliminating the cause of the decrease in the speech recognition rate. FIG. 7 shows a schematic configuration diagram thereof. The conventional vehicle voice recognition device includes a signal processing control unit 112, a sound processing unit 114, a microphone 116, a voice recognition unit 118, a voice synthesis unit 120, a speaker 122, a communication control unit 124, and a trigger switch unit 12.
6, body ECU 130, navigation ECU 1
50. And power window 132,
A sunroof 134, an air conditioner 136, an audio device 138, a door lock 140, and a speedometer 142 are connected to the body ECU 130.

[0003] In the above configuration, when unrecognition occurs in the recognition result of the voice recognition unit 118, the signal processing control unit 112 specifies the cause of the unrecognition. For example, when it is determined that the surrounding environment of the microphone 116 is deteriorated when the voice is buried in the noise, the body ECU 130 improves the input environment of the microphone 116. For example, when the power window 132 is open, the window 132 is closed. This is an example in which speech recognition is performed again to improve the recognition rate.

Another example is a speech recognition apparatus disclosed in Japanese Patent Application Laid-Open No. 9-134193. This is an example in which a speech recognition algorithm is selected according to the situation to improve the recognition rate. For example, the central processing unit estimates the usage environment based on information from various sensors. Then, a program suitable for the use environment is selected and executed from among the plurality of speech recognition programs. For example, it is determined from the vehicle speed sensor whether or not the vehicle is traveling. If the vehicle is traveling, a voice recognition program having high noise immunity against noise such as road noise and wind noise is executed. This allows
This is an example in which the recognition rate is improved.

[0005] There are other papers suggesting an improvement in the recognition rate. For example, the document "MULTI-STYLE FORROBUST ISOLATED-W
ORD SPEECH RECONGNITION ”(Richard P. Lippmann, Edw
ard A. Martin, Douglas B. Paul) ICASSP'87, "AM
ACROSCOPIC ANALYSYS OF AN EMOTIONAL SPEECH CORPUS
”(JEH Noad, SP Whiteside and PD Green), E
uroSpeech'97 ”etc. These are speech recognition acoustic models created from voices recorded under various mental states,
It states that a higher recognition rate can be obtained in that situation than an acoustic model created from normal speech only.

[0006]

However, Japanese Patent Laid-Open No.
The speech recognition device and the vehicle speech recognition device disclosed in Japanese Patent Application Laid-Open No. 1-126092 are examples in which a physical cause inside and outside the vehicle compartment that reduces the speech recognition rate is eliminated to increase the recognition rate. Therefore, there is a disadvantage that the recognition rate cannot be improved when the cause of the decrease in the voice recognition rate is the driver state. Also, Japanese Patent Laid-Open No. 9-1
The voice recognition device disclosed in Japanese Patent No. 34193 is a device that selects a voice recognition algorithm in accordance with physical conditions inside and outside a vehicle such as noise to improve a recognition rate. Therefore, even in this case, it is not a device that can respond to a change in the driver's state.

[0007] Also, the document "MULTI-STYLE FOR ROBUST ISOLA"
TED-WORD SPEECH RECONGNITION ”(Richard P. Lippman
n, Edward A. Martin, Douglas B. Paul) ICASSP'87
Is a method of creating one type of average acoustic model from voices recorded under various mental states, thereby responding to changes in voice quality. However, under stress caused by driving, the driver not only changes voice quality but also changes words and phrases such as unnecessary words. It is considered that the recognition rate can be further improved by using a speech recognition dictionary that takes this unnecessary word into consideration. That is, there is ample room for improvement and progress.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a speech recognition dictionary having dictionaries corresponding to various mental states, and to have the mental state of a speaker during speech recognition. It is to improve the recognition rate by estimating and using a dictionary corresponding to the mental state. It is another object of the present invention to apply it to a voice input device to perform appropriate control on various devices.

[0009]

A voice input device according to a first aspect of the present invention is a voice input device having voice input means and voice recognition means for recognizing input voice and controlling various devices. A mental state estimating apparatus for estimating a speaker's mental state by a sensor device and an output of the sensor device, and a speech recognition dictionary group having a plurality of dictionaries created based on a plurality of acoustic models and a plurality of language models. And a dictionary selecting device for selecting an appropriate dictionary from the speech recognition dictionary group according to the output of the mental state estimating device. Note that the plurality of dictionaries created based on a plurality of acoustic models and a plurality of language models means a speech dictionary created based on speech recorded under various environments. Hereinafter, the dictionary is used in the meaning of the voice dictionary.

[0010] According to the voice input device of the second aspect,
The dictionary selecting device is characterized by having a correspondence table between a driver's mental state and a plurality of dictionaries of a voice recognition dictionary group.
According to the voice input device of the third aspect, when the speaker is in the own vehicle, the acoustic model is a vehicle interior acoustic model. According to a fourth aspect of the present invention, there is provided a voice input device, wherein the sensor device has a vehicle sensor for detecting a traveling state of the vehicle. According to the voice input device of the fifth aspect, the sensor device has an obstacle sensor for detecting an obstacle outside the own vehicle.

[0011]

According to the voice input device of the first aspect, the sensor device detects and outputs, for example, a speaker state, for example, a heart rate, a sweating amount, an operation amount, and the like of the speaker. The mental state estimation device estimates the mental state of the speaker based on the output of the sensor device. The mental state is, for example, a state where a work load is applied, a degree of tension, a margin, an emotion, and the like of the speaker when the environment is changed. For example, the heart rate and the amount of palm sweat correlate with the speaker's tension. If the sensor device is, for example, a camera device, the emotion of the speaker can be estimated by image processing. next,
The dictionary selecting device selects an appropriate dictionary corresponding to the mental state from a speech recognition dictionary group including a plurality of dictionaries, according to the mental state. Then, it is output to the speech recognition device. The speech recognition device performs speech recognition using the dictionary. Since the speech recognition device uses the dictionary according to the mood of the speaker, the speech can be more accurately recognized. That is, the speech recognition rate is improved. Therefore, if a voice input device is formed with these components, various devices can be operated more accurately.

[0012] According to the voice input device of the second aspect,
The dictionary selection device has a correspondence table in which a mental state of a speaker is associated with a plurality of dictionaries in a speech recognition dictionary group. Therefore, an appropriate dictionary can be instructed immediately according to the output of the mental state estimation device. Accordingly, the voice input device controls the various devices by performing the voice recognition quickly.

According to the voice input device of the third aspect,
When the speaker is in the vehicle, the acoustic model uses the vehicle interior acoustic model. The vehicle interior acoustic model is an acoustic model that takes into account vehicle interior reflection of a speaker's voice, own vehicle engine sound, wind noise, and the like. Since the vehicle interior acoustic model is used, speech recognition can be performed with high accuracy even in the own vehicle. Therefore, the voice input device operates the on-vehicle device with high accuracy.

According to the voice input device of the fourth aspect,
The sensor device has a host vehicle sensor for detecting the host vehicle state. For example, when the speaker is a driver, the mental state changes depending on the running state. For example, during acceleration or high-speed running, the degree of tension increases, and the driving margin decreases. That is, the mental state of the speaker can be estimated by the own vehicle sensor, for example, the vehicle speed sensor. This further reinforces the estimation from speaker sensors, such as heart rate and sweat detection. For example, even if the driving margin is medium as estimated from the speaker sensor, the driving margin is corrected to be small if the traveling speed is faster than a predetermined value, for example. Therefore, the recognition rate in the vehicle compartment can be further improved. That is, the voice input device controls the on-vehicle device more accurately.

According to the voice input device of the fifth aspect,
The sensor device has an obstacle sensor that detects an obstacle outside the vehicle. For example, when the speaker is a driver, the tension of the mental state increases when an obstacle is found during traveling. At this time, the obstacle sensor can estimate the mental state of the speaker (driver) faster than the speaker sensor and the own vehicle sensor. Because speaker sensors such as heart rate detection,
This is because, for example, the reaction of the own vehicle sensor such as lane change detection has a predetermined period of delay with respect to tension.

Therefore, the obstacle sensor can change the speech recognition dictionary at the earliest when an obstacle is found. Alternatively, the dictionary for speech recognition can be changed prior to the tension. That is, the delay does not lower the recognition rate (due to an inappropriate dictionary). That is, a high recognition rate can be maintained even when an obstacle is found. Here, this obstacle sensor is desirably used in combination with the speaker sensor and the vehicle sensor. If three persons are used together, it becomes a voice input device which can cope with various changes. The obstacle sensor is, for example, a camera device, a laser radar device, or the like. The obstacles include not only so-called obstacles on the road but also vehicles ahead and side vehicles. It also includes travel path components such as guardrails.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS (First Embodiment) An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows an embodiment of a voice input device according to the present invention. The figure is a system configuration diagram. The voice input device of the present embodiment includes a sensor device 5, a mental state estimation device 10, a dictionary selection device 15, a microphone 20 as voice input means, a voice recognition device 25 as voice recognition means, an output device 35, a plurality of dictionaries. And a voice recognition dictionary group 40 composed of Dictionary correspondence table 15a
Are included in the dictionary selection device 15.

The sensor device 5 is a sensor that detects, for example, the speaker's heart rate and the amount of sweat in the palm. Mental state estimation device 1
Numeral 0 is a device that receives the output and estimates the mental state of the speaker. For example, a margin, which is one of the mental states, is estimated to be large, medium, or small according to the output (heart rate) of the sensor device 5.
The dictionary correspondence table 15a is a correspondence table of a dictionary and its margin. For example, a dictionary 41 (dictionary A) corresponds to a small margin, a dictionary 42 (dictionary B) corresponds to a medium margin, and a dictionary 43 (dictionary C) corresponds to a large margin.

The dictionary selection device 15 is a device that notifies an appropriate dictionary, for example, a dictionary number, from the dictionary correspondence table 15a according to the estimated value. The speech recognition dictionary group 40 is a dictionary 4 which is a plurality of speech dictionaries formed under various environments, that is, based on a plurality of acoustic models and a plurality of language models.
, 42, 43,. Then, the voice recognition device 25 transmits the dictionary of the designated dictionary number, for example, the dictionary 4
1 is a device for recognizing the voice from the microphone 20 using the device 1 and outputting the result to the output device 35 connected thereto. The voice input device according to the present embodiment is configured by a computer system including a CPU for executing arithmetic operations, a ROM for storing programs, a RAM for storing data, an external memory such as a hard disk as a peripheral device, and an interface. I have. The mental state estimation device 10 and the dictionary selection device 15 are formed by a program executed by the CPU and peripheral devices. Also, the dictionary correspondence table 15a and the speech recognition dictionary group 4
Numeral 0 is formed in a peripheral device of the computer system, for example, an external memory unit.

FIG. 2 shows the operation of the voice input device of this embodiment. The figure is a flowchart. The voice input device of the present embodiment is started when a recognition switch (not shown) is turned on. First, in step S10, the mental state estimating device 10 is operated and the output of the sensor device 5 is read. For example,
Read the speaker's heart rate. Next, the process proceeds to step S11, in which, for example, the mental state of the speaker according to the heart rate is estimated. For example, the margin of the speaker is estimated to be large, medium, or small according to the heart rate. Then, in step S12, for example, a dictionary number corresponding to the estimated value is selected from the dictionary correspondence table 15a.
For example, if the margin is small, the dictionary 41 is selected.

Next, the process proceeds to step S13, where the microphone 20
From the speaker. Then, in step S14, the voice is recognized using the dictionary specified by the dictionary number.
For example, speech recognition of, for example, HMM (Hidden Markov Model) method is performed using the dictionary 41. Then, the result is output to the output device 35.

At this time, the output value is, for example, a series of recognized character codes, and the output device is, for example, an audio device or an air conditioner. When the output device is an audio device, the recognition result is, for example, switch-on of a TV device or the like, such as designation of a channel, designation of a volume, and the like. The voice input device is operated, for example, according to such a flowchart.

As described above, the voice input device of the present embodiment detects the speaker state (mental state) by the sensor device and performs voice recognition using a dictionary corresponding to the mental state. Therefore,
The recognition rate can be improved as compared with the conventional voice recognition using a normal dictionary. In addition, since the control device is controlled based on the recognition result, it can be controlled more accurately.

Second Embodiment The first embodiment is an example in which the present invention is applied to a general voice input device. This embodiment is an example applied to an on-vehicle navigation system. FIG. 3 shows a navigation system according to the present embodiment. The difference from the first embodiment is that the sensor device 5 of the first embodiment is a vehicle speed sensor 5a, and the mental state of the driver is estimated from the sensor output. And the output device 35
Is a navigation device 36 and a display device 37 for displaying the result to the driver.

The navigation system according to the present embodiment has a vehicle speed sensor 5a as a sensor device and a mental state estimation device 1
0, dictionary selection device 15, microphone 20, speech recognition device 2
5, dictionary correspondence table 15a, navigation device 3
6, a display device 37 for displaying the result, and a speech recognition dictionary group 40 including a plurality of dictionaries 41, 42, 43.

The vehicle speed sensor 5a is, for example, a sensor for detecting the rotation speed of the axle. The mental state estimation device 10 estimates the mental state of the driver from the output of the vehicle speed sensor 5a.
For example, the driving margin is estimated. For example, a vehicle speed of 10 km /
If the speed is less than h, it is estimated that the operation margin is large if 10 km / h to 50 km, and if the speed is 50 km / h or more, the operation margin is small. The dictionary correspondence table 15a is a correspondence table between the dictionary and the mental state. For example, a dictionary 41 (dictionary A) for a small driving margin (high vehicle speed), a dictionary 42 (dictionary B) for a medium driving margin (medium vehicle speed), and a dictionary for a large driving margin (small vehicle speed). 43 (dictionary C) are supported (FIG. 4).

The dictionaries A, B, and C are a vehicle interior acoustic model and a language model learned from the voice recorded in the vehicle interior according to the mental state at each vehicle speed. It is stored in. The dictionary selecting device 15 is a device that selects and notifies the dictionaries 41, 42, and 43 corresponding to the mental state according to the dictionary correspondence table 15a. still,
The voice recognition device 25 is a device equivalent to that of the first embodiment. That is, it is a device that recognizes the voice from the microphone 20 using a designated dictionary, for example, the dictionary 41, and outputs the result to the connected navigation device 36. The navigation device 36 and the display device 37 of the present embodiment are
CPU, external storage device, input device, interface, R
It is realized as a computer system including a display such as an OM, a RAM, and a liquid crystal.

FIG. 5 shows the operation of the navigation system of this embodiment. The figure is a flowchart. Here, for example, it is assumed that a destination is already set in the system. Then, in the case of obtaining the estimated required time to the destination, an example will be described in which the driver pronounces the "estimated time", recognizes its voice, and displays the result. The navigation system according to the present embodiment is started when a recognition switch (not shown) is turned on. First, step S
At step 20, the output of the vehicle speed sensor 5a is read, and the driving margin, which is the mental state of the driver, is estimated (operation of the mental state estimation device). For example, if the vehicle speed is 10 Km / h or less, the driving margin is large.
If it is equal to or higher than Km / h, it is estimated that the operating margin is small.

Next, the process proceeds to step S21, where a dictionary corresponding to the estimated value is selected from the dictionary correspondence table 15a (FIG. 4) (operation of the dictionary selection device). For example, if the driving margin is large, the dictionary A is selected, and if the driving margin is small, the dictionary C is selected and its number is notified. Next, the process proceeds to step S22 to obtain a sound from the microphone 20. For example, the pronunciation “predicted time” is obtained. Then, in step S23, the voice is recognized using, for example, the dictionary 41 selected according to the driving margin. This speech recognition is also performed, for example, by the HMM (Hidden Markov Model) method.

At this time, the output value of the speech recognition device 25 is, for example, a series of recognized character codes. Then, it outputs it to the navigation device 36. And step s
At 24, the navigation device 36 issues the command “estimated time”.
, That is, the remaining time to the destination is displayed on the display device 37, and the process ends. Alternatively, the speech is synthesized by a voice synthesizer (not shown), the driver is notified, and the processing ends. The navigation system employing the voice input device of the present embodiment operates, for example, according to such a flowchart.

As described above, the voice input device of the present embodiment detects the traveling state of the vehicle and estimates the mental state of the driver based on the detected traveling state. Then, speech recognition is performed using a dictionary corresponding to the mental state. The dictionary is a dictionary based on a vehicle interior acoustic model and a vehicle interior language model. Therefore, the recognition rate in the vehicle compartment can be improved as compared with the related art. Therefore, the voice input device controls the navigation device more accurately than before.

(Modification) The present invention can be implemented in various other forms. For example, although the mental state estimation device 10 and the dictionary selection device 15 of the first and second embodiments are constituted by a computer device (not shown) and its program and its peripheral devices, they may be formed independently. The actual form does not matter as long as it has the function.

Further, the sensor device 5 of the first embodiment is an example in which a sensor for detecting the state of a speaker is used to directly estimate the mental state of the speaker. Further, the sensor device of the second embodiment is an example in which a sensor (vehicle speed sensor 5a) for detecting the state of the own vehicle is used, and the mental state of the driver (speaker) is indirectly estimated from the output thereof. The sensor device 5 may have both functions. That is, the state of the speaker (driver) and the state of the own vehicle may be detected, and the mental state of the speaker may be comprehensively estimated from both. For example, even if it is determined that the driving margin, which is a mental state once, is large when the vehicle speed is 5 km / h or less, if the heart rate and the amount of sweating of the palm are large, the driving margin is corrected to medium or small. You may. This is because the driver may be nervous due to other factors. With this configuration, the mental state can be more accurately estimated. Therefore, the device can be controlled more accurately.

Further, an obstacle detection sensor may be added to the sensor device 5. This is because, if the driver is nervous due to an obstacle or the like, the obstacle sensor can detect it faster than other sensors. Because the speaker (driver) becomes nervous, the speaker sensor,
This is because it takes a predetermined time for the own vehicle sensor to detect it. For example, heart rate detection, lane change detection, and the like have a response delay of, for example, reflexes to the occurrence of tension. Therefore, in order to respond quickly to tension caused by an obstacle or the like, an obstacle sensor such as a laser radar device, a camera device, or an ultrasonic device may be added.

When an obstacle sensor is added to the sensor device 5, step S20 in the flowchart of FIG. 5 is changed to steps S20a and S2b shown in FIG. That is, in step S20a, three outputs of the speaker sensor, the own vehicle sensor, and the obstacle sensor are read. Next, step S20b
Then, the mental state of the driver is estimated from the three outputs. For example, priorities are assigned in the order of the obstacle sensor, the vehicle speed sensor, and the speaker sensor, and the mental state is estimated according to the priority. And
The estimated value may be sent to the next step S21.
By doing so, it is possible to change the dictionary for speech recognition at the earliest. Alternatively, the dictionary for speech recognition can be changed prior to the tension. That is, the recognition rate is not reduced (by an inappropriate dictionary) due to the delay of the speaker's reflexes. That is, the voice input device maintains a high recognition rate even when an obstacle is found.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram of a voice input device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of the voice input device according to the first embodiment of the present invention.

FIG. 3 is a system configuration diagram of a navigation system according to a second embodiment of the present invention.

FIG. 4 is a correspondence table between a mental state and a dictionary used in the second embodiment of the present invention.

FIG. 5 is a flowchart showing an operation of the navigation system according to the second embodiment of the present invention.

FIG. 6 is a modified step of the flowchart according to a modified example of the second embodiment of the present invention.

FIG. 7 is a system configuration diagram of a voice recognition device that is a conventional voice input device.

[Explanation of symbols]

 Reference Signs List 5 sensor device 10 mental state estimation device 15 dictionary selection device 15a dictionary correspondence table 20 microphone 25 voice recognition device 35 output device 40 voice recognition dictionary group

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) B60R 21/00 621 B60R 21/00 621E 622B 622 622C 622F 624B 624 624C 624E 624F G10L 3/00 571V G10L 15 / 06 521T 15/20 531Q 15/00 551J 15/28 551Q 15/24 561C 571Q F term (reference) 3D037 FA01 FA09 FA16 FA23 FB09 FB10 5D015 AA01 AA06 BB02 HH23 KK01 KK04 LL05 LL06 LL12

Claims (5)

[Claims] 1. A voice input device comprising a voice input means and a voice recognition means for recognizing an input voice and controlling various devices, comprising: a sensor device; A mental state estimating device for estimating a state, a speech recognition dictionary group having a plurality of dictionaries created based on a plurality of acoustic models and a plurality of language models, and according to an output of the mental state estimating device, A dictionary selection device for selecting an appropriate dictionary from a group of speech recognition dictionaries. 2. The speech input device according to claim 1, wherein the dictionary selection device has a correspondence table between a mental state of a speaker and the plurality of dictionaries of the speech recognition dictionary group. 3. The voice input device according to claim 1, wherein when the speaker is in a vehicle, the acoustic model is a vehicle interior acoustic model. 4. The voice input device according to claim 3, wherein said sensor device has a vehicle sensor for detecting a traveling state of the vehicle. 5. The voice input device according to claim 3, wherein said sensor device has an obstacle sensor for detecting an obstacle outside the host vehicle.