JP2017090613A - Voice recognition control system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice recognition control system capable of readily providing guide information relating to a function unknown to an occupant to improve convenience.SOLUTION: A voice recognition controller 1 includes: a voice recognition unit 3 for recognizing an utterer and a content of an utterance on the basis of at least a voice input from a microphone array 21; a gesture detection unit 4 for detecting a gesture of the utterer expressing an object of the utterance on the basis of an image in a vehicle cabin that is photographed by an indoor camera 22; and a guide unit 7 for outputting a voice guide or a video guide for guiding how to use the object expressed by the gesture when the content of the utterance recognized by the voice recognition unit 3 contains a predetermined voice command.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a voice recognition control system for controlling an in-vehicle device with a voice of a vehicle occupant.

  2. Description of the Related Art Conventionally, guidance systems have been developed that provide voice guidance for operating various on-vehicle devices mounted on a vehicle. That is, the guide information explaining specific usage is provided by voice to passengers unfamiliar with the operation of various in-vehicle devices. For example, a technique for outputting a voice guide related to an operation method of an automatic locking / unlocking device at the time of opening / closing a vehicle door is known (see Patent Document 1). As a result, the occupant can not only save time and effort to check the user manual, but also can easily increase the proficiency level of operation and improve the convenience of various in-vehicle devices.

JP2008-255753

  On the other hand, in-vehicle devices in recent years have been multi-functionalized to meet the needs for convenience, comfort, and adaptability to the natural environment, and even the presence of newly added functions may not be known to passengers. . Such guide information relating to unknown functions is difficult to provide with existing guidance systems, and it has also been difficult for passengers to extract such guide information.

  One of the purposes of this case was created in view of the above problems, and can easily provide guide information regarding functions unknown to the occupant, and can improve the convenience of voice recognition control. Is to provide a system. It is not limited to this purpose, and is an operational effect derived from each configuration shown in “Mode for Carrying Out the Invention” to be described later. Can be positioned as

(1) The voice recognition control system disclosed here is a voice recognition control system that controls an in-vehicle device using a voice of a vehicle occupant as an input signal. The system includes a speech recognition unit that recognizes a speaker and utterance contents based on at least the speech. In addition, a gesture detection unit is provided for detecting the gesture of the speaker representing the utterance target based on the vehicle interior image captured by the indoor camera. In addition, when the utterance content recognized by the voice recognition unit includes a predetermined voice command, a guide unit that outputs a voice guide or a video guide for guiding how to use the object represented by the gesture is provided.
In addition, when the said utterance content does not contain the said predetermined | prescribed voice command, it is preferable that the said guide part controls the operating state of the said object according to the said utterance content.

(2) It is preferable that the gesture is a gesture of pointing a finger toward the target.
(3) It is preferable that the guide unit changes an information amount of the voice guide or the video guide according to the speaker.
For example, when the speaker is a driver, it is preferable to increase the amount of information and provide specialized guidance. On the other hand, when the speaker is an occupant other than the driver, it is preferable to reduce the amount of information and perform basic guidance.

(4) It is preferable that the guide unit changes the information amount according to the accumulated boarding time of the speaker.
(5) It is preferable that the guide unit outputs the voice guide or the video guide on the condition that the speaker is a driver or an assistant.
(6) It is preferable to further include a vehicle speed detector that detects the vehicle speed. In this case, it is preferable that the guide unit outputs the audio guide or the video guide on condition that the vehicle speed detected by the vehicle speed detection unit is equal to or lower than a predetermined vehicle speed.

  By combining voice input and gesture input to output a voice guide or video guide, it is possible to inform the occupant how to use the in-vehicle device, and in particular, it is possible to easily provide guide information regarding functions unknown to the occupant. it can. Further, the occupant can draw out a voice guide and a video guide related to the operation method and function of the target simply by issuing a predetermined voice command while making a gesture representing the target of the utterance. Therefore, the convenience of the in-vehicle device can be improved.

1 is a schematic top view of a vehicle to which a voice recognition control system is applied. It is a schematic diagram which shows the structure of a speech recognition control system. It is a flowchart for demonstrating the control content of a speech recognition control system. It is a figure which shows the state which the passenger | crew is requesting the guide regarding switches.

  A speech recognition control system as an embodiment will be described with reference to the drawings. Note that the embodiment described below is merely an example, and there is no intention to exclude various modifications and technical applications that are not explicitly described in the following embodiment. Each configuration of the present embodiment can be implemented with various modifications without departing from the spirit thereof. Further, they can be selected as necessary, or can be appropriately combined.

[1. Device configuration]
The voice recognition control system of this embodiment is applied to the vehicle 10 shown in FIG. A driver's seat 14 and a passenger seat 15 are provided in the passenger compartment of the vehicle 10, and an instrument panel (instrument panel, dashboard) is disposed on the front side of the passenger compartment. A steering device and instruments are arranged in front of the driver's seat 14 and a glove box is arranged in front of the passenger seat 15 in the portion of the instrument panel facing the passenger compartment. In the center of the instrument panel in the vehicle width direction, a multi-communication type display device 16 and a button-type switch 17 for providing a user interface such as a car navigation function and an audio visual function are provided. The positions of the display device 16 and the switches 17 are diagonally left forward from the viewpoint of the driver sitting in the driver's seat 14 and diagonally forward right from the viewpoint of the passenger (assistant) sitting in the passenger seat 15.

  The display device 16 is an electronic control including a general-purpose video display device (display screen) having a touch panel, a speaker (acoustic device), a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. An electronic device provided with a device (computer). The display device 16 is connected to in-vehicle devices such as the navigation device 11, the air conditioner device 12, the car audio device 13, and the multimedia system, and can function as an input / output device for various in-vehicle devices. For example, route information, map information, traffic jam information, etc. provided from the navigation device 11 to the destination can be displayed on the display screen of the display device 16, and voice guidance can also be output. In addition, the display screen of the display device 16 is used for operating a program received by an in-vehicle terrestrial digital broadcast tuner, video content of an information recording medium, video taken by a rear view camera, an air conditioner device 12 or a car audio device 13. Various audiovisual information such as an interface can be reproduced and displayed.

  The switches 17 are composed of a plurality of keys (buttons) having built-in light emitting elements such as LEDs (Light Emitting Diodes) and organic ELs (Organic Electro-Luminescence). The audio device 13 is connected to a vehicle-mounted device such as a multimedia system, and can function as an input device for various vehicle-mounted devices. One of the functions of various in-vehicle devices can be arbitrarily set for each button. For example, when a passenger of the vehicle 10 frequently uses the car audio device 13, it is possible to give a music selection function or a speaker volume adjustment function to one of the buttons of the switches 17. In addition, when the navigation device 11 is frequently used, it is possible to provide a startup function of the navigation device 11.

  In addition, the vehicle 10 is equipped with a voice recognition control device 1 that controls various in-vehicle devices using an occupant's voice as an input signal. The speech recognition control device 1 is an electronic device (ECU, electronic control device) in which a processor such as a CPU or MPU (Micro Processing Unit) and a ROM, RAM, nonvolatile memory, etc. are integrated. The processor here is, for example, a processing device (processor) including a control unit (control circuit), an arithmetic unit (arithmetic circuit), a cache memory (register), and the like. The ROM, RAM, and nonvolatile memory are memory devices that store programs and working data. The contents of the control performed by the voice recognition control apparatus 1 are recorded in ROM, RAM, nonvolatile memory, and removable media as firmware and application programs. When the program is executed, the contents of the program are expanded in the memory space in the RAM and executed by the processor.

  As shown in FIG. 2, examples of the input device of the voice recognition control device 1 include a microphone array 21, an indoor camera 22, a vehicle speed sensor 23, and the like. The microphone array 21 is an audio input device in which a plurality of microphones are arranged in a predetermined arrangement, and the indoor camera 22 is a wide-angle video camera capable of photographing the entire vehicle interior. The microphone array 21 and the indoor camera 22 are built in, for example, the ceiling surface at the center in the vehicle width direction. The vehicle speed sensor 23 is a sensor that outputs a pulse signal corresponding to the rotational speed of the wheel. On the other hand, examples of the output device of the voice recognition control device 1 include a navigation device 11, an air conditioner device 12, a car audio device 13, a display device 16, and switches 17. The voice recognition control device 1 controls various on-vehicle devices based on the voice input from the microphone array 21, the image taken by the indoor camera 22, and the pulse information detected by the vehicle speed sensor 23.

[2. Control configuration]
The voice recognition control device 1 has a function (so-called “how to use guide function”) for outputting a voice guide and a video guide for guiding how to use various in-vehicle devices based on a passenger's voice and a vehicle interior image. For example, when some kind of voice is input from the microphone array 21, the utterer who utters the voice is first recognized and the utterance content is recognized. Moreover, based on the image image | photographed with the indoor camera 22, the gesture made by the speaker at the same time as the utterance is detected, and the object (the object specified by the gesture) represented by the gesture is detected. When the utterance content is “request for guide”, a voice guide or a video guide for guiding how to use the object represented by the gesture is provided to the occupant. On the other hand, if the utterance content is a “control command”, control is performed on the object represented by the gesture according to the utterance content.

Specific examples of voice guides provided to passengers are shown below.
・ "This button is a car navigation start button"
・ "This switch is an eco-mode selection switch"
・ "This lever is a steering heater start lever"
・ "The menu screen of the car navigation system, audio system, and air conditioner system is displayed on this display"
In addition, specific examples of the video guide provided to the occupant include displaying the contents of the audio guide in characters and playing and displaying an illustrated guide as described in the user manual of the vehicle 10. It is done.

  As an element for carrying out the control as described above, the voice recognition control device 1 is provided with a vehicle speed detection unit 2, a voice recognition unit 3, a gesture detection unit 4, a database 5, a person identification unit 6, and a guide unit 7. . These indicate some functions of a program executed by the speech recognition control apparatus 1 and are realized by software. However, some or all of the functions may be realized by hardware (electronic control circuit), or may be realized by using software and hardware together.

The database 5 is a storage device in which various general data relating to speech recognition are recorded and stored. Here, acoustic models and language models used in speech recognition are recorded and stored. The acoustic model and the language model are created in advance based on the voice of the standard speaker. In addition, about the structure of a specific acoustic model and a language model, it can employ | adopt with reference to a well-known technique (for example, Unexamined-Japanese-Patent No. 2002-189492 etc.).
The vehicle speed detection unit 2 acquires (detects and calculates) the vehicle speed based on the pulse signal output from the vehicle speed sensor 23. The information on the vehicle speed acquired here is transmitted to the guide unit 7.

  The voice recognition unit 3 recognizes a speaker (speech position) and utterance contents based on at least the voice input from the microphone array 21. Here, whether the speaker is a person (driver) seated in the driver's seat 14, a person (passenger) seated in the passenger seat 15, or another passenger (passenger in the rear seat) Is determined. The position of the speaker can be specified based on the size and delay of a plurality of audio signals detected by the microphone array 21. Alternatively, the position of the speaker can be specified by analyzing an image captured by the indoor camera 22 and comparing the movement of the lips of the person existing in the image with the timing at which the sound is detected.

  The utterance content is classified and recognized as one of three types: “Guide request”, “Control command”, and “Other”. For example, if the utterance is “What is this? ""What was that? ""Explain? ”Is determined to be“ Guide Request ”. On the other hand, if the utterance content includes voice commands such as “on”, “off”, “activate”, and “stop”, it is determined that the utterance content is a “control command”. If the utterance content does not include any of the above voice commands, it is determined that the utterance content corresponds to “other”. A specific method of speech recognition is arbitrary, and a known speech recognition technique can be employed. For example, after a phoneme included in speech is analyzed based on an acoustic model, a word or phrase consisting of a series of phonemes is analyzed based on a language model, and its semantic content is recognized. The information of the speaker and the content of the utterance recognized here is transmitted to the gesture detection unit 4 and the guide unit 7.

  The gesture detection unit 4 detects a gesture by a person (that is, a speaker) existing at an utterance position based on an image photographed by the indoor camera 22. Here, a gesture representing an utterance target is detected as the gesture of the speaker. Specific examples of gestures include a gesture that points a finger at a target and a gesture that looks at a target (a gesture that turns a line of sight). The “utterance target” represented by this gesture is displayed on various display devices 16 such as various in-vehicle devices mounted on the vehicle 10, its operation buttons, switches 17, indicators displayed on the instrument panel. Includes icons.

  The object can be detected by estimating from the position of the speaker recognized by the voice recognition unit 3 what is placed ahead of the virtual line extending in the direction indicated by the gesture. When detecting the gesture of the pointing finger as a gesture, the position of the hand is estimated by image analysis, and the target is accurately detected by extending a virtual line in the direction of the finger based on the position of the speaker's hand in the passenger compartment. be able to. On the other hand, when detecting the gaze as a gesture, the position and orientation of the face are estimated by image analysis, and the target is accurately detected by extending the virtual line in the gaze direction based on the position of the speaker's face in the vehicle interior. can do. Information on the target detected here is transmitted to the guide unit 7.

  The person identifying unit 6 identifies a person who has emitted the sound based on at least the sound input from the microphone array 21. Here, the person who speaks is detected, and the time (cumulative boarding time) that the person has boarded the vehicle 10 is measured. As a method for identifying a person, a method for identifying in real time when sound is detected, and a relationship between the person and the seating position when someone gets on the vehicle 10, and a sound source of the detected sound And a method of identifying a person based on the position.

  In the former case, it is possible to specify a person based on a waveform pattern, a frequency pattern, a voiceprint pattern, etc. included in the voice. Alternatively, it is also possible to extract a human face from an image photographed by the indoor camera 22 and specify a person whose lip movement matches the voice timing. In the latter case, a person may be specified by analyzing an image captured by the indoor camera 22 (for example, face authentication), or the person is uttered with some sound, and the same method as the former is used. The person may be specified. The information of the person specified here is transmitted to the guide unit 7.

The guide unit 7 controls various in-vehicle devices based on the speaker and speech content recognized by the speech recognition unit 3 and the gesture detected by the gesture detection unit 4. The guide part 7 mainly has two functions.
The first function is a function (hands-free control function) for controlling an object specified by a gesture by voice. When the utterance content recognized by the voice recognition unit 3 does not include a voice command meaning “request for guide” and includes a voice command meaning “control command”, the guide unit 7 To control. If the utterance content does not include a voice command that means “request for guide” or a voice command that means “control command”, the voice command is canceled without being controlled. The

The second function is a function (guide function) for outputting a voice guide or a video guide for guiding how to use the object specified by the gesture to the display device 16. When the utterance content recognized by the voice recognition unit 3 includes a voice command meaning “request for guide”, the guide unit 7 provides the occupant with a voice guide and a video guide regarding the object.
The guide unit 7 of the present embodiment is based on the condition that the speaker is a driver or an assistant and that the vehicle speed detected by the vehicle speed detection unit 2 is equal to or lower than a predetermined vehicle speed (for example, 10 km / h or lower). Activate the guide function. That is, the guide function can be executed when the “request for guide” by the driver or the assistant is recognized together with the gesture. This prevents unnecessary guidance by the passenger in the rear seat. The guide function can be executed while the vehicle 10 is stopped. As a result, the guide function is prevented from being activated at a timing not intended by the driver while the vehicle 10 is traveling.

  The guidance level (amount and quality of guidance) provided by the guide unit 7 is changed according to the speaker. That is, the guide unit 7 of the present embodiment has a function of changing the information amount of the audio guide or the video guide according to the speaker. For example, if the target is the start button of the navigation device 11 and the speaker is the assistant, the name of the target and information on the basic and basic usage will be displayed, such as “This button is the start button of the car navigation system”. provide. On the other hand, if the speaker is a driver, “This button is the start button for car navigation. If you press and hold it, the car navigation system will be restarted. "

  Moreover, the guide part 7 of this embodiment has a function which changes a guidance level according to a speaker's accumulated boarding time. For example, when the accumulated boarding time of the driver who is the speaker is relatively short and the driving experience of the vehicle 10 is inexperienced, “This button is a car navigation start button. A long-press will restart the car navigation system. ” On the other hand, if the driver's accumulated boarding time is relatively long and he / she thinks he / she is proficient at operating various in-vehicle devices, he / she can provide basic and basic usage information such as “Reset by long press”. Omitted and provides advanced usage information.

[3. flowchart]
FIG. 3 is an example of a flowchart for explaining the control contents executed by the speech recognition control apparatus 1. First, voice information detected by the microphone array 21, image information taken by the indoor camera 22, and pulse information from the vehicle speed sensor 23 are inputted to the voice recognition control device 1 (step A1), and whether or not voice is inputted. Is determined (step A2). Here, if some kind of voice is input, the voice recognition unit 3 recognizes the speaker and the content of the utterance based on at least the voice (step A3). In addition, the gesture detection unit 4 detects the gesture of the speaker based on the image captured by the indoor camera 22 (step A4). Similarly, the person specifying unit 6 specifies the person who has spoken based on the input voice and image (step A5).

  In the guide unit 7, it is determined whether the speaker is a driver or an assistant (step A6), and it is determined whether the vehicle speed detected by the vehicle speed detection unit 2 is equal to or lower than a predetermined vehicle speed. (Step A7). In addition, it is determined whether or not the utterance content recognized by the voice recognition unit 3 includes a voice command meaning “request for guide” (step A8). When all these conditions are satisfied, the guide unit 7 sets a guidance level corresponding to the seating position of the speaker and the accumulated boarding time (step A9), and voice for guiding the usage of the target specified by the gesture. A guide and a video guide are output to the display device 16 (step A10).

  On the other hand, if any of the conditions determined in steps A6, A7, and A8 is not satisfied, whether or not the utterance content recognized by the voice recognition unit 3 includes a voice command meaning “control command” is determined. Determination is made (step A11). When this condition is satisfied, the operating state of the gesture target is controlled. If this condition is not satisfied, the voice command is canceled (cancelled) without being controlled.

[4. Action, effect]
As shown in FIG. 4, while the driver points to one of the switches 17 while the vehicle is stopped, “What is this? The speech recognition control device 1 recognizes the speaker (utterance position) and the content of the speech and identifies the person who spoke. Also, from the driver's gesture, one switch with a finger is recognized as an utterance target.
"What was this?" ] Means a “guide request”, and the voice recognition control device 1 outputs a voice guide and a video guide for guiding how to use the switch to the display device 16. At this time, the guidance level is determined according to the accumulated boarding time of the driver. Thereby, information commensurate with the skill level of the driver regarding how to use the switches 17 is provided, and convenience is improved.

  (1) In this way, by combining voice input and gesture input and outputting a voice guide and video guide for guiding how to use the gesture target, the usage of the in-vehicle device and its operation buttons can be given to the occupant. I can inform you. Further, it is not necessary to actually operate the in-vehicle device, and guide information regarding functions unknown to the occupant can be easily provided. Furthermore, the occupant can easily draw out a voice guide and a video guide related to the operation method and function of the target simply by issuing a predetermined voice command while making a gesture representing the target of the utterance. Therefore, the convenience of the vehicle and the in-vehicle device can be improved.

  (2) Moreover, by using the gesture of pointing a finger at the object as a trigger for starting the guide, it is possible to easily teach the occupant how to use the in-vehicle device, and the convenience can be improved. In addition, compared to the case of using a gesture such as a line of sight or a face direction, it is easier to specify the utterance target, so that it is possible to make it difficult for the guide to malfunction or to detect a gesture erroneously. Therefore, the convenience of the vehicle and the in-vehicle device can be improved.

  (3) In the voice recognition control device 1 described above, the information amount of the guide is changed according to the speaker. For example, when the speaker is a driver, the amount of information is increased and specialized guidance is implemented. On the other hand, when the speaker is an occupant (assistant) other than the driver, the amount of information is reduced and basic guidance is performed. Thus, by increasing or decreasing the amount of information according to the speaker (speaking position), it is possible to provide a guide function that meets the passenger's needs.

  (4) In the voice recognition control device 1 described above, the information amount of the guide is changed according to the accumulated boarding time of the speaker. For example, both basic guidance and specialized guidance are implemented for a driver who has little experience in driving the vehicle 10. On the other hand, only specialized guidance is provided for a driver skilled in driving the vehicle 10. Thereby, a guide function commensurate with the skill level and knowledge amount of the speaker can be provided.

(5) The above guide function is implemented on condition that the speaker is a driver or an assistant. As a result, it is possible to prohibit the guide function from being activated by a person on the rear seat, and to prevent erroneous recognition of voice commands and erroneous detection of gestures.
(6) Moreover, said guide function is implemented on condition that a vehicle speed is below a predetermined vehicle speed. Thereby, in the state where the vehicle 10 is traveling at a medium to high speed, it is possible to limit the activation of the guide function, and it is possible to more reliably prevent erroneous recognition of voice commands and erroneous detection of gestures.

[5. Modified example]
In the above-described embodiment, the case where the gesture of pointing the finger at the object is a gesture representing the utterance target has been described in detail, but the specific type of gesture is not limited to this. For example, an occupant turned his gaze toward the subject, saying, “What ’s this? The control configuration may be such that guide information relating to the target is provided when “
In the above-described embodiment, the guide for one of the switches 17 has been described in detail. However, at the same time as providing the audio guide and the video guide, the light emitting element incorporated in the target switch may be blinked. Good. By making the object in the guide conspicuous, it is possible to clearly indicate to which occupant other than the speaker the object of the guide relates, and the convenience can be further enhanced.

  Further, in the above-described embodiment, all processes from speech recognition to control of the control target are managed in an integrated manner by the speech recognition control device 1, but some or all of the functions of the speech recognition control device 1 are controlled by the vehicle. Relocation to the outside of 10 is also conceivable. For example, the voice recognition control device 1 can be connected to a network such as the Internet, a mobile phone wireless communication network, and other digital wireless communication networks, and a part or all of the functions of the voice recognition control device 1 are mounted on a server on the network. May be. Thereby, management and update of the database 5 become easy, and speech recognition accuracy and gesture recognition accuracy can be improved.

DESCRIPTION OF SYMBOLS 1 Voice recognition control apparatus 2 Vehicle speed detection part 3 Voice recognition part 4 Gesture detection part 5 Database 6 Person specific part 7 Guide part 10 Vehicle 11 Navigation apparatus 12 Air conditioner apparatus 13 Car audio apparatus 14 Driver's seat 15 Passenger seat 16 Display apparatus 17 Switches 21 Microphone array 22 Indoor camera 23 Vehicle speed sensor

Claims (6)

In a voice recognition control system that controls an in-vehicle device using the voice of a vehicle occupant as an input signal,
A voice recognition unit for recognizing a speaker and utterance content based on at least the voice;
A gesture detection unit that detects a gesture of the speaker representing an utterance target based on an image of a passenger compartment captured by an indoor camera;
When the utterance content recognized by the voice recognition unit includes a predetermined voice command, a guide unit that outputs a voice guide or a video guide for guiding how to use the target represented by the gesture;
A voice recognition control system comprising: The voice recognition control system according to claim 1, wherein the gesture is a gesture of pointing a finger toward the object. The voice recognition control system according to claim 1, wherein the guide unit changes an information amount of the voice guide or the video guide according to the speaker. The voice recognition control system according to claim 3, wherein the guide unit changes the information amount according to the accumulated boarding time of the speaker. The audio according to any one of claims 1 to 4, wherein the guide unit outputs the audio guide or the video guide on condition that the speaker is a driver or an assistant. Recognition control system. A vehicle speed detector for detecting the vehicle speed;
The said guide part outputs the said audio | voice guide or the said image | video guide on condition that the said vehicle speed detected by the said vehicle speed detection part is below a predetermined vehicle speed, The one of Claims 1-5 characterized by the above-mentioned. The speech recognition control system according to item 1.

Images