JP6604151B2 - Speech recognition control system - Google Patents

Description

  The present invention relates to a voice recognition control system for controlling an in-vehicle device with a voice of a vehicle occupant.

  2. Description of the Related Art Conventionally, there has been proposed an information guide device that provides various information to a driver by using a voice or line of sight emitted by the driver of the vehicle as an input operation. For example, an apparatus is known that answers the name of a mountain ahead of the driver's line of sight when the driver utters “What is that mountain?” By voice (see Patent Document 1). Also, when the driver speaks “What is the recommendation for that store?”, A technology that searches the facility ahead of the driver's line of sight using the map data of the car navigation system and outputs a guidance text for the facility Is also known (see Patent Document 2). By using voice input and line-of-sight input together, it is possible to provide desired information without reducing the driver's concentration.

JP 2003-329463 A JP 2009-031065

  However, the type of information required by the occupant may differ for each occupant. For example, when the destination of the vehicle is a large commercial facility, the driver wants to know information such as the location of the parking lot, the size of the parking space, the height limit, and the parking fee. There is. On the other hand, passengers other than the driver do not need information related to the parking lot, and may want to know store information and business hours information in the facility. The existing technology has room for improvement because the degree of appropriateness of information provided to passengers is not fully evaluated.

  One of the purposes of this case is to provide a voice recognition control system that has been developed in view of the above-described problems and has improved convenience by providing appropriate information to passengers. . It should be noted that the present invention is not limited to this purpose, and is an operational effect that is derived from each configuration shown in “Mode for Carrying Out the Invention” to be described later. Can be positioned as a purpose.

  (1) The voice recognition control system disclosed here is a voice recognition control system that controls an in-vehicle device using a voice of a vehicle occupant as an input signal. The system includes a speech recognition unit that recognizes an utterance position and utterance content based on at least the voice, a gesture detection unit that detects a gesture of a person at the utterance position based on an image of a vehicle interior captured by an indoor camera, Is provided. In addition, a database for storing a plurality of information related to the facility designated by the gesture is provided. Furthermore, a control unit is provided that selects and outputs a part of the plurality of information based on the utterance content and the utterance position.

(2) It is preferable that the control unit outputs parking / stopping facility information of the facility when the utterance position is a driver's seat.
(3) It is preferable that the said control part outputs the business content information of the said facility, when the said speech position is other than the said driver's seat.
(4) It is preferable that the said control part outputs the said information according to the time when the said audio | voice was input. For example, it is preferable to output business hours information when the time is outside the business hours of the facility.

  In addition, it is preferable to provide the vehicle speed detection part which detects a vehicle speed. For example, it is preferable that the control unit selects and outputs some of the plurality of information according to the vehicle speed detected by the vehicle speed detection unit. The vehicle speed information may be a condition for “outputting” some of the plurality of information, or may be a condition for “selecting” some of the plurality of information.

  Information corresponding to the utterance position as well as the utterance content can be provided, and convenience can be improved.

1 is a top view of a vehicle to which a voice recognition control system is applied. It is a figure which shows the structure of a speech recognition control system. It is a figure which shows the relationship between the direction of a gesture, and the object of guidance. It is a table | surface which shows the relationship between the object of guidance, utterance content, utterance position, and the kind of information. It is a flowchart for demonstrating the control content of a speech recognition control system.

  A speech recognition control system as an embodiment will be described with reference to the drawings. Note that the embodiment described below is merely an example, and there is no intention to exclude various modifications and technical applications that are not explicitly described in the following embodiment. Each configuration of the present embodiment can be implemented with various modifications without departing from the spirit thereof. Further, they can be selected as necessary, or can be appropriately combined.

[1. Device configuration]
The voice recognition control system of this embodiment is applied to the vehicle 10 shown in FIG. A driver's seat 14 and a passenger seat 15 are provided in the passenger compartment of the vehicle 10, and an instrument panel (instrument panel, dashboard) is disposed on the front side of the passenger compartment. A steering device and instruments are arranged in front of the driver's seat 14 and a glove box is arranged in front of the passenger seat 15 in the portion of the instrument panel facing the passenger compartment. In addition, a multi-communication type display device 16 that provides a user interface such as a car navigation function and an audio visual function is provided at the center of the instrument panel in the vehicle width direction. The position of the display device 16 is diagonally forward left from the viewpoint of the driver sitting in the driver's seat 14 and diagonally forward right from the viewpoint of the passenger (assistant) sitting in the passenger seat 15.

  The display device 16 is an electronic control including a general-purpose video display device (display screen) having a touch panel, a speaker (acoustic device), a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. An electronic device provided with a device (computer). The display device 16 is connected to in-vehicle devices such as the navigation device 11, the air conditioner device 12, the car audio device 13, and the multimedia system, and can function as an input / output device for various in-vehicle devices. For example, route information, map information, traffic jam information, etc. provided from the navigation device 11 to the destination can be displayed on the display screen of the display device 16, and voice guidance can also be output. In addition, the display screen of the display device 16 is used for operating a program received by an in-vehicle terrestrial digital broadcast tuner, video content of an information recording medium, video taken by a rear view camera, an air conditioner device 12 or a car audio device 13. Various audiovisual information such as an interface can be reproduced and displayed.

  In addition, the vehicle 10 is equipped with a voice recognition control device 1 that controls various in-vehicle devices using an occupant's voice as an input signal. The speech recognition control device 1 is an electronic device (ECU, electronic control device) in which a processor such as a CPU or MPU (Micro Processing Unit) and a ROM, RAM, nonvolatile memory, etc. are integrated. The processor here is, for example, a processing device (processor) including a control unit (control circuit), an arithmetic unit (arithmetic circuit), a cache memory (register), and the like. The ROM, RAM, and nonvolatile memory are memory devices that store programs and working data. The contents of the control performed by the voice recognition control apparatus 1 are recorded in ROM, RAM, nonvolatile memory, and removable media as firmware and application programs. When the program is executed, the contents of the program are expanded in the memory space in the RAM and executed by the processor.

  As shown in FIG. 2, examples of the input device of the voice recognition control device 1 include a microphone array 21, an indoor camera 22, a vehicle speed sensor 23, and the like. The microphone array 21 is an audio input device in which a plurality of microphones are arranged in a predetermined arrangement, and the indoor camera 22 is a wide-angle video camera capable of photographing the entire vehicle interior. The microphone array 21 and the indoor camera 22 are built in, for example, the ceiling surface at the center in the vehicle width direction. The vehicle speed sensor 23 is a sensor that outputs a pulse signal corresponding to the rotational speed of the wheel. On the other hand, examples of the output device (control target) of the voice recognition control device 1 include a navigation device 11, an air conditioner device 12, a car audio device 13, a display device 16, and the like. The voice recognition control device 1 controls various on-vehicle devices based on the voice input from the microphone array 21, the image taken by the indoor camera 22, and the pulse information detected by the vehicle speed sensor 23.

[2. Control configuration]
The voice recognition control device 1 has a function of providing information related to a facility designated by a gesture based on a passenger's voice and gesture. The facility here means a building or a structure existing outside the vehicle 10 and includes, for example, a building, equipment, a factory, a park, a stadium, a station, etc., and is preferably a map built in the navigation device 11. Includes POI (Point Of Interest) recorded in the information. For example, when some kind of sound is input from the microphone array 21, the utterance position where the sound is uttered is recognized and the utterance content is recognized. Further, based on the image taken by the indoor camera 22, a gesture made by the person at the utterance position is detected, and a facility represented by the gesture (a facility designated by the gesture) is detected. Specific examples of gestures include a gesture that points to the facility with a finger and a gesture that looks at the facility (a gesture that turns the line of sight). When the utterance content is “request for guidance”, information about the facility is provided to the occupant. At this time, the type of information provided to the occupant is set according to the utterance position.

As elements for carrying out the control as described above, the voice recognition control device 1 is provided with a vehicle speed detection unit 2, a voice recognition unit 3, a gesture detection unit 4, a database 5, and a control unit 6. These indicate some functions of a program executed by the speech recognition control apparatus 1 and are realized by software. However, some or all of the functions may be realized by hardware (electronic control circuit), or may be realized by using software and hardware together.
The vehicle speed detection unit 2 acquires (detects and calculates) the vehicle speed based on the pulse signal output from the vehicle speed sensor 23. The vehicle speed information acquired here is transmitted to the gesture detection unit 4 and the control unit 6.

  The voice recognition unit 3 recognizes the utterance position and the utterance content based on at least the voice input from the microphone array 21. Here, for example, whether the speaker is a person (driver) seated in the driver's seat 14, a person seated in the passenger seat 15 (assistant), or other passengers (passenger in the rear seat) ) Is determined. The position of the speaker can be specified based on the size and delay of a plurality of audio signals detected by the microphone array 21. Alternatively, the utterance position can be specified by analyzing an image photographed by the indoor camera 22 and comparing the movement of the lips of the person existing in the image with the timing when the sound is detected.

  The utterance content is classified and recognized as one of three types of “request for guidance”, “command for control”, and “other”. For example, the utterance content is “What is that? ""What was that? ""Explain? ”Is determined to be“ guidance request ”. On the other hand, if the utterance content includes voice commands such as “on”, “off”, “activate”, and “stop”, it is determined that the utterance content is a “control command”. If the utterance content does not include any of the above voice commands, it is determined that the utterance content corresponds to “other”. A specific method of speech recognition is arbitrary, and a known speech recognition technique can be employed. For example, after a phoneme included in speech is analyzed based on an acoustic model, a word or phrase consisting of a series of phonemes is analyzed based on a language model, and its semantic content is recognized. The information on the utterance position and the utterance content recognized here is transmitted to the gesture detection unit 4 and the control unit 6.

  The gesture detection unit 4 detects a gesture by a person (that is, a speaker) existing at an utterance position and an utterance target designated by the gesture based on an image taken by the indoor camera 22. The target of the utterance in the “request for guidance” is a facility outside the vehicle 10. Further, the utterance target in the “control command” includes various in-vehicle devices mounted on the vehicle 10 and operation buttons thereof, indicators displayed on the instrument panel, icons displayed on the display device 16, and the like. It is.

  The target of the utterance can be detected by estimating from the utterance position recognized by the voice recognition unit 3 what is arranged ahead of the virtual line extending in the direction indicated by the gesture. When detecting the gesture of the pointing finger as a gesture, the position of the hand is estimated by image analysis, and the target is accurately detected by extending a virtual line in the direction of the finger based on the position of the speaker's hand in the passenger compartment. be able to. On the other hand, when detecting the gaze as a gesture, the position and orientation of the face are estimated by image analysis, and the target is accurately detected by extending the virtual line in the gaze direction based on the position of the speaker's face in the vehicle interior. can do. Information on the target detected here is transmitted to the control unit 6.

  As shown in FIG. 3, the gesture detection unit 4 according to the present embodiment, when the utterance content is “guidance request”, uses the position of the vehicle 10 at the time of the utterance as a reference, and moves in the direction indicated by the gesture. Identify facilities that exist in A publicly known method can be adopted as a facility identification method using a gesture. For example, the position of the vehicle 10 at the time of utterance can be specified by the navigation device 11, and the facility can be specified from map information built in the navigation device 11 (see Patent Documents 1 and 2).

  The database 5 is a storage device in which various general data relating to speech recognition are recorded and stored. Here, acoustic models and language models used in speech recognition are recorded and stored. The acoustic model and the language model are created in advance based on the voice of the standard speaker. Note that a specific acoustic model and language model can be created based on a known technique (for example, JP-A-2002-189492).

  The database 5 records and stores the relationship between the type of information provided to the occupant when the utterance content is “guidance request”, and the utterance content and utterance position. In the database 5 of the present embodiment, as shown in FIG. 4, the relationship between the facility to be uttered, the utterance content, the utterance position combination, and the type of information provided to the passenger is recorded and stored. . FIG. 4 shows that, for example, when there is a request for guidance for the facility A, if the utterance position is the driver's seat 14, parking / stopping facility information of the facility A is provided. Further, it is shown that if the utterance position is the passenger seat 15, the business content information of the facility A is provided, and if the utterance position is another seat, the general information of the facility A is provided.

Based on the utterance position and utterance content recognized by the speech recognition unit 3 and the utterance target detected by the gesture detection unit 4, the control unit 6 uses various in-vehicle devices using the relationship recorded and stored in the database 5. Is to control. The control unit 6 mainly has two functions.
The first function is a function (hands-free control function) for controlling a controlled object by voice. The control unit 6 controls the operation state of the utterance target when the utterance content recognized by the voice recognition unit 3 includes a voice command meaning “control command”. Note that the specific method of the hands-free control function is arbitrary, and a known method can be adopted.

  The second function is a function (guidance function) for outputting information that the passenger wants to know to the display device 16. When the utterance content recognized by the voice recognition unit 3 does not include a voice command meaning “control command” and includes a voice command meaning “request for guidance”, the control unit 6 Providing passengers with information about a facility. At this time, the information provided to the occupant is selected according to the utterance position from among a lot of information regarding the facility. If the utterance content does not include a voice command that means “request for guidance” or a voice command that means “control command”, the voice command is canceled (cancelled) without controlling the utterance target. )

[3. flowchart]
FIG. 5 is an example of a flowchart for explaining the control contents executed by the speech recognition control apparatus 1. First, voice information detected by the microphone array 21, image information taken by the indoor camera 22, and pulse information from the vehicle speed sensor 23 are inputted to the voice recognition control device 1 (step A1), and whether or not voice is inputted. Is determined (step A2). Here, when some kind of voice is input, the voice recognition unit 3 recognizes the utterance position and the utterance content based on at least the voice (step A3). The gesture detection unit 4 detects the gesture of the person at the utterance position based on the image taken by the indoor camera 22 (step A4), and specifies the utterance target specified by the gesture (step A5). .

  The control unit 6 determines whether or not the utterance content recognized by the voice recognition unit 3 includes a voice command meaning “control command” (step A6). If this condition is satisfied, a gesture is performed. The operating state of the designated utterance target (for example, the navigation device 11 or the air conditioner device 12) is controlled (step A7).

  If the condition of step A6 is not satisfied, it is determined whether or not the utterance content recognized by the voice recognition unit 3 includes a voice command meaning “request for guidance” (step A8). When this condition is satisfied and the utterance position is the driver's seat 14, parking facility information for the driver is selected and output, and voice guidance and video regarding the utterance target (for example, the facility A and the facility B) are output. Guidance is provided from the display device 16 (steps A9 and A10). On the other hand, when the utterance position is the passenger seat 15, the business content information for the assistant is selected and output. If the utterance position is neither the driver's seat 14 nor the passenger seat 15, general information on the facility is selected and output. If the condition in step A8 is not satisfied, the voice command is canceled without controlling the utterance target.

[4. Action, effect]
(1) In the voice recognition control device 1 described above, when providing information using both voice input and line-of-sight input, a part of a plurality of pieces of information is selectively output according to the utterance position. Thus, by selecting the information provided to the occupant according to the utterance position, not only the utterance contents but also information corresponding to the utterance position can be provided, and convenience can be improved.

  (2) For example, when a driver seated in the driver's seat 14 requests guidance about the facility A, parking / stop facility information of the facility A is provided. Thereby, the driver can confirm whether or not parking is possible before actually moving the vehicle 10 to the parking lot of the facility A. You can also check parking fees and opening hours. Therefore, it is possible to acquire information useful for the parking and stopping operation of the vehicle, and it is possible to improve convenience.

  (3) On the other hand, when the assistant seated in the passenger seat 15 requests guidance about the facility A, the business content information of the facility A is provided. As a result, the assistant can preempt information on business hours and advantageous information before actually visiting the facility A. Therefore, information useful for facility use can be acquired, and convenience can be improved.

[5. Modified example]
In the above-described embodiment, as shown in FIG. 4, the type of information is selected based on the utterance content and the utterance position. However, the type of information may be selected in consideration of the utterance time. For example, if the utterance time is within the business hours of the facility, the business content information of the facility may be provided, and if the utterance time is outside the business hours of the facility, only the business time information may be provided. Thus, by changing the type of information according to the time zone, more appropriate information can be provided for the occupant, and convenience can be further improved.

  Further, in the above-described embodiment, all processes from speech recognition to control of the control target are managed in an integrated manner by the speech recognition control device 1, but some or all of the functions of the speech recognition control device 1 are controlled by the vehicle. Relocation to the outside of 10 is also conceivable. For example, the voice recognition control device 1 can be connected to a network such as the Internet, a mobile phone wireless communication network, and other digital wireless communication networks, and a part or all of the functions of the voice recognition control device 1 are mounted on a server on the network. May be. Thereby, management and update of the database 5 become easy, and speech recognition accuracy and gesture recognition accuracy can be improved.

  In the control of the above-described embodiment, a vehicle speed condition may be added to the guide function execution condition. For example, the guidance function may be performed on the condition that the vehicle speed detected by the vehicle speed detection unit 2 is a predetermined vehicle speed or less (for example, 10 km / h or less). Thereby, in the state where the vehicle 10 is traveling at a medium to high speed, it is possible to limit the activation of the guidance function, and it is possible to more reliably prevent erroneous recognition of voice commands and erroneous detection of gestures.

  Alternatively, the type of information provided to the occupant may be selected according to the vehicle speed. For example, when the driver requests guidance to the facility A and the vehicle speed is equal to or less than a predetermined slow speed (for example, 20 km / h or less), the driver parks the vehicle 10 at the facility A. It is judged that it is going to provide parking and stopping facilities information. On the other hand, when the vehicle speed exceeds the slow vehicle speed, it is determined that there is no intention of parking and stopping, and the business content information of the facility A is provided. Thus, by selecting and changing the type of information according to the vehicle speed, it is possible to provide information corresponding to the traveling state of the vehicle 10 and improve convenience.

DESCRIPTION OF SYMBOLS 1 Voice recognition control apparatus 2 Vehicle speed detection part 3 Voice recognition part 4 Gesture detection part 5 Database 6 Control part 10 Vehicle 11 Navigation apparatus 12 Air conditioner apparatus 13 Car audio apparatus 14 Driver's seat 15 Passenger's seat 16 Display apparatus 21 Microphone array 22 Indoor camera 23 Vehicle speed sensor

Claims (4)

In a voice recognition control system that controls an in-vehicle device using the voice of a vehicle occupant as an input signal,
A voice recognition unit for recognizing the utterance position and the utterance content based on at least the voice;
A gesture detection unit that detects a gesture of a person at the utterance position based on an image of a passenger compartment captured by an indoor camera;
A database for storing a plurality of pieces of information related to the facility designated by the gesture;
A control unit that selects and outputs a part of the plurality of information based on the utterance content and the utterance position;
A voice recognition control system comprising: The voice recognition control system according to claim 1, wherein the control unit outputs parking / stopping facility information of the facility when the utterance position is a driver's seat. The voice recognition control system according to claim 1, wherein the control unit outputs business content information of the facility when the utterance position is other than the driver's seat. The said control part outputs the said information according to the time when the said audio | voice was input, The speech recognition control system of any one of Claims 1-3 characterized by the above-mentioned.

Images