JP2019079083A - Driving support apparatus - Google Patents

Abstract

To reduce the difficulty of an occupant in talking to a driving support apparatus, as one example.SOLUTION: A driving support apparatus according to the embodiment includes: a sight line detection unit that detects an orientation of a sight line of an occupant from a face image of the occupant included in a picked-up image of an imaging apparatus for picking up images of the inside of a vehicle; and a light emission control unit that changes the light emission mode of a light emitting apparatus when the sight line detection unit detects that the sight line of the occupant is directed to the light emitting apparatus installed in the vehicle.SELECTED DRAWING: Figure 4

Description

  An embodiment of the present invention relates to a driving support device.

  2. Description of the Related Art Conventionally, there is known a technology of a driving support device that recognizes an instruction by a passenger's voice. In addition, there is known a driving support device that determines the presence or absence of a passenger's speech from an image obtained by imaging the passenger and performs voice recognition only when it is determined that the user is speaking.

JP-A-11-352987

  However, conventionally, when the occupant utters a voice for an instruction, there has been a case in which the driver assistance device feels "unspeakable to talk".

  The driving assistance apparatus according to the embodiment of the present invention includes, as an example, a gaze detection unit that detects the direction of the occupant's gaze from the occupant's face image included in a captured image of an imaging device that captures the inside of the vehicle; And a light emission control unit configured to change a light emission mode of the light emitting device when it is detected that the line of sight of the occupant is facing the light emitting device installed in the vehicle. Therefore, according to the driving support device according to the embodiment, since the light emitting device is operated to respond to the line of sight of the passenger, it is possible to reduce the difficulty of the passenger talking to the driving support device, for example.

  In the driving support apparatus, as an example, the light emission control unit controls the light emitting device to emit light in the direction in which the occupant is present when the sight line detection unit detects that the line of sight of the occupant is facing the light emitting device. Therefore, according to the driving support device according to the embodiment, as one example, it is possible to more clearly indicate to the occupant that the line of sight is detected.

  The driving support apparatus further includes, as an example, a speech detection unit that detects a speech of a passenger, and the light emission control unit emits light in the direction in which the passenger who made a speech is present when the speech detection unit detects a speech of the passenger Control the light emitting device. Therefore, according to the driving support device according to the embodiment, as one example, it is possible to more clearly indicate that the user is detecting an utterance.

  In the driving support device, as an example, the light emission control unit detects that the sight line detector detects that the line of sight of one passenger is facing the light emitting device, and the sight line detector detects the line of sight of a plurality of occupants The light emitting device emits light in a different manner from the case where it is detected that the light emitting device is facing. Therefore, according to the driving support device according to the embodiment, the light emitting device operates to respond to the eyes of a plurality of occupants, thereby reducing difficulty in talking to the driving assistance device of each occupant as an example. be able to.

  In the above driving support apparatus, as an example, the light emission control unit causes the light emitting device to emit light in different modes when the vehicle on which the light emitting device is installed is under automatic driving control and when the vehicle is manually driven. . Therefore, according to the driving support device according to the embodiment, as an example, the occupant can easily check whether the vehicle is under automatic driving control.

FIG. 1 is an exemplary perspective view in which a part of a cabin of a vehicle according to a first embodiment is seen through. FIG. 2 is a view showing an example of the arrangement of the imaging device according to the first embodiment. FIG. 3 is a diagram illustrating an example of a hardware configuration of the driving support system according to the first embodiment. FIG. 4 is a block diagram illustrating an example of functions of the ECU according to the first embodiment. FIG. 5 is a diagram illustrating an example of a captured image according to the first embodiment. FIG. 6 is a view showing an example of a light emission mode of the light emitting device according to the first embodiment. FIG. 7 is a flowchart of an example of a control process of the light emitting device according to the first embodiment. FIG. 8 is a view showing an example of the light emission mode of the light emitting device according to the first modification. FIG. 9 is an exemplary perspective view in which a part of a cabin of a vehicle according to a second embodiment is seen through. FIG. 10 is a diagram illustrating an example of a captured image according to the second embodiment. FIG. 11 is a diagram illustrating an example of a light emission mode of the light emitting device according to the second embodiment. FIG. 12 is a diagram illustrating another example of the light emission mode of the light emitting device according to the second embodiment. FIG. 13 is a flowchart of an example of a control process of the light emitting device according to the second embodiment. FIG. 14 is a block diagram illustrating an example of functions of the ECU according to the third embodiment. FIG. 15 is a flowchart of an example of a control process of the light emitting device according to the third embodiment.

(Embodiment 1)
In the present embodiment, the driving assistance device controls the light emitting device mounted on the vehicle to emit light when the line of sight of the occupant is directed, and the driving assistance felt when the occupant emits a voice for an instruction Reduce the difficulty of talking to the device. Hereinafter, the example which mounted the driving assistance device of this embodiment in the vehicle 1 is given and demonstrated.

  FIG. 1 is an exemplary perspective view in which a part of a compartment 2a of a vehicle 1 according to the present embodiment is seen through. The vehicle 1 may be, for example, an internal combustion engine automobile, an electric automobile, a fuel cell automobile, a hybrid automobile or the like, or may be an automobile provided with another drive source. In addition, the vehicle 1 can be mounted with various transmissions, and can be mounted with various devices necessary for driving an internal combustion engine or a motor, such as a system or components.

  Further, as illustrated in FIG. 1, the vehicle 1 is, for example, a four-wheeled vehicle, and has two left and right front wheels 3F and two left and right two rear wheels 3R. All of these four wheels 3 can be configured to be steerable. The system, the number, the layout, and the like of devices related to the driving of the wheels 3 in the vehicle 1 can be set variously.

  As illustrated in FIG. 1, the vehicle body 2 constitutes a passenger compartment 2 a in which a passenger (not shown) rides. A steering unit 4, an acceleration operation unit 5, a braking operation unit 6, a shift operation unit 7 and the like are provided in the passenger compartment 2a in a state of facing the driver's seat 40a (driver's seat) as a passenger.

  The steering unit 4 is, for example, a steering wheel (handle) that protrudes from the dashboard 12. In addition, the acceleration operation unit 5 is, for example, an accelerator pedal positioned under the driver's foot. The braking operation unit 6 is, for example, a brake pedal positioned under the driver's foot. Further, the shift operation unit 7 is, for example, a shift lever that protrudes from the center console. The steering unit 4, the acceleration operation unit 5, the braking operation unit 6, and the transmission operation unit 7 are not limited to these.

  In addition, a light emitting device 30 is installed inside the vehicle 1. In the present embodiment, the light emitting device 30 is installed on the dashboard 12 in the passenger compartment 2a. The light emitting device 30 can change the mode of light emission. More specifically, the light emitting device 30 includes, for example, a plurality of LEDs (light emitting diodes) in a dome-shaped cover, and displays light emitting units of various shapes by lighting the LEDs. In addition, the light emitting device 30 can change the color of light. The light emitting device 30 is controlled by an ECU (electronic control unit) described later. The installation position of the light emitting device 30 in the present embodiment is an example, and the present invention is not limited to this. Further, the configuration of the light emitting device 30 is not limited to this, and a liquid crystal panel or the like may be used.

  Further, a monitor device 11 is provided in the passenger compartment 2a. The monitor device 11 is provided with a display device (shown in FIG. 3) and an audio output device (shown in FIG. 3). The audio output device is, for example, a speaker. The display device is, for example, a liquid crystal display (LCD), an organic electroluminescent display (OELD), or the like. The display device is covered with a transparent operation input unit (shown in FIG. 3) such as a touch panel, for example.

  Further, on the ceiling of the vehicle body 2, a voice input unit 24 is provided. The voice input unit 24 is, for example, a microphone, and can input a voice of a passenger in the passenger compartment 2a. The installation position of the voice input unit 24 shown in FIG. 1 is an example, and the present invention is not limited to this.

  Further, an imaging device 15 for imaging the inside of the vehicle 1 is installed on a steering wheel column (shown in FIG. 2) connecting the steering unit 4 and the dashboard 12. The imaging device 15 is, for example, a CCD (Charge Coupled Device) camera or the like.

  FIG. 2 is a view showing an example of the arrangement of the imaging device 15 according to the present embodiment. In the present embodiment, the imaging device 15 is installed on the handle column 41. The viewing angle and the attitude of the imaging device 15 are adjusted such that the face of the occupant 42a seated on the seat 40a is positioned at the center of the visual field. The imaging device 15 captures the inside of the vehicle 1 and sequentially outputs captured images, which are image data obtained by imaging, to an ECU described later. Further, the driver in the present embodiment refers to an occupant 42 a of the occupants of the vehicle 1 sitting on the seat 40 a facing the steering unit 4. When the vehicle 1 is under automatic driving control, the driver may not necessarily control the driving of the vehicle 1.

  FIG. 3 is a diagram showing an example of a hardware configuration of the driving support system 10 according to the present embodiment. As shown in FIG. 3, in the driving support system 10 mounted in the vehicle 1, the brake system 18, the steering angle sensor 19 (angle sensor), the accelerator sensor 20, in addition to the ECU 14, the monitor device 11, the light emitting device 30, etc. The shift sensor 21, the wheel speed sensor 22, the steering system 13, the voice input unit 24 and the like are electrically connected via an in-vehicle network 23 as a telecommunication line. The in-vehicle network 23 is configured as, for example, a CAN (controller area network). The voice input unit 24 may be directly connected to the ECU 14 without passing through the in-vehicle network 23.

  The ECU 14 controls the light emission mode of the light emitting device 30. Further, the ECU 14 acquires a captured image from the imaging device 15. Further, the ECU 14 controls the steering system 13, the brake system 18 and the like by transmitting control signals through the in-vehicle network 23. In addition, the ECU 14 detects the torque sensor 13b, the brake sensor 18b, the steering angle sensor 19, the accelerator sensor 20, the shift sensor 21, the wheel speed sensor 22, the voice input unit 24, etc. An instruction signal (a control signal, an operation signal, an input signal, data) of the input unit 8c or the like is received. The ECU 14 is an example of the driving support device in the present embodiment.

  The ECU 14 has, for example, a CPU 14a (central processing unit), a ROM 14b (read only memory), a RAM 14c (random access memory), a display control unit 14d, an audio control unit 14e, an SSD 14f (solid state drive, flash memory) and the like. ing.

  The CPU 14a reads a program installed and stored in a non-volatile storage device such as the ROM 14b, and executes arithmetic processing according to the program. In addition, the display control unit 14d mainly performs image processing using image data obtained by the imaging device 15, composition of image data displayed by the display device 8a, and the like among the arithmetic processing in the ECU 14. Further, the voice control unit 14 e mainly performs processing of voice data output from the voice output device 8 b among the arithmetic processing in the ECU 14.

  The steering system 13 also steers at least two wheels 3. The steering system 13 has an actuator 13a and a torque sensor 13b.

  The brake system 18 also has an actuator 18a and a brake sensor 18b. The brake system 18 applies a braking force to the wheel 3 and thus to the vehicle 1 via the actuator 18a.

  The configuration, arrangement, electrical connection form and the like of the various sensors and actuators described above are merely examples, and can be set (changed) in various ways.

  FIG. 4 is a block diagram showing an example of the function of the ECU 14 according to the present embodiment. As shown in FIG. 4, the ECU 14 includes a storage unit 140, an acquisition unit 141, a gaze detection unit 142, an utterance detection unit 143, a light emission control unit 144, a voice recognition mode switching unit 145, and a voice recognition unit 146. And

  The CPU 14a stores the configuration of the acquisition unit 141, the sight line detection unit 142, the speech detection unit 143, the light emission control unit 144, the voice recognition mode switching unit 145, and the voice recognition unit 146 in the ROM 14b. It is realized by executing a program. Note that these configurations may be configured to be realized by a hardware circuit.

  The storage unit 140 stores the setting value of the speech recognition mode. The setting value of the voice recognition mode is a setting value of the sensitivity of the voice recognition function that receives input of a command by voice. The setting value of the speech recognition mode is, for example, three stages of “low”, “medium”, and “high”. The voice recognition unit 146 described later executes voice recognition of the occupant according to the setting value of the voice recognition mode stored in the storage unit 140. The setting values of the speech recognition mode in the present embodiment are an example, and the present invention is not limited to this. In addition, the storage unit 140 is configured of, for example, a storage device such as an SSD 14 f.

  The acquisition unit 141 acquires a captured image from the imaging device 15.

  The sight line detection unit 142 detects the direction of the sight line of the occupant 42 a from the face image of the occupant 42 a included in the captured image acquired by the acquisition unit 141.

  More specifically, the line-of-sight detection unit 142 detects, for example, the direction of the line of sight of the occupant 42a by specifying the direction of the face of the occupant 42a, the position of the eyes, the direction in which the eyes are facing, etc. . Further, the line-of-sight detection unit 142 determines whether the detected line of sight of the occupant 42 a faces the light emitting device 30. It is assumed that in what direction the line of sight of the occupant 42 a in the captured image is oriented, it is registered in advance that it is determined that the light emitting device 30 is facing. For example, the imaging range of the imaging device 15 and the positional relationship of the light emitting device 30 may be registered in the storage unit 140 in advance. Further, the line-of-sight detection unit 142 determines whether or not the detected line of sight of the occupant 42 a faces the light-emitting device 30 stored in advance. Note that the method of detecting the direction of the line of sight by the line-of-sight detection unit 142 is not limited to this, and another known method may be adopted.

  FIG. 5 is a view showing an example of a captured image 60 according to the present embodiment. The gaze detection unit 142 detects the gaze D <b> 1 of the occupant 42 a from the face image of the occupant 42 a included in the captured image 60. In the present embodiment, when the line of sight D1 of the occupant 42a faces diagonally lower left of the captured image 60, the line of sight detection unit 142 determines that the line of sight D1 of the occupant 42a faces the light emitting device 30. The positional relationship between the occupant 42a and the light emitting device 30 is an example, and the present invention is not limited to this.

  Returning to FIG. 4, the speech detection unit 143 detects the speech of the occupant 42a. More specifically, the utterance detection unit 143 according to the present embodiment detects the movement of the mouth of the occupant 42a from the face image of the occupant 42a included in the captured image 60 acquired by the acquisition unit 141, whereby the occupant 42a speaks Detect what you did. Generally, noise such as radio sound is present in the car, but since the speech detection unit 143 detects movement of the mouth of the occupant 42a from the captured image 60, false detection of noise or the like as speech of the occupant 42a is It can be suppressed. For this reason, the speech detection unit 143 of the present embodiment can detect that the occupant 42a has made a speech with high accuracy.

  Further, in the present embodiment, since the light emitting device 30 is installed in the car, the face of the occupant 42 a can be easily directed to the light emitting device 30 when speaking. Therefore, the face of the occupant 42a can be easily imaged by the imaging device 15, and the utterance detection unit 143 can detect the utterance of the occupant 42a with higher accuracy.

  The light emission control unit 144 changes the light emission mode of the light emitting device 30 based on the line of sight of the occupant. More specifically, the light emission control unit 144 of the present embodiment changes the light emission mode of the light emitting device 30 when the sight line detection unit 142 detects that the line of sight of the occupant 42 a faces the light emitting device 30. Further, the light emission control unit 144 controls the light emitting device 30 to emit light in the direction in which the occupant 42a is present when the speech detection unit 143 detects an utterance of the occupant 42a.

  FIG. 6 is a view showing an example of the light emission mode of the light emitting device 30 according to the present embodiment. In the example shown in FIG. 6, the light emitting device 30 causes the circular light emitting portion 301a to emit light, and the other places are turned off. In the present embodiment, it is assumed that the light emitting device 30 is in the extinguished state in a state in which the occupant 42a does not detect that the line of sight D1 is directed to the light emitting device 30, and the utterance of the occupant 42a is not detected. .

  The light emission control unit 144 displays a circular light emitting unit 301 a that emits blue light in the direction in which the occupant 42 a is present, when the sight line detection unit 142 detects that the sight line D 1 of the occupant 42 a faces the light emitting device 30. Control the light emitting device 30. The direction in which the occupant 42a is present is assumed to be registered in advance. For example, the positional relationship between the position where the light emitting device 30 is installed and the seat 40 a on which the occupant 42 a is seated may be registered in the storage unit 140 in advance.

  The light emission control unit 144 further controls the light emitting device 30 to display a circular light emission unit 301 a that emits orange light in the direction in which the occupant 42 a is present, when the utterance of the occupant 42 a is detected. The shape and color of the light emitting unit 301 a are an example, and the present invention is not limited to this.

  The light emission control unit 144 changes the light emission mode of the light emitting device 30 based on the line of sight D1 of the occupant 42a and the speech so that the light emitting device 30 responds to the line of sight D1 of the occupant 42a and the speech To work. By operating the light emitting device 30 in this manner, it is possible to reduce the difficulty in speaking when the occupant 42a inputs an instruction by voice.

  Referring back to FIG. 4, the voice recognition mode switching unit 145 switches the setting value of the voice recognition mode based on the line of sight D1 or the utterance of the occupant 42a. Further, in the present embodiment, when the line of sight D1 of the occupant 42a is not directed to the light emitting device 30, and the occupant 42a is not speaking, the setting value of the voice recognition mode stored in the storage unit 140 is “low It shall be

  More specifically, the voice recognition mode switching unit 145 sets the voice recognition mode stored in the storage unit 140 when the sight line detection unit 142 detects that the sight line D1 of the occupant 42a is facing the light emitting device 30. Change the value to "Medium". In the present embodiment, when the occupant 42a directs the sight line D1 to the light emitting device 30, it is assumed that the occupant 42a intends to input an instruction by voice. For example, the occupant 42a may be previously instructed to look at the light emitting device 30 when inputting an instruction by voice. In addition, when the speech detection unit 143 detects the speech of the occupant 42a, the speech recognition mode switching unit 145 changes the setting value of the speech recognition mode stored in the storage unit 140 to "high".

  The voice recognition unit 146 performs voice recognition processing on the voice input to the voice input unit 24 and specifies the content of the command. More specifically, the speech recognition unit 146 increases the speech recognition sensitivity as the setting value of the speech recognition mode stored in the storage unit 140 is higher. For example, the higher the setting value of the speech recognition mode, the speech recognition unit 146 processes the word extraction with high accuracy, assuming that the input sound is not noise but is more likely to be an instruction by the voice of the occupant 42a. I do. The voice recognition unit 146 can identify a section that is highly likely to contain the command of the occupant 42 a from the input voice according to the setting value of the voice recognition mode, so voice recognition is performed with higher accuracy. be able to. In addition, it is assumed that the speech recognition unit 146 always executes speech recognition processing while the ECU 14 is activated. The speech recognition method is not limited to this, and other known methods can be adopted.

  Next, control processing of the light emitting device 30 in the ECU 14 of the present embodiment configured as described above will be described. FIG. 7 is a flow chart showing an example of the procedure of control processing of the light emitting device 30 according to the present embodiment. The processing of this flowchart is repeatedly executed while the ECU 14 is activated.

  The acquisition unit 141 acquires the captured image 60 from the imaging device 15 (S1). Next, the sight line detection unit 142 detects the direction of the sight line D1 of the occupant 42a from the face image of the occupant 42a included in the captured image 60 acquired by the acquisition unit 141. When the sight line detection unit 142 detects that the sight line D1 of the occupant 42a is facing the light emitting device 30 (S2 "Yes"), the light emission control unit 144 emits light in blue in the direction in which the occupant 42a is present. Is controlled to display the light emitting device 30 (S3).

  The voice recognition mode switching unit 145 also sets the voice recognition mode setting value stored in the storage unit 140 when the sight line detection unit 142 detects that the sight line D1 of the occupant 42 a is facing the light emitting device 30. Change to medium (S4).

  Then, the speech detection unit 143 detects whether or not the occupant 42a has made a speech by detecting the movement of the mouth of the occupant 42a from the face image of the occupant 42a included in the captured image 60 acquired by the acquisition unit 141. Do. When the speech detection unit 143 detects the speech of the occupant 42a (S5 "Yes"), the light emission control unit 144 causes the light emitting device 30 to display the light emission unit 301a that emits orange light in the direction in which the occupant 42a is present. Control (S6).

  Further, when the speech detection unit 143 detects the speech of the occupant 42a, the speech recognition mode switching unit 145 changes the setting value of the speech recognition mode stored in the storage unit 140 to "high" (S7).

  When the sight line detection unit 142 does not detect that the sight line of the occupant 42 a is facing the light emitting device 30 from the captured image 60 (S2 “No”), the processes of S3 and S4 are not performed, and the process proceeds to the process of S5. . When the speech detection unit 143 does not detect the speech of the occupant 42a from the captured image 60 (S5 “No”), the processes of S6 and S7 are not performed, and the process of this flowchart ends.

  As described above, according to the ECU 14 of the present embodiment, when the light emission control unit 144 detects that the sight line detection unit 142 is pointing the sight line D1 of the occupant 42 a, the light emission control unit 144 In order to change the mode, the light emitting device 30 operates to respond to the line of sight D1 of the occupant 42a. Therefore, according to the ECU 14 of the present embodiment, it is possible to reduce the difficulty in talking to the ECU 14 of the occupant 42a.

  Further, according to the ECU 14 of the present embodiment, the light emission control unit 144 emits light in the direction in which the occupant 42a is present when the sight line detection unit 142 detects that the sight line D1 of the occupant 42a faces the light emitting device 30. In order to control the light emitting device 30, it is possible to show the occupant 42a more clearly that the line of sight D1 is detected.

  The light emission control unit 144 controls the light emitting device 30 so as to emit light in the direction in which the occupant 42a is present, when the utterance detection unit 143 detects the utterance of the occupant 42a. Therefore, according to the ECU 14 of the present embodiment, it is possible to clearly indicate to the occupant 42 a that the utterance is detected.

  In addition, by causing the light emitting device 30 to emit light in the direction in which the occupant 42 a is present, it is possible to produce an effect that makes the occupant 42 a feel as if the light emitting device 30 is looking at or recognizing the occupant 42 a. it can. Further, the light emitting device 30 responds to the line of sight D1 or the like of the occupant 42a and emits light in the direction in which the occupant 42a is present, whereby it is possible to express an animal-like response and soften the inorganicity.

(Modification 1)
In the first embodiment, one light emitting unit 301 a is displayed on the light emitting device 30, but the light emission mode of the light emitting device 30 is not limited to this. FIG. 8 is a view showing an example of the light emission mode of the light emitting device 30 according to the present modification. As shown in FIG. 8, the light emission control unit 144 detects the direction in which the occupant 42 a is present when it is detected that the occupant 42 a has turned the line of sight D 1 to the light emitting device 30 or when an utterance of the occupant 42 a is detected. The light emitting device 30 may be controlled to display the two circular light emitting portions 301d and the light emitting portions 301e that emit blue light. When the configuration of this modification is adopted, the light emitting portions 301d and 301e are shaped like the eyes of the light emitting device 30, so it is more strongly felt by the occupant 42a that they respond to the line of sight D1 or the voice of the occupant 42a. It becomes easy to do.

  Further, the light emission control unit 144 may control the light emission unit to move circumferentially along the edge of the light emitting device 30 without being limited to the light emission mode shown in FIG. You may control to blink. Further, in the first embodiment, it is assumed that the light emitting device 30 does not emit light in a state in which the occupant 42a does not detect that the line of sight D1 is directed to the light emitting device 30, and the utterance of the occupant 42a is not detected. However, the light emitting device 30 may emit light also when the sight line D1 or an utterance is not detected.

(Modification 2)
In the first embodiment, the light emission control unit 144 changes the light emission mode of the light emitting device 30 based on the line of sight D1 and the utterance of the occupant 42a, but may change the light emission mode based on only the line of sight D1. . When the configuration of the present modification is adopted, the processes of S1 to S4 of the flowchart shown in FIG. 7 are executed, and the processes of S5 to S7 are not executed.

(Modification 3)
In the first embodiment, when the speech of the occupant 42a is detected, the setting value of the voice recognition mode is set to "high" regardless of whether the line of sight D1 of the occupant 42a is directed to the light emitting device 30 or not. However, when the line of sight D1 of the occupant 42a is directed to the light emitting device 30, the setting value of the voice recognition mode may be set higher. More specifically, in the voice recognition mode switching unit 145, the line of sight D1 of the occupant 42a emits light than when the line of sight D1 of the occupant 42a is not directed to the light emitting device 30 and the speech of the occupant 42a is detected. The setting value of the voice recognition mode may be set to a high value when directed to the device 30 and when an utterance of the occupant 42a is detected. When the configuration of the present modification is adopted, it is possible to further increase the sensitivity of voice recognition when the occupant 42a speaks with the intention of inputting a command by voice.

(Modification 4)
In the first embodiment, the speech detection unit 143 detects the speech of the occupant 42a from the captured image 60, but may detect the speech of the occupant 42a by voice. For example, the speech detection unit 143 may estimate the direction of the sound source from the voice input from the voice input unit 24, and may determine that the passenger 42a is speaking if the voice is from the direction in which the occupant 42a is present.

(Modification 5)
In the ECU 14 of the first embodiment, the setting value of the voice recognition mode is changed based on the line of sight D1 or the speech of the occupant 42a, but the voice recognition function "on" or "on" based on the line of sight D1 or the speech of the occupant 42a. A configuration to switch off may be adopted. For example, when the occupant 42a does not detect that the line of sight D1 is directed to the light emitting device 30, and the utterance of the occupant 42a is not detected, the voice recognition function is off, that is, not accepting voice input. It is good. Then, when it is detected that the occupant 42a has turned the line of sight D1 to the light emitting device 30, or when an utterance of the occupant 42a is detected, the voice recognition function is turned on, that is, the voice input is accepted. As well.

Second Embodiment
In the first embodiment, the light emission mode of the light emitting device 30 is changed according to the line of sight D1 or the utterance of the occupant 42a who is the driver. In the present embodiment, the ECU 14 changes the light emission mode of the light emitting device 30 not only by the driver but also by the line of sight or speech of another occupant.

  FIG. 9 is an exemplary perspective view showing a state in which a part of the cabin 2a of the vehicle 1 according to the present embodiment is seen through. In the vehicle 1 of the present embodiment, an imaging device 1015 is provided instead of the imaging device 15 of the first embodiment. The imaging device 1015 is a wide-angle camera, and is set such that the face of each occupant sitting in the driver's seat, the passenger's seat, and the rear seat in the vehicle 1 is included in the imaging range.

  FIG. 10 is a view showing an example of a captured image 1060 according to the present embodiment. As shown in FIG. 10, the captured image 1060 includes face images of the occupants 42a to 42d seated on the seats 40a to 40c. The seats 40a to 40c are simply referred to as the seats 40 unless otherwise specified. Hereinafter, the occupants 42 a to 42 d are simply referred to as the occupant 42 unless otherwise specified. The number of occupants 42 shown in FIG. 10 is an example, and is not limited to this.

  In the present embodiment, as shown in FIG. 9, the imaging device 1015 is provided above the monitor device 11 of the vehicle 1. The installation position of the imaging device 1015 shown in FIG. 9 is an example, and the present invention is not limited to this. For example, the imaging device 1015 may be installed on the dashboard 12.

  Further, the ECU 14 of the present embodiment acquires a captured image from the imaging device 1015. The other configuration of the vehicle 1 is the same as the configuration of the first embodiment described in FIGS. 1 and 2. The vehicle 1 may adopt a configuration including both the imaging device 15 of the first embodiment and the imaging device 1015.

  Further, as in the first embodiment, the ECU 14 according to the present embodiment includes the storage unit 140, the acquisition unit 141, the gaze detection unit 142, the speech detection unit 143, the light emission control unit 144, and the voice recognition mode switching unit 145. And a speech recognition unit 146.

  The storage unit 140, the acquisition unit 141, the voice recognition mode switching unit 145, and the voice recognition unit 146 of the present embodiment have the same functions as in the first embodiment.

  The gaze detection unit 142 of the present embodiment detects the direction of the gaze of the occupant 42 from the face image of the occupant 42 included in the captured image 1060 acquired by the acquisition unit 141. In the present embodiment, when the captured image 1060 includes face images of the plurality of occupants 42, the gaze detection unit 142 detects the direction of the line of sight of each of the plurality of occupants 42. The sight line detection unit 142 determines whether the sight lines of the plurality of detected occupants 42 face the light emitting device 30.

  In the example of the captured image 1060 illustrated in FIG. 10, the sight line detection unit 142 determines that the sight line D1 of the occupant 42a and the sight line D2 of the occupant 42b face the light emitting device 30. Hereinafter, when the line of sight D1 and the line of sight D2 are not particularly distinguished from each other, it is referred to as a line of sight D.

  The speech detection unit 143 of the present embodiment detects that the occupant 42 has made a speech by detecting the movement of the mouth of the occupant 42 from the face image of the occupant 42 included in the captured image 1060 acquired by the acquisition unit 141 Do. In the present embodiment, when face images of a plurality of occupants 42 are included in the captured image 1060, the utterance detection unit 143 detects the respective utterances of the plurality of occupants 42.

  In the light emission control unit 144 of the present embodiment, in addition to the functions of the first embodiment, the case where the sight line detection unit 142 detects that the sight line D of one occupant 42 is directed to the light emitting device 30; In the case where it is detected that the line of sight D of the plurality of occupants 42 faces the light emitting device 30, the light emitting device 30 is caused to emit light in a different mode.

  More specifically, the light emission control unit 144 emits light so that when the sight line detection unit 142 detects that the sight line D of one occupant 42 is directed to the light emitting device 30, the light emission control unit 144 emits light in the direction in which the occupant 42 is present. Control the device 30; In addition, when the light emission control unit 144 detects that the line of sight D of the two occupants 42 faces the light emitting device 30, the light emission control unit 144 emits light in the respective directions in which the two occupants 42 are present. Control. In addition, the light emission control unit 144 controls the light emitting device 30 to display an annular light emitting unit when the sight line detection unit 142 detects that the sight lines D of three or more persons are facing the light emitting device 30.

  In the light emission control unit 144 of this embodiment, different modes are used between the case where the speech detection unit 143 detects the speech of one occupant 42 and the case where the speech detection unit 143 detects speech of a plurality of occupants 42. Causes the light emitting device 30 to emit light. More specifically, the light emission control unit 144 controls the light emitting device 30 to emit light in the direction in which the occupant 42 is present, when the speech detection unit 143 detects an utterance of one occupant 42. Further, the light emission control unit 144 controls the light emitting device 30 to emit light in the respective directions in which the two occupants 42 are present, when the speech detection unit 143 detects the utterance of the two occupants 42. In addition, the light emission control unit 144 controls the light emitting device 30 to display the annular light emitting unit when the speech detection unit 143 detects the speech of three or more occupants 42.

  FIG. 11 is a view showing an example of the light emission mode of the light emitting device 30 according to the present embodiment. As in the first embodiment, the light emission control unit 144 according to the present embodiment detects blue when the sight line detection unit 142 detects that the sight line D1 of the occupant 42a is directed to the light emitting device 30. The light emitting device 30 is controlled to display a circular light emitting unit 301 a that emits light. The light emission control unit 144 of the present embodiment further has a circular shape that emits blue light in the direction in which the occupant 42b is present, when the gaze detection unit 142 detects that the gaze D2 of the occupant 42b is directed to the light emitting device 30. The light emitting device 30 is controlled to display the light emitting unit 301 b.

  Moreover, FIG. 12 is a figure which shows another example of the light emission aspect of the light-emitting device 30 concerning this embodiment. In the example illustrated in FIG. 12, it is assumed that the sight line detection unit 142 detects that the sight lines D1 to D4 of the occupants 42a to 42d are facing the light emitting device 30. In such a case, the light emission control unit 144 of the present embodiment controls the light emitting device 30 to display an annular light emitting unit 301 c that emits blue light along the edge of the light emitting device 30. In addition, when the speech detection unit 143 detects the speech of the occupant 42, the light emission control unit 144 displays the orange light emission units 301a to 301c according to the number of the detected speakers. Control.

  The light emission control unit 144 changes the light emission mode of the light emitting device 30 based on the line of sight D and the utterance of each occupant 42, whereby the light emitting device 30 responds to the line of sight D and the utterance of each occupant 42, Each operates to respond. In addition, the shape and color of light emission part 301a-301c shown to FIG. 11 and FIG. 12 are an example, It is not limited to this. Hereinafter, the light emitting units 301 a to 301 c will be referred to as the light emitting unit 301 when not particularly distinguished.

  Next, control processing of the light emitting device 30 in the ECU 14 of the present embodiment configured as described above will be described. FIG. 13 is a flowchart showing an example of the procedure of control processing of the light emitting device 30 according to the present embodiment.

  The acquisition unit 141 acquires a captured image 1060 from the imaging device 1015 (S21).

  Next, the sight line detection unit 142 detects the direction of the sight line D of each occupant 42 from the face image of the occupant 42 included in the captured image 1060 acquired by the acquisition unit 141. When the sight line detection unit 142 detects that the sight line D of any one of the occupants 42 is facing the light emitting device 30 (S22 “Yes”), the number of sight lines facing the light emitting device 30 is one If there is (S23 “Yes”), the light emission control unit 144 displays the light emitting device 30 such that one light emitting unit 301 emitting blue light is displayed in the direction in which the occupant 42 who directs the sight line D to the light emitting device 30 is present. It controls (S24).

  Further, when the number of sight lines facing the light emitting device 30 detected by the sight line detection unit 142 is two (S23 “No”, S25 “Yes”), the light emission control unit 144 selects the sight line D from the light emitting device 30. The light emitting device 30 is controlled to display the light emitting portions 301 emitting blue light one by one in the two directions in which the two occupants 42 facing are in (S26).

  In addition, when the number of sight lines facing the light emitting device 30 detected by the sight detection unit 142 is three or more (S25 “No”), the light emission control unit 144 displays the blue annular light emitting unit 301c. The light emitting device 30 is controlled to do this (S27).

  Further, the voice recognition mode switching unit 145 sets the voice recognition mode stored in the storage unit 140 when the sight line detection unit 142 detects that the sight line D of any one of the occupants 42 faces the light emitting device 30. The value is changed to "medium" (S28).

  When the sight line detection unit 142 does not detect that the sight line D of any of the occupants 42 faces the light emitting device 30 (S22 “No”), the processes of S23 to S28 are not performed, and the process of S29 is performed. move on.

  Next, the utterance detection unit 143 detects whether or not the occupant 42 speaks by detecting the movement of the occupant 42's mouth from the face image of the occupant 42 included in the captured image 1060 acquired by the acquisition unit 141. to decide. When the speech detection unit 143 detects the speech of any of the occupants 42 (S29 "Yes"), and when the number of the occupants 42 whose speech is detected is one (S30 "Yes"), the light emission control unit 144 The light emitting device 30 is controlled to display the light emitting unit 301 that emits orange light in the direction in which the occupant 42 whose utterance is detected is present (S31).

  In addition, when the number of occupants 42 whose utterances are detected is two (S30 “No”, S32 “Yes”), the light emission control unit 144 is operated in two directions in which the two occupants 42 whose utterances are detected are present. The light emitting device 30 is controlled to display the light emitting units 301 that emit orange light (S33).

  In addition, when the number of occupants 42 whose utterance is detected is three or more (S32 “No”), the light emission control unit 144 controls the light emitting device 30 to display the orange annular light emitting unit 301c ( S34).

  Further, the voice recognition mode switching unit 145 changes the setting value of the voice recognition mode stored in the storage unit 140 to "high" when the voice detection unit 143 detects a voice of any one of the occupants 42 (S35). ).

  When the speech detection unit 143 does not detect any speech of the occupant 42 (S29 “No”), the process of S30 to S35 is not performed, and the process of this flowchart ends.

  As described above, according to the ECU 14 of the present embodiment, the light emission control unit 144 causes the light emission device 30 to emit light in a mode that differs depending on the number of sight lines D of the occupant 42 facing the light emission device 30. It operates to respond to the gaze D of the plurality of occupants 42 respectively. For this reason, according to the ECU 14 of the present embodiment, it is possible to reduce the difficulty in speaking by each occupant 42.

(Modification 6)
In the ECU 14 of the second embodiment, the occupant 42 of the vehicle 1 is imaged by the imaging device 1015 installed in the monitor device 11 at the center of the dashboard 12. However, the installation position of the imaging device 1015 is not limited thereto. . For example, one imaging device may be installed at each of the positions facing the occupants 42a to 42d seated in the respective seats 40a to 40c.

(Modification 7)
In the second embodiment, the light emitting device 30 has an annular light emitting portion in the case where there are three or more occupants 42 whose line of sight D is directed to the light emitting device 30, or in the case where there are three or more occupants 42 speaking. Although 301c is displayed, the light emission mode is not limited to this. For example, regardless of the number of occupants 42, circular light emitting units 301 may be displayed in the direction in which each occupant 42 is present.

(Embodiment 3)
In the present embodiment, the ECU 14 changes the light emission mode of the light emitting device 30 according to whether the vehicle 1 is under automatic driving control or manual driving.

  FIG. 14 is a block diagram showing an example of the function of the ECU 14 according to the present embodiment. As shown in FIG. 14, the ECU 14 includes a storage unit 1140, an acquisition unit 141, a gaze detection unit 142, an utterance detection unit 143, a light emission control unit 1144, a voice recognition mode switching unit 145, and a voice recognition unit 146. And an operation control unit 147.

  The acquisition unit 141, the gaze detection unit 142, the speech detection unit 143, the speech recognition mode switching unit 145, and the speech recognition unit 146 according to the present embodiment have the same functions as in the first embodiment.

  The storage unit 1140 of the present embodiment stores the setting value of the voice recognition mode and the automatic driving flag. The setting values of the speech recognition mode are the same as in the first embodiment.

  The automatic driving flag is a flag indicating whether the vehicle 1 is in automatic driving control or in manual driving. For example, when the vehicle 1 is under automatic driving control, the automatic driving flag is "1", and when the vehicle 1 is manually driving, the automatic driving flag is "0". In the present embodiment, the automatic driving control means that steering, acceleration, and braking of the vehicle 1 are controlled by a driving control unit 147 described later. In addition, the method of determining whether the vehicle 1 is under automatic driving control is not limited to the automatic driving flag.

  The operation control unit 147 executes automatic operation control of the vehicle 1. More specifically, the operation control unit 147 drives the vehicle 1 by controlling the steering system 13, the brake system 18, the acceleration operation unit 5, and the like. The operation control unit 147 starts the automatic operation control, for example, when the occupant 42a performs an operation to start the automatic operation control from the operation input unit 8c, and the automatic operation control when the operation to end the automatic operation control is performed. Finish. In addition, when an instruction of a destination, an instruction of stopping, or the like is input by voice during automatic driving control, the operation control unit 147 controls the vehicle 1 according to the instruction.

  Further, the operation control unit 147 sets the automatic operation flag of the storage unit 1140 to “1” when starting the automatic operation control. Further, the operation control unit 147 sets the automatic operation flag of the storage unit 1140 to “0” when ending the automatic operation control. The type of automatic driving control and the conditions for starting or ending automatic driving control are not limited to these.

  Further, the operation control unit 147 is realized by the CPU 14a executing a program stored in the ROM 14b, as in the other functions. The operation control unit 147 may be configured to be realized by a hardware circuit.

  The light emission control unit 1144 according to the present embodiment causes the light emitting device 30 to emit light in different modes in the case where the automatic driving control of the vehicle 1 is performed and in the case where the vehicle 1 is manually operated. More specifically, the light emission control unit 1144 causes the light emitting device 30 to emit light in different colors depending on whether the automatic operation flag stored in the storage unit 1140 is “1” or “0”. For example, when the automatic driving flag stored in the storage unit 1140 is “1”, the light emission control unit 1144 causes the entire light emitting device 30 to emit light in yellow. In addition, when the automatic driving flag stored in the storage unit 1140 is “0”, the light emission control unit 1144 causes the entire light emitting device 30 to emit green light. When the light emission control unit 1144 changes the light emission mode of the light emitting device 30, the occupant 42 can easily check whether the vehicle 1 is under automatic driving control.

  The method of determining whether the vehicle 1 is in automatic driving control in the present embodiment is an example, and the light emission control unit 1144 may determine the control state of the vehicle 1 by another method. The light emission mode of the light emitting device 30 in the present embodiment is an example, and the present invention is not limited to this.

  Next, control processing of the light emitting device 30 in the ECU 14 of the present embodiment configured as described above will be described. FIG. 15 is a flow chart showing an example of the procedure of control processing of the light emitting device 30 according to the present embodiment.

  The light emission control unit 1144 according to the present embodiment refers to the automatic driving flag stored in the storage unit 1140, and determines whether the vehicle 1 is under automatic driving control (S41).

  If the automatic driving flag is "1", that is, if the vehicle 1 is under automatic driving control (S41 "Yes"), the light emission control unit 1144 causes the light emitting device 30 to emit light yellow (S42).

  When the automatic driving flag is “0”, that is, when the vehicle 1 is not under automatic driving control (S41 “No”), the light emission control unit 1144 causes the light emitting device 30 to emit green light (S43).

  The subsequent processing from acquisition of the captured image 60 in S1 to processing for setting the voice recognition mode in S7 to "high" are the same as in the first embodiment.

  As described above, in the ECU 14 of the present embodiment, since the light emitting device 30 emits light in different modes when the vehicle 1 is under automatic driving control and when the vehicle 1 is under manual driving, the vehicle 1 is automatic. The occupant 42 can easily confirm whether the operation control is in progress.

  For this reason, according to the ECU 14 of the present embodiment, it is possible to reduce that the occupant 42 erroneously inputs a voice command for automatic driving control when the vehicle 1 is in a manual operation. Further, according to the ECU 14 of the present embodiment, when the vehicle 1 is under automatic driving control, it is possible to reduce that the passenger 42 erroneously inputs a voice command for manual driving.

  As mentioned above, although the embodiment of the present invention was illustrated, the above-mentioned embodiment and modification are an example to the last, and limiting the scope of the invention is not intended. The above embodiment and modifications can be implemented in other various forms, and various omissions, replacements, combinations, and changes can be made without departing from the scope of the invention. In addition, the configurations and shapes of the embodiments and the modifications may be partially replaced and implemented.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 10 ... Driving assistance system, 14 ... ECU, 15 ... Imaging device, 24 ... Voice input part, 30 ... Light-emitting device, 40, 40a-40c ... Seat, 42, 42a-42d ... Occupant, 60, 1060 ... Captured image, 140, 1140 ... storage unit, 141 ... acquisition unit, 142 ... gaze detection unit, 143 ... speech detection unit, 144, 1144 ... emission control unit, 145 ... speech recognition mode switching unit, 146 ... speech recognition unit, 147 ... driving control unit, 301, 301a to 301e ... light emitting unit, D, D1 to D4 ... line of sight.

Claims (5)

A line-of-sight detection unit that detects the direction of the line of sight of the occupant from a face image of the occupant included in a captured image of an imaging device that captures the inside of the vehicle;
A light emission control unit that changes a light emission mode of the light emitting device when the sight line detection unit detects that the line of sight of the occupant is facing a light emitting device installed in the vehicle;
Driving support device equipped with. The light emission control unit controls the light emitting device to emit light in a direction in which the occupant is present, when the sight line detection unit detects that the line of sight of the occupant is facing the light emitting device.
The driving support device according to claim 1. And a speech detection unit that detects the speech of the occupant.
The light emission control unit controls the light emitting device to emit light in a direction in which the occupant who has made an utterance is emitted when the utterance detection unit detects an utterance of the occupant.
The driving support device according to claim 1. The light emission control unit is configured such that the line of sight detection unit detects that the line of sight of one of the occupants faces the light emitting device, and the line of sight detection unit directs the lines of sight of the plurality of occupants to the light emitting device The light emitting device is made to emit light in a different mode from the case of detecting that
The driving assistance device according to any one of claims 1 to 3. The light emission control unit causes the light emitting device to emit light in different modes when the vehicle in which the light emitting device is installed is under automatic driving control and when the vehicle is manually driven.
The driving assistance device according to any one of claims 1 to 4.

Images