JP7436277B2 - Crew emotion transmission device - Google Patents

Description

The present invention relates to an occupant emotion transmission device.

In recent years, techniques for estimating the emotions of vehicle occupants have become known. For example, Patent Document 1 discloses an information output device that supports driving by considering the mental and physical condition of a passenger, and which detects the heart rate of the driver and the passenger measured by a heart rate sensor, and the drowsiness reference heart rate. , an information output device is disclosed that calculates the drowsiness level of each of a driver and a fellow passenger based on a drowsiness prediction parameter, and outputs driving support information based on the calculated drowsiness level. The information output device described in Patent Document 1 is configured to visually or audibly transmit driving support information to a driver using at least one of a display unit that displays driving support information and an audio output unit that outputs driving support information by voice. It is composed of

Japanese Patent Application Publication No. 2016-184282

Here, in the case of manual driving, even if the driver does not feel any discomfort in his/her own driving, the driving may be undesirable for the passenger. In this case, it may be difficult for the passenger to tell the driver. Furthermore, in the case of automatic driving, even if the driver involved in setting the driving control parameters does not feel uncomfortable with the driving, the driving may be undesirable for other occupants. In this case, it may be difficult for other passengers to tell the driver. Furthermore, if one of the passengers feels unwell, there may be a case where the driver or the person driving the vehicle wants to be informed without the other passengers knowing. Such a situation may cause mental stress to the passenger who is unable to speak up. Furthermore, it would be meaningful for the driver and the person driving the vehicle to be able to know the emotions of a fellow passenger in a casual manner without being understood by the fellow passenger.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to enable a driving principal such as a vehicle driver to know the emotions of a fellow passenger in a manner that is not understood by the fellow passenger. The objective is to provide a device for conveying occupant emotions.

In order to solve the above problems, according to one aspect of the present invention, an emotion estimator that estimates the emotion of a vehicle occupant based on biometric information of the vehicle occupant, and a method that does not understand the vehicle occupant, calculate the estimated emotion when driving. An occupant emotion communication device is provided, which includes a notification control unit that notifies a subject.

The notification control unit may notify the driver of the estimated emotion through the driver's vision.

The notification control unit may notify the driver of the estimated emotion through the driver's vision by switching the display of a display device that presents information other than emotion.

The display device that presents information other than emotions may be a speedometer, tachometer, remaining fuel monitor, voltage monitor, or water temperature monitor.

The notification control unit may notify the driver of the estimated emotion through the visual sense of the driver by performing a directional display.

The notification control unit may notify the driving subject of the estimated emotion through the hearing of the driving subject.

The notification control unit may notify the driver of the estimated emotion through the auditory senses of the driver by generating a sound that can be recognized only by the driver.

The notification control unit may notify the driving subject of the estimated emotion through the driving subject's sense of touch, smell, or taste.

The notification control unit may notify the driver of the estimated emotion through the driver's five senses by driving a device worn by the driver.

As described above, according to the present invention, the person driving the vehicle, such as the driver of the vehicle, can learn the emotions of the passenger in a manner that is not understood by the passenger.

1 is a schematic diagram showing an example of the configuration of an occupant emotion transmission device according to a first embodiment of the present invention. FIG. 2 is an explanatory diagram showing a circular model for emotion estimation. FIG. 3 is an explanatory diagram showing an example of a first display method by the occupant emotion transmission device according to the embodiment. FIG. 7 is an explanatory diagram showing an example of a second display method by the occupant emotion transmission device according to the embodiment. FIG. 7 is an explanatory diagram showing an example of a third display method by the occupant emotion transmission device according to the embodiment. FIG. 6 is an explanatory diagram showing an example of a fourth display method by the occupant emotion transmission device according to the embodiment. FIG. 3 is an explanatory diagram illustrating an example of displaying information on a passenger's emotions using a speedometer. FIG. 2 is an explanatory diagram showing an example of an orientation display monitor. FIG. 2 is a flowchart showing an example of control operation by the control device of the occupant emotion transmission device according to the embodiment; FIG. FIG. 2 is a block diagram illustrating an example of the configuration of a passenger emotion transmission device using a wearable device or a mobile terminal device as a vibration generating device.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that, in this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals and redundant explanation will be omitted.

<1. Configuration example of occupant emotion transmission device>
First, a configuration example of an occupant emotion transmission device according to an embodiment of the present invention will be described. FIG. 1 is a schematic diagram showing a configuration example of an occupant emotion transmission device 10 according to the present embodiment.

The occupant emotion communication device 10 according to this embodiment includes a multi-function display 80, an emotion estimation section 40, and a notification control section 50. The occupant emotion transmitting device 10 according to the present embodiment transmits information to the occupants other than the driver of the vehicle 1 estimated by the emotion estimation unit 40 by displaying the emotion in the multi-function display 80 of the vehicle 1 controlled by the notification control unit 50. The device is designed to notify the driver of the driver's emotions in a way that is not understood by the vehicle occupant.

Note that in this specification, the term "driving person" refers to a driver during manual driving or a passenger involved in setting driving control parameters for automatic driving. The occupants involved in setting the driving control parameters for autonomous driving are, for example, the occupants who set the inter-vehicle distance, acceleration, etc. for automatic driving control including driving support control, and the occupants who performed driving operations during learning of automatic driving control parameters. One example is the crew. Moreover, when simply referred to as "passenger", it means a passenger other than the driver.

In addition, "preventing other occupants from understanding" does not mean that other occupants do not notice the notification to the driver at all, but also that some actions toward the driver may affect the emotions of the occupants. This includes not being able to understand the meaning of notifying someone.

The multi-function display 80 displays information about the vehicle 1 other than the emotions of the occupants, such as vehicle speed, engine speed, shift position, remaining amount of fuel, and distance from the preceding vehicle. Furthermore, when the function for displaying the occupant's emotion is activated, the multi-function display 80 displays a display to notify the driver of the occupant's emotion in a way that is not understood by the occupant.

The multi-function display 80 illustrated in FIG. 1 includes a speedometer 81 that displays the vehicle speed, a tachometer 83 that displays the engine speed, a first shift position monitor 85 and a second shift position monitor 87 that display the shift position, and a fuel A fuel remaining amount monitor 89 that displays the remaining amount of battery, a voltage monitor 91 that displays the battery voltage, a water temperature monitor 93 that displays the engine cooling water temperature, a warning display monitor 95 that displays various warnings, and a direction display monitor 97. There is.

However, the display contents and arrangement positions of these monitors are merely examples, and some of these monitors may be omitted, and monitors displaying other contents may be further provided. For example, the multi-function display 80 may include a monitor that displays the distance between the vehicle and the preceding vehicle, and a monitor that displays the charge/discharge status of the battery.

In the occupant emotion transmission device 10 according to the present embodiment, the emotion estimation section 40 and the notification control section 50 are configured as functions of the control device 15. The occupant emotion transmission device 10 according to this embodiment further includes a vehicle information acquisition section 20 and a display switching section 30. Part or all of the control device 15 includes, for example, an arithmetic processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or a storage element such as a RAM (Random Access Memory) or a ROM (Read Only Memory). Prepared and configured. Part or all of the control device 15 may be configured with something that can be updated, such as firmware, or may be a program module or the like executed by a command from a CPU or the like. In addition, the control device 15 also uses a HDD (Hard Disk Drive), CD (Compact Disc), DVD (Digital Versatile Disc), SSD (Solid State Drive), USB ( (Universal Serial Bus) may be equipped with a storage medium such as flash or a storage device.

The vehicle information acquisition section 20, display switching section 30, emotion estimation section 40, and notification control section 50 of the control device 15 according to the present embodiment are configured as functions realized by execution of a program by an arithmetic processing device. Note that the control device 15 may be configured by one control device, or a plurality of control devices may be configured to be able to communicate with each other.

The vehicle information acquisition unit 20 has a function of acquiring information about the vehicle 1 to be displayed on the multi-function display 80, such as vehicle speed, engine speed, shift position, remaining amount of fuel, and battery voltage. For example, the vehicle information acquisition unit 20 acquires the vehicle speed, engine rotation speed, shift position, and fuel based on sensor signals from the vehicle speed sensor 21, engine rotation speed sensor 23, shift position sensor 25, and other sensors. Obtain information such as remaining power and battery voltage. The vehicle information acquisition unit 20 may acquire information about the vehicle 1 based on a message signal from another control device.

Further, the vehicle information acquisition unit 20 may acquire information on the driving environment of the vehicle based on signals transmitted from an external camera, radar sensor, LiDAR, etc. (not shown). Information on the driving environment of the vehicle includes, for example, the distance between the vehicle in front or the following vehicle and the vehicle, the distance between the vehicle and pedestrians, bicycles, etc., and the distance between the vehicle and surrounding objects such as obstacles and buildings. It includes information such as distance, brightness outside or inside the vehicle, and surrounding environment (in the forest, in the city, etc.). Further, the vehicle information acquisition unit 20 may acquire information about the driving scene of the vehicle (driving on an expressway, driving on an ordinary road, traffic jam, etc.) based on a signal transmitted from the navigation system.

The display switching unit 30 has a function of switching whether or not to execute notification control for displaying the occupant's emotions on the multi-function display 80. In the example shown in FIG. 1, the control device 15 is connected to a changeover switch 31 and an operation input section 33. The changeover switch 31 is operated by the driver to start or stop the notification control function that displays the occupant's emotions on the multi-function display 80. The changeover switch 31 may be a mechanical switch provided on a dashboard or the like, or may have a function such as a touch panel. The operation input unit 33 is an operation device for receiving operation input from a driver or a passenger, and includes, for example, at least one of a touch panel, an operation dial, a button switch, and the like. The operation input unit 33 may be configured to receive operation input by voice or gesture from the driver. The changeover switch 31 and the operation input section 33 may be integrated.

The display switching unit 30 switches whether or not to execute notification control for displaying the occupant's emotions on the multi-function display 80 based on an on/off switching operation of the changeover switch 31 by a driver or the like. In other words, the display switching unit 30 notifies the driver of the occupant's emotion through his/her vision by switching the display of the display device that presents information other than the occupant's emotion. The display switching unit 30 also determines which seat the emotion of the passenger sitting in is to be displayed, and what kind of emotion is to be displayed, based on the input operation to the operation input unit 33 by the driver or the like. Alternatively, set how to display it.

The emotion estimation unit 40 has a function of estimating the emotion of the occupant based on the occupant's biological information. The emotion estimating unit 40 estimates the occupant's biometric information based on the biometric information output from the biometric information detectors 41, 43, and 45 for detecting the biometric information of the occupant in each seat 3, 5, and 7 set in the vehicle 1. Estimate emotions. The biological information detectors 41, 43, and 45 may be non-wearable detectors that are not attached to the human body, or may be wearable detectors that are attached to the human body. In order to prevent the occupant from knowing that the occupant's emotions are being estimated, it is preferable that the biological information detectors 41, 43, and 45 are non-wearable detectors.

Examples of the non-wearable detector include an imaging camera, a radio Doppler sensor, and a pulse sensor. The imaging camera can detect, for example, the heartbeat and body temperature of the occupant. A radio Doppler sensor and a pulse sensor can detect the pulse of the passenger. Further, the non-wearable detector may be an electrode embedded in the steering wheel to measure the heartbeat or electrocardiogram of the occupant. In addition, the non-worn type detector is a pressure measuring device embedded in each seat 3, 5, 7 to measure the sitting pressure distribution while the occupant is seated on each seat 3, 5, 7. It's okay. Further, the non-worn type detector may be a device that detects changes in the position of the seat belt in order to measure the heartbeat or respiration of the occupant. Further, the non-wearable detector may be a TOF (Time of Flight) sensor for detecting information on the position of the occupant (biological position). Further, the non-wearable detector may be a thermography for measuring the surface temperature of the skin of the occupant.

Moreover, as a wearable detector, a wearable device connected to the control device 13 via wired or wireless communication means is exemplified. For example, a wearable device can be worn on a driver's wrist or arm to measure the driver's heartbeat.

Note that the biological information detected by the biological information detectors 41, 43, and 45 is not limited to the above-mentioned examples, and may be other biological information such as skin potential, blood pressure, cerebral blood flow, and electroencephalogram. .

The biological information detectors 41, 43, and 45 are connected to the control device 13 directly or via a communication line such as a CAN (Controller Area Network) or a LIN (Local Inter-Net). Alternatively, the biological information detectors 41, 43, and 45 communicate with the control device via wireless communication means such as Bluetooth (registered trademark), NFC (Near Field Communication), WiFi (wireless fidelity), and wireless LAN (Local Area Network). It may be configured such that it can communicate with 15.

The emotion estimation unit 40 estimates the emotion of the occupant based on the detection signals output from the biological information detectors 41, 43, and 45. The emotion estimation method based on biological information is not particularly limited, and may be any appropriate emotion estimation method. For example, the emotion estimation unit 40 classifies emotions into four categories: excitement, relief, boredom, or tension on a two-dimensional plane of “comfort-discomfort” and “alertness-drowsiness” shown in FIG. Emotions are estimated using a method that can classify the emotions of the occupants using a ring model (Russell's ring model) in which the robots are arranged in a ring. Specifically, the emotion estimation unit 40 may estimate the occupant's emotion by determining whether the sympathetic nerve or parasympathetic nerve is dominant and perform an autonomic nervous balance evaluation, or may estimate the occupant's emotion by using FACS theory based on the occupant's facial expression. The emotion of the person may be estimated. At this time, the emotion estimation unit 40 may estimate the level of emotion (degree of emotion/degree of condition).

Furthermore, the emotion estimation unit 40 may construct an emotion estimation model that specifies each person by performing machine learning on the emotion estimation results, and estimate the emotions of the occupant. For example, the emotion estimation unit 40 acquires at least one of the information on the driving state of the vehicle 1, the information on the driving environment, or the information on the driving scene from the vehicle information acquisition unit 20, and uses the information on the driving state of the vehicle 1. The factors that cause the autonomic nervous balance to be disrupted may be learned for each individual occupant by using the occupant's emotions estimated to be the same as the input data. As such an emotion estimation model, a learning model constructed in advance may be used. Further, the identification of an individual may be performed by extracting the feature amount of the person's face based on image data acquired by an imaging camera. By estimating the emotion of the occupant using the emotion estimation model, the emotion estimation unit 40 can estimate a sign of a change in the emotion of the occupant and estimate the emotion.

The notification control unit 50 has a function of notifying the driver of the occupant's emotion estimated by the emotion estimation unit 40 by a method that is not understood by the occupant. The notification control unit 50 notifies the driver of the occupant's emotions through the driver's five senses in a manner that is not understood by the occupant. In the present embodiment, the notification control unit 50 notifies the driver of the occupant's emotions through the visual sense of the driver by displaying the information on the multi-function display 80. The notification control unit 50 controls the display on the multi-function display 80 according to the display content and display method set by the display switching unit 30. For example, while the function of notifying the occupant's emotion through the tachometer 83 is set to the execution state, the notification control unit 50 executes control to notify the occupant's emotion.

By using the display on the multi-function display 80, which is a display device normally used in the vehicle 1, to notify the driver of the occupant's emotions, the driver can be informed of the occupant's emotions without feeling strange or being understood by the driver. can be communicated to other people. Hereinafter, an example of a method of notifying the driver of the occupant's emotions by displaying on the multi-function display 80 in a manner that is not understood by the occupant will be described.

(Example of first display method)
FIG. 3 is an explanatory diagram showing an example of the first display method. In the first display method, the notification control unit 50 uses the display of the tachometer 83 that displays the engine speed to transmit the emotions of the occupant to the driver.

The tachometer 83 shown in FIG. 3 is a pointer meter that displays the engine speed with a needle 83a in a normal usage state (a state in which the function to notify the occupant's emotion is off). When the function that notifies occupants of their emotions is on, displays 1, 2, 3, and 4 indicating the engine speed are assigned to the front passenger seat occupant, rear left seat occupant, rear middle seat occupant, and rear right seat occupant, respectively. ing. However, information on crew assignments is not displayed on the display, and only needs to be understood by the driver.

In the example shown in FIG. 3, the hand 83a instructs the display of a numerical value corresponding to the occupant exhibiting the estimated emotion so that the occupant exhibiting the estimated emotion can be identified. For example, if the driver is set to be notified only when the occupant's emotion is one of excitement, relief, boredom, or nervousness, the notification control unit 50 , the needle 83a may be used to instruct the display of a numerical value corresponding to the relevant occupant. Alternatively, the corresponding numerical value may be lit, blinking, or displayed in a specific color without operating the hand 83a. In addition, when the setting is made to notify the driver of multiple emotions of each occupant, the notification control unit 50 changes the display color, lighting state, or background color of the numerical value corresponding to the applicable occupant. It may be changed. Thereby, the display can be visually recognized only by the driver, and the emotions of the passenger can be transmitted to the driver without the passenger noticing.

Alternatively, when notifying the occupants' emotions through the tachometer 83, the notification control unit 50 may cause the needle 83a to instruct the display of a numerical value corresponding to the number of occupants who have experienced a predetermined specific emotion. The numerical value may be displayed by lighting, blinking, or in a specific color. In this case as well, if it is set to notify the driving principal of the number of occupants who have expressed the emotion regarding multiple emotions, the notification control unit 50 may notify the number of occupants who have expressed the emotion in question. The display color or lighting state of the corresponding numerical value may be changed. Thereby, it is possible to inform the driver of the vehicle how many of the fellow passengers are feeling a predetermined emotion without the occupants noticing.

Even if the tachometer that displays the engine speed is a digital meter with a variable bar length, the bar display can be used in the same way as the needle in the above explanation to convey the emotions of the occupants to the driver. can be done.

(Example of second display method)
FIG. 4 is an explanatory diagram showing an example of the second display method. In the second display method, the notification control unit 50 uses the display of the first shift position monitor 85 that displays the shift position to transmit the emotions of the occupant to the driver.

The first shift position monitor 85 shown in FIG. 4 is a monitor that numerically displays the shift position in a normal usage state (a state in which the function of notifying the occupant's emotion is off). In the display when the function to notify occupants' emotions is on, numerical values are assigned to each of the front passenger seat occupant, rear left seat occupant, rear center occupant, and rear right seat occupant. Further, the first shift position monitor 85 is configured so that auxiliary displays 85a to 85d light up depending on the emotion. For example, auxiliary displays 85a-85d are each assigned to display emotions of excitement, relief, boredom, or nervousness, and the level of these emotions. However, information regarding the allocation of occupant or emotion display is not displayed on the display, and it is sufficient for the driver to understand the information.

In the example shown in FIG. 4, numerical values corresponding to the occupants exhibiting the estimated emotions are displayed so that the occupants exhibiting the estimated emotions can be identified. Further, if the setting is made to notify the driver of a plurality of emotions of each occupant, the notification control unit 50 lights up one of the auxiliary displays 85a to 85d corresponding to the emotion of the occupant. The notification control unit 50 may display the emotional level together with or alternately with the numerical value corresponding to the occupant. Thereby, the display can be visually recognized only by the driver, and the emotions of the passenger can be transmitted to the driver without the passenger noticing.

Note that if the configuration is such that the color of the numerical value, the lighting state, or the background color is changed depending on the passenger's emotion, the auxiliary displays 85a to 85d may be omitted. Further, even if the driver is set to be notified only when the occupant's emotion becomes one of a specific emotion such as excitement, relief, boredom, or nervousness, the auxiliary display 85a to 85d may be omitted. Alternatively, when notifying the occupant's emotion through the first shift position monitor 85, the notification control unit 50 may display a numerical value corresponding to the number of occupants who have a pre-designated specific emotion. In this case as well, if it is set to notify the driving principal of the number of occupants who have expressed the emotion regarding multiple emotions, the notification control unit 50 may notify the number of occupants who have expressed the emotion in question. The display color or lighting state of the corresponding numerical value may be changed. Thereby, it is possible to inform the driver of the vehicle how many of the fellow passengers are feeling a predetermined emotion without the occupants noticing.

(Example of third display method)
FIG. 5 is an explanatory diagram showing an example of the third display method. In the third display method, the notification control unit 50 uses the display of the second shift position monitor 87 that displays the shift position to transmit the emotions of the occupant to the driver.

The second shift position monitor 87 shown in FIG. 5 is a monitor that displays the shift position in a normal usage state (a state in which the function of notifying the occupant's emotion is off). In the display when the function to notify occupants' emotions is on, numerical values are assigned to each of the front passenger seat occupant, rear left seat occupant, rear center occupant, and rear right seat occupant. Further, the second shift position monitor 87 is configured so that auxiliary displays 87a to 87d light up depending on the emotion. For example, auxiliary displays 87a-87d are assigned to display emotions of excitement, relief, boredom, or nervousness, respectively. However, information regarding the allocation of occupant or emotion display is not displayed on the display, and it is sufficient for the driver to understand the information.

In the example shown in FIG. 5, numerical values corresponding to the occupants exhibiting the estimated emotions are displayed so that the occupants exhibiting the estimated emotions can be identified. Further, if the setting is made to notify the driver of a plurality of emotions of each occupant, the notification control unit 50 lights up one of the auxiliary displays 87a to 87d corresponding to the emotion of the occupant. Thereby, the display can be visually recognized only by the driver, and the emotions of the passenger can be transmitted to the driver without the passenger noticing.

Note that if the configuration is such that the color of the numerical value, the lighting state, or the background color is changed depending on the emotion of the occupant, the auxiliary displays 87a to 87d may be omitted. Furthermore, even if the driver is set to be notified only when the occupant's emotion becomes one of a specific emotion such as excitement, relief, boredom, or nervousness, the auxiliary displays 87a to 87d may be omitted. Alternatively, when notifying the occupants' emotions through the second shift position monitor 87, the notification control unit 50 may display a numerical value corresponding to the number of occupants who have a predetermined specific emotion. In this case as well, if it is set to notify the driving principal of the number of occupants who have expressed the emotion regarding multiple emotions, the notification control unit 50 may notify the number of occupants who have expressed the emotion in question. The display color or lighting state of the corresponding numerical value may be changed. Thereby, it is possible to inform the driver of the vehicle how many of the fellow passengers are feeling a predetermined emotion without the occupants noticing.

(Example of fourth display method)
FIG. 6 is an explanatory diagram showing an example of the fourth display method. In the fourth display method, the notification control unit 50 uses the display of the remaining fuel amount monitor 89 that displays the shift position to convey the emotions of the occupant to the driver.

The remaining fuel amount monitor 89 shown in FIG. 6 is a monitor in which a needle 89a indicates the remaining fuel level in a normal usage state (a state in which the function of notifying the occupant's emotion is off). When the function for notifying occupants' emotions is on, a level display is assigned to each of the front passenger seat occupant, rear left seat occupant, rear center occupant, and rear right seat occupant. In the example shown in FIG. 6, the front passenger seat occupant is assigned a display of "F (Full)," and the rear center passenger is assigned a display of "M (Mid)." Additionally, the rear left seat occupant is assigned a display indicating an intermediate level between "F" and "M", and the rear right seat occupant is assigned a display indicating an intermediate level between "M" and "L (Low)". ing. Further, the remaining fuel amount monitor 89 includes an auxiliary display 89b whose color or lighting state changes depending on the emotion. However, information regarding the allocation of occupant or emotion display is not displayed on the display, and it is sufficient for the driver to understand the information.

In the example shown in FIG. 6, the hand 89a is caused to indicate a level display corresponding to the occupant exhibiting the estimated emotion so that the occupant exhibiting the estimated emotion can be identified. Further, if the setting is made to notify the driver of a plurality of emotions of each occupant, the notification control unit 50 causes the auxiliary display 89b to be displayed in a color or lighting state corresponding to the emotion of the occupant. . Thereby, the display can be visually recognized only by the driver, and the emotions of the passenger can be transmitted to the driver without the passenger noticing.

Note that if the level display color, lighting state, or background color is configured to change depending on the passenger's emotion, the auxiliary display 89b may be omitted. Furthermore, even if the driver is set to be notified only when the occupant's emotion is one of a specific emotion such as excitement, relief, boredom, or nervousness, the auxiliary display 87b is omitted. may have been done. Alternatively, when notifying the emotions of the occupants through the remaining fuel level monitor 89, the notification control unit 50 may turn on a level display corresponding to the number of occupants who have a pre-specified specific emotion. In this case as well, if it is set to notify the driving principal of the number of occupants who have expressed the emotion regarding multiple emotions, the notification control unit 50 may notify the number of occupants who have expressed the emotion in question. The color or lighting state of the corresponding level display may be changed. Thereby, it is possible to inform the driver of the vehicle how many of the fellow passengers are feeling a predetermined emotion without the occupants noticing.

Note that some or all of the examples of the first to fourth display methods described so far may be applied to the speedometer 81, voltage monitor 91, water temperature monitor 93, and warning display monitor 95 provided in the multi-function display 80. You may. Further, each of the first to fourth display methods may be performed alone or may be performed in an appropriate combination. Further, by appropriately combining the first to fourth display methods, the level of emotion may be displayed in addition to the position and emotion of the occupant.

In addition, when conveying emotions to the driver using displays important for vehicle safety, such as the speedometer 81 and warning display monitor 95, when displaying information about the emotions of the occupants, it is necessary to It is preferable to display the information simultaneously. FIG. 7 is an explanatory diagram showing an example of displaying information on the occupant's emotions using the speedometer 81. The left side of FIG. 7 shows an example of a normally displayed vehicle speed display, and the right side of FIG. 7 shows an example of displaying emotional information while simultaneously displaying the vehicle speed.

As shown in FIG. 7, when displaying information on the occupant's emotions using the speedometer 81, the notification control unit 50 displays the needle 81a indicating the vehicle speed as is, and the indicator 82a indicating the emotional level of the occupant. and level display (0, 1, 2, 3) is additionally displayed. At the same time, the notification control unit 50 displays the position of the passenger (seat position) using the monitor lamp 82b. As a result, information on the occupant's emotions can be transmitted to the driver while maintaining the display of the vehicle speed, which is important for vehicle safety. This display method is not limited to the example shown in FIG. For example, when displaying information about the occupant's emotion, the display may be changed so that the pointer 81a indicating the vehicle speed is used as a pointer indicating the level of the occupant's emotion, and the vehicle speed may be separately displayed digitally.

(Example of fifth display method)
In the fifth display method, the notification control unit 50 uses the display of the orientation display monitor 97 provided in the multi-function display 80 to transmit the emotions of the occupant to the driver. As shown in FIG. 1, the orientation display monitor 97 is a monitor in which the lighting display of the first to fourth lamps 97a to 97d is visible only to the driver. For example, as shown in FIG. 8, the orientation display monitor 97 is constructed by arranging an LCF (Light Control Film) 98 in front of the first to fourth lamps 97a to 97d.

The orientation display monitor 97 shown in FIG. 1 is a monitor that is not used in normal use (the function of notifying the passenger's emotions is off). The first to fourth lamps 97a to 97d are assigned to the front passenger seat occupant, rear left seat occupant, rear center occupant, and rear right seat occupant, respectively. Further, the first to fourth lamps 97a to 97d are configured so that their colors or lighting states change depending on the emotion. However, information regarding the allocation of occupant or emotion display is not displayed on the display, and it is sufficient for the driver to understand the information.

In the example of the fifth display method, a lamp corresponding to an occupant exhibiting an emotion is lit so that the occupant exhibiting the estimated emotion can be identified. Further, when the setting is made to notify the driver of a plurality of emotions of each occupant, the notification control unit 50 changes the lamp to a color or a lighting state corresponding to the emotion of the occupant. Thereby, the display can be visually recognized only by the driver, and the emotions of the passenger can be transmitted to the driver without the passenger noticing.

Note that the lamp color, lighting state, or background color may be changed depending on the passenger's emotion. Further, when notifying the emotions of the occupants through the orientation display monitor 97, the notification control unit 50 may turn on lamps corresponding to the number of occupants who have a predetermined specific emotion. In this case as well, if it is set to notify the driving principal of the number of occupants who have expressed the emotion regarding multiple emotions, the notification control unit 50 may notify the number of occupants who have expressed the emotion in question. The color and lighting state of the corresponding lamp may also be changed. Thereby, it is possible to inform the driver of the vehicle how many of the fellow passengers are feeling a predetermined emotion without the occupants noticing.

<2. Operation example of occupant emotion transmission device>
Next, an example of the operation of the occupant emotion transmission device 10 according to the present embodiment will be described. FIG. 9 is a flowchart schematically showing an example of control operation by the control device 15 of the occupant emotion transmission device 10. In the following operation example, an example will be described in which the emotion estimation unit 40 estimates an emotion using an emotion estimation model that is generated by machine learning and identifies an individual. Further, in the following operation example, an example will be described in which the driving subject is notified of the emotion using a display on the multi-function display 80 in a manner that is not understood by the occupant.

First, the emotion estimation unit 40 of the control device 15 acquires biometric information of the vehicle occupant based on detection signals transmitted from the biometric information detectors 41, 43, and 45 (step S11). At this time, the emotion estimation unit 40 also detects the seat position of the detected occupant. Furthermore, the emotion estimating unit 40 may specify the detected individual occupant. Individual occupants may be identified, for example, by analyzing facial features based on image data acquired by an imaging camera.

Next, the emotion estimation unit 40 inputs the acquired biological information data into the emotion estimation model, and estimates the occupant's emotion from the acquired biological information data (step S13). The emotion estimating unit 40 estimates the emotion of the passenger so that the type of emotion can be classified according to the above-mentioned circular model of emotions for each seat position where the passenger is present. The output of the passenger's emotion estimated using the emotion estimation model may also include an estimation result of the passenger's emotional level.

Next, the emotion estimating unit 40 estimates at least the results of estimating the emotions of each occupant and the information on the driving state of the vehicle 1, the information on the driving environment, or the information on the driving scene at the time of detection of the biological information used for estimating the emotion. One is input into the emotion estimation model and machine learning is performed (step S15). As a result, machine learning is performed using the currently input emotion estimation results and vehicle driving state information as new training data, and the emotion estimation model is adapted. At this time, by adding to the input data the attributes, facial expressions, biological changes, etc. of individual occupants identified using an imaging camera, it is possible to estimate emotions tailored to each occupant, and to estimate the individual occupant's past. The emotion is estimated based on the average value or worst value of the data.

Next, the notification control unit 50 of the control device 15 determines whether the display switching unit 30 is set to execute notification control for notifying the occupant's emotion (step S17). If the notification control for notifying the occupant's emotion is not set to be executed (S17/No), the notification control unit 50 ends this routine and returns to step S11.

On the other hand, if the notification control for notifying the occupant's emotion is set to be executed (S17/Yes), the notification control unit 50 uses information about the occupant's emotion estimated by the emotion estimating unit 40 in step S13, Display control information for controlling the display on the multi-function display 80 is generated based on the setting information by the display switching unit 30 (step S19). Specifically, the notification control unit 50 controls the display switching unit 30 to determine which seat the emotion of the passenger sitting in should be displayed, what type of emotion should be displayed, or what method should be used to display the emotion. Refer to the information set by. Then, the notification control unit 50 generates display control information for performing a predetermined display on a desired monitor based on information on the estimated emotion of the occupant in light of the information set by the display switching unit 30. do.

Next, the notification control unit 50 generates a control signal based on the generated display control information, and outputs the control signal to the multi-function display 80 (step S21). As a result, a display based on the estimated emotion of the occupant is displayed on the set monitor portion of the multi-function display 80. Although the content of such display can be understood by the driver, it is difficult for the passenger to understand the display, so the driver must know the passenger's emotions so that the passenger does not understand. Can be done.

As described above, according to the occupant emotion transmission device 10 according to the present embodiment, the emotions of occupants other than the driver who are detected using the biological information detectors 41, 43, and 45 are not understood by the occupant. The driver will be notified depending on the method. Therefore, even if it is difficult for the occupant to tell the driver that the current driving condition is undesirable, the driver can inform the driver without the occupant noticing. Further, the passenger who feels unwell can notify the driver of the vehicle without the other passengers knowing that he or she has felt unwell. Therefore, the mental stress of the fellow passenger can be reduced, and the driver can also casually reflect the stress on the driving operation.

Further, the occupant emotion transmitting device 10 according to the present embodiment is configured to notify the driver of the occupant's emotion using the display on the multi-function display 80 that corresponds to the display on the instrument panel. For this reason, it is difficult to see from the front passenger seat or the rear seat, and the driver's emotions are displayed on the display that is visible during normal driving operations, so the occupant's emotions are displayed without being noticed by other passengers. You can know emotions. In particular, in the occupant emotion transmitting device 10 according to the present embodiment, the occupant's emotion is notified by using the display of a monitor normally used to display the state of the vehicle 1, such as a tachometer or a fuel level gauge. Furthermore, it is possible to notify the occupants of their emotions without the occupants being noticed.

In addition, in the occupant emotion communication device 10 according to the present embodiment, by notifying the occupant's emotion using the orientation display monitor 97 that is visible only to the driver, the occupant's emotion can be further communicated without being noticed by surrounding occupants. Can communicate emotions.

Although the occupant emotion communication device 10 according to the above embodiment conveys the emotion of the occupant to the driver in a manner that is not understood by the occupant by displaying it on the multi-function display 80, the display device is limited to the multi-function display 80. I can't. For example, it may be displayed on a head up display (HUD), a head mounted display (HMD), a wearable display device, smart glasses, or the like. By displaying the information on personally usable devices such as wearable display devices and smart glasses, it is possible to directly notify the driver of the vehicle occupant's emotions without the vehicle occupant noticing.

Further, in the occupant emotion transmitting device 10 according to the embodiment described above, the display switching unit 30 may change the display method or display device at predetermined time intervals or the number of times of display. As a result, the method or display device for displaying the occupant's emotions is changed at an appropriate timing, so it is possible to further reduce the possibility that the occupant will notice.

Furthermore, in the occupant emotion communication device 10 according to the above embodiment, the notification control unit 50 changes the display mode such as the display color, design, font, lighting brightness, etc. when displaying the occupant's emotion. , which seat the occupant is in, or what kind of emotion the driver is feeling may be communicated to the driver.

<3. Other embodiments>
Up to this point, an example of the configuration of the occupant emotion transmitting device 10 according to the present embodiment has been described, but the occupant emotion transmitting device 10 can be modified in various ways. Hereinafter, some of the occupant emotion transmitting devices 10 according to modified examples will be explained.

(3-1. Auditory notification)
The occupant emotion transmitting device 10 according to the first modification is configured to notify the driver of the occupant's emotions through the driver's hearing in a manner that the occupant does not understand. The passenger emotion transmission device 10 includes a device that can be used by an individual, such as a wearable device, as an output device for transmitting the passenger's emotions through the driver's auditory senses. Examples of such output devices include wearable devices worn by the driver, including earphones, bone conduction devices, and the like. When the output device is a wearable device, the output device connects the control device 15 to, for example, Bluetooth (registered trademark), NFC (Near Field Communication), wifi (wireless fidelity), wireless LAN (Local Area Network), etc. The connection may be made via a wireless communication means, or may be connected via a wired communication means.

The notification control unit 50 generates an audio signal for transmitting information about the emotion of the occupant in which seat and the content of the emotion to the driver, and transmits it to the output device. The signal may be a sound signal instead of a voice, as long as the sound can be recognized in advance by the driver. With such a device that can be used by individuals, it is possible to directly convey the emotions of the occupant to the driver without the occupant noticing.

Alternatively, the occupant emotion communication device 10 may use a directional speaker or a proximity speaker that can be heard only by the driver to notify the driver of the passenger's emotions through hearing so that the driver does not understand. . For example, the notification control unit 50 generates a tone signal that can be recognized in advance by the driving person at a volume that reaches only the driving person or with a sound that has a directionality that only the driving person can hear. let For example, by appropriately combining differentiators based on single sounds, continuous sounds, pattern sounds, tones, types of sounds, etc., it is possible to determine which seat the occupant is in, or what kind of emotion the driver is feeling. Can be communicated to the subject. As long as the voice cannot be heard by other occupants, it may be possible to generate a voice that directly indicates which seat the occupant is in and what kind of emotion he or she is feeling. Even when such a directional speaker or a proximity speaker is used, the emotions of the occupant can be transmitted to the driver without being understood by the occupant.

(3-2. Tactile notification)
The occupant emotion transmitting device 10 according to the second modification is configured to notify the driver of the occupant's emotions through the driver's tactile sense in a manner that is not understood by the occupant. The occupant emotion transmission device 10 includes at least one of a heating device, a vibration generator, an air blowing device, or a far-infrared heating device as an output device for transmitting the emotions of the passenger through the tactile sense of the driver.

When the output device is a heating device or a vibration generating device, for example, heat sources or vibration generating sources are arranged inside the seat of the driver's seat and are appropriately divided into the passenger seat, rear left seat, rear center seat, and rear right seat, The notification control unit 50 operates a heat source or a vibration generation source corresponding to the occupant whose emotion is to be notified. Furthermore, the notification control unit 50 adjusts the amount of heat generated or the intensity of vibration depending on the content of the emotion. If the driver understands in advance the classification of the seating surface, the position of the occupant, and the relationship between the amount of heat generated or the intensity of vibration and the content of the emotion, the driver can know the emotion of the occupant without being noticed by other occupants. be able to.

Further, when the output device is an air blowing device, air injection nozzles are attached to the driver's seat, the front passenger seat, the rear left seat, the rear center seat, and the rear right seat at appropriate positions, and the notification control unit 50 notifies the emotion. Air is injected from the air injection nozzle corresponding to the target occupant. For example, the injection location may vary depending on the location of the occupant, such as the neck, back of the head, waist, or thigh. Further, the notification control unit 50 adjusts the strength (flow rate) of the air to be ejected depending on the content of the emotion. If the driver understands in advance the relationship between the injection position and the passenger's position, as well as the strength of the air and the content of the emotion, the driver can know the passenger's emotion without being noticed by other passengers.

Further, when the output device is an air blowing device, air injection nozzles are attached to the driver's seat, the front passenger seat, the rear left seat, the rear center seat, and the rear right seat at appropriate positions, and the notification control unit 50 notifies the emotion. Air is injected from the air injection nozzle corresponding to the target occupant. For example, the injection locations may vary depending on the occupant, such as the neck, back of the head, waist, thighs, etc. Further, the notification control unit 50 adjusts the strength (flow rate) of the air to be ejected depending on the content of the emotion. If the driver understands in advance the relationship between the injection position and the passenger's position, as well as the strength of the air and the content of the emotion, the driver can know the passenger's emotion without being noticed by other passengers.

In addition, if the output device is a far-infrared heat generating device, devices capable of emitting far-infrared rays are installed toward the driver's seat with different radiation positions for each of the passenger seat, rear left seat, rear center seat, and rear right seat. The notification control unit 50 causes far infrared rays to be emitted from a device corresponding to the occupant whose emotion is to be notified. For example, the radiation position may vary depending on the occupant, such as the neck, face, arm, thigh, etc. Furthermore, the notification control unit 50 adjusts the intensity of the far-infrared rays to be emitted depending on the content of the emotion. If the driver understands in advance the relationship between the radiation position and the occupant's position, as well as the intensity of the air and the content of the emotion, the driver can know the occupant's emotion without other occupants noticing.

Note that the device for transmitting the emotions of the passenger to the driver through the sense of touch may be a wearable device or a mobile terminal device worn by the driver. By using wearable devices and mobile terminal devices, it is possible to directly convey the emotions of the passenger to the driver without the other passengers knowing.

FIG. 10 is a block diagram illustrating a configuration example of an occupant emotion transmission device 100 using a wearable device or a mobile terminal device as a vibration generating device. The occupant emotion communication device 100 uses a vibration generating device 101 as a notification means instead of a display device such as a multi-function display. The vibration generating device 101 may be any device that can be carried or worn by the driver, such as a wristwatch-type wearable device, a mobile terminal device such as a smart phone, or a wireless key equipped with a vibrator.

The notification control unit 50 generates vibrations according to the intensity or pattern set according to the passenger whose emotion is to be notified and the content of the estimated emotion. If the driver understands in advance the relationship between the occupant's position and emotional content and the intensity or pattern of vibration, the driver can know the occupant's emotion without other occupants noticing. With the passenger emotion transmission device 100 using such a vibration generating device 101, for example, if the driver who was involved in setting the control parameters for automatic driving moves to a seat other than the driver's seat, such as the rear seat, Even if there is, it is possible to know the emotions of other passengers without them noticing.

(3-3. Notification by smell)
The occupant emotion transmitting device 10 according to the third modification is configured to notify the driver of the occupant's emotions through the driver's sense of smell in a manner that is not understood by the occupant. The occupant emotion transmitting device 10 includes an aroma diffusion device as an output device for transmitting the occupant's emotions through the driver's sense of smell. The aroma diffuser may be, for example, a device that is connected to a cigarette socket and receives power supply and a drive signal, or a device that operates by receiving a drive signal through wireless communication means.

For example, the aroma diffusion device is equipped with an air pump and a plurality of aroma cartridges, and can emit at least as many types of aromas as the number of passengers. The notification control unit 50 selects and releases a fragrance depending on the passenger whose emotion is to be notified. This allows the driver to know the emotions of the occupants. For example, by emitting a fragrance when the occupant feels uncomfortable, the unpleasant emotion of the occupant can be alleviated. In this case, the passenger will feel the scent, but the passenger will not be able to understand that the scent represents the passenger's unpleasant feelings.

Further, the aroma diffuser may be a device that is configured in the shape of a helmet and is worn by the driver. With a helmet-type aroma diffuser, by diffusing the aroma inside the helmet, it can be used even in saddle-type vehicles, two-wheeled vehicles, motorcycles with sidecars, and vehicles with separate spaces for each seat. A small amount of fragrance can reliably convey the emotions of the occupant to the driver without being noticed by other occupants.

(3-4. Notification by taste)
The occupant emotion transmitting device 10 according to the fourth modification is configured to notify the driver of the occupant's emotions through the taste of the driver in a manner that is not understood by the occupant. The occupant emotion transmitting device 10 includes a mouthpiece-type driving device, an electronic cigarette, a tumbler, and other devices that can control the taste, as an output device for transmitting the emotions of the passenger through the driver's taste.

The output device includes, for example, a cartridge containing a plurality of tastes and a driving means for emitting tastes from the cartridge, and can generate at least as many types of tastes as the number of passengers. The driving means may be, for example, a heating device, a blower device, an extrusion device, or the like. The notification control unit 50 generates a taste depending on the occupant whose emotion is to be notified. This allows the driver to know the emotions of the occupants.

In this way, even when the occupant emotion transmission device 10 notifies the occupant of the occupant's emotion through five senses other than vision, it is possible to transmit the occupant's emotion to the driver without the occupant understanding. Therefore, even if it is difficult for the occupant to tell the driver that the current driving condition is undesirable, the driver can inform the driver without the occupant noticing. Further, the passenger who feels unwell can notify the driver of the vehicle without the other passengers knowing that he or she has felt unwell. Therefore, the mental stress of the fellow passenger can be reduced, and the driver can also casually reflect the stress on the driving operation.

Although preferred embodiments of the present invention have been described above in detail with reference to the accompanying drawings, the present invention is not limited to such examples. It is clear that a person with ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea stated in the claims. It is understood that these also naturally fall within the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 10... Occupant emotion transmission device, 15... Control device, 30... Display switching unit, 40... Emotion estimation unit, 50... Notification control unit, 80... Multi-function display, 81... Speedometer, 83... Tachometer, 85... First Shift position monitor, 87...Second shift position monitor, 89...Remaining fuel level monitor, 91...Voltage monitor, 93...Water temperature monitor

Claims (8)

an emotion estimation unit that estimates the emotion of an occupant other than the vehicle driver based on biological information of the occupant;
A display device that presents information other than emotions and that is important for vehicle safety uses the display of the display device to display the estimated information through the driver's vision in a manner that is not understood by the occupant. a notification control unit that notifies the driving subject of the emotion that has been expressed,
The notification control unit additionally displays information indicating the estimated emotion while maintaining the display important for vehicle safety on the display device . The occupant emotion transmitting device according to claim 1 , wherein the display device that presents information other than emotions is a speedometer, a tachometer, a remaining fuel monitor, a voltage monitor, or a water temperature monitor. The occupant emotion transmitting device according to claim 1 , wherein the notification control unit notifies the estimated emotion through the visual sense of the driving subject by performing a directional display. The occupant emotion transmission device according to claim 1, wherein the notification control unit further notifies the driver of the estimated emotion through the driver's hearing. The occupant emotion transmitting device according to claim 4 , wherein the notification control unit notifies the driver of the estimated emotion through the auditory sense of the driver by generating a sound that can be recognized only by the driver. The occupant emotion transmission device according to claim 1, wherein the notification control unit further notifies the driver of the estimated emotion through the driver's sense of touch, smell, or taste. The occupant emotion transmission device according to claim 1, wherein the notification control unit notifies the driver of the estimated emotion through the five senses of the driver by further driving a device worn by the driver. According to any one of claims 1 to 7, the notification control unit changes the notification method or the notification device to be used from among the plurality of notification methods and notification devices for each predetermined time interval or the number of notifications. passenger emotion communication device.

Images