JP2021107801A - Automatic driving vehicle, image display method and program - Google Patents

Abstract

To provide an automatic driving vehicle that facilitates an occupant to select information and reduces operational cumbersomeness.SOLUTION: An HMI control part 10 of an automatic driving vehicle includes: a display part for displaying an image indicating a surrounding situation of a vehicle; an occupant feeling estimation part 101 for estimating an occupant feeling of an occupant in a cabin; a driving intention recognition part 102 for recognizing a driving intention of the vehicle; and a display control part 103 for outputting to the display part an image indicating a surrounding situation that is produced by varying an information amount according to the occupant feeling and driving intention. More specifically, the occupant feeling estimation part estimates an occupant feeling degree quantitatively representing the occupant feeling from a facial expression of the occupant imaged by a driver monitoring camera. In a case where the occupant feeling degree is higher than a given value, the display control part outputs to the display part a map image produced by increasing an additional information amount.SELECTED DRAWING: Figure 3

Description

The present invention relates to an autonomous vehicle, an image display method, and a program for displaying peripheral information of a traveling vehicle on an in-vehicle display.

In recent years, in order to reduce the burden on the driver and realize comfortable vehicle operation, a driving support technology called automatic driving has been enthusiastically proposed.
In the international standard for autonomous driving, levels over multiple stages are set hierarchically according to the lightness / presence / absence of driver-required tasks and the degree of automation. In addition, even within a certain level, a group of requirements for realizing automation can be realized side by side, and divisions (categories) over a plurality of stages are set hierarchically.
For example, in level 4 (highly automatic driving) and level 5 (fully automatic driving) automatic driving, the driver does not need to override the driving operation.

As described above, the driver does not need to be involved in the driving operation during the automatic driving of level 4 or higher. Therefore, like other occupants, the driver can utilize the vehicle infotainment system and the like.
For example, Patent Document 1 discloses an autonomous driving vehicle that displays a traveling route of a vehicle or the like on a display device of the vehicle in a plurality of different display forms.

Japanese Unexamined Patent Publication No. 2019-32307

According to the technology of Patent Document 1, the environment of the vehicle is posted in a plurality of different display forms, and a plurality of driving operations corresponding to the environment can be selected. Therefore, the optimum driving operation during automatic driving can be selected. Can be done.
However, the display is switched by the occupant making a selection, which is troublesome depending on the situation.

An object of the present invention is to provide an autonomous driving vehicle that facilitates information selection by an occupant and reduces the troublesome operation.

In order to solve the above problems, the autonomous driving vehicle of the present invention includes a display unit that displays an image showing the surrounding situation of the vehicle, an occupant emotion estimation unit that estimates the occupant's emotions in the vehicle, and driving of the vehicle. It is provided with a driving intention recognition unit that recognizes an intention, and a display control unit that outputs the image showing the peripheral situation in which the amount of information is varied according to the occupant's feelings and the driving intention to the display unit.

According to the self-driving vehicle of the present invention, since the amount of information of the display device is changed according to the situation of the occupant and the driving intention, the occupant can easily select the information and the troublesomeness of the information selection can be reduced.

It is a control block block diagram of the self-driving vehicle of an embodiment. It is a figure which shows an example of the passenger compartment front structure of the self-driving vehicle. It is a block diagram explaining the structure of the HMI control unit. It is a figure explaining the relationship between the emotional degree of an occupant, the clarity of a driving intention, and the map display. It is a flow chart which shows the processing outline of the HMI control unit. It is a processing flow diagram when the occupant emotion estimation unit analyzes an emotion from a occupant's facial expression or movement. It is a processing flow diagram when the occupant emotion estimation unit analyzes an emotion from the conversation of an occupant. This is a display example of the display format of "Camera View". This is a display example of a display format in which the occupant's emotional level is high and the amount of additional information is large. This is a display example of "3D Real Map" with high map detail. It is a flow chart of the detailed processing of a driving intention recognition unit and a display control unit. This is a display example of the display format of "Simple Map". This is a display example with a traffic light (traffic light not participating) added. This is a display example with trees (traffic non-participation markers) added. It is a figure explaining the relationship between a virtual viewpoint and a display range of a map image in a bird's-eye view. It is a processing flow diagram which changes a virtual viewpoint of a map image in a bird's-eye view.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a control block configuration diagram of the autonomous driving vehicle M of the embodiment.
The self-driving vehicle M includes a vehicle powered by an internal combustion engine such as a gasoline engine, an electric vehicle powered by an electric motor, a hybrid vehicle having both an internal combustion engine and an electric motor, and the like. Is.

The automatic driving vehicle M is equipped with a vehicle control device 1, an outside world sensor 2, a navigation device 3, a communication device 4, an HMI (Human Machine Interface) 5, a traveling drive device 6, a braking device 7, a steering device 8, and a vehicle sensor 9. Has been done.

The external world sensor 2, the navigation device 3, the communication device 4, the HMI 5, the traveling drive device 6, the brake device 7, the steering device 8, and the vehicle sensor 9 each have data with and from the vehicle control device 1 via a communication medium. It is configured to be communicable.

The outside world sensor 2 has a function of detecting outside world information about a target including an object or a sign existing around the autonomous driving vehicle M, and includes a camera 21, a radar 22, and a rider 23.
The camera 21 is composed of a plurality of CMOS cameras and CCD cameras, and has a function of capturing an image of the traveling direction of the autonomous driving vehicle M. The captured image of the camera 21 that periodically and repeatedly captures the front, right rear side, and left rear side of the autonomous driving vehicle M in the traveling direction is notified to the outside world recognition unit 11, which will be described later.

The radar 22 irradiates the target including the preceding vehicle to be followed traveling in front of the autonomous driving vehicle M with the radar wave, and receives the radar wave reflected by the target to reach the target. It has a function to acquire distribution information of the target including the direction of the target and the target. As the radar wave, a laser, a microwave, a millimeter wave, an ultrasonic wave or the like can be appropriately used.
Further, for example, three radars 22 are provided on the front side of the autonomous driving vehicle M, and two radars 22 are provided on the rear side. The distribution information of the target by the radar 22 is notified to the outside world recognition unit 11.

The rider 23 (LIDAR: Light Detection and Ranging) detects the presence or absence of a target and the distance to the target by measuring the time required to detect the scattered light with respect to the irradiation light. For example, two riders 23 are provided on the front side of the autonomous driving vehicle M, and three riders 23 are provided on the rear side. The distribution information of the target by the rider 23 is notified to the outside world recognition unit 11.

The navigation device 3 has a function of mapping the current position on a map and providing route guidance to the destination.

The communication device 4 has a function of communicating via a wireless communication medium such as a cellular network, a Wi-Fi network, Bluetooth (registered trademark), or DSRC (Dedicated Short Range Communication).
Further, the communication device 4 is a communication unit in the smart entry key system.

The HMI 5 is a group of devices that are controlled by the HMI control unit 10 described later and for communicating between the vehicle control device 1 and the occupants of the autonomous driving vehicle M.
The HMI 5 may be a sensor that detects the amount of operation of the accelerator pedal, the brake pedal, the steering wheel, and the like.
In the autonomous driving vehicle M of the embodiment, the HMI 5 will be described as including the information panel 51, the in-vehicle microphone 52, and the driver monitor camera 53.

The information panel 51 is controlled by the HMI control unit 10 described later, and is a display unit that displays an image showing the surrounding situation of the vehicle and various information to the occupants in the vehicle interior. The information panel 51 is not particularly limited, but is composed of, for example, an LCD (Liquid Crystal Display), an organic EL (Electroluminescence), a HUD (Head Up Display), or the like.
The configuration of the information panel 51 will be described with reference to FIG.

FIG. 2 is a diagram showing an example of a passenger compartment front structure of an autonomous driving vehicle.
As shown in FIG. 2, the HMI 5 of the autonomous driving vehicle M is provided in the vehicle width direction (Y direction) so as to face the meter panel 54 provided at a position facing the driver's seat and the driver's seat and the passenger seat. It includes a horizontally long information panel 51, a right panel 55R provided on the driver's seat side in the vehicle width direction, and a left panel 55L provided on the passenger seat side in the vehicle width direction.

On the meter panel 54, for example, a speedometer, a tachometer, an odometer, shift position information, lighting status information of lights, and the like are displayed.
On the information panel 51, for example, map information around the automatic driving vehicle M, current position information of the automatic driving vehicle M on the map, and traffic information (including signal information) regarding the current traveling route / planned route of the automatic driving vehicle M are displayed. , Traffic participant information about traffic participants (including pedestrians, bicycles, motorcycles, other vehicles, etc.) existing around the automatic driving vehicle M, various information such as messages sent to the traffic participants, and the like are displayed.

On the right panel 55R, image information on the rear and lower sides of the vehicle on the right side, which is captured by the camera 21 provided on the right side of the autonomous driving vehicle M, is displayed.
On the left panel 55L, image information of the rear and lower parts on the left side of the vehicle captured by the camera 21 provided on the left side of the autonomous driving vehicle M is displayed.

Returning to FIG. 1, the in-vehicle microphone 52 is a sound collecting device that detects the voice of the occupant of the autonomous driving vehicle M.
The driver monitor camera 53 is a digital camera that uses a solid-state image sensor such as a CCD or CMOS, and is provided on an information panel 51, a rear-view mirror, and a steering boss (all not shown), and is at least seated in the driver's seat. Image the driver.
The audio information collected by the in-vehicle microphone 52 and the image information or video information captured by the driver monitor camera 53 are notified to the HMI control unit 10.

The traveling drive device 6 outputs a driving force (torque) for the autonomous driving vehicle M to travel to the drive wheels. The traveling drive device 6 has an internal combustion engine engine and an engine ECU (Electronic Control Unit) or a traveling motor and a motor ECU, and controls traveling driving force according to information notified from a traveling control unit 16 described later.

The braking device 7 is an electric servo braking device or an electronically controlled hydraulic braking device that brakes the autonomous driving vehicle M, and controls the braking force according to the information notified from the traveling control unit 16.

The steering device 8 changes the direction of the steering wheel of the autonomous driving vehicle M according to the information notified from the automatic driving control unit 15 described later or the steering angle of the steering wheel.

The vehicle sensor 9 is a group of sensors that detect various information regarding the vehicle state of the autonomous driving vehicle M.
The vehicle sensor 9 is a seating sensor (not shown) that detects whether or not an occupant is seated in the driver's seat, and a handle grip sensor (not shown) that detects whether or not the driver is gripping the handle. It is composed of a doorknob sensor (not shown) which is a sensor for detecting that an occupant of the automatic driving vehicle M has grasped (touched the door handle) the doorknob of the vehicle door.

Further, the vehicle sensor 9 detects a vehicle speed sensor for detecting vehicle speed, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting an angular velocity around a vertical axis, an orientation sensor for detecting the direction of the vehicle, and an inclination angle of the vehicle, which are not shown. It has various sensors related to automatic operation such as an inclination angle sensor.

Next, the configuration of the vehicle control device 1 will be described.
The vehicle control device 1 is composed of one or more processors or hardware having equivalent functions, and an ECU (Electronic Control Unit) in which a processor such as a CPU (Central Processing Unit), a storage device, and a communication interface are connected by an internal bus. ), Or MPU (Micro-Processing Unit), etc. are combined and configured.
The vehicle control device 1 functions by executing a CPU program.

The automatic driving control unit 15 is a control unit for autonomously controlling and traveling the automatic driving vehicle M without the driver performing a driving operation.
The automatic driving control unit 15 obtains a traveling track of the automatic driving vehicle M based on the action plan information 132 of the storage unit 13, and controls the traveling control unit 16 so that the automatic driving vehicle M travels on this traveling track.

The action plan information 132 includes external world information regarding a target including objects and signs existing around the automatically driven vehicle M recognized by the camera 21, radar 22 and rider 23 by the external world recognition unit 11, and inertia in the own vehicle position recognition unit 12. Position information calculated by navigation, position information acquired from GNSS (Global Navigation Satellite System), lane center and boundary information, road information (three-dimensional position information, etc.), traffic regulation information, address The action plan generation unit (not shown) is generated from the high-precision map information 131 of the storage unit 13 that includes information and the like.

The driving support control unit 14 performs transition control by overriding or handover in manual driving and 5-level automatic driving control.
For example, when the driving support control unit 14 monitors the steering wheel grip sensor (not shown) during level 2 automatic driving (driving support) and detects that the driver has released the steering wheel for a predetermined time, the driving support control unit 14 detects that the driver has released the steering wheel. Cancel the warning or automatic operation.

Based on the operation instruction from the automatic driving control unit 15, the traveling control unit 16 communicates with the traveling driving device 6, the braking device 7, and the steering device 8 to control the traveling of the automatic driving vehicle M.

The HMI control unit 10 displays the control status of the autonomous driving vehicle M by the automatic driving control unit 15 and the driving support control unit 14 on the information panel 51, and acquires information from the in-vehicle microphone 52 and the driver monitor camera 53 for automatic driving control. It is a control unit that notifies the unit 15 and the driving support control unit 14 and controls communication between the vehicle control device 1 and the occupant of the autonomous driving vehicle M.
In the embodiment, a case where a map is displayed on the information panel 51 and communication between the vehicle control device 1 and the occupant of the autonomous driving vehicle M is performed will be described.

The configuration of the HMI control unit 10 will be described with reference to FIG.
The HMI control unit 10 includes an occupant emotion estimation unit 101 that estimates the emotional degree of the occupant in the vehicle, a driving intention recognition unit 102 that recognizes the driving intention of the vehicle, and an occupant emotional degree estimated by the occupant emotion estimation unit 101. The display control unit 103 outputs a map image in which the amount of information to be displayed is varied according to the driving intention recognized by the driving intention recognition unit 102 to the information panel 51.
Here, the occupant's emotional level is a positive or negative value that quantitatively indicates the occupant's emotion and indicates the degree (scale) between emotional uplift / mood excitement / positive and mood calm / depression / negative. Means (positive value means uplifting).

Hereinafter, the HMI control unit 10 (occupant emotion estimation unit 101, driving intention recognition unit 102, and display control unit 103) of the embodiment will be described in detail, but before that, the map image to be displayed on the information panel 51 of the embodiment will be described. The outline of the variable processing of the amount of information will be described with reference to FIG.

FIG. 4 is a diagram showing the relationship between the “passenger's emotional level”, the “clarity of driving intention”, and the map display.
In the map image displayed on the information panel 51, as parameters of the amount of information, the degree of detail of the map such as the display of detailed roads, the three-dimensional display, and the display of non-participating traffic markers, and the detailed information of stores and roads (road width, lane). There is an amount of additional information such as number).

In the display of the map image of the embodiment, for example, the "3D Real Map" display of FIG. 8C, which will be described later, is a map in a display format in which the degree of detail of the map is high and the amount of additional information is large.
Further, the "Simple Map" display of FIG. 10A is a map in a display format in which the degree of detail of the map is low and the amount of additional information is low.
The "Camera View" display of FIG. 8A, which displays the surrounding image by the camera 21 (see FIG. 1), is a map in a display format in which the degree of detail of the map and the amount of additional information are moderate.
The information panel 51 displays a map image in a display format according to the degree of detail of the map and the amount of additional information.

In order to display map images having different display formats, the HMI control unit 10 has the occupant (driver) "the point he wants to go to is decided", "the isotropic direction he wants to go to the restaurant is decided", and "the purpose". The state of "clarity of driving intention" such as "none" is associated with the degree of detail of the map, and the degree of emotion of the occupant (driver) is associated with the amount of additional information on the map.

Specifically, the HMI control unit 10 associates the emotional level of the occupant estimated by the occupant emotion estimation unit 101 with the amount of additional information on the map as the "emotional degree of the occupant", and when the emotional degree of the occupant is large, the emotion Increase the amount of additional information on the map according to the degree.
Further, the HMI control unit 10 associates the movement purpose or the like recognized by the driving intention recognition unit 102 with the degree of detail of the map as "clarity of the driving intention", and when the clarity of the driving intention is large, it is clear. Increase the level of detail of the map according to gender.
Then, the HMI control unit 10 displays, in the display control unit 103, a map image in a display format having an additional information amount and a level of detail corresponding to the “occupant's emotional degree” and the “clarity of the driving intention” on the information panel 51. Output.

Hereinafter, the processing of the HMI control unit 10 (occupant emotion estimation unit 101, driving intention recognition unit 102, and display control unit 103) will be described in detail.
FIG. 5 is a flow chart showing a processing outline of the HMI control unit 10.

In step S51, the occupant emotion estimation unit 101 of the HMI control unit 10 estimates the occupant's emotional degree based on the captured image of the occupant in the vehicle captured by the driver monitor camera 53.
In step S52, the occupant emotion estimation unit 101 obtains the amount of additional information on the map corresponding to the occupant's emotional degree detected in step S51.

In step S53, the driving intention recognition unit 102 of the HMI control unit 10 recognizes the "clarity of the driving intention" of the moving tower of the vehicle based on the set value of the navigation device 3 and the like.
In step S54, the driving intention recognition unit 102 obtains the degree of detail of the map corresponding to the “clarity of driving intention” recognized in step S53.

In step S55, the display control unit 103 of the HMI control unit 10 has a predetermined degree of detail and an amount of additional information from the amount of additional information of the map obtained in step S52 and the amount of additional information of the map obtained in step S54. A map image in the display format is generated from the captured image of the camera 21 (see FIG. 1) or the map information of the vehicle position. The map information of the vehicle position can be obtained from the own vehicle position recognition unit 12, the high-precision map information 131, or the navigation device 3.

In step S56, the display control unit 103 outputs the map image generated in step S55 to the information panel 51.

Next, the detailed processing of the occupant emotion estimation unit 101 and the display control unit 103 related to steps S51, S52, and S55 of FIG. 5 will be described with reference to the flow chart of FIG.
In step S61, the occupant emotion estimation unit 101 uses the driver monitor camera 53 to image the occupant in the vehicle.

In step S62, the occupant emotion estimation unit 101 extracts the occupant's face image from the captured image of the occupant in step S61.
In step S63, the occupant emotion estimation unit 101 analyzes emotions such as anger, disgust, fear, happiness, sadness, and surprise by using a supervised machine learning model for the facial image.
Then, in step S64, the analyzed emotion, that is, the facial expression is estimated as the emotion of happiness and the analyzed value as the emotion level.

Unlike the above steps S62 to S63, the emotional degree may be obtained as follows.
In step S62, the occupant emotion estimation unit 101 extracts the occupant's movement (body movement) from the captured image of the occupant in step S61.
In step S63, the occupant emotion estimation unit 101 analyzes the agility (change amount) of the movement.
Then, in step S64, the magnitude of the agility of the analyzed movement is estimated as the emotional degree.

In step S65, the display control unit 103 determines whether or not the emotional degree estimated from the facial expression or body movement of the occupant in step S64 is equal to or greater than a predetermined threshold value. If the emotional level is equal to or higher than the threshold value (Yes in S65), it is assumed that the facial expression of the occupant or the body movement of the occupant is active, and the process proceeds to step S66. If the emotional level is not equal to or higher than the threshold value (No in S65), it is assumed that the facial expression of the occupant or the body movement of the occupant is negative, and the process proceeds to step S67.

In step S66, the display control unit 103 increases the level of the amount of additional information on the map according to the difference from the threshold value of the emotional degree, and proceeds to step S68.
In step S67, the display control unit 103 reduces the level of the amount of additional information on the map according to the difference from the threshold value of the emotional degree, and proceeds to step S68.
In step S68, the display control unit 103 determines the display format of the map based on the level of the amount of additional information of the map, and generates a map image.

In FIG. 6, the case where the emotion analysis is performed from the facial expression or the movement of the occupant has been described, but the emotion analysis may be performed from the conversation of the occupant. The process of analyzing emotions from the conversation of the occupants will be described with reference to the flow chart of FIG.

In step S71, the occupant emotion estimation unit 101 acquires the voice in the vehicle by the in-vehicle microphone 52.
In step S72, the occupant emotion estimation unit 101 extracts the voice of the occupant (driver) from the acquired voice in the vehicle.

In step S73, the occupant emotion estimation unit 101 analyzes the occupant's voice.
In this voice sentiment analysis, the voice waveform of the acquired voice of the occupant is acoustically analyzed, recognized and decoded into a text string, and this text string is morphologically analyzed to extract a word (morpheme). Then, the emotion value of the text string is calculated by referring to the emotion dictionary for each word.
The number of predetermined keywords included in the recognition-decoded text string may be used as the emotion value, or the number of sentences included in the recognition-decoded text string may be used as the number of utterances and used as the emotion value.

In addition, the conversation between the driver and the occupant other than the driver may be recognized, and the length and number of conversations may be used as emotion values. The driver and the occupants other than the driver can be identified by voice authentication.
Further, the magnitude of the intonation of the voice and the range of the voice may be used as the emotion value.

In step S74, the occupant emotion estimation unit 101 estimates the magnitude of the emotion value obtained from the voice as the emotion level.

Since steps S65 to S68 are the same as those in FIG. 6, description thereof will be omitted here.

Next, an example of a map image in a display format in which the amount of additional information on the map is increased according to changes in the emotional level is shown.
FIG. 8A shows a case where the image around the vehicle captured by the camera 21 (see FIG. 1) is displayed on the information panel 51 (“Camera View” display), and is a display example in a display format in which the amount of additional information is moderate. be.

FIG. 8B is a display example of a display format in which the occupant's emotional level is higher and the amount of additional information is larger than that of the “Camera View” display of FIG. 8A. Specifically, the rectangular recommendation information 51a and 51b are displayed as an overlay on the “Camera View” display of FIG. 8A.
The recommendation information 51a and 51b are related information of famous stores at the destination. In addition, it is related information of the cooking store detected when the emotional level is calculated from the voice of the occupant.

FIG. 8C shows the “3D Real Map” display with high map detail, and the circular recommendation information 51c, 51d, 51e, 51f of the famous store of the destination according to the high emotional level due to approaching the destination. Is displayed as an overlay.
The recommendation information 51c, 51d, 51e, and 51f are displayed in a circle having an area corresponding to the recognition level of the store.

Next, detailed processing of the driving intention recognition unit 102 and the display control unit 103 related to steps S53, S54, and S55 of FIG. 5 will be described with reference to the flow chart of FIG.
In step S91, the driving intention recognition unit 102 determines whether or not the destination has been input to the navigation device 3, and if the destination has been input (Yes in S91), the process proceeds to step S92 and the destination is set. If it has not been input (No in S91), the process proceeds to step S93.
In step S92, the driving intention recognition unit 102 sets the clarity of the driving intention to a predetermined value 1, and proceeds to step S98.

In step S93, the driving intention recognition unit 102 determines whether or not a target genre has been input to the navigation device 3, and if the genre has been input (Yes in S93), the process proceeds to step S94 and the genre is input. If not (No in S93), the process proceeds to step S95.
In step S94, the driving intention recognition unit 102 sets the clarity of the driving intention to a predetermined value 2, and proceeds to step S98.

In step S95, the driving intention recognition unit 102 determines whether or not the navigation device 3 has been operated, and if it is operated (Yes in S95), proceeds to step S96, and if it is not operated (No in S95). , Step S97.
In step S96, the driving intention recognition unit 102 sets the clarity of the driving intention to a predetermined value 3, and proceeds to step S98.

In step S97, the driving intention recognition unit 102 sets the clarity of the driving intention to a predetermined value 4, and proceeds to step S98.
The predetermined value 4 of the clarity of the driving intention is a level indicating the degree of detail of the map when the navigation is not operated in step S95, but the driver does not request the map display of the navigation device 3 and the map. The level of detail is low.

In step S98, the driving intention recognition unit 102 determines the degree of detail of the map according to the clarity of the driving intention set in steps S92, S94, S96, and S97.
That is, the degree of detail of the map is increased in the order of the predetermined value 4, the predetermined value 3, the predetermined value 2, and the predetermined value 1, so that the degree of detail of the map is maximized when the predetermined value is 1.

In step S99, the display control unit 103 generates a map image based on the degree of detail of the map determined in step S98.

Further, in step S99, the display control unit 103 displays the traffic participants around the autonomous driving vehicle M in the map image, and the display control unit 103 has a large degree of detail of the map determined in step S98. The traffic guide may be displayed as the map becomes more detailed, and an image of the traffic participant may be output as the map becomes more detailed.

Further, in step S99, the display control unit 103 determines the distance to the destination input to the navigation device 3 when the destination is input to the navigation device 3 by the driving intention recognition unit 102 (Yes in S91). The display range of the map may be narrowed according to the decrease in the distance to the destination to reduce the amount of information on the map.

Next, an example of a map image in which the degree of detail of the map is changed according to the change in the clarity of the driving intention is shown.
FIG. 10A is a map image of a map in a display format of “Simple Map” in which the degree of detail of the map is low and the amount of additional information is low.
FIG. 10B is a map image in which the driving intention is more clear than the map image of FIG. 10A, and the traffic lights 51g and 51h are added as traffic non-participating object markers to improve the detail of the map.
FIG. 10C is a map image in which the driving intention is more clear than the map image of FIG. 10A, and trees 51j and 51k are added as traffic non-participating object markers to improve the level of detail of the map.

Next, in a bird's-eye view in which the map image displayed on the information panel 51 is displayed so as to look down from diagonally above, an example in which the information amount (detail degree / additional information amount) of the map is changed and displayed will be described.

FIG. 11 is a diagram for explaining the relationship between the virtual viewpoint and the display range of the map image by the bird's-eye view displayed on the information panel 51.
As shown in FIG. 11, when the angle of view in the virtual viewpoint is fixed, the display range A2 of the virtual viewpoint P2 located higher than the virtual viewpoint P1 becomes wider than the display range A1 of the virtual viewpoint P1, and the information panel 51 The amount of information of the map displayed in is larger in the virtual viewpoint P2 than in the virtual viewpoint P1.

Therefore, the display control unit 103 of the HMI control unit 10 of the embodiment displays a bird's-eye view as a map image in which the position of the virtual viewpoint is increased as the amount of information on the map increases according to the flow diagram of FIG.
In step S121, the display control unit 103 determines whether or not the emotional degree is equal to or higher than the predetermined value, and if it is equal to or higher than the predetermined value (Yes in S121), the process proceeds to step S123, and if it is not equal to or higher than the predetermined value, the process proceeds to step S123. (No in S121), the process proceeds to step S122.

In step S122, the display control unit 103 determines whether or not the clarity of the driving intention is equal to or greater than the predetermined value, and if it is equal to or greater than the predetermined value (Yes in S122), the process proceeds to step S123 and is equal to or greater than the predetermined value. If not (No in S122), the process proceeds to step S125.

In step S123, the display control unit 103 obtains the increment amount of the emotional degree and the clarity of the driving intention, and proceeds to step S124.
In step S124, the display control unit 103 raises the virtual viewpoint position according to the increment amount obtained in step S123, and proceeds to step S126.

In step S125, the display control unit 103 sets the virtual viewpoint position to a standard value, and proceeds to step S126.
In step S126, a bird's-eye view map image is generated based on the virtual viewpoint position set in step S124 or step S125.

M Self-driving vehicle 1 Vehicle control device 2 External sensor 21 Camera 22 Radar 23 Rider 3 Navigation device 4 Communication device 5 HMI
51 Information panel (display)
52 In-vehicle microphone 53 Driver monitor camera 54 Meter panel 55R Right panel 55L Left panel 6 Driving drive device 7 Brake device 8 Steering device 9 Vehicle sensor 10 HMI control unit 101 Passenger emotion estimation unit 102 Driving intention recognition unit 103 Display control unit 11 External world recognition Unit 12 Vehicle position recognition unit 13 Storage unit 131 High-precision map information 132 Action plan information 14 Driving support control unit 15 Automatic driving control unit 16 Driving control unit

Claims (9)

A display unit that displays an image showing the surrounding conditions of the vehicle,
The occupant emotion estimation unit that estimates the occupant emotions of the occupants in the vehicle, and the occupant emotion estimation unit.
A driving intention recognition unit that recognizes the driving intention of the vehicle,
A display control unit that outputs the image showing the peripheral situation in which the amount of information is changed according to the occupant's emotion and the driving intention to the display unit.
An autonomous vehicle characterized by being equipped with. The display unit displays a map image and
The occupant emotion estimation unit estimates the occupant emotion level that quantitatively indicates the occupant emotion from the facial expression of the occupant or the movement of the occupant's body captured by the driver monitor camera.
The autonomous driving vehicle according to claim 1, wherein the display control unit outputs a map image in which the amount of additional information is increased to the display unit when the occupant emotion level is higher than a predetermined value. The display unit displays a map image and
The occupant emotion estimation unit acquires the occupant's voice with a sound collector, estimates the occupant emotion degree that quantitatively indicates the occupant's emotion from the sound range or the number of utterances of the voice, and then estimates the occupant's emotion level.
The claim is characterized in that the display control unit outputs a map image in which the amount of additional information is increased to the display unit when the occupant emotion level estimated from the sound range of the voice or the number of utterances is higher than a predetermined value. Item 1. The self-driving vehicle according to item 1. The display unit displays a map image and
The driving intention recognition unit recognizes the driving intention according to the destination or purpose set in the navigation device.
The autonomous driving vehicle according to claim 1, wherein the display control unit outputs a map image in which the degree of detail of the map is increased according to the clarity of the driving intention to the display unit. The map image includes a map image of a predetermined range of the surrounding situation of the vehicle as seen from a predetermined virtual viewpoint.
7. Self-driving vehicle. Map images include images of traffic participants, road conditions and traffic markers in the surrounding conditions.
The autonomous driving vehicle according to claim 4, wherein the display control unit outputs an image of the traffic target as the degree of detail of the map increases, and further outputs an image of a traffic participant. When the driving intention recognition unit recognizes that the destination has been set in the navigation device,
The claim is characterized in that the display control unit calculates a distance to the destination, narrows the display range of the map according to the decrease in the distance, and outputs a map image in which the amount of information on the map is reduced. The self-driving vehicle according to 4. A program that displays an image showing the surrounding conditions of the vehicle.
The vehicle's computer is used as an occupant emotion estimation unit that estimates the occupant's emotions in the vehicle.
A driving intention recognition unit that recognizes the driving intention of the vehicle,
A display control unit that outputs the image showing the surrounding situation in which the amount of information is changed according to the occupant's emotion and the driving intention to the display unit.
A program that functions as. It is an image display method that displays an image showing the surrounding conditions of the vehicle.
The occupant emotion estimation unit estimates the occupant emotions of the occupants in the vehicle.
The driving intention recognition unit recognizes the driving intention of the vehicle and recognizes the driving intention of the vehicle.
An image display method for outputting to the display unit the image showing the surrounding situation in which the amount of information is varied according to the occupant's emotion and the driving intention by the display control unit.

Images