WO2022039146A1 - Display control device - Google Patents

Abstract

This display control device comprises a display unit (S102, S105, S202, S204, S207, S303, S306, S402, S404, S502, S504) and an area determination unit (S101). The display unit generates an ambient image showing a situation around a vehicle having the display control device on the basis of a result of detection by a sensor installed in the vehicle, and displays the ambient image in a display region provided in a position visible to a driver. The area determination unit determines if there is any undetectable area around the vehicle that cannot be detected by the sensor due to blockage by an obstacle. If it is determined by the area determination unit that there is an undetectable area, the display unit displays an indication of the undetectable area on the ambient image.

Description

Display control device Cross-reference of related applications

This international application claims priority based on Japanese Patent Application No. 2020-139249 filed with the Japan Patent Office on August 20, 2020, and Japanese Patent Application No. 2020-139249. The entire contents are incorporated in this international application by reference.

This disclosure relates to a display control device.

There is a display device that displays the situation around the own vehicle so that the driver can recognize the situation around the own vehicle. For example, Patent Document 1 describes a display device in which a display representing the own vehicle is fixedly displayed in the vicinity of the lower end portion in the center of the screen to indicate the situation around the own vehicle.

Japanese Unexamined Patent Publication No. 2018-92290

However, as a result of the inventor's detailed examination, the following issues were found. That is, in the vehicle display device described in Patent Document 1, it is difficult for the driver to grasp to what extent the own vehicle can recognize the situation around the own vehicle in the first place. Generally, in order to recognize the situation around the own vehicle, information such as the presence of another vehicle is detected from a plurality of sensors. Here, for example, by being shielded by an obstacle such as another vehicle traveling around the own vehicle, a region that cannot be accurately detected even within the detection range of the sensor may be temporarily generated. As a result, although there is a possibility that another vehicle exists in the area around the own vehicle, it may not be properly displayed on the display device.

One aspect of the present disclosure provides a display control device that makes it easier for the driver to grasp an area that cannot be accurately detected due to being shielded by an obstacle.

One aspect of the present disclosure is a display control device, which includes a display unit and an area determination unit. The display unit generates a surrounding image showing the surrounding situation of the own vehicle based on the detection result by the sensor mounted on the own vehicle, and displays the surrounding image in the display area provided at the position visible to the driver. It is configured as follows. The area determination unit is configured to determine whether there is an undetectable area around the own vehicle that cannot be detected by the sensor due to being shielded by an obstacle. When the area determination unit determines that there is an undetectable area, the display unit displays a display indicating the undetectable area on the surrounding image.

According to such a configuration, a display control device is provided that makes it easy for the driver to grasp an area that cannot be accurately detected due to being shielded by an obstacle.

It is a block diagram which shows the structure of an in-vehicle system. It is a flowchart which shows the undetectable area display process. It is the figure which looked at the scene where the undetectable area is generated from the upper part of the road. It is an example of the surrounding image displayed in the scene shown in FIG. In the scene shown in FIG. 3, this is an example of a surrounding image displayed when a motorcycle is present around the own vehicle instead of a truck. It is a flowchart which shows the 1st process. 7A, 7B, and 7C are views of a scene in which another vehicle enters an undetectable area and then exits from the undetectable area, as viewed from above the road. 8A, 8B and 8C are examples of surrounding images displayed in the scenes shown in FIGS. 7A, 7B and 7C, respectively. 9A and 9B are diagrams showing how other vehicles disappear from the surrounding image when they go out of the display range of the surrounding image. It is a figure which shows the change of the display mode of another vehicle which disappears from the surrounding image by going out of the display range of the surrounding image. 7 is a diagram showing an example of a change in the display mode of another vehicle entering an undetectable region in the scenes shown in FIGS. 7A, 7B and 7C. 7 is a diagram showing another example of a change in the display mode of another vehicle entering an undetectable area in the scenes shown in FIGS. 7A, 7B and 7C. It is a flowchart which shows the 2nd process. It is a figure which shows the change of the display mode of the undetectable area in a normal time. It is a figure which shows the change of the display mode of the undetectable area when the vehicle speed of the own vehicle is a predetermined vehicle speed threshold value V or more, or the vehicle density is a predetermined density threshold value K or more. It is a flowchart which shows the 3rd process. This is an example of a surrounding image displayed when there is a request for a lane change by the own vehicle to the lane on the undetectable area side. It is a flowchart which shows the 4th process. It is the figure which looked at the scene which the other vehicle traveling in the adjacent lane of the own lane is located relatively behind the own vehicle from the upper part of the road. It is an example of the surrounding image displayed in the scene shown in FIG. It is another example of the surrounding image displayed in the scene shown in FIG.

Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings.

[1. Constitution]
The in-vehicle system 100 shown in FIG. 1 is mounted on a vehicle such as an automobile. The in-vehicle system 100 includes a front camera 2a, a rear camera 2b, a left camera 2c, a right camera 2d, a front ranging sensor 3a, a rear ranging sensor 3b, a left ranging sensor 3c, and a right ranging sensor. It includes 3d, an inter-vehicle communication device 4, a navigation device 5, a display device 6, a vehicle control ECU 30, an automatic driving ECU 40, and an HCU 50. These are connected to, for example, an in-vehicle LAN. The ECU is an abbreviation for Electronic Control Unit. HCU is an abbreviation for Human Machine Interface Control Unit. Hereinafter, the vehicle on which the in-vehicle system 100 is mounted is referred to as a own vehicle.

The front camera 2a is an image sensor for capturing a predetermined front image pickup area in front of the own vehicle. The rear camera 2b is an image sensor for capturing a predetermined rear image pickup region behind the own vehicle. The left camera 2c is an image sensor for capturing a predetermined left side imaging region on the left side of the own vehicle. The right camera 2d is an image sensor for capturing a predetermined right side imaging region on the right side of the own vehicle.

Each camera 2a to 2d outputs each captured image to the in-vehicle LAN. Each captured image is used for processing to detect road markings such as other vehicles and road markings.

The front ranging sensor 3a is a sensor for detecting the relative position of an object existing in a predetermined front ranging area in front of the own vehicle with respect to the own vehicle. The rear ranging sensor 3b is a sensor for detecting the relative position of an object existing in a predetermined rear ranging region behind the own vehicle with respect to the own vehicle. The left ranging sensor 3c is a sensor for detecting the relative position of an object existing in a predetermined left ranging area on the left side of the own vehicle with respect to the own vehicle. The right ranging sensor 3d is a sensor for detecting the relative position of an object existing in a predetermined right ranging region on the right side of the own vehicle with respect to the own vehicle.

In this embodiment, millimeter-wave radar is used as each ranging sensor 3a to 3d. The millimeter wave radar transmits millimeter waves to each ranging area and receives reflected waves from an object existing in each ranging area. The millimeter wave radar detects the relative position and relative velocity of the object existing in each ranging area based on the reception result. In addition, for example, sonar, LIDAR, or the like may be used instead of the millimeter wave radar or together with the millimeter wave radar. In addition, LIDAR is an abbreviation for Light Detection and Ringing.

Each distance measurement sensor 3a to 3d outputs each detected distance measurement information to the in-vehicle LAN. Each distance measurement information is used for processing such as detecting the relative position of another vehicle.

The vehicle-to-vehicle communication device 4 is a communication device for performing wireless communication with other vehicles other than the own vehicle. The vehicle-to-vehicle communication device 4 wirelessly performs vehicle-to-vehicle communication with a plurality of other vehicles existing in the vicinity of the own vehicle. Specifically, the vehicle-to-vehicle communication device 4 receives the status information of the other vehicle from each of the plurality of other vehicles and transmits the status information of the own vehicle by broadcasting. The situation information is information indicating the running situation in the source vehicle such as the current position, the running speed, and the running direction. The situation information may also include, for example, the width of the vehicle, the length of the vehicle, and the like.

The vehicle-to-vehicle communication device 4 determines whether the received status information is the status information received from another vehicle existing in any of the regions around the own vehicle. Specifically, the vehicle-to-vehicle communication device 4 has a predetermined front communication area in front of the own vehicle, a predetermined rear communication area in the rear of the own vehicle, and a left side of the own vehicle based on the information of the current position included in the situation information. It is determined which of the predetermined left-side communication area on the side and the predetermined right-side communication area on the right side of the own vehicle has received the status information from another vehicle existing in which communication area.

The vehicle-to-vehicle communication device 4 outputs each status information with data indicating which communication area exists from another vehicle to the in-vehicle LAN. Each situation information is used for processing for predicting the traveling route of another vehicle and the like.

The navigation device 5 detects the position of the own vehicle specified by latitude and longitude based on a GPS signal or the like received via a GPS antenna. Further, map data is recorded in the navigation device 5. The map data includes detailed road information to the extent that the vehicle is located in which lane on the road based on the position of the vehicle, and various data necessary for displaying the map. GPS is an abbreviation for Global Positioning System.

The navigation device 5 outputs the detected position of the own vehicle and necessary map data to the in-vehicle LAN.

The display device 6 is a display for displaying an image, and is provided at a position that can be visually recognized by the driver of the own vehicle. In this embodiment, a speedometer liquid crystal display is used as the display device 6. The liquid crystal display of the speedometer is installed, for example, at a position facing the driver. The display device 6 displays a surrounding image described later. A CID, a head-up display, or the like may be used as the display device 6. CID is an abbreviation for Center Information Display.

The vehicle control ECU 30 is an electronic control device that controls acceleration / deceleration of the own vehicle and steering control. Examples of the vehicle control ECU 30 include a steering system ECU that controls the operation of a steering system such as steering, a braking system ECU that controls a braking system such as a brake, and a drive system ECU that controls a drive system such as an engine. The vehicle control ECU 30 acquires detection signals output from various sensors such as an accelerator position sensor, a brake pedal force sensor, a steering angle sensor, and a wheel speed sensor mounted on the own vehicle, and acquires a power steering motor, a brake actuator, and an electronically controlled throttle. The control signal is output to each driving control device such as.

The automatic driving ECU 40 realizes an automatic driving function capable of executing a driving operation on behalf of the driver by controlling the vehicle control ECU 30. The automatic driving referred to here is not limited to the automatic driving in which the driving operation is executed on behalf of the driver, and only some driving operations such as acceleration / deceleration operation and steering operation are executed on behalf of the driver. Including automatic driving.

The automatic driving ECU 40 includes the captured images output by the cameras 2a to 2d, the distance measurement information output by the distance measurement sensors 3a to 3d, and the status information output by the vehicle-to-vehicle communication device 4. The driving environment of the own vehicle is recognized based on the position of the own vehicle and the map data output by the navigation device 5. For example, the automatic driving ECU 40 recognizes the shape and moving state of an object around the own vehicle, and recognizes the shape of the road marking around the own vehicle. Then, by combining with the vehicle position and map data of the own vehicle, a virtual space that reproduces the actual driving environment in three dimensions is generated. Further, the automatic driving ECU 40 generates a driving plan for automatically driving the own vehicle by the automatic driving function based on the recognized driving environment. Based on the travel plan, it is decided to perform steering for lane following and lane change, acceleration / deceleration for speed adjustment, and sudden braking for collision avoidance.

The HCU 50 generates a surrounding image showing the surrounding situation of the own vehicle, and executes a display control process for displaying the surrounding image on the display device 6. The HCU 50 is mainly composed of a well-known microcomputer having a CPU, ROM, ROM, flash memory and the like. The HCU 50 may include one microcomputer or a plurality of microcomputers. The HCU 50 executes a program stored in ROM, which is a non-transitional substantive recording medium. When the program is executed, the method corresponding to the program is executed.

The HCU 50 generates a surrounding image based on the driving environment recognized by the automatic driving ECU 40 and the information used for recognizing the driving environment. The information used for recognizing the driving environment includes, for example, the detection results of the cameras 2a to 2d and the distance measuring sensors 3a to 3d, the situation information output by the vehicle-to-vehicle communication device 4, and the information output by the navigation device 5. The location and map data of the own vehicle are included. As will be described later, the surrounding image includes a map background showing the shape of the road around the own vehicle, a display of the own vehicle, a display of another vehicle, and the like. The surrounding image is generated as a virtual viewpoint, for example, a bird's-eye view image from the rear of the own vehicle. The range of the image is set with the detection range of various sensors as a guide.

[2. process]
Hereinafter, the display control process executed by the HCU 50 will be described. The display control process includes the following undetectable area display process and the first to fourth processes.

[2-1. Undetectable area display processing]
The undetectable area display process shown in FIG. 2 is repeatedly executed at a predetermined cycle while the ignition switch of the own vehicle is turned on.

First, in S101, the HCU 50 determines whether or not there is an undetectable area around the own vehicle that cannot be detected by the sensor due to being shielded by an obstacle. For example, in the scene shown in FIG. 3, in the right-hand distance measuring region of the own vehicle 10, the vehicle is shielded by another vehicle 11 traveling in the adjacent lane of the own lane in which the own vehicle 10 travels, so that the vehicle is adjacent to the own lane by two. An undetectable region 20 is generated on the lane of the vehicle. The undetectable region referred to here is a region where the desired detection accuracy cannot be guaranteed by the sensor.

The HCU 50 returns to S101 when it is determined in S101 that there is no undetectable region.

On the other hand, when the HCU 50 determines in S101 that there is an undetectable area, it shifts to S102 and displays a display indicating the undetectable area on the surrounding image. For example, in the scene shown in FIG. 3, as shown in FIG. 4, a display 20a indicating an undetectable area is newly added on the surrounding image in which the display 10a of the own vehicle, the display 11a of another vehicle, and the road shape are already displayed. Make it appear in. Specifically, the display 20a indicating the undetectable region is displayed in a portion corresponding to the undetectable region on the surrounding image. As shown in FIG. 4, for example, the display 20a indicating the undetectable region is displayed in the central portion in the width direction of the lane on the lane two adjacent to the own lane. The display 20a showing the undetectable region has a pattern that makes it easy for the driver to intuitively understand that the existence of another vehicle is ambiguous, for example, a pattern that is painted in light gray like a haze and has a vague outline. The display mode of the display indicating the undetectable area is not limited to this. For example, the color of the display indicating the undetectable area does not have to be gray, and the outline may be clear.

Further, the display indicating the undetectable area may be changed according to the size of another vehicle that becomes an obstacle so that the driver can easily grasp the size of the undetectable area intuitively. For example, when the other vehicle 11 is a truck as shown in FIG. 4, the display 20a indicating the undetectable area is displayed relatively large, and the other vehicle 11 is a motorcycle smaller than the truck as shown in FIG. In some cases, the display 20a indicating the undetectable area may be displayed relatively small.

Subsequently, in S103, the HCU 50 requests the start of the first to fourth processes described later.

Subsequently, in S104, the HCU 50 determines whether or not the undetectable region has disappeared. For example, from the scene shown in FIG. 3, when the other vehicle 11 is greatly separated from the own vehicle 10 and the area that cannot be detected by the sensor disappears due to being shielded by the other vehicle 11 around the own vehicle 10, it is detected. It is determined that the impossible region 20 has disappeared.

The HCU 50 returns to S104 when it is determined in S104 that the undetectable region has not disappeared.

On the other hand, when the HCU 50 determines in S104 that the undetectable region has disappeared, the HCU 50 shifts to S105 and eliminates the display indicating the undetectable region from the surrounding image.

Subsequently, the HCU 50 requests the end of the first to fourth processes requested to be started in S103 in S106, and then ends the undetectable area display process in FIG. 2.

Hereinafter, the first to fourth processes started in S103 in the undetectable area display process of FIG. 2 will be described in order.

[2-2. 1st processing]
First, the first process will be described with reference to the flowchart of FIG. The first process is executed every time the start is requested in the undetectable area display process of FIG.

First, in S201, the HCU 50 determines whether or not there is another vehicle entering the undetectable area.

When the HCU 50 determines in S201 that there is another vehicle entering the undetectable area, it shifts to S202 and eliminates the other vehicle entering the undetectable area from the surrounding image.

As shown in FIG. 7A, when the other vehicle 12 traveling on the lane two adjacent to the own lane is located behind the undetectable region 20, the other vehicle 12 can be detected by the sensor. As shown in 8A, the display 12a of another vehicle is displayed on the surrounding image. However, as shown in FIG. 7B, when the other vehicle 12 enters the undetectable region 20, the sensor cannot detect the other vehicle 12. Therefore, as shown in FIG. 8B, the display 12a of another vehicle cannot be displayed on the surrounding image.

At this time, the HCU 50 shows how the display of the other vehicle disappears from the surrounding image by entering the undetectable area and the display of the other vehicle disappearing from the surrounding image by going out of the display range of the surrounding image. Make it different from how it disappears.

Specifically, as shown in FIGS. 9A and 9B, the HCU 50 instantly disappears from the display 13a of another vehicle that disappears from the surrounding image when it goes out of the display range of the surrounding image. Let me. That is, as shown in FIG. 10, the HCU 50 eliminates the display 13a of the other vehicle at the same time when the display 13a of the other vehicle is determined to be out of the display range at time t1.

On the other hand, regarding the display of the other vehicle that disappears from the surrounding image by entering the undetectable area, the HCU 50 determines that the other vehicle enters the undetectable area at time t2 as shown in FIG. After that, it is slowly faded out from the surrounding image over a certain period of time T1.

In this way, the HCU 50 is longer than the display of the other vehicle that disappears from the surrounding image by entering the undetectable area and the display of the other vehicle that disappears from the surrounding image by going out of the display range of the surrounding image. It will disappear over time. Therefore, another vehicle that disappears from the surrounding image by entering the undetectable area has a long process until it disappears from the surrounding image, and the driver can be alerted to the existence of the other vehicle.

Note that the disappearance of other vehicles that disappear from the surrounding image by entering the undetectable area is not limited to the above. For example, the display 12a of the other vehicle may disappear while blinking, or the display 12a of the other vehicle may gradually disappear from the upper side on the display 20a side indicating the undetectable region in the scene shown in FIG. 8A. The outer frame may disappear after the outer frame other than the display 12a fades out.

Returning to the flowchart of FIG. 6, the HCU 50 also changes the display mode of the display indicating the undetectable region in S202 as shown in FIGS. 8A and 8B. Specifically, the display 20a indicating the undetectable region is changed from the gray haze in the normal state to the red haze.

As described above, the HCU 50 sets the display mode of the display indicating the undetectable area to the mode in which it can be seen that another vehicle exists, specifically, before it is determined that there is another vehicle entering the undetectable area. Change to a display mode that is emphasized rather than the display mode of. Therefore, the driver can grasp that there is a high possibility that another vehicle exists in the undetectable area.

The display mode after the change of the display indicating the undetectable area is not limited to the above. For example, the display indicating the undetectable area may be maintained in a slowly blinking state, or only the outer frame of the display indicating another vehicle may be displayed at an arbitrary position in the undetectable area.

Subsequently, in S203, the HCU 50 determines whether or not another vehicle that has entered the undetectable area has left the undetectable area.

When the HCU 50 determines in S203 that another vehicle has left the undetectable area, the HCU 50 shifts to S204 and causes the display of the other vehicle leaving the undetectable area to appear on the surrounding image. That is, as shown in FIG. 7C, when the other vehicle 12 traveling on the lane two adjacent to the own lane comes out ahead of the undetectable region 20, the other vehicle 12 can be detected by the sensor. Therefore, as shown in FIG. 8C, the display 12a of the other vehicle 12 is displayed on the surrounding image.

At this time, the HCU 50 shows how the display of the other vehicle appears on the surrounding image by leaving the undetectable area and the display of the other vehicle appearing on the surrounding image by entering the display range of the surrounding image. Make it different from how it appears.

Specifically, the HCU 50 displays the display of another vehicle that appears on the surrounding image when it enters the display range of the surrounding image, and the display of the other vehicle that disappears from the surrounding image when it goes out of the display range of the peripheral image. Similar to how it disappears, it is within the display range and appears instantly.

On the other hand, regarding the display of the other vehicle appearing on the surrounding image by leaving the undetectable area, the HCU 50 determines that the other vehicle leaves the undetectable area at time t3 as shown in FIG. After that, it is slowly faded into the surrounding image over a certain period of time T2.

Further, the HCU 50 increases the time T1 required for the display of another vehicle that disappears from the surrounding image to disappear by entering the undetectable area to disappear from the undetectable area of the other vehicle that appears on the surrounding image. The time it takes for the display to appear is longer than T2. This is because it is preferable that the display of the other vehicle that appears on the surrounding image by leaving the undetectable area appears quickly in order to promptly notify the driver of the existence of the other vehicle. As shown in FIG. 12, the display of the other vehicle appearing on the surrounding image by leaving the undetectable area may appear at the same time as it is determined that the other vehicle leaves the undetectable area at time t3. ..

The appearance of other vehicles appearing on the surrounding image by leaving the undetectable area is not limited to the above. For example, the other vehicle may appear while blinking, or the display 12a of the other vehicle may gradually appear from the lower side, which is the display 20a side indicating the undetectable region in the scene shown in FIG. 8C, and the other vehicle may appear. After the outer frame of the display 12a appears, a part other than the outer frame may fade in.

Returning to the flowchart of FIG. 6, the HCU 50 also changes the display mode of the display indicating the undetectable area in S204. Specifically, as shown in FIGS. 8B and 8C, the display 20a indicating the undetectable region is returned from the red haze to the normal gray haze.

On the other hand, when the HCU 50 determines in S203 that the other vehicle has not left the undetectable area, it shifts to S205 and determines whether or not the end request of the first process has been made.

The HCU 50 returns to S203 when it is determined in S205 that the end request of the first process has not been made.

On the other hand, when the HCU 50 determines that the end request of the first process has been made in S205, the HCU 50 ends the first process of FIG.

Returning to S201, when it is determined in S201 that there is no other vehicle entering the undetectable area, the HCU 50 shifts to S206 and determines whether or not there is another vehicle exiting the undetectable area.

When the HCU 50 determines in S206 that there is another vehicle leaving the undetectable area, it shifts to S207, causes the other vehicle leaving the undetectable area to appear on the surrounding image, and shifts to S208.

On the other hand, if it is determined in S206 that there is no other vehicle leaving the undetectable area, the HCU 50 shifts to S208 as it is.

Subsequently, the HCU 50 determines in S208 whether or not the end request for the first process has been made.

The HCU 50 returns to S201 when it is determined in S208 that the end request for the first process has not been made.

On the other hand, when the HCU 50 determines in S208 that the end request for the first process has been made, the HCU 50 ends the first process in FIG.

[2-3. Second process]
Next, the second process will be described with reference to the flowchart of FIG. Similar to the first process, the second process is also executed every time the start is requested in the undetectable area display process of FIG.

First, in S301, the HCU 50 determines whether or not the vehicle speed of the own vehicle is equal to or less than a predetermined vehicle speed threshold value V.

When the HCU 50 determines in S301 that the vehicle speed of the own vehicle is not equal to or less than the predetermined vehicle speed threshold value V, the HCU 50 shifts to S302 and determines whether or not the vehicle density around the own vehicle is equal to or higher than the predetermined density threshold value K. do. Specifically, the HCU 50 determines whether or not the vehicle density of the own lane and the adjacent lane of the own lane is equal to or higher than a predetermined density threshold value K.

When the HCU 50 determines in S302 that the vehicle density is not equal to or higher than the predetermined density threshold value K, the HCU 50 shifts to S308.

On the other hand, the HCU 50 shifts to S303 when it is determined in S301 that the vehicle speed of the own vehicle is equal to or less than the predetermined vehicle speed threshold value V, and when it is determined in S302 that the vehicle density is equal to or higher than the predetermined density threshold value K. .. In S303, the HCU 50 changes the display duration of the display indicating the undetectable area. Specifically, the HCU 50 operates as follows because it is determined that there is no undetectable region after displaying a display indicating the undetectable region on the surrounding image. That is, the HCU 50 determines that the time until the display indicating the undetectable region disappears from the surrounding image is not equal to or less than the predetermined vehicle speed threshold value V and the vehicle density is not equal to or more than the predetermined density threshold value K. Change to be longer than.

Specifically, when the vehicle speed of the own vehicle is not equal to or less than the predetermined vehicle speed threshold value V and the vehicle density is not equal to or higher than the predetermined density threshold value K, the undetectable region is formed at time t4 and t6 as shown in FIG. If it is determined to be present, the HCU 50 operates as follows. That is, the HCU 50 takes T3 for a certain period of time to make a display indicating an undetectable region appear on the surrounding image. After that, when it is determined that the undetectable region has disappeared at times t5 and t7, the HCU 50 takes T4 for a certain period of time to eliminate the display indicating the undetectable region from the surrounding image.

On the other hand, when it is determined that the vehicle speed of the own vehicle is equal to or less than the predetermined vehicle speed threshold value V, or when it is determined that the vehicle density is equal to or higher than the predetermined density threshold value K, the HCU 50 operates as follows. do. That is, as shown in FIG. 15, the HCU 50 takes a constant time to eliminate the display indicating the undetectable region from the surrounding image after it is determined that the undetectable region has disappeared at times t5 and t7. Change to a constant time T5 that is longer than the time T4.

When the vehicle speed of the own vehicle is equal to or less than the predetermined vehicle speed threshold value V, it is assumed that the vehicle is congested or congested, for example, on the expressway. In addition, on general roads, it is assumed that the vehicle is congested or congested, and that the vehicle is stopped due to waiting for a traffic light or the like. When the vehicle density is equal to or higher than the predetermined density threshold value K, it is assumed that the road is congested or congested regardless of whether it is an expressway or a general road.

In such a case, if another vehicle frequently passes around the own vehicle, the undetectable area frequently appears and disappears repeatedly, and the driver may find it annoying.

Therefore, by lengthening the display duration of the undetectable area as described above in such a case, the repetition of the appearance and disappearance of the undetectable area becomes inconspicuous, and the troublesomeness is reduced.

The vehicle speed threshold value V and the density threshold value K may be different depending on whether the road on which the own vehicle travels is an expressway or a general road.

Alternatively, instead of changing the display duration of the display indicating the undetectable area in S303, the HCU 50 changes the display mode of the display indicating the undetectable area from the gray haze in the normal time to a display mode that is less noticeable than in the normal time. For example, it may be changed to a gray haze with higher transparency than usual.

Returning to the flowchart of FIG. 13, the HCU 50 subsequently determines in S304 whether or not the vehicle speed of the own vehicle is equal to or less than the predetermined vehicle speed threshold value V.

When the HCU 50 determines in S304 that the vehicle speed of the own vehicle is not equal to or less than the predetermined vehicle speed threshold value V, the HCU 50 shifts to S305 and determines whether or not the vehicle density around the own vehicle is equal to or higher than the predetermined density threshold value K. do.

When the HCU 50 determines in S305 that the vehicle density is not equal to or higher than the predetermined density threshold value K, the HCU 50 shifts to S306, and when the display duration of the display indicating the undetectable region is changed in S303, the display duration is changed. Change to undo. On the other hand, when the display mode indicating the undetectable region is changed in S303, the HCU 50 is changed so that the display mode indicating the undetectable region is returned to the original display mode. After that, the HCU 50 shifts to S308.

On the other hand, the HCU 50 shifts to S307 when it is determined in S304 that the vehicle speed of the own vehicle is equal to or less than the predetermined vehicle speed threshold value V, or when it is determined in S305 that the vehicle density is equal to or higher than the predetermined density threshold value K. , It is determined whether or not the end request of the second process is made.

The HCU 50 returns to S304 when it is determined in S307 that the end request for the second process has not been made.

On the other hand, when it is determined in S307 that the end request for the second process has been made, the HCU 50 ends the second process in FIG. 13.

Returning to S308, the HCU 50 determines whether or not the end request for the second process has been made.

The HCU 50 returns to S301 when it is determined in S308 that the end request for the second process has not been made.

On the other hand, when it is determined in S307 that the end request for the second process has been made, the HCU 50 ends the second process in FIG. 13.

[2-4. Third process]
Next, the third process will be described with reference to the flowchart of FIG. Similar to the first process and the second process, the third process is also executed every time the start is requested in the undetectable area display process of FIG.

First, in S401, the HCU 50 determines whether or not there is a request for the lane change by the own vehicle to the lane on the undetectable region side. For example, in the scene shown in FIG. 3, the HCU 50 determines whether or not there is a request to change the lane of the own vehicle 10 to the right side ranging region side where the undetectable region 20 exists.

The HCU50 shifts to S406 when it is determined in S401 that there is no request to change lanes.

On the other hand, when the HCU 50 determines that there is a lane change request in S401, the HCU 50 shifts to S402, and the display mode of the display indicating the undetectable area is the display mode before it is determined that there is a lane change request. Change to a more emphasized display mode. Specifically, the HCU 50 blinks the display 20a indicating the undetectable region, as shown in FIG. As a result, it is possible to notify the driver that another vehicle may exist in the undetectable area on the side where the own vehicle intends to change lanes, and call the driver's attention.

The display mode after the change of the display indicating the undetectable area is not limited to the above. For example, the display indicating the undetectable area may be changed to another emphasized color.

Subsequently, the HCU 50 determines whether or not the request for changing lanes has been completed in S403. For example, the HCU 50 determines that the lane change request has been completed when the lane change request is withdrawn without the lane change, or when the lane change is performed and the lane change is completed.

When the HCU 50 determines that the lane change request has been completed in S403, the HCU 50 shifts to S404, changes the display mode of the display indicating the undetectable area to return to the original display mode, and then shifts to S406. .. Specifically, the HCU 50 ends the blinking of the display indicating the undetectable region in S404.

On the other hand, if the HCU 50 determines in S403 that the lane change request has not been completed, it shifts to S405 and determines whether or not the end request for the third process has been made.

The HCU 50 returns to S403 when it is determined in S405 that the end request for the third process has not been made.

On the other hand, when the HCU 50 determines in S405 that the end request for the third process has been made, the HCU 50 ends the third process in FIG.

Returning to S406, the HCU 50 determines whether or not the end request for the third process has been made.

HCU50 returns to S401 when it is determined in S406 that the end request of the third process has not been made.

On the other hand, when the HCU 50 determines in S406 that the end request for the third process has been made, the HCU 50 ends the third process in FIG.

[2-5. 4th process]
Next, the fourth process will be described with reference to the flowchart of FIG. The fourth process is also executed every time the start is requested in the undetectable area display process of FIG. 2, similarly to the first process to the third process.

First, in S501, the HCU 50 determines whether or not at least a part of the display indicating the undetectable area is located outside the display range of the surrounding image on the rear side of the own vehicle.

Here, in the normal state, as shown in FIG. 4, in order to make it easier to grasp the situation on the front side, which is the traveling direction side of the own vehicle, the bird's-eye view so that the display 10a of the own vehicle is located on the lower side in the surrounding image. The virtual viewpoint position of the image is fixed.

Then, for example, as shown in FIG. 19, when the other vehicle 11 traveling in the adjacent lane of the own lane is located relatively behind the own vehicle 10, the portion on the rear side of the undetectable region 20 is the own vehicle 10. It exists in a distant position behind. In this case, as shown in FIG. 20, a portion on the rear side, which is at least a part of the display 20a indicating the undetectable area, is located outside the display range of the surrounding image.

When the HCU 50 determines that at least a part of the display indicating the undetectable area in S501 is located outside the display range of the surrounding image on the rear side of the own vehicle, the HCU 50 shifts to S502 and virtualizes the bird's-eye view image. Change the viewpoint position. Specifically, as shown in FIG. 21, the HCU 50 displays a peripheral image on the rear side of the display 20a in which at least a part of the display 20a indicating an undetectable area is displayed on the peripheral image of the display 10a of the own vehicle. The virtual viewpoint position of the bird's-eye view image is shifted backward so that it is above the position when it is not located outside the range, for example, near the center. As a result, the display 20a indicating the undetectable area is within the display range of the surrounding image, and the driver can easily grasp the existence of the undetectable area behind the own vehicle.

Subsequently, the HCU 50 determines whether or not to end the change of the virtual viewpoint position of the bird's-eye view image in S503. Specifically, the HCU 50 virtually determines the bird's-eye view image based on whether or not the display indicating the undetectable area falls within the display range of the surrounding image even if the virtual viewpoint position of the bird's-eye view image is restored. Determine whether to end the change of the viewpoint position.

When the HCU50 determines in S503 that the change of the virtual viewpoint position of the bird's-eye view image is completed, the HCU 50 shifts to S504, undoes the change of the virtual viewpoint position of the bird's-eye view image, and then shifts to S507. ..

On the other hand, if it is determined in S503 that the change of the virtual viewpoint position of the bird's-eye view image is not completed, the HCU 50 shifts to S505 and determines whether or not the end request of the fourth process is made.

The HCU 50 returns to S503 when it is determined in S505 that the end request for the fourth process has not been made.

On the other hand, when the HCU 50 determines that the end request of the fourth process is made in S505, the HCU 50 shifts to S506, restores the change of the virtual viewpoint position of the bird's-eye view image, and then changes the position of the virtual viewpoint in FIG. 4 Finish the process.

Returning to S501, the HCU 50 skips S502 to S504 when it is determined in S501 that at least a part of the display indicating the undetectable area is not located outside the display range of the surrounding image on the rear side of the own vehicle. Then, it shifts to S507.

In S507, the HCU 50 determines whether or not the end request for the fourth process has been made.

The HCU 50 returns to S501 when it is determined in S507 that the end request for the fourth process has not been made.

On the other hand, when the HCU 50 determines that the end request of the fourth process has been made in S507, the HCU 50 ends the fourth process of FIG.

[3. effect]
According to the embodiment described in detail above, the following effects can be obtained.

(3a) When the HCU 50 determines that there is an undetectable area that cannot be detected by the sensor due to being shielded by an obstacle, the HCU 50 displays a display indicating the undetectable area on the surrounding image. Therefore, the driver can grasp the area that cannot be accurately detected due to being shielded by the obstacle.

(3b) The HCU 50 disappears from the display of the vehicle disappearing from the surrounding image by entering the undetectable area, and disappears from the surrounding image by going out of the display range of the surrounding image. And make it different. Therefore, whether the disappearing vehicle is a vehicle that disappears from the surrounding image by entering the undetectable area that the driver should be careful of, or the vehicle goes out of the display range of the surrounding image that the driver does not have to pay attention to. This allows the driver to intuitively distinguish whether the vehicle disappears from the surrounding image.

(3c) The HCU 50 takes a longer time than the display of the vehicle that disappears from the surrounding image by entering the undetectable area and the display of the vehicle that disappears from the surrounding image by going out of the display range of the surrounding image. To extinguish. Therefore, for a vehicle that disappears from the surrounding image by entering the undetectable region, the process from the surrounding image to disappearing is long, and the driver can be alerted to the existence of the vehicle.

(3d) The HCU 50 has a display of a vehicle appearing on the surrounding image by leaving the undetectable area and a display of a vehicle appearing on the surrounding image by entering the display range of the surrounding image. And make it different. Therefore, the driver can intuitively distinguish whether the appearing vehicle has exited the undetectable area or has appeared on the surrounding image by entering the display range of the surrounding image.

(3e) The HCU 50 causes a vehicle that appears on the surrounding image by leaving the undetectable area to appear over a longer time than a vehicle that appears on the surrounding image by entering the display range of the surrounding image. Therefore, for a vehicle that appears on the surrounding image by leaving the undetectable area, the process until it appears on the surrounding image is long, and the driver can be alerted to the existence of the vehicle.

(3f) In the HCU 50, the time required for the display of the vehicle that disappears from the surrounding image by entering the undetectable area disappears, and the display of the vehicle that appears on the surrounding image by leaving the undetectable area is displayed. Make it longer than it takes to appear. Vehicles that disappear from the surrounding image by entering the undetectable area are preferably extinguished over a relatively long time in order to alert the driver about their existence, whereas they leave the undetectable area. This is because it is preferable that the vehicle appearing on the surrounding image appears quickly in order to inform the driver of its existence quickly.

(3g) When it is determined that there is a vehicle entering the undetectable area, the HCU 50 displays the display mode indicating the undetectable region before it is determined that there is a vehicle entering the undetectable region. Change to a different display mode. Therefore, the driver can grasp that there is a high possibility that another vehicle exists in the undetectable area.

(3h) When it is determined that the vehicle speed of the own vehicle is equal to or less than a predetermined vehicle speed threshold value, the HCU 50 determines that there is no undetectable region after displaying a display indicating the undetectable region on the surrounding image. The time until the display indicating the undetectable area disappears from the surrounding image is changed so as to be longer than when the vehicle speed of the own vehicle is not equal to or less than the predetermined vehicle speed threshold value. Alternatively, when it is determined that the vehicle speed of the own vehicle is equal to or less than the predetermined vehicle speed threshold value, the HCU 50 is more conspicuous than the case where the vehicle speed of the own vehicle is not equal to or less than the predetermined vehicle speed threshold value. Change to no display mode. Therefore, for example, in a traffic jam or a crowded scene, the undetectable area is conspicuous even if the appearance and disappearance of the undetectable area are frequently repeated due to the frequent passage of other vehicles around the own vehicle. Since there is no such thing, the hassle is reduced.

(3i) When it is determined that the vehicle density is equal to or higher than a predetermined density threshold value, the HCU 50 determines that there is no undetectable region after displaying the undetectable region on the surrounding image, thereby forming the undetectable region. The time until the indicated display disappears from the surrounding image is changed so that the vehicle density is longer than when the vehicle density is not equal to or higher than the predetermined density threshold value. Alternatively, the HCU 50 makes the display mode of the display indicating the undetectable region less conspicuous than the case where the vehicle density is not equal to or higher than the predetermined density threshold value when it is determined that the vehicle density is equal to or higher than the predetermined density threshold value. change. Therefore, as in the above (3h), even if the appearance and disappearance of the undetectable region are frequently repeated, the undetectable region is inconspicuous and the troublesomeness is reduced.

(3j) When it is determined that there is a request for lane change by the own vehicle to the lane on the undetectable area side, the HCU 50 determines that there is a request for lane change in the display mode of the display indicating the undetectable area. Change to a more emphasized display mode than the previous display mode. Therefore, it is possible to notify the driver that another vehicle may exist in the undetectable area on the side where the own vehicle intends to change lanes, and call the driver's attention.

(3k) The HCU 50 is displayed on the lower side of the surrounding image when it is determined that at least a part of the display indicating the undetectable area is located outside the display range of the surrounding image on the rear side of the own vehicle. The position of the display of the own vehicle on the surrounding image is higher than the position when at least a part of the display indicating the undetectable area is not located outside the display range of the surrounding image on the rear side of the own vehicle. In addition, the virtual viewpoint position of the bird's-eye view image is changed. Therefore, even if the undetectable area is behind the own vehicle, the driver can easily grasp the existence of the undetectable area on the surrounding image.

In this embodiment, the HCU 50 corresponds to the display control device. Further, S102, S105, S202, S204, S207, S303, S306, S402, S404, S502, S504, and S506 correspond to the processing as the display unit. Further, S101 corresponds to the processing as the area determination unit, S201 corresponds to the processing as the approach determination unit, S301 and S304 correspond to the processing as the vehicle speed determination unit, and S302 and S305 serve as the vehicle density determination unit. Corresponds to processing. S401 corresponds to the processing as the lane change determination unit, and S501 corresponds to the processing as the position determination unit.

[4. Other embodiments]
Although the embodiments of the present disclosure have been described above, it is needless to say that the present disclosure is not limited to the above-described embodiments and can take various forms.

(4a) In the above embodiment, when the HCU 50 determines that there is an undetectable region, a display indicating the undetectable region is displayed on the surrounding image. However, when it is determined that there is an undetectable area and there is a vehicle entering the undetectable area, the HCU 50 may display a display indicating the undetectable area on the surrounding image. Further, when it is determined that there is an undetectable area and it is determined that there is a request for the own vehicle to change lanes to the lane on the undetectable area side, the HCU 50 displays a display indicating the undetectable area on the surrounding image. It may be displayed in.

(4b) In the above embodiment, when the HCU 50 determines that at least a part of the display indicating the undetectable region is located outside the display range of the surrounding image on the rear side of the own vehicle, as follows. It worked. That is, the HCU 50 shifts the virtual viewpoint position of the bird's-eye view image backward so that all of the display 20a indicating the undetectable area is within the display range of the surrounding image. However, the HCU 50 may shift the virtual viewpoint position of the bird's-eye view image so that not all but a part of the display 20a indicating the undetectable area is within the display range of the surrounding image.

(4c) Further, in the above embodiment, the HCU 50 is at least one case where it is determined that the vehicle speed of the own vehicle is equal to or less than the predetermined vehicle speed threshold value V or the vehicle density is determined to be equal to or higher than the predetermined density threshold value K. In a certain range around the own vehicle, for example, when there are a plurality of undetectable areas on the same lane, the plurality of undetectable areas may be integrated and displayed. That is, when the road is congested or congested, it is assumed that other vehicles as obstacles are continuously present on the road. In such a case, the HCU 50 may integrate a plurality of undetectable regions corresponding to each of a plurality of connected vehicles and display them on a surrounding image.

(4d) In the above embodiment, the HCU 50 divides the periphery of the own vehicle into four regions of front, rear, left side, and right side, and acquires detection results from each sensor corresponding to the four regions. .. However, the division of the detection range by the sensor is not limited to this. For example, the HCU 50 may divide the periphery of the own vehicle into two and acquire the detection result from each sensor corresponding to the two regions.

(4e) The functions of one component in the above embodiment may be dispersed as a plurality of components, or the functions of the plurality of components may be integrated into one component. Further, a part of the configuration of the above embodiment may be omitted. Further, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment.

(4f) In the present disclosure, in addition to the above-mentioned display control device, a system having the display control device as a component, a program for operating a computer as the display control device, and a non-transitional substantive record recording this program. It can be realized in various forms such as a medium and a display control method.

Claims (15)

1. A surrounding image showing the surrounding situation of the own vehicle is generated based on the detection result by the sensor mounted on the own vehicle, and the surrounding image is displayed in a display area provided at a position visible to the driver. The configured display units (S102, S105, S202, S204, S207, S303, S306, S402, S404, S502, S504) and
   An area determination unit (S101) configured to determine whether or not there is an undetectable area that cannot be detected by the sensor due to being shielded by an obstacle around the own vehicle.
   Equipped with
   The display unit is a display control device that displays a display indicating the undetectable area on the surrounding image when the area determination unit determines that the undetectable area exists.
2. The display unit displays how the display of the vehicle disappears from the surrounding image by entering the undetectable area and the display of the vehicle disappearing from the surrounding image by going out of the display range of the peripheral image. The display control device according to claim 1, which is different from the way in which the image disappears.
3. The display unit displays the display of the vehicle that disappears from the surrounding image by entering the undetectable region, and the display of the vehicle that disappears from the surrounding image by going out of the display range of the peripheral image. The display control device according to claim 2, which is extinguished over a longer period of time.
4. The display unit displays how the vehicle appears on the surrounding image by exiting the undetectable area and the vehicle appearing on the surrounding image by entering the display range of the peripheral image. The display control device according to any one of claims 1 to 3, which is different from the appearance of the above.
5. The display unit has a longer time than the vehicle appearing on the surrounding image by entering the display range of the peripheral image of the vehicle appearing on the surrounding image by leaving the undetectable region. The display control device according to claim 4, wherein the display control device is made to appear.
6. The display unit increases the time required for the display of the vehicle that disappears from the surrounding image by entering the undetectable area to disappear from the undetectable area, and the vehicle that appears on the surrounding image by leaving the undetectable area. The display control device according to any one of claims 1 to 5, wherein the display is made longer than the time required for the display to appear.
7. Further, an approach determination unit (S201) configured to determine whether or not there is a vehicle entering the undetectable area is provided.
   When the display unit determines that there is a vehicle entering the undetectable area by the approach determination unit, the display mode of the display indicating the undetectable area is displayed on the undetectable area by the approach determination unit. The display control device according to any one of claims 1 to 6, wherein the display mode is changed to a display mode different from the display mode before it is determined that there is an approaching vehicle.
8. Further, an approach determination unit (S201) configured to determine whether or not there is a vehicle entering the undetectable area is provided.
   When the area determination unit determines that the display unit has the undetectable area and the approach determination unit determines that the vehicle enters the undetectable area, the display unit determines the undetectable area. The display control device according to any one of claims 1 to 6, wherein the indicated display is displayed on the surrounding image.
9. Further, a vehicle speed determination unit (S301, S304) configured to determine whether the vehicle speed of the own vehicle is equal to or lower than a predetermined vehicle speed threshold value is provided.
   When the vehicle speed determination unit determines that the vehicle speed of the own vehicle is equal to or lower than the predetermined vehicle speed threshold value, the display unit displays a display mode indicating the undetectable region, wherein the vehicle speed of the own vehicle is the same. The display control device according to any one of claims 1 to 8, wherein the display mode is changed to be less conspicuous than when the vehicle speed is not equal to or less than a predetermined vehicle speed threshold value.
10. Further, a vehicle speed determination unit (S301, S304) configured to determine whether the vehicle speed of the own vehicle is equal to or lower than a predetermined vehicle speed threshold value is provided.
    When the vehicle speed determination unit determines that the vehicle speed of the own vehicle is equal to or lower than the predetermined vehicle speed threshold value, the display unit displays a display indicating the undetectable region on the surrounding image, and then displays the region. The time until the display indicating the undetectable area disappears from the surrounding image due to the determination by the determination unit that the undetectable area does not exist is longer than the time when the vehicle speed of the own vehicle is not equal to or less than the predetermined vehicle speed threshold value. The display control device according to any one of claims 1 to 8, wherein the display control device is changed so as to be longer.
11. Further, a vehicle density determination unit (S302, S305) configured to determine whether the vehicle density around the own vehicle is equal to or higher than a predetermined density threshold value is further provided.
    When the vehicle density determination unit determines that the vehicle density is equal to or higher than the predetermined density threshold value, the display unit displays a display mode indicating the undetectable region, and the vehicle density is the predetermined density. The display control device according to any one of claims 1 to 10, wherein the display mode is changed to be less conspicuous than when the display mode is not equal to or higher than the threshold value.
12. Further, a vehicle density determination unit (S302, S305) configured to determine whether the vehicle density around the own vehicle is equal to or higher than a predetermined density threshold value is further provided.
    When the vehicle density determination unit determines that the vehicle density is equal to or higher than the predetermined density threshold value, the display unit displays the undetectable region on the surrounding image and then the region determination unit. The time until the display indicating the undetectable region disappears from the surrounding image due to the determination that there is no undetectable region is changed so as to be longer than when the vehicle density is not equal to or higher than the predetermined density threshold value. The display control device according to any one of claims 1 to 10.
13. Further, a lane change determination unit (S401) configured to determine whether or not there is a request for a lane change by the own vehicle to the lane on the undetectable region side is provided.
    When the lane change determination unit determines that the lane change request is made, the display unit displays the display mode indicating the undetectable area before it is determined that the lane change request is made. The display control device according to any one of claims 1 to 12, wherein the display mode is changed to a display mode that is emphasized rather than the display mode.
14. Further, a lane change determination unit (S401) configured to determine whether or not there is a request for a lane change by the own vehicle to the lane on the undetectable region side is provided.
    When the area determination unit determines that the display unit has the undetectable area and the lane change determination unit determines that there is a request for the lane change, the display unit displays a display indicating the undetectable area. The display control device according to any one of claims 1 to 12, which is displayed on the surrounding image.
15. The display unit generates the surrounding image as a bird's-eye view image from the rear of the own vehicle.
    Further, a position determination unit (S501) configured to determine whether at least a part of the display indicating the undetectable area is located outside the display range of the surrounding image on the rear side of the own vehicle is further provided.
    When it is determined by the position determination unit that at least a part of the display indicating the undetectable area is located outside the display range of the surrounding image on the rear side of the own vehicle, the display unit is the periphery. The position of the display of the own vehicle displayed on the lower side of the image on the surrounding image is such that at least a part of the display indicating the undetectable area is outside the display range of the surrounding image on the rear side of the own vehicle. The display control device according to any one of claims 1 to 14, wherein the virtual viewpoint position of the bird's-eye view image is changed so as to be above the position when the position is not located.

Images