JP7349888B2 - Driving support method and in-vehicle device - Google Patents

Description

The present invention relates to a driving support method for supporting the driving operations of a driver in a vehicle, a roadside device installed on a road, and an in-vehicle device installed in a vehicle.

In recent years, technological development has been progressing toward the practical application and widespread use of automatic driving systems that support the driving of self-driving cars. This self-driving car is controlled to travel along a preset travel route, but if an obstacle such as a traffic accident (an event that obstructs traffic) occurs in the direction of travel of the vehicle, the It is desirable to change the driving route to a route that detours around the point where the event occurred.

However, depending on the situation of the failure event, there may be no need to change the route, and it may be difficult for the automatic driving ECU to determine whether or not a route change is necessary. For example, if the traffic accident is relatively minor, it is possible to pass by the spot where the traffic accident occurred, and in this case there is no need to change the route. Therefore, it is desirable to have the driver decide whether or not to change the route. In particular, when a display device mounted on a vehicle displays an image of a point ahead in the direction of travel of the vehicle, the driver can instantly check the specific situation at that point and decide whether or not to change the route. can be done quickly.

Conventionally, as a technology for displaying images of a point ahead in the direction of travel of the vehicle on a display device mounted on such a vehicle, a display device mounted on the vehicle has conventionally been used to display images that are located around the vehicle's travel route. 2. Description of the Related Art There is a known technique for acquiring and displaying a video of a location from a distribution server (see Patent Document 1).

Patent No. 4004798

Now, a safe driving support wireless system using an ITS (Intelligent Transport System) uses communication between a roadside unit installed on the road and a vehicle (road-vehicle communication) to monitor road conditions. The system provides various information to the vehicle. Therefore, it is conceivable to mount a camera on the roadside device and provide images of the road from the roadside device to the vehicle.

However, when images from preset shooting points, that is, images shot by roadside devices installed at each intersection, are distributed to vehicles as in conventional technology, many images are displayed on the vehicle's display device. This makes it impossible for drivers to immediately distinguish between images showing troublesome events such as traffic accidents. For this reason, there is a problem in that the driver cannot quickly confirm the specific situation of the trouble event that should be noted, and the response to the trouble event, such as determining whether or not a route change is necessary, is delayed.

Therefore, the present invention provides driving support that allows a driver to immediately identify a video showing a cautionary event occurring on the road, and quickly confirm the specific situation of the cautionable event. The main object of the present invention is to provide a method, a roadside device, and a vehicle-mounted device.

In the driving support method of the present invention, a roadside device installed on a road photographs the road around the device using a camera, an in-vehicle device mounted on a vehicle acquires the image photographed by the roadside device, and A display device mounted on a vehicle displays the image, and when the roadside device detects a specific event occurring on a road around the device, the display device highlights the image in which the specific event is displayed. If the in-vehicle device determines that the driver of the own vehicle is looking aside, the display device displays the image displayed at the edge of the driver's field of vision among the images currently being displayed. The configuration is such that it is highlighted .

Further, the in-vehicle device of the present invention receives an image captured by the roadside device of a surrounding road from the roadside device directly or via a distribution device, and when a specific event is detected in the roadside device. , a communication unit that receives information to that effect directly or via another roadside device, displays the image on a display device mounted on the vehicle, and receives information that the specific event has been detected. , a processor that highlights the image showing the specific event, and when the processor determines that the driver of the own vehicle is inattentive, the processor highlights the driver's field of view in the image being displayed. The image displayed at the edge of the screen is highlighted .
Further, the in-vehicle device of the present invention receives an image captured by the roadside device of a surrounding road from the roadside device directly or via a distribution device, and when a specific event is detected in the roadside device. , a communication unit that receives information to that effect directly or via another roadside device, displays the image on a display device mounted on the vehicle, and receives information that the specific event has been detected. , a processor that highlights the video showing the specific event, and the display device is an AR display that displays an AR screen on which a virtual object is superimposed on the real space that the driver can see, The image is displayed as the virtual object on a space outside the vehicle that can be seen through the glass, and further includes a navigation display that displays a navigation screen in which the traveling direction of the vehicle is drawn on a map, and the image is drawn on the image on the AR screen. The mark image of the moving body and the mark image of the moving body drawn on the map on the navigation screen are configured to have the same color.

According to the present invention, a driver can quickly identify a video showing a specific event and view the video, thereby quickly confirming the specific situation of the specific event that occurred on the road.

Overall configuration diagram of the driving support system according to the first embodiment An explanatory diagram showing an overview of the operations of the vehicle 1 and the roadside device 6 according to the first embodiment An explanatory diagram showing an example of a navigation screen displayed on the navigation display 4 mounted on the vehicle 1 according to the first embodiment An explanatory diagram showing a case where a failure event occurs on an example of the navigation screen according to the first embodiment An explanatory diagram showing another example of the navigation screen according to the first embodiment An explanatory diagram showing a case where the detection status of a moving object changes in another example of the navigation screen according to the first embodiment A block diagram showing a schematic configuration of the roadside machine 6 according to the first embodiment A block diagram showing a schematic configuration of a vehicle 1 according to a first embodiment A sequence diagram showing an overview of the operations of the in-vehicle terminal 2, roadside device 6, and server 7 according to the first embodiment A flow diagram showing the operation procedure of the roadside machine 6 according to the first embodiment A flow diagram showing the operation procedure of the in-vehicle terminal 2 and automatic driving ECU 3 according to the first embodiment An explanatory diagram showing an overview of the operations of the vehicle 1 and the roadside machine 6 according to the first modification of the first embodiment An explanatory diagram showing a navigation screen displayed on the navigation display 4 according to the first modification of the first embodiment An explanatory diagram showing an example of a navigation screen by the navigation display 4 mounted on the vehicle 1 and a front AR screen 201 by the front AR display 35 according to the second modification of the first embodiment An explanatory diagram showing a case where a failure event occurs on an example of the navigation screen and front AR screen according to the second modification of the first embodiment A block diagram showing a schematic configuration of a vehicle 1 according to a second modification of the first embodiment An explanatory diagram showing an overview of the operations of the vehicle 1 and the roadside device 6 according to the second embodiment An explanatory diagram showing a navigation screen on the navigation display 4 mounted on the vehicle 1 according to the second embodiment A block diagram showing a schematic configuration of a vehicle 1 according to a second embodiment A flow diagram showing the operation procedure of the in-vehicle terminal 2 according to the second embodiment An explanatory diagram showing an overview of the operation of the vehicle 1 according to the third embodiment An explanatory diagram showing a navigation screen on the navigation display 4 mounted on the vehicle 1 according to the third embodiment A block diagram showing a schematic configuration of a vehicle 1 according to a third embodiment A flow diagram showing the operation procedure of the in-vehicle terminal 2 according to the third embodiment An explanatory diagram showing a navigation screen by the navigation display 4 mounted on the vehicle 1 and a front AR screen 201 by the front AR display 35 according to the fourth embodiment A block diagram showing a schematic configuration of a vehicle 1 according to a fourth embodiment A flow diagram showing the operation procedure of the in-vehicle terminal 2 according to the fourth embodiment

A first invention made to solve the above problem is that a roadside device installed on a road photographs the road around the device using a camera, and an in-vehicle device mounted on a vehicle captures the image taken by the roadside device. the display device mounted on the vehicle displays the video, and when the roadside device detects a specific event occurring on the road around the roadside device, the display device displays the specific event. When the in-vehicle device determines that the driver of the own vehicle is inattentive by highlighting the video showing the event, the display device displays the video at the edge of the driver's field of vision in the video being displayed. The configuration is such that the image displayed on the screen is highlighted .

According to this, a driver can quickly identify a video showing a specific event and view the video to quickly confirm the specific situation of the specific event that occurred on the road. Further, according to this, when the driver is looking aside, it is possible to direct the driver's line of sight to an image that the driver cannot immediately see and draw his attention to it.

Further, in a second invention, the in-vehicle device acquires a plurality of the images taken by the roadside device installed at each of a plurality of intersections on the travel route of the vehicle, and the display device acquires a plurality of images taken by the roadside device installed at each of a plurality of intersections on the travel route of the vehicle. Each of the images is displayed at a position on the screen corresponding to the position of the roadside device that captured the image.

According to this, the driver can quickly check the conditions of multiple intersections ahead of the vehicle's travel route.

Further, in a third invention, the in-vehicle device or the roadside device sets a display priority for each of the plurality of videos, and the display device displays the video based on the priority. .

According to this, the visibility of high-priority images increases, allowing the driver to make even more appropriate decisions.

Further, in a fourth aspect of the present invention, the display device is configured to display the video having a higher priority in a larger size than the video having a lower priority.

According to this, the driver's line of sight can be guided to a video with a high priority.

In a fifth aspect of the present invention, the display device displays the video having a high priority over a video display frame of the video having a low priority.

According to this, the driver sees less video. Therefore, you can quickly check the specific situation of a moving object or obstacle event that requires attention using images that show the moving object or obstacle event that requires attention, without being bothered by images that require little attention. .

Further, in the sixth invention, the in-vehicle device or the roadside device sets the priority for each video based on the importance of the event captured in the video and the distance from the vehicle to the shooting point. composition.

According to this, images from points near the vehicle are displayed with priority over images from points far from the vehicle, and images that show important events are displayed with priority over images that do not show important events. Therefore, the driver can make an appropriate judgment according to the degree of emergency.

In a seventh aspect of the present invention, the display device displays a mark image representing the moving direction of the moving body on the video image.

According to this, the driver can instantly recognize the moving direction of the moving object and easily determine whether or not the moving object requires attention.

Further, in an eighth aspect of the present invention, the specific event is an obstacle event that becomes an obstacle to vehicle passage.

According to this, the driver can quickly determine whether or not a route change is necessary.

Further, in a ninth aspect of the invention, when the in-vehicle device detects a moving object existing around the own vehicle and determines that there is a possibility of transitioning to a highly dangerous state depending on the behavior of the own vehicle, , the display device is configured to display an image of the moving object taken by a camera mounted on the own vehicle.

According to this, by viewing the video of the moving object, the driver can immediately recognize that the vehicle is transitioning into a highly dangerous state, and can quickly perform driving operations to avoid the danger.

In a tenth invention, the roadside device transmits the video to a distribution device, and the in-vehicle device receives the video from the distribution device.

According to this, since video is distributed from the roadside device to the vehicle-mounted device via the distribution device, it is possible to reduce the load of road-to-vehicle communication between the roadside device and the vehicle-mounted device.

Further, in an eleventh invention, the roadside device transmits video link information of its own device to the in-vehicle device directly or via another roadside device, and the in-vehicle device transmits video link information of its own device to the in-vehicle device based on the video link information. and receives the video from the distribution device.

According to this, it is possible to reliably receive a video shot by a required roadside device from the distribution device.

Further, in the twelfth invention, a roadside device installed on a road photographs the road around the device using a camera, and an in-vehicle device mounted on a vehicle acquires the image photographed by the roadside device, and A display device mounted on a vehicle displays the image, and when the roadside device detects a specific event occurring on a road around the device, the display device highlights the image in which the specific event is displayed. The display device is an AR display that displays an AR screen in which a virtual object is superimposed on the real space visible to the driver, and the image is displayed as the virtual object on the outside space of the vehicle that is visible to the driver through the window glass. The display further includes a navigation display that displays a navigation screen in which the traveling direction of the vehicle is drawn on a map, and a mark image of a moving object drawn on the image on the AR screen and a mark image drawn on the map on the navigation screen. and the mark image of the moving object that has been marked are of the same color .

According to this, the image can be displayed in a large size near the actual position, so the visibility of the image can be improved. Further, according to this, the driver can immediately recognize the correspondence between the moving object displayed on the navigation screen and the moving object shown in the image of the AR screen.

Further , the thirteenth invention is configured such that the AR screen displays a guide image that associates the video displayed on the AR screen with the mark image of the moving object displayed on the navigation screen. do.

According to this, the driver can immediately recognize the correspondence between the image displayed on the front AR screen and the mark image of the moving object displayed on the navigation screen.

Further, the fourteenth invention is such that when a roadside device receives an image of a surrounding road from the roadside device directly or via a distribution device, and when a specific event is detected in the roadside device, A communication unit that receives information to that effect directly or via another roadside device, displays the image on a display device mounted on the vehicle, and receives information that the specific event has been detected, a processor that highlights the image showing the specific event, and when the processor determines that the driver of the own vehicle is inattentive, the processor selects a part of the image that shows the driver's field of vision out of the image being displayed. The image displayed at the edge is displayed in a highlighted manner.

According to this, similar to the first invention, a driver can immediately identify a video showing a specific event and, by viewing the video, quickly determine the specific situation of the specific event that occurred on the road. It can be confirmed.

Further, the fifteenth invention is such that when a roadside device receives an image of a surrounding road from the roadside device directly or via a distribution device, and when a specific event is detected in the roadside device, A communication unit that receives information to that effect directly or via another roadside device, displays the image on a display device mounted on the vehicle, and receives information that the specific event has been detected, a processor that highlights the image showing the specific event, and the display device is an AR display that displays an AR screen in which a virtual object is superimposed on the real space visible to the driver, and The image is displayed as the virtual object in the space outside the vehicle that can be seen beyond the vehicle, and further includes a navigation display that displays a navigation screen on which the traveling direction of the vehicle is drawn on a map, and the movement drawn on the image on the AR screen is provided. The mark image of the body and the mark image of the moving body drawn on the map on the navigation screen are configured to have the same color .

According to this, like the twelfth invention, the image can be displayed in a large size near the actual position, so the visibility of the image can be improved. Further, the driver can immediately recognize the correspondence between the moving object displayed on the navigation screen and the moving object shown in the image on the AR screen.

Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is an overall configuration diagram of a driving support system according to a first embodiment.

This driving support system supports the driver of the vehicle 1 in emergency driving operations by providing an on-vehicle terminal 2 with an image taken of a road out of line of sight from the vehicle 1 (self-driving vehicle). This driving support system includes an in-vehicle terminal 2 (on-vehicle device), an automatic driving ECU 3 (driving control device), a navigation display 4 (display device) mounted on a vehicle 1, and a pedestrian terminal 5 (walking device) carried by a pedestrian. A roadside device 6 (roadside device) installed on a road, and a server 7 (distribution device).

ITS communication is performed between the in-vehicle terminal 2, the pedestrian terminal 5, and the roadside device 6. This ITS communication is wireless communication using a frequency band (for example, a 700 MHz band or a 5.8 GHz band) employed in a safe driving support wireless system using an ITS (Intelligent Transport System). In this ITS communication, messages containing required information such as location information of the vehicle 1 and pedestrians are sent and received.

It should be noted that among these ITS communications, the communication performed between the vehicle-mounted terminals 2 is referred to as vehicle-to-vehicle communication, and the communication performed between the roadside device 6 and the vehicle-mounted terminal 2 is referred to as road-to-vehicle communication. Furthermore, the in-vehicle terminal 2 and the roadside device 6 can also perform ITS communication (pedestrian-to-vehicle communication, road-to-pedestrian communication) with the pedestrian terminal 5.

The in-vehicle terminal 2 sends and receives messages including position information, etc. to and from other in-vehicle terminals 2 through ITS communication (vehicle-to-vehicle communication), determines the risk of collision between the vehicles 1, and determines the risk of collision. If there is, an action is taken to alert the driver. Note that the alerting operation is preferably performed using a car navigation device (not shown) connected to the in-vehicle terminal 2. In addition, the vehicle-mounted terminal 2 sends and receives messages to and from the pedestrian terminal 5 through ITS communication (pedestrian-to-vehicle communication), and determines the risk of collision between the pedestrian and the vehicle 1.

The automatic driving ECU 3 detects obstacles around the vehicle 1 based on sensor outputs, detects the state of the vehicle 1, and controls the running of the vehicle 1.

The roadside device 6 notifies the in-vehicle terminal 2 and the pedestrian terminal 5 of the presence of vehicles 1 and pedestrians located around the device through ITS communication (road-to-vehicle communication, road-to-pedestrian communication). This makes it possible to prevent collisions when making right or left turns at intersections that are out of line of sight. In addition, the roadside device 6 also distributes traffic information to the in-vehicle terminal 2 and pedestrian terminal 5.

Furthermore, the roadside device 6 includes an antenna 11, a radar 12, and a camera 13. The antenna 11 transmits and receives radio waves for ITS communication. The radar 12 detects moving objects (pedestrians and vehicles 1) existing on roads around the radar 12 by detecting reflected waves of emitted radio waves, and measures the direction and distance of the moving objects. The camera 13 photographs the roads around the device itself. Note that, in addition to the visible light camera, a night vision camera (infrared light camera) may be used as the camera 13. This makes it possible to obtain highly visible images even at locations with little illumination light at night.

Furthermore, the roadside device 6 communicates (roadside communication) with other roadside devices 6 installed in the vicinity via a dedicated roadside network (wired or wireless) or a network such as cellular communication. .

The server 7 is connected to the roadside machine 6. This server 7 receives the video taken by the camera 13 of the roadside device 6 from the roadside device 6, and distributes the video to the in-vehicle terminal 2. Communication is performed between the server 7 and the in-vehicle terminal 2 using wireless LAN communication or cellular communication. The roadside device 6 notifies the in-vehicle terminal 2 of the link information (address) allocated to each camera 13 of the roadside device 6. The in-vehicle terminal 2 can acquire the video of the camera 13 of the roadside device 6 by accessing the server 7 based on the link information and requesting distribution of the video of the camera 13 of the desired roadside device 6 . Thereby, the image taken by the camera 13 of the roadside device 6 is displayed on the navigation display 4 mounted on the vehicle 1.

Note that in this embodiment, the video shot by the roadside device 6 is once transmitted to the video distribution server, and the video is distributed from the video distribution server to the in-vehicle terminal 2. It is also possible to directly deliver the video to 2. In this case, wireless LAN communication such as WiFi (registered trademark) may be performed between the roadside device 6 and the vehicle-mounted terminal 2.

Further, the video shot by the roadside device 6 may be distributed to the vehicle-mounted terminal 2 via another roadside device 6 by multi-hop communication.

Next, an overview of the operations of the vehicle 1 and the roadside machine 6 according to the first embodiment will be described. FIG. 2 is an explanatory diagram showing an overview of the operations of the vehicle 1 and the roadside device 6.

The roadside machine 6 is installed at each intersection. In this embodiment, the video captured by the camera 13 of the roadside device 6 is distributed to the in-vehicle terminal 2 via the server 7. The in-vehicle terminal 2 acquires an image of the non-line-of-sight road taken by a roadside device 6 installed at each intersection, and causes the navigation display 4 to display the image.

Further, in this embodiment, the roadside device 6 detects moving objects (vehicles, pedestrians) existing on the road around the intersection based on the detection results of the radar 12, and detects moving objects existing on the road. Acquire position information (moving object detection processing). Note that a moving object existing on the road may be detected by image recognition of an image taken by the camera 13.

Further, in the present embodiment, the roadside device 6 detects that an obstacle event (an event that becomes an obstacle to traffic) has occurred on the road around the intersection, in which it is desirable to change the route of the vehicle 1 (an obstacle event detection process). This failure event detection process is performed based on the detection results of the radar 12 and the video captured by the camera 13. In this failure event detection process, position information of the failure event occurrence point is obtained.

Here, the hindrance events include, for example, traffic accidents, traffic jams, road construction, crowds (for example, groups of children on their way to and from school), emergency vehicles, obstacles, fallen trees, landslides, flooding, and the like. Furthermore, the fact that a person requiring attention (dangerous person), for example, a person who has frequently taken dangerous actions such as jumping out in the past, is walking may be detected as an obstacle event. Further, a vehicle requiring attention (dangerous vehicle) may be detected as an obstacle event. Here, vehicles that require attention include, for example, vehicles whose running speed exceeds a predetermined value, vehicles that drive in a meandering direction, vehicles that accelerate or decelerate more than necessary (including sudden accelerations and sudden stops), vehicles that change lanes more than necessary, and vehicles that use unnecessary lights ( (including hazard lights, etc.), and vehicles operated by elderly people, people with disabilities, and drivers who are not confident in driving (including vehicles with beginner's marks and driving school vehicles), etc. be.

In addition, in this embodiment, by detecting an obstacle event that obstructs vehicle passage and notifying the driver of the occurrence of the specific event, the driver can quickly determine whether or not to change the route. However, an event other than a failure event may be defined as a specific event, and the driver may be notified of the occurrence of the specific event. For example, in the embodiment described later, two-wheeled vehicles (e.g., including motorcycles, bicycles, etc.) that may cause an accident when a vehicle turns left, or other vehicles that may collide from behind, are identified as specific events. The driver is notified of the occurrence of a specific event.

Furthermore, in the present embodiment, during normal times, that is, when no fault event has occurred, a normal message is transmitted from the roadside device 6 to the in-vehicle terminal 2. Furthermore, when the roadside device 6 detects a fault event (such as a collision), a fault event notification message is sent from the roadside device 6 to the in-vehicle terminal 2 via road-to-vehicle communication, and then another message is sent via road-to-road communication. It is transmitted to the roadside device 6. Further, when the roadside device 6 receives a failure event notification message from another roadside device 6, a failure event transfer message is transmitted from the roadside device 6 to the vehicle-mounted terminal 2.

In the example shown in FIG. 2, an obstacle event (traffic accident, traffic jam, etc.) occurs at the next intersection that the vehicle 1 sees from the nearest intersection and goes straight ahead. In this case, the roadside device 6 installed at the next intersection detects the occurrence of a fault event, and the roadside device 6 sends a fault event notification message to the roadside device 6 installed at the nearest intersection. A fault event transfer message is transmitted from the roadside device 6 to the in-vehicle terminal 2.

When the in-vehicle terminal 2 receives the failure event notification message from the roadside device 6, it performs control to highlight the video showing the failure event among the images of each non-line-of-sight road at each intersection displayed on the navigation display 4. be exposed. Thereby, the driver can immediately recognize the occurrence of a trouble event and quickly determine whether or not a route change is necessary. Note that as a method for highlighting the video, it is sufficient to display the video in an enlarged manner, or to change the thickness and color of the video display frame, as in the example shown below. Further, as another highlighting method, for example, the video display frame may be changed from a dotted line to a solid line.

Furthermore, in the present embodiment, upon receiving a failure event notification message from the roadside device 6, the automatic driving ECU 3 operates based on the location information of the failure event occurrence point and the location information of the destination included in the failure event notification message. Then, a detour route that detours around the point where the failure event occurs and heads to the destination is generated (driving route planning process). Then, the automatic driving ECU 3 controls the own vehicle to travel along the generated detour route. Further, a detour route is displayed on the navigation display 4.

Here, if an obstacle event occurs around the planned route during automatic driving, the automatic driving ECU 3 will proceed along the detour route without inquiring the driver about the necessity of changing the route (outputting an alert). It is also possible to start control for driving the host vehicle. However, depending on the situation of the failure event, there may be no need to change the route, and it may be difficult for the automatic driving ECU 3 to determine whether or not a route change is necessary. For example, if the traffic accident is relatively minor, it is possible to pass by the area where the traffic accident occurred, and in this case there is no need to change the route.

Therefore, in the present embodiment, a video showing the fault event is displayed on the navigation display 4, and the driver views the video showing the fault event, confirms the specific situation of the fault event, and determines whether a route change is necessary. and perform necessary operations such as changing the route. Note that the same applies when a failure event occurs while the driver cancels automatic driving and is performing driving operations himself/herself.

Next, a navigation screen displayed on the navigation display 4 mounted on the vehicle 1 according to the first embodiment will be described. FIG. 3 is an explanatory diagram showing an example of a navigation screen. FIG. 4 is an explanatory diagram showing a case where a failure event occurs on an example of the navigation screen. FIG. 5 is an explanatory diagram showing another example of the navigation screen. FIG. 6 is an explanatory diagram illustrating a case where the detection status of a moving object changes in another example of the navigation screen.

In the vehicle 1, a navigation screen 101 is displayed on the navigation display 4. This navigation screen 101 displays a map 102 around the planned route of the own vehicle. On the map 102, a mark image 111 representing the current position and planned route (direction of travel) of the host vehicle is displayed. Further, on the map 102, a mark image 112 representing a moving object detected by the roadside device 6 is displayed.

Furthermore, in the present embodiment, images 121 and 122 of non-line-of-sight roads at intersections captured by the camera 13 of the roadside device 6 are displayed on the navigation screen 101. The images 121 and 122 are displayed in a superimposed manner on the map 102 in an area other than the center of the image where the current position of the host vehicle is displayed. Note that a map may be displayed at the center of the image, and an image may be displayed around the map.

Furthermore, in the present embodiment, a plurality of images 121 and 122 of roads outside the intersection are displayed on the navigation screen 101. The plurality of images 121, 122 are displayed at a position on the screen corresponding to the shooting point, that is, the position of the roadside device 6 that took the images 121, 122. Specifically, on the navigation screen 101, the current position of the own vehicle is set at the lower center of the screen, and the travel route of the own vehicle is displayed above it. Therefore, for example, if the shooting point of the video is located on the left side of the vehicle, the video will be displayed on the left side of the screen. On the other hand, if the video shooting point is located on the right side of the vehicle, the video is displayed on the right side of the screen. Additionally, if the location where the video was taken is far from the vehicle, the video will be displayed at the top of the screen. On the other hand, if the location where the video was taken is close to the vehicle, the video will be displayed at the bottom of the screen.

Furthermore, in this embodiment, among the plurality of videos, videos with a higher priority are displayed with priority over videos with a lower priority. In particular, in this embodiment, the priority for each video is set based on both the importance of the event shown in the video and the distance from the vehicle to the roadside device 6 serving as the shooting point. Specifically, the priority of a video increases as the degree of importance increases, and decreases as the distance increases. Note that the importance level relates to the level of importance of an event that has occurred, and the priority level relates to the level of priority when displaying a video.

In the example shown in FIG. 3, the navigation screen 101 displays an image 121 of the non-sight road on the left at the most recent intersection, and an image 122 of the non-sight road on the right at the next intersection when going straight through the most recent intersection. has been done. Here, in normal times, that is, when no disturbance event such as a traffic accident occurs, the importance of the events shown in images 121 and 122 is equal, so the priority of each image 121 and 122 is lower than that of the own vehicle. It is set according to the distance to the roadside device 6 that is the shooting point. Specifically, the image 121 of the nearest intersection near the host vehicle is given priority over the image 122 of the next intersection located further away, and the image 121 is displayed larger than the image 122.

Note that the image of the right-side non-line-of-sight road at the most recent intersection and the image of the left-side non-line-of-sight road at the next intersection after going straight through the most recent intersection are not displayed because there are no moving objects on the road.

On the other hand, as shown in FIG. 4, when an obstacle event (traffic accident) occurs at an intersection, the priority of the image 123 of the intersection where the obstacle event has occurred becomes higher. In the example shown in FIG. 4, a collision occurs at the intersection following the most recent intersection. Therefore, the image 123 showing the failure event is displayed enlarged from the normal image 122 (see FIG. 3). Further, in the video 123 in which a fault event is captured, although the distance from the vehicle to the shooting point is long, the importance level increases due to the occurrence of the fault event, so its priority is higher than that of the other videos 121. Therefore, the image 123 showing the trouble event is displayed larger than the image 121 of the nearest intersection. This allows the driver to immediately recognize that an obstacle event (traffic accident) has occurred at the intersection following the most recent intersection, stop driving straight through the most recent intersection, and instead turn right or It is possible to determine a route change, such as turning left.

Here, since the display priority for each video changes depending on the distance from the vehicle to the roadside device 6 serving as the shooting point, the process of setting the display priority for each video is performed on the in-vehicle terminal 2. However, it is also possible for the roadside device 6 to perform the process of setting the priority.

Note that when a trouble event occurs in this way, the driver may be automatically guided along an appropriate route. This automatic guidance can be realized, for example, by the driver inputting a destination and map information into the in-vehicle terminal 2, and the in-vehicle terminal 2 or the automatic driving ECU 3 automatically selecting a route that avoids the relevant location. .

Further, on the navigation screen 101, when an obstacle event occurs at an intersection, a mark image 113 representing the point where the obstacle event occurs is displayed blinking on the map 102. Note that among the mark images 111 of moving bodies, the mark image 111 of the moving body related to the failure event may be displayed blinking.

Further, on the navigation screen 101, when an obstacle event occurs at an intersection, a guide image 114 is displayed that associates a video 123 showing the obstacle event with a mark image 113 representing the point where the obstacle event occurred displayed on the map 102. Ru.

Further, on the navigation screen 101, when an obstacle event occurs at an intersection, a video display frame 141 in which an image 123 showing the obstacle event is displayed is highlighted. Specifically, the video display frame 141 is displayed as a thick line in a predetermined color (for example, red).

As described above, in this embodiment, an image of an intersection is displayed. Furthermore, the display form of the video changes depending on the priority of the video. This allows the driver to quickly determine whether or not a route change is necessary. Further, even when the automatic driving ECU 3 executes a route change and proceeds on a detour route, the driver can recognize the cause of the detour by viewing the video showing the obstacle event.

Further, in this embodiment, at least the nearest intersection and the next intersection are subject to video display, but there are many roads that are out of line of sight at each intersection when viewed from the vehicle 1, and it is necessary to display all images. , it becomes extremely troublesome. Therefore, in this embodiment, the display form of the video is changed depending on the necessity of display, that is, the priority of the video. Specifically, if there is no moving object in the video, there is no need to display the video, so the video is not displayed.

In addition, in this embodiment, if the priority of the video of the intersection in front is low and the priority of the video of the intersection one ahead is high, the video of the intersection one ahead with high priority is given priority. It is superimposed on the image of the intersection in the foreground, where the intensity is low.

For example, if there is no moving object at the intersection in front of you, but there is a moving object at the intersection immediately ahead, the image of the next intersection with a higher priority will be the image of the intersection ahead of you with a lower priority. displayed on top. In addition, if an obstacle event does not occur at the intersection in front, but an obstacle event occurs at the intersection one ahead, the image of the intersection one ahead, which has a higher priority, will be displayed at the intersection one ahead, which has a lower priority. will be superimposed on the image of the intersection.

Further, the image of the intersection one ahead is displayed in a small size within the image display frame in which the image of the intersection in front is displayed. This reduces the amount of video the driver sees. Therefore, you can quickly check the specific situation of a moving object or obstacle event that requires attention using images that show the moving object or obstacle event that requires attention, without being bothered by images that require little attention. .

In the example shown in FIG. 5, normally, an image 121 of a non-line-of-sight road on the left side at the most recent intersection and an image 124 of a non-sight-sight road on the right side at the most recent intersection are displayed on the navigation screen.

On the other hand, as shown in FIG. 6, when a moving object is detected in the image 122 of the right-side non-line-of-sight road at the next intersection after going straight through the nearest intersection, the image of the intersection where the moving object was detected is 122 is displayed in priority to the image 124 (see FIG. 5) of the right-side non-line-of-sight road at the nearest intersection. Specifically, the image 124 (see FIG. 5) of the right-side non-line-of-sight road at the nearest intersection where no moving object is detected is not displayed (blackout), and only the image display frame 141 thereof is displayed. On the other hand, the image 122 of the intersection where the moving object was detected is displayed in an overlapping manner within the image display frame 141 for the image of the most recent intersection.

In this case, videos with a high priority, such as videos showing a specific event or videos showing a moving object, are displayed with priority, while videos in which a specific event does not occur or videos without a moving object are displayed with priority. , becomes hidden and invisible.

Further, in this embodiment, in order to reduce the troublesomeness of displaying a large number of videos, the display format of the video is changed, but there may be cases where changing the display format of the video alone is not enough. Therefore, characters may be displayed to guide the position of the moving object (vehicle 1, person, etc.) related to the trouble event. For example, the text ``A person requiring attention is moving xm ahead of your vehicle, and there is a traffic jam ym ahead of you'' is displayed.

In the present embodiment, the images 121 and 122 of non-line-of-sight roads displayed on the navigation screen include a mark image 131 pointing to a moving object (pedestrian, vehicle 1, etc.), specifically, a mark image 131 pointing to a moving object (pedestrian, vehicle 1, etc.) Frame images are drawn in a superimposed manner. This allows the driver to instantly recognize the moving object. Further, a mark image 132 (arrow image) representing the moving direction of the moving body is drawn superimposed on the video. This allows the driver to instantly recognize the moving direction of the moving object and easily determine whether or not the moving object requires attention.

Furthermore, in this embodiment, the same moving object is displayed in the same color in the moving object mark image 131 displayed on the video 121 and the moving object mark image 112 displayed on the map 102. This allows the driver to immediately recognize the correspondence between the moving objects displayed on the image 121 and the moving objects displayed on the map 102.

Note that in the present embodiment, the navigation display 4 displays the image of the road beyond the line of sight of the intersection taken by the roadside device 6 superimposed on the map, but the screen configuration is not limited to this. . For example, the image taken by the roadside device 6 may be displayed superimposed on the image taken of the front of the host vehicle. In this case, the vehicle 1 is equipped with a camera that photographs the front of the own vehicle, and the image photographed by the camera is displayed on the navigation display 4.

Next, a schematic configuration of the roadside machine 6 according to the first embodiment will be described. FIG. 7 is a block diagram showing a schematic configuration of the roadside machine 6. As shown in FIG.

The roadside device 6 includes, in addition to the radar 12 and the camera 13, an ITS communication section 21, a roadside communication section 22, a server communication section 23, a memory 24, and a processor 25.

The ITS communication unit 14 broadcasts messages to the in-vehicle terminal 2 through ITS communication (road-to-vehicle communication), and receives messages transmitted from the in-vehicle terminal 2.

The road-to-road communication unit 22 communicates with adjacent roadside devices 6 via a dedicated roadside network (wired or wireless) or a network such as cellular communication.

The server communication unit 23 communicates with the server 7 via the network.

The memory 24 stores programs executed by the processor 25 and the like.

The processor 25 executes programs stored in the memory 24 to perform various processes related to information collection. In this embodiment, the processor 25 performs a moving object detection process, a moving object display information generation process, a mark image synthesis process, a failure event detection process, and the like.

In the moving object detection process, the processor 25 detects a moving object (vehicle 1, pedestrian) based on the detection result of the radar 12, and acquires position information and movement information of the moving object. Specifically, the processor 25 first obtains, based on the detection results of the radar 12, relative position information of the mobile body with respect to the road terminal, that is, the direction in which the mobile body exists as seen from the road terminal; Calculate the distance from the road terminal to the mobile object. Next, the processor 25 uses the absolute position information (latitude, longitude) of the mobile body based on the relative position information (azimuth, distance) of the mobile body and the position information (latitude, longitude) of the installation position of the road terminal. , longitude). Further, the processor 25 obtains movement information of the moving body, that is, the moving speed and direction of the moving body, based on the change status of the position information of the moving body.

In the moving object display information generation process, the processor 25 generates display information for each moving object detected on the video, that is, the display position (coordinates, size) on the video of a mark image (frame image) pointing to the moving object, and Generates information about display colors. Specifically, the processor 25 acquires the position of the moving object on the video based on the position information and movement information of the moving object obtained in the moving object detection process. Note that the moving object may be detected from the video by image recognition of the video and the position of the moving object may be acquired. Further, the processor 25 assigns a display color to each moving object detected on the video. At this time, if a plurality of moving objects are detected, the display color for each moving object is set so that the display colors of the mark images do not overlap.

In the mark image synthesis process, the processor 25 generates a composite image in which a mark image pointing to a moving body is superimposed on the image captured by the camera 13. At this time, a mark image is drawn in the display color for each moving object acquired in the moving object display information generation process. The video generated by this mark image synthesis process is uploaded to the server 7.

In the obstacle event detection process, the processor 25 detects the occurrence of an obstacle event (an event that becomes an obstacle to traffic) that makes it desirable for the vehicle 1 to change its route on the road around the intersection. At this time, it is determined whether or not a failure event has occurred based on the position, moving direction, and moving speed of each moving object obtained in the moving object detection process.

Next, a schematic configuration of the vehicle 1 according to the first embodiment will be described. FIG. 8 is a block diagram showing a schematic configuration of the vehicle 1. As shown in FIG.

The vehicle 1 includes a sensor 31, a steering ECU 32, a drive ECU 33, and a brake ECU 34 in addition to an on-vehicle terminal 2, an automatic driving ECU 3, and a navigation display 4.

The in-vehicle terminal 2 includes an ITS communication section 41, a wireless communication section 42, a positioning section 43, a memory 44, and a processor 45.

The ITS communication unit 41 broadcasts messages to other vehicle-mounted terminals 2 through ITS communication (vehicle-to-vehicle communication), and receives messages transmitted from other vehicle-mounted terminals 2. Further, the ITS communication unit 41 transmits messages to the roadside device 6 and receives messages transmitted from the roadside device 6 through ITS communication (road-to-vehicle communication).

The wireless communication unit 42 communicates with the server 7 using wireless communication such as cellular communication or wireless LAN communication.

The positioning unit 43 measures the position of its own device using a satellite positioning system such as GNSS (Global Navigation Satellite System), that is, GPS (Global Positioning System) or QZSS (Quasi-Zenith Satellite System), and obtains position information of its own device. Get (latitude, longitude).

The memory 44 stores map information, programs executed by the processor 45, and the like.

The processor 45 executes programs stored in the memory 44 to perform various processes related to information collection. In this embodiment, the processor 45 performs collision determination processing, video reception control processing, video display control processing, screen display control processing, and the like.

In the collision determination process, the processor uses the position information of other vehicles and pedestrians included in messages received from the in-vehicle terminal 2 and pedestrian terminal 5 of other vehicles, the position information of the own vehicle acquired by the positioning unit 43, etc. The system determines the possibility of a collision between the vehicle and another vehicle or pedestrian. If it is determined in this collision determination process that there is a risk of collision, control is performed to execute a predetermined warning action for the user.

In the video reception control process, the processor 45 accesses the server 7 based on the video link information included in the message received from the roadside device 6, sends a request for video distribution to the video distribution server, and receives the request from the server 7. Controls the wireless communication unit to receive the distributed video.

In the video display control process, the processor 45 performs control to display the video received from the video distribution server on the navigation screen of the navigation display 4. Specifically, the images taken by each roadside device 6 installed at the nearest intersection and the intersection beyond that are used for navigation in a size that corresponds to the distance from the current position of the own vehicle to the roadside device 6 (shooting point). displayed on the screen. Furthermore, the video in which the failure event was captured is highlighted.

In the screen display control process, the processor 45 controls the navigation screen of the navigation display 4. Specifically, a mark image representing a moving object is displayed on a map on a navigation screen. In particular, the mark image of the moving object related to the failure event is highlighted. Specifically, for example, a mark image of a moving object related to a failure event is displayed in a blinking manner.

The automatic driving ECU 3 is connected to the steering ECU 32, the driving ECU 33, and the brake ECU 34, and controls the steering ECU 32, the driving ECU 33, and the brake ECU 34 based on the detection result of the sensor 31, so that the vehicle 1 can automatically operate (autonomous driving). Realize.

Here, the sensor 31 is a radar, lidar, camera, or the like, and detects moving objects existing on the road around the own vehicle. Further, the steering ECU 32 controls the steering mechanism of the own vehicle, the drive ECU 33 controls the drive mechanism (engine, electric motor, etc.) of the own vehicle, and the braking ECU 34 controls the braking mechanism of the own vehicle. It controls the

Next, an overview of the operations of the in-vehicle terminal 2, roadside device 6, and server 7 according to the first embodiment will be explained. FIG. 9 is a sequence diagram showing an overview of the operations of the in-vehicle terminal 2, the roadside device 6, and the server 7.

In this embodiment, images of a non-line-of-sight road taken by two roadside devices 6 installed at two intersections lined up in the direction of travel of the vehicle 1 are displayed on the navigation display 4 of the vehicle 1. Here, the roadside unit 6 installed at the nearest intersection when viewed from the vehicle 1 is referred to as a first roadside unit 6, and the roadside unit 6 installed at the next intersection after the nearest intersection is referred to as a second roadside unit. It is called 6.

The in-vehicle terminal 2 receives the video link information of the first roadside device 6 from the first roadside device 6, and transmits the video shot by the first roadside device 6 from the server 7 based on the video link information. The image received by the first roadside device 6 is displayed on the navigation display 4. In addition, the in-vehicle terminal 2 receives the video link information of the second roadside device 6 via the first roadside device 6, and based on the video link information, The image is received from the server 7, and the image photographed by the second roadside device 6 is displayed on the navigation display 4.

At this time, during normal times, that is, when no failure event has occurred, a normal message is transmitted from the roadside device 6 to the vehicle-mounted terminal 2. This normal message includes, as normal notification information, video link information of the own device (first roadside device 6), specifically link information of the video distribution server that distributes the video of the roadside device 6. , the position information and display information of the moving body detected by the roadside device 6.

Note that the display information of the moving object includes information regarding the display position (coordinates, size) and display color of a mark image (frame image) pointing to the moving object on the video. By notifying the in-vehicle terminal 2 of the display color of each moving object, the mark image of the moving object displayed on the navigation image can be displayed in the same color as the mark image of the moving object drawn on the video.

On the other hand, when a failure event occurs, a message of failure event notification is transmitted from the roadside device 6. Similar to the normal message, this message includes video link information, position information of the moving body, and display information as notification information during normal times. Furthermore, the failure event occurrence notification message includes video link information of the roadside device 6, location information of the failure event occurrence point, and location information of the roadside device 6 that detected the failure event as notification information regarding the failure event. It will be done.

Furthermore, a failure event notification message transmitted from the roadside device 6 installed at the next intersection after the most recent intersection is relayed by the roadside device 6 installed at the most recent intersection, and then received by the in-vehicle terminal 2. At this time, when the roadside device 6 installed at the nearest intersection receives the fault event notification message sent from the roadside device 6 installed at the next intersection, it generates a fault event forwarding message and sends it to the on-vehicle terminal. Send to.

This fault event transfer message is obtained by adding normal notification information of the own device to the notification information included in the fault event notification message. Specifically, the failure event transfer message includes video link information of the first roadside device 6 and location information and display of the mobile object as notification information regarding the first roadside device 6 (own device) during normal times. Contains information and. Further, the message includes, as normal notification information regarding the second roadside device 6, video link information of the second roadside device 6, and position information and display information of the mobile object. The message also includes, as notification information regarding the failure event, video link information of the second roadside device 6, location information of the failure event occurrence point, and location information of the second roadside device 6 that detected the failure event. is included.

Next, the operation procedure of the roadside machine 6 according to the first embodiment will be explained. FIG. 10 is a flowchart showing the operation procedure of the roadside machine 6.

In the roadside device 6, the processor 25 detects moving objects (vehicles 1, pedestrians) existing on the road around the intersection based on the detection results of the radar 12, and detects moving objects (vehicles 1, pedestrians) on the road around the intersection. It is determined whether a vehicle (vehicle 1, pedestrian) exists (moving object detection process) (ST101).

Here, if a moving object exists on the road around the intersection (Yes in ST101), based on the detection results of the radar 12, the moving object's position information (latitude, longitude) and movement information (moving direction, (moving speed) (ST102).

Furthermore, the processor 25 instructs the camera 13 to photograph the road where the moving object is present (ST103). The camera 13 photographs the road on which the moving object is located, according to instructions from the processor 25.

Next, the processor 25 acquires the position of the moving object on the image taken by the camera 13 based on the moving object's position information (latitude, longitude) and movement information (moving direction, moving speed), and Display information for superimposing and drawing a mark image (frame image) pointing to the moving object shown in the image on the video is generated (moving object display information generation process) (ST104).

Next, the processor 25 generates a composite video in which a mark image pointing to the moving body is superimposed on the video captured by the camera 13 based on the display information (mark image composition processing) (ST105).

Next, the server communication unit 23 transmits the processed video, that is, the composite video in which the mark image of the moving object is superimposed, to the server 7 (video upload) (ST106). Note that the image of the moving object and the display information for superimposing the mark image of the moving object on the image are sent to the server 7, and the server 7 superimposes the mark image of the moving object on the image. A drawing process may also be performed.

Next, the processor 25 determines whether a predetermined obstacle event has occurred around the intersection based on the position information and movement information of the moving object acquired in the moving object detection process (obstruction event detection process). ST107).

Here, if a fault event occurs (Yes in ST107), processor 25 generates a fault event notification message. Then, the ITS communication unit 21 transmits a message notifying the occurrence of a fault event to the in-vehicle terminal 2, and the road-to-road communication unit 22 transmits a message notifying the occurrence of a fault event to the adjacent roadside device 6 ( ST108).

On the other hand, if no failure event has occurred (No in ST107), then the road-to-road communication unit 22 determines whether or not a failure event notification message has been received from the surrounding roadside device 6 ( ST109).

Here, if a failure event notification message is received (Yes in ST109), processor 25 generates a failure event notification transfer message. Then, the ITS communication unit 21 transmits a failure event transfer message to the in-vehicle terminal 2 (ST110).

On the other hand, if the failure event notification message has not been received (No in ST109), the processor generates a normal message. Then, the ITS communication unit 21 transmits a normal message to the in-vehicle terminal 2 (ST111).

Next, the operation procedure of the on-vehicle terminal 2 and the automatic driving ECU 3 according to the first embodiment will be explained. FIG. 11 is a flow diagram showing the operation procedure of the in-vehicle terminal 2 and the automatic driving ECU 3.

In the in-vehicle terminal 2, when the ITS communication unit 41 receives the message from the roadside device 6 (Yes in ST201), the processor acquires the video link information included in the received message, the location information of the failure event occurrence point, etc. (ST202). Next, the processor transmits a request for video distribution to the video distribution server based on the video link information, and controls the wireless communication unit to receive the video distributed from the video distribution server (video reception control processing ) (ST203).

Next, the processor 45 performs control to display the video received from the video distribution server on the navigation screen of the navigation display 4 (video display control process) (ST204). Specifically, the images taken by each roadside device 6 installed at the nearest intersection and the intersection beyond that are used for navigation in a size that corresponds to the distance from the current position of the own vehicle to the roadside device 6 (shooting point). displayed on the screen. Furthermore, if a fault event has occurred, the video in which the fault event was captured is highlighted. At this time, the mark image of the moving object appearing in the video is also highlighted. This allows the driver to quickly determine whether or not a route change is necessary.

The automatic driving ECU 3 acquires the location information of the failure event occurrence point included in the message received from the roadside device 6 from the in-vehicle terminal 2 (ST301). Next, the automatic driving ECU 3 generates a detour route to the destination, bypassing the failure event occurrence point, based on the location information of the failure event occurrence point, the destination location information, etc. (driving route planning process) (ST302). Then, the automatic driving ECU 3 displays the generated detour route on the navigation screen of the navigation display 4 (ST303).

Next, the automatic driving ECU 3 controls the traveling of the own vehicle so that the own vehicle detours around the failure event occurrence point according to the detour route (travel control process) (ST304). At this time, the automatic driving ECU 3 performs an operation (outputting an alert) to inquire of the driver whether or not a route change is necessary. Perform the operation.

(First modification of the first embodiment)
Next, a first modification of the first embodiment will be described. Note that the points not particularly mentioned here are the same as those in the previous embodiment. FIG. 12 is an explanatory diagram showing an overview of the operations of the vehicle 1 and the roadside machine 6 according to the first modification of the first embodiment.

In the first embodiment, the video shot by the roadside device 6 located in front of the vehicle 1 in the direction of travel is distributed to the in-vehicle terminal 2. The video shot by the roadside device 6 located there is distributed to the in-vehicle terminal 2 of the vehicle 1.

Furthermore, in this modification, the roadside device 6 detects an emergency vehicle existing around the roadside device 6 as an obstruction event (an event that obstructs the passage of the vehicle 1). Specifically, based on the vehicle information included in the message received from the in-vehicle terminal 2 of the emergency vehicle through road-to-vehicle communication, it is determined whether the vehicle that is the source of the message is an emergency vehicle.

When the roadside device 6 detects an emergency vehicle existing around the self-device, it transmits a failure event notification message to the in-vehicle terminal 2 of the vehicle 1, either directly or via another roadside device 6. This failure event notification message includes video link information, emergency vehicle position information, and the like.

When the in-vehicle terminal 2 of the vehicle 1 receives a failure event notification message from the roadside device 6, it determines whether an emergency vehicle is approaching from behind the own vehicle based on the position information of the emergency vehicle included in the message. If an emergency vehicle is approaching from behind the own vehicle, a video showing the emergency vehicle is received from the server 7 based on the video link information included in the received message, and the video is used for navigation. Display on screen.

Next, a navigation screen displayed on the navigation display 4 according to the first modification of the first embodiment will be described. FIG. 13 is an explanatory diagram showing the navigation screen.

In this modification, the navigation screen 101 of the navigation display 4 displays an image 121 taken by the roadside device 6 in front of the own vehicle, as in the first embodiment (see FIG. 3), and also displays the image 121 of the own vehicle. When an emergency vehicle is detected by the roadside device 6 behind the vehicle, an image 125 captured by the roadside device 6 behind the own vehicle is displayed. An emergency vehicle appears in the video 125 taken by the roadside device 6 behind this.

Further, the video 125 photographed by the roadside device 6 at the rear is displayed with priority over the other videos 121, and specifically, is displayed larger than the other videos 121. Furthermore, the video 125 taken by the roadside machine 6 at the rear is highlighted. Specifically, the video display frame 141 is displayed as a thick line in a predetermined color (for example, red). This allows the driver to immediately recognize that an emergency vehicle is approaching from behind the vehicle and quickly perform driving operations to yield to the emergency vehicle, such as stopping or changing course. can.

Further, on the navigation screen 101, a mark image 115 representing an emergency vehicle is displayed superimposed on the position of the emergency vehicle on the map. The mark image 115 representing this emergency vehicle is highlighted, and more specifically, is displayed blinking. Furthermore, on the navigation screen 101, characters 142 that urge the driver to yield to an emergency vehicle are displayed, such as characters 142 that say "Stop at the side of the road."

In addition, in this modification, an emergency vehicle is detected and the image 125 showing the emergency vehicle is displayed on the vehicle 1 located in front of the emergency vehicle in the direction of travel. For example, a vehicle whose traveling speed exceeds a predetermined value or a vehicle traveling in a meandering manner may be detected, and an image showing the vehicle requiring attention may be displayed.

Furthermore, as the video 125 showing the emergency vehicle, the video taken by the roadside device 6 at the rear is displayed on the navigation screen 101. The captured video may also be displayed.

(Second modification of the first embodiment)
Next, a second modification of the first embodiment will be described. Note that the points not particularly mentioned here are the same as those in the previous embodiment. FIG. 14 is an explanatory diagram showing an example of a navigation screen on the navigation display 4 mounted on the vehicle 1 and a front AR screen 201 on the front AR display 35 according to the second modification of the first embodiment. FIG. 15 is an explanatory diagram illustrating a case where a failure event occurs on an example of the navigation screen and the front AR screen. FIG. 16 is a block diagram showing a schematic configuration of the vehicle 1. As shown in FIG.

In the first embodiment, a navigation display 4 is provided in the vehicle 1, and an image photographed by a roadside device 6 is displayed on a navigation screen 101 of the navigation display 4. On the other hand, in this modification, as shown in FIG. 16, in addition to the navigation display 4, a front AR display 35 (display device) is provided. A front AR screen 201 is displayed superimposed on the windshield 1, and images 121 and 122 taken by the roadside device 6 are displayed on the front AR screen 201.

The front AR display 35 is, for example, a head-up display, and includes a display device such as a projector or a liquid crystal display panel, and a transparent panel that reflects a display image (projection image) of the display device in a semi-transparent state.

Note that in this embodiment, the front AR display 35 is provided as an AR display device, and the AR screen is displayed superimposed on the windshield of the vehicle 1. You can also do this. Further, although a head-up display is shown as an example of the AR display device, other AR display devices such as a head-mounted display are also possible.

On the front AR screen 201 provided by the front AR display 35, images 121 and 122 taken by the roadside device 6 are superimposed and displayed as virtual objects on the real space outside the vehicle that can be seen through the window glass. On this front AR screen 201, a plurality of images 121 and 122 taken by the roadside device 6 are displayed in the same display format as in the first embodiment (see FIG. 3).

The plurality of images 121, 122 are displayed at a position on the screen corresponding to the shooting point, that is, the position of the roadside device 6 that took the images 121, 122. For example, if the shooting point of the video is located on the left side of the vehicle, the video will be displayed on the left side of the screen. On the other hand, if the video shooting point is located on the right side of the vehicle, the video is displayed on the right side of the screen. Furthermore, if the point at which the video was taken is far from the vehicle, the video is displayed at the back (upper side) of the screen. On the other hand, if the location where the video was taken is close to the vehicle, the video will be displayed in the front (bottom) of the screen.

Further, on the front AR screen 201, as in the first embodiment, a mark image 131 (detection frame) pointing to a moving object in the video and a mark image 132 (arrow) indicating the moving direction of the moving object are displayed on the videos 121, 122. will be superimposed on the screen.

On the other hand, on the navigation screen 101 on the navigation display 4, a mark image 111 representing the current position and traveling direction of the own vehicle and a mark image 112 representing a moving object are displayed superimposed on the map 102, as in the first embodiment. Ru.

In addition, in this modification, the mark image 131 of a moving object displayed on the image 121 of the front AR screen 201 and the mark image 112 of a moving object displayed on the map 102 of the navigation screen 101 are the same moving object. are displayed in the same color. Thereby, the driver can immediately recognize the correspondence between the mark image 112 of the moving object displayed on the navigation screen 101 and the moving objects shown in the images 121 and 122 on the front AR screen 201.

Further, as shown in FIG. 15, in this modification, when an obstacle event occurs at an intersection, an image 123 showing the obstacle event is highlighted on the front AR screen 201. Further, a guide image 211 is displayed that associates the video 123 showing the failure event with the mark image 113 of the failure event occurrence point displayed on the navigation screen 101. Thereby, the driver can immediately recognize the correspondence between the image 123 displayed on the front AR screen 201 and the mark image 113 of the trouble occurrence point displayed on the navigation screen 101. That is, the driver can check the specific situation of the trouble occurrence point displayed on the map 102 on the image 123, and conversely, the driver can check the location of the trouble occurrence point displayed on the image 123 on the map 102. You can check with.

In addition, the video taken by the roadside device 6 has an animation effect such that the image 123 showing the failure event is displayed on the front AR screen 201 by zooming up (pop-up) from the mark image 113 of the failure event occurrence point on the navigation screen 101. It may be possible to perform screen control that provides the following.

Furthermore, in this modification, a mark image 212 (arrow) representing the direction of travel of the own vehicle (going straight, turning right, and turning left) at the nearest intersection is displayed on the front AR screen. In the mark image 212 representing the traveling direction, a mark image (x mark) is displayed if it is not appropriate to proceed due to the occurrence of an obstacle event. In the example shown in FIG. 15, an obstacle event has occurred at the intersection after going straight through the nearest intersection, so a mark image (x mark) indicating that going straight is not appropriate is displayed in the mark image 212 for going straight. .

(Second embodiment)
Next, a second embodiment will be described. Note that the points not particularly mentioned here are the same as those in the previous embodiment. FIG. 17 is an explanatory diagram showing an overview of the operations of the vehicle 1 and the roadside device 6 according to the second embodiment.

In the embodiment described above, when the occurrence of a trouble event is detected, control is performed to change the display form of the video, specifically, to highlight the video showing the trouble event. On the other hand, in this embodiment, a moving object existing around the own vehicle is detected, and depending on the behavior of the own vehicle, especially the direction of travel (going straight, turning right, turning left), there is a possibility that the moving object will transition to a highly dangerous state. If there is a vehicle, an image of the moving object will be displayed on the navigation screen.

In particular, in this embodiment, when the vehicle 1 makes a left turn at an intersection, in order to prevent an accident involving a two-wheeled vehicle such as a bicycle that attempts to pass by the vehicle 1, when the vehicle 1 is traveling in a left turn, When a moving object (such as a two-wheeled vehicle) approaching from behind is detected on the left side of the vehicle, an image of the moving object is displayed on the navigation screen.

In this embodiment, an example in which the vehicle 1 drives on the left side has been described, but when the vehicle 1 drives on the right side, the left and right cases are reversed.

Next, a navigation screen on the navigation display 4 mounted on the vehicle 1 according to the second embodiment will be explained. FIG. 18 is an explanatory diagram showing the navigation screen.

In this embodiment, when a moving object (such as a two-wheeled vehicle) approaching from behind is detected on the left side of the own vehicle when the traveling direction of the own vehicle is a left turn, the image 126 showing the moving object is It is displayed on a navigation screen 101 on the display 4.

At this time, on the navigation screen 101, an image 126 showing the moving object is highlighted. Specifically, the video display frame 141 is displayed as a thick line in a predetermined color (for example, red). Further, a mark image 116 representing a moving object on the left side of the vehicle body is displayed at the position of the moving object on the map 102. The mark image 116 of this moving object is displayed in a highlighted manner, specifically, in a blinking manner. Thereby, it is possible to alert the driver to the moving object present on the left side of the vehicle body. Therefore, accidents involving two-wheeled vehicles (bicycles, etc.) when turning left can be prevented.

Note that in this embodiment, both the images 121 and 122 of the intersection in front taken by the roadside device 6 and the image 126 of the left side of the vehicle body taken by the camera of the own vehicle are displayed on the navigation screen 101. However, the image of the intersection ahead and the image of the left side of the vehicle may be switched depending on the behavior of the own vehicle (going straight, turning right, turning left). Specifically, if the vehicle is going straight or turning right, an image of the intersection ahead is displayed; if the vehicle is turning left, an image of the left side of the vehicle is displayed. so that

Next, a schematic configuration of the vehicle 1 according to the second embodiment will be described. FIG. 19 is a block diagram showing a schematic configuration of the vehicle 1. As shown in FIG.

The vehicle 1 includes an on-vehicle terminal 2, an automatic driving ECU 3, a navigation display 4, etc., as in the first embodiment (see FIG. 8), and is also provided with a camera 36.

The camera 36 photographs the surroundings of the own vehicle (at least the left side of the vehicle body).

The in-vehicle terminal 2 includes an ITS communication section 41, a wireless communication section 42, a positioning section 43, a memory 44, and a processor 45, as in the first embodiment (see FIG. 8). Further, as in the first embodiment, the processor 45 performs collision determination processing, video reception control processing, video display control processing, screen display control processing, etc., and in addition, performs moving body detection processing.

In the moving object detection process, the processor 45 detects a moving object (such as a bicycle) present on the left side of the own vehicle based on the detection result of a sensor (radar, lidar, etc.), and detects a moving object (such as a bicycle) on the left side of the own vehicle. Determine whether a moving object exists. Note that a moving object present on the left side of the vehicle body may be detected based on an image taken by a camera.

In the video display control process, the processor 45 performs control to display an image of the left side of the vehicle body taken by the camera of the vehicle on the navigation screen when the traveling direction of the vehicle is a left turn. At this time, an image of a moving object (such as a bicycle) on the left side of the vehicle is highlighted. Specifically, the video display frame is drawn with a thick line of a predetermined color (for example, red).

In the screen display control process, the processor 45 performs control to display a mark image (detection frame) pointing to a moving object on the left side of the vehicle body on the map of the navigation screen. At this time, the mark image of the moving object is highlighted. Specifically, the mark image of the moving object is displayed blinking.

Note that in this embodiment, the in-vehicle terminal 2 performs the process of detecting a moving body present on the left side of the vehicle 1 (mobile body detection process), but the roadside device 6 performs the moving body detection process. You can do it like this.

Next, the operating procedure of the in-vehicle terminal 2 and automatic driving ECU 3 according to the second embodiment will be explained. FIG. 20 is a flowchart showing the operating procedure of the on-vehicle terminal 2 and the automatic driving ECU 3. Note that the same processing as in the first embodiment (see FIG. 11) is performed in this embodiment as well.

The automatic driving ECU 3 detects a moving object (such as a bicycle) existing on the left side of the vehicle's body based on the detection result of a sensor (radar, lidar, etc.), and determines that the mobile object is present on the left side of the vehicle's body. It is determined whether or not (moving object detection processing) (ST311).

Here, if a moving object exists on the left side of the vehicle body (Yes in ST311), the automatic driving ECU 3 sends a moving object detection notification to the in-vehicle terminal 2 to the effect that a moving object exists on the left side of the vehicle body. (ST312).

When the in-vehicle terminal 2 receives the mobile object detection notification from the automatic driving ECU 3 (Yes in ST211), the processor 45 acquires information regarding the traveling direction of the own vehicle from the automatic driving ECU 3, and determines that the traveling direction of the own vehicle is a left turn. It is determined whether or not (ST212).

Here, if the traveling direction of the own vehicle is a left turn (Yes in ST212), the processor 45 instructs the camera 36 to photograph the left side of the vehicle where the moving object is present (ST213). The camera 36 photographs the left side of the vehicle where the moving object is located, in accordance with instructions from the vehicle-mounted terminal 2. Next, the processor 45 performs control to display an image of the left side of the vehicle body taken by the camera 36 of the own vehicle on the navigation screen (image display control process) (ST214). At this time, an image of a moving object (such as a bicycle) on the left side of the vehicle is highlighted.

Note that in this embodiment, the image displayed on the navigation screen is controlled according to the behavior of the own vehicle (whether it is turning left or not); however, in addition to the behavior of the own vehicle, The image displayed on the navigation screen may be controlled according to the behavior of other vehicles. Specifically, depending on the positional relationship between the own vehicle and other vehicles, either the image of the intersection in front taken by the roadside device 6 or the image around the vehicle body taken by the camera of the own vehicle is used. Alternatively, both the image of the intersection in front and the image of the surrounding area of the vehicle may be displayed. In addition, for example, in addition to cases where a two-wheeled vehicle is traveling on the left side of the own vehicle, other vehicles may be driving in an unnatural manner such as meandering, or other vehicles may be traveling at a speed exceeding a predetermined value. When the vehicle is in the vehicle, images around the vehicle body may be displayed preferentially.

(Third embodiment)
Next, a third embodiment will be described. Note that the points not particularly mentioned here are the same as those in the previous embodiment. FIG. 21 is an explanatory diagram showing an outline of the operation of the vehicle 1 according to the third embodiment.

In the embodiment described above, when the roadside device 6 detects the occurrence of an obstacle event, an image showing the obstacle event is displayed on the navigation display 4 of the vehicle 1. On the other hand, in this embodiment, when it is determined in the collision determination that there is a possibility of a collision with another vehicle (for example, a truck) approaching from behind the host vehicle, an image showing the other vehicle is displayed on the navigation display 4. to be done.

Note that in this embodiment, as in the first embodiment, collisions between the own vehicle and another vehicle or a pedestrian are detected based on the position information of the other vehicle acquired from the in-vehicle terminal 2 of the other vehicle through vehicle-to-vehicle communication. A collision determination is performed to determine the possibility.

Next, a navigation screen displayed on the navigation display 4 mounted on the vehicle 1 according to the third embodiment will be explained. FIG. 22 is an explanatory diagram showing the navigation screen 101.

In this embodiment, when it is determined in the collision determination that there is a possibility of a collision with another vehicle (for example, a truck) approaching from behind the host vehicle, an image 127 showing the other vehicle is displayed on the navigation screen 101. Ru.

At this time, on the navigation screen 101, an image 127 showing another vehicle approaching from behind the host vehicle is highlighted. Specifically, the video display frame 141 is displayed as a thick line in a predetermined color (for example, red). Further, a mark image 117 representing the predicted accident point is displayed at the position of the predicted accident point on the map 102. The mark image 117 of the predicted accident point is highlighted, and more specifically, is displayed blinking. Thereby, it is possible to alert the driver to other vehicles with which there is a possibility of collision.

Next, a schematic configuration of the vehicle 1 according to the third embodiment will be described. FIG. 23 is a block diagram showing a schematic configuration of the vehicle 1. As shown in FIG.

The vehicle 1 is equipped with an on-vehicle terminal 2, an automatic driving ECU 3, a navigation display 4, etc., as in the first embodiment (see FIG. 8), and is also equipped with a camera 37.

The camera 37 photographs the surroundings of the own vehicle (at least the rear of the vehicle).

The in-vehicle terminal 2 includes an ITS communication section 41, a wireless communication section 42, a positioning section 43, a memory 44, and a processor 45, as in the first embodiment (see FIG. 8). Further, the processor 45 performs collision determination processing, video reception control processing, video display control processing, screen display control processing, etc., as in the first embodiment.

In the video display control process, when the processor 45 determines through the collision determination process that there is a possibility of a collision with another vehicle approaching from behind, the processor 45 starts the camera 37 and displays the camera 37 so that the camera 37 displays the own vehicle. The system takes a picture of the rear of the vehicle, obtains an image of another vehicle approaching from behind, and displays that image on the navigation screen.

Next, the operation procedure of the in-vehicle terminal 2 according to the third embodiment will be explained. FIG. 24 is a flow diagram showing the operation procedure of the in-vehicle terminal 2. Note that the same processing as in the first embodiment (see FIG. 11) is performed in this embodiment as well.

The vehicle-mounted terminal 2 determines whether or not there is a possibility of a collision with another vehicle approaching from behind the host vehicle, based on the determination result in the collision determination process (ST221).

Here, if there is a possibility of a collision with another vehicle approaching from behind (ST221: Yes), the processor instructs the camera to photograph the rear of the vehicle where the moving object is present (ST222). The camera photographs the rear of the vehicle where the moving object is located, according to instructions from the processor.

Next, in the vehicle-mounted terminal 2, the processor performs control to display an image of the rear of the vehicle body taken by the camera of the own vehicle on the navigation screen (image display control process) (ST223). At this time, an image of a moving object (such as a truck) behind the vehicle is highlighted.

In addition, in this embodiment, when it is determined that there is a possibility of a collision with another vehicle in the collision determination based on the position information of the other vehicle obtained through vehicle-to-vehicle communication, the image of the rear of the vehicle that shows the other vehicle is Although it is displayed on the navigation display 4, the vehicle 1 is equipped with a sensor (such as a radar) that detects another vehicle approaching from behind the vehicle 1, and measures the distance between the vehicle 1 and the other vehicle. Regardless of the result of the collision determination, when the distance between the host vehicle and the other vehicle becomes less than or equal to a predetermined value, an image of the rear of the vehicle body showing the other vehicle may be displayed.

(Fourth embodiment)
Next, a fourth embodiment will be described. Note that the points not particularly mentioned here are the same as those in the previous embodiment. FIG. 25 is an explanatory diagram showing a navigation screen on the navigation display 4 mounted on the vehicle 1 and a front AR screen 201 on the front AR display 35 according to the fourth embodiment.

In this embodiment, similarly to the second modification of the first embodiment (see FIG. 14), images 121 and 122 taken by the roadside device 6 are displayed on the front AR screen 201 by the front AR display 35. .

In addition, in the above embodiment, the conditions outside the vehicle (external factors), that is, the occurrence of an obstacle event, moving objects (such as bicycles) around the own vehicle, and moving objects (such as trucks) that may collide with the own vehicle, are also discussed. etc.) as a trigger event to perform video display control that highlights images related to external factors. On the other hand, in this embodiment, video display control is performed to highlight a predetermined video using the driver's condition (internal factor) as a trigger event. In particular, in this embodiment, when the driver is looking aside, for example, when the driver is looking at the screen of a smartphone in hand, the image displayed on the front AR screen 201 is less than the driver's field of view. The video 121 displayed at the edge is highlighted.

In the example shown in FIG. 25, an image 121 of a non-line-of-sight road to the left where a moving object is present is used as an image of the most recent intersection, and a line-of-sight image 121 to the right where a moving object is present is used as an image of the next intersection after the most recent intersection. An image 122 of an outside road is displayed.

Here, if it is determined that the driver is looking inattentive because the driver is looking at the screen of the smartphone in hand, the most recent images 121 and 122 displayed on the front AR screen 201 An image 121 of a non-line-of-sight road to the left of the intersection is displayed at the edge of the driver's field of vision. Therefore, this video 121 is highlighted. Specifically, a video display frame 141 in which a video 121 of the nearest intersection is displayed is drawn with a thick line of a predetermined color (for example, red). As a result, the driver is guided to view the highlighted image 121 on the front AR screen 201, and the driver's attention can be alerted to a moving object that is present on the road outside the line of sight at the nearest intersection.

Next, a schematic configuration of the vehicle 1 according to the fourth embodiment will be described. FIG. 26 is a block diagram showing a schematic configuration of the vehicle 1. As shown in FIG.

Similarly to the second modification of the first embodiment (see FIG. 8), the vehicle 1 includes an on-vehicle terminal 2, an automatic driving ECU 3, a navigation display 4, a front AR display 35, and the like. It is equipped with an in-vehicle camera 38.

The in-vehicle camera 38 photographs the interior of the own vehicle (at least the range that includes the driver's face).

The in-vehicle terminal 2 includes an ITS communication section 41, a wireless communication section 42, a positioning section 43, a memory 44, and a processor 45, as in the first embodiment (see FIG. 8). Further, the processor 45 performs collision determination processing, video reception control processing, video display control processing, screen display control processing, etc., as in the first embodiment, and in addition, performs inattentiveness determination processing.

In the inattentiveness determination process, the processor 45 detects the direction of the driver's line of sight based on the image of the driver captured by the in-vehicle camera 38, and determines whether the driver is inattentive based on the direction of the driver's line of sight. Determine whether or not. Here, looking aside is when the driver is not looking in front of the vehicle through the windshield. For example, when the driver is looking at the screen of a smartphone in hand, the driver's line of sight is diagonally downward. In some cases.

In the video display control process, the processor 45 displays a video at the edge of the driver's field of vision based on the direction of the driver's line of sight obtained in the inattentiveness determination process and the display position of the video displayed on the front AR screen. Select the video that appears and highlight it.

Next, the operation procedure of the in-vehicle terminal 2 according to the fourth embodiment will be explained. FIG. 27 is a flow diagram showing the operation procedure of the in-vehicle terminal 2. Note that the same processing as in the first embodiment (see FIG. 11) is performed in this embodiment as well.

In the vehicle-mounted terminal 2, the processor detects the direction of the driver's line of sight based on the video of the driver captured by the in-vehicle camera (ST231). Then, based on the direction of the driver's line of sight, it is determined whether the driver is looking aside (ST232).

Here, if the driver is looking aside (Yes in ST232), the processor highlights the image of the road beyond the line of sight of the nearest intersection among the images displayed on the front AR screen (ST233). ). At this time, the mark image of the moving object appearing in the video is also highlighted.

As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in the present disclosure is not limited to this, and can also be applied to embodiments in which changes, replacements, additions, omissions, etc. are made. Furthermore, it is also possible to create a new embodiment by combining the components described in the above embodiments.

For example, in the present disclosure, in order to reduce the communication load related to video distribution, when a moving object is present on the road around an intersection, the roadside device 6 transmits a video of the moving object to the server 7 ( In addition, the in-vehicle terminal 2 is configured to receive (upload) videos taken of moving objects from the server 7, but in order to further reduce the communication load related to video distribution, uploading and downloading of this video is required. However, this is limited to cases where a disturbance event (traffic accident, traffic jam, road construction, crowd, emergency vehicle, etc.) is detected, or when a highly dangerous situation is detected due to the behavior of other vehicles or pedestrians in the vicinity. You can do it like this. Conversely, if the communication environment is good, video uploading and downloading may be performed at all times.

In addition, in order to reduce the communication load associated with video distribution, depending on the importance (urgency) of the video, we can display the video as a video or still image, change the resolution, or change the display range of the video. You may. For example, if the image is of high importance, such as when the detected moving object is highly dangerous, the image may be displayed as a video, the image may be displayed in high resolution, and the image may be zoomed to the moving object. Display a narrower area of the image that has been uploaded. Conversely, when the importance of the video is low, the video is displayed as a still image, the video is displayed at low resolution, or a wide range of video is displayed.

In addition, in order to reduce the communication load associated with video distribution, the frame rate when displaying video as a video and the update rate when displaying video as a still image are changed depending on the speed of the vehicle and other factors. Good too. Specifically, when the vehicle is traveling at a relatively high speed, such as during normal driving, the video frame rate and still image update rate are set high, and when the vehicle is traveling at a relatively high speed, such as during traffic congestion. If it is slow, set the video frame rate and still image update rate low. Note that the frame rate of moving images and the update rate of still images may be changed depending on the location, surrounding conditions, and the like. For example, the frame rate of moving images and the update rate of still images may be set high at intersections or in places where there are many traffic accidents. Further, for example, when a dangerous person or vehicle is nearby, the frame rate of the video and the update rate of the still image may be set high.

Further, in order to reduce the communication load related to video distribution, the resolution of the video of the intersection displayed on the screen may be changed depending on the distance from the vehicle. Specifically, images of intersections close to the vehicle are displayed in high resolution, and images of intersections far from the vehicle are displayed in low resolution. Note that, like the frame rate of moving images and the update rate of still images, the resolution of video may also be changed depending on the location, surrounding conditions, and the like.

Furthermore, in the present disclosure, for example, the in-vehicle terminal 2 and the roadside device 6 may be linked with the cloud. Specifically, the video information collected by the in-vehicle terminal 2 and the roadside device 6 is sent to a cloud-side device, and the cloud-side device analyzes locations, dates, and times where traffic accidents and aggressive driving are likely to occur. However, vehicles traveling near the location obtained as a result of the analysis may be notified. This makes it possible to reduce traffic accidents, aggressive driving, and the like. Further, when a dangerous vehicle is detected, for example, by automatically notifying the police etc. immediately, it is possible to prevent damage caused by aggressive driving.

The driving support method, roadside device, and in-vehicle device according to the present invention allow a driver to immediately identify a video showing a cautionary event occurring on the road, and to help the driver identify the specific situation of the cautionable event. The present invention has an effect that can be confirmed quickly, and is useful as a driving support method for supporting the driving operations of a driver in a vehicle, a roadside device installed on a road, an on-vehicle device mounted on a vehicle, and the like.

1 Vehicle 2 In-vehicle terminal (in-vehicle device)
3 Automatic driving ECU (driving control unit)
4 Navigation display (display device)
5 Pedestrian terminal (pedestrian device)
6 Roadside equipment (roadside equipment)
7 Server (distribution device)
12 Radar 13 Camera 35 Front AR display (display device)
36 Camera 37 Camera 38 In-vehicle camera 101 Navigation screen 102 Maps 111 to 117 Mark images 121 to 127 Videos 131, 132 Mark images 141 Video display frame 201 Front AR screen 211 Guide image

Claims (15)

A roadside device installed on the road uses a camera to take pictures of the road around it,
An on-vehicle device mounted on a vehicle acquires the image captured by the roadside device,
a display device mounted on the vehicle displays the video;
When the roadside device detects a specific event that occurs on the road around the device,
the display device highlights the video showing the specific event;
When the in-vehicle device determines that the driver of the own vehicle is inattentive,
A driving support method characterized in that the display device highlights the image displayed at the edge of the driver's field of vision among the images being displayed. the in-vehicle device acquires a plurality of images taken by the roadside device installed at each of a plurality of intersections on the travel route of the vehicle;
2. The driving support method according to claim 1, wherein the display device displays each of the plurality of videos at a position on the screen corresponding to a position of the roadside device that captured the video. The in-vehicle device or the roadside device sets a display priority for each of the plurality of videos,
The driving support method according to claim 2, wherein the display device displays the video based on the priority. 3. The driving support method according to claim 2, wherein the display device displays the video having a higher priority in a larger size than the video having a lower priority. 3. The driving support method according to claim 2, wherein the display device displays the video with a high priority over a video display frame of the video with a low priority. Claim 3, wherein the in-vehicle device or the roadside device sets the priority for each video based on the importance of an event captured in the video and the distance from the vehicle to a shooting point. 6. The driving support method according to claim 5. 2. The driving support method according to claim 1, wherein the display device displays a mark image representing a moving direction of the moving object on the video image. 2. The driving support method according to claim 1, wherein the specific event is an obstruction event that obstructs the passage of a vehicle. When the in-vehicle device detects a moving object existing around the own vehicle and determines that there is a possibility of transitioning to a highly dangerous state depending on the behavior of the own vehicle,
2. The driving support method according to claim 1, wherein the display device displays an image of the moving object taken by a camera mounted on the own vehicle. the roadside device transmits the video to a distribution device,
The driving support method according to claim 1, wherein the in-vehicle device receives the video from the distribution device. The roadside device transmits video link information of the own device to the in-vehicle device directly or via another roadside device,
The driving support method according to claim 10, wherein the in-vehicle device receives the video from the distribution device based on the video link information. A roadside device installed on the road uses a camera to take pictures of the road around it,
An on-vehicle device mounted on a vehicle acquires the image captured by the roadside device,
a display device mounted on the vehicle displays the video;
When the roadside device detects a specific event that occurs on the road around the device,
the display device highlights the image showing the specific event;
The display device is an AR display that displays an AR screen on which a virtual object is superimposed on the real space visible to the driver,
The image is displayed as the virtual object in a space outside the vehicle that can be seen by the driver through the window glass ,
Furthermore, it is equipped with a navigation display that shows the direction of travel of the vehicle on a map.
A driving support method characterized in that a mark image of a moving body drawn on a video on the AR screen and a mark image of a moving body drawn on a map on the navigation screen are the same color. 13. The AR screen displays a guide image that associates the video displayed on the AR screen with a mark image of a moving object displayed on the navigation screen. driving assistance method. A roadside device receives images of surrounding roads from the roadside device directly or via a distribution device, and when a specific event is detected in the roadside device, information to that effect is sent directly or via a distribution device. a communication unit that receives data via another roadside device;
a processor that displays the image on a display device mounted on the vehicle and, upon receiving information indicating that the specific event has been detected, highlights the image that shows the specific event ;
If the processor determines that the driver of the own vehicle is looking aside, the processor highlights the image displayed at the edge of the driver's field of vision among the images being displayed. Device. A roadside device receives images of surrounding roads from the roadside device directly or via a distribution device, and when a specific event is detected in the roadside device, information to that effect is sent directly or via a distribution device. a communication unit that receives data via another roadside device;
a processor that displays the image on a display device mounted on the vehicle and, upon receiving information indicating that the specific event has been detected, highlights the image that shows the specific event;
The display device is an AR display that displays an AR screen on which a virtual object is superimposed on the real space visible to the driver,
The image is displayed as the virtual object in a space outside the vehicle that can be seen by the driver through the window glass,
Furthermore, it is equipped with a navigation display that shows the direction of travel of the vehicle on a map.
An in-vehicle device characterized in that a mark image of a moving body drawn on a video on the AR screen and a mark image of a moving body drawn on a map on the navigation screen are the same color.

Images