JP7491059B2 - Driving assistance control device, driving assistance control program - Google Patents

Description

The present invention relates to a driving assistance control device and a driving assistance control program for supporting the movement of a moving object such as a vehicle when providing remote driving assistance for autonomous driving in a specific location, taking into consideration accidents that may occur in the location.

In recent years, developments have been progressing in the area of autonomous vehicles, particularly in the area of autonomous driving.

Along with the development of such autonomous vehicles, progress is being made with the development of fail-operational technology to ensure the safety of the vehicle if an abnormality or failure occurs in the onboard system during autonomous driving.

One type of fail-operational technology is a technology that, in the case of autonomous driving, when a predetermined operating program is not anticipated and the operating program is blocked, making it impossible to continue autonomous driving, operates the moving object from a remote location to avoid areas where autonomous driving is difficult, move the object to a location where autonomous driving is possible, and restore autonomous driving.

Patent document 1 describes how, when remotely controlling a moving object capable of autonomous driving, the autonomous driving trajectory is correctly superimposed on the remote control screen.

More specifically, Patent Document 1 describes a technology that holds trajectory information indicating the autonomous driving trajectory of a moving object and camera information including the position, orientation, and viewing angle of a camera installed on the moving object, stores the received position of the moving object and the time of its capture, and upon receiving an image captured by the camera and its capture time, displays the received image and stores the capture time, estimates the position and orientation of the moving object at the time of image capture based on the image capture time and the position of the moving object at each capture time, identifies a portion of the trajectory that is included within the range of the image based on the trajectory information, camera information, and estimated position and orientation of the moving object, converts the coordinates of the identified portion of the trajectory into coordinates on the image based on the trajectory information, camera information, and estimated position and orientation of the moving object, and displays an image with the trajectory superimposed on the converted coordinate position.

JP 2019-016188 A

However, when providing remote assistance, simply superimposing the autonomous driving trajectory that suggests the destination onto an image, as in the technology of Patent Document 1, makes it difficult to provide remote operation that takes into account the risk of accidents that may occur in the surrounding area.

The present invention aims to provide a driving assistance control device and a driving assistance control program that can provide a party receiving remote assistance with appropriate information that enables the party to safely pass through a driving site by utilizing past accident data.

The driving assistance control device of the present invention is a driving assistance control device that remotely supports the driving of a vehicle performing automatic driving and provides driving assistance information when an autonomous driving impossible factor occurs in the vehicle, and has a receiving unit that receives vehicle information including at least position information indicating the driving position transmitted by the vehicle in which the autonomous driving impossible factor occurs and a surrounding image of the driving position when the autonomous driving impossible factor occurs, an accident pattern database in which accident pattern data is stored in which a plurality of past accident cases are associated with traffic conditions including at least one of road structure, vehicle behavior, and other party information, an analysis unit that reads out accident pattern data related to the vehicle information received by the receiving unit from the accident pattern database based on the vehicle information received by the receiving unit, and superimposes the analysis result obtained by analyzing the read accident pattern data on the surrounding image included in the vehicle information received by the receiving unit to generate driving assistance auxiliary information, and a providing unit that provides the driving assistance auxiliary information generated by the analysis unit to the vehicle.

The driving assistance control device according to the present invention is characterized in that it operates a computer as each part of the driving assistance control device.

According to the present invention, it is possible to provide the remotely assisted party with appropriate information that enables the party to safely pass through a driving site by utilizing past accident data.

1 is a schematic diagram of a driving assistance system including a driving assistance control device that assists in autonomous driving of a vehicle according to a first embodiment. 1 is a functional block diagram for executing control for providing driving assistance information executed by a driving assistance control device in response to the occurrence of an autonomous driving impossible factor from an automatic driving control device according to a first embodiment. FIG. 13 is an example of a structural diagram visualizing the storage state of cases classified by traffic conditions in the accident pattern database 156. 1A is a front view of an image of a vehicle in the forward driving direction, and FIG. 1B is a driving assistance information image. 4 is a control flowchart for generating driving assistance information in the driving assistance control device according to the first embodiment. 1 is a front view of a presented image in a driving support auxiliary information presentation unit according to Example 1 of the first embodiment. FIG. 13 is a front view of a presented image in a driving support auxiliary information presenting unit according to Example 2 of the first embodiment. FIG. 13 is a front view of a presented image in a driving support auxiliary information presenting unit according to Example 3 of the first embodiment. FIG. 13A is a characteristic diagram relating to Example 4 of the first embodiment, in which (A) is a two-dimensional characteristic diagram of vehicle speed vs. accident frequency, (B) is a clustering characteristic diagram based on the three axes of vehicle speed, visibility, and traffic volume, and (C) is a characteristic diagram of vehicle speed vs. accident frequency for a cluster specialized for when an autonomous vehicle is stopped. 13 is a control flowchart showing the flow of a clustering process according to Example 4 of the first embodiment; 13A is a characteristic diagram relating to Example 5 of the first embodiment, in which (A) is a vehicle speed-accident frequency characteristic diagram by speed (three types), (B) is a vehicle speed-accident frequency characteristic diagram for accident-presence data and accident-free data, (C) is a characteristic diagram in which a deceleration distribution is superimposed on the vehicle speed-accident frequency characteristic diagram, and (D) is a legend diagram of a box plot used as the deceleration distribution. 13A and 13B are images presented by a driving support auxiliary information presentation unit according to Example 6 of the first embodiment, in which (A) is a front view in the vehicle travel direction, and (B) is a plan view map. 13A and 13B are images presented by a driving support auxiliary information presentation unit according to Example 7 of the first embodiment, in which (A) is a front view in the vehicle travel direction, and (B) is a plan view map. 13A and 13B are images presented by a driving support auxiliary information presentation unit according to Example 8 of the first embodiment, in which (A) is a front view in the vehicle travel direction, and (B) is a plan view map. 13A and 13B are images presented by a driving support auxiliary information presentation unit according to Example 9 of the first embodiment, in which (A) is a front view in the vehicle travel direction, and (B) is a plan view map. 13A and 13B are images presented by a driving support auxiliary information presentation unit according to a tenth example of the first embodiment, in which (A) is a front view in the vehicle travel direction, and (B) is a plan view map. 13A and 13B are images presented by a driving support auxiliary information presentation unit according to an eleventh embodiment of the first embodiment, in which (A) is a front view in the vehicle travel direction, and (B) is a plan view map. FIG. 11 is a block diagram of an automatic driving control device according to a second embodiment, showing a pattern form for receiving driving route information. 10 is a control flowchart for generating driving assistance information in a driving assistance control device according to a second embodiment.

{First embodiment}

Figure 1 shows a schematic diagram of a driving assistance system including a driving assistance control device 12 that assists the driving of an autonomous vehicle (hereinafter sometimes simply referred to as vehicle 10) according to a first embodiment.

The vehicle 10 is equipped with a vehicle control device 14 and an automatic driving control device 20.

The vehicle control device 14 performs control including the drive system (engine control, etc.) and the electrical system (fault diagnosis using status detection sensors for each part, etc.) when the vehicle 10 is running.

A group of cameras (FIG. 1 shows, as an example, a front camera 16A, a left front camera 16B, a left rear camera 16C, a right front camera 16D, a right rear camera 16E, and a rear camera 16F) that capture images of the surroundings of the vehicle 10 are connected to the vehicle control device 14 (collectively referred to as the "camera group 16"). In addition, a radar group 18 equipped with multiple millimeter wave radars and LIDARs is connected to the vehicle control device 14.

The autonomous driving control device 20 determines the driving operation to the destination based on the information required for autonomous driving from the vehicle control device 14 (e.g., the detection information from the camera group 16 and the radar group 18) and instructs the vehicle control device 14.

The automatic driving control device 20 is capable of communicating with the driving assistance control device 12 via the wireless communication device 22A of the network 22.

The driving assistance control device 12 collects driving history information from each vehicle 10 during automated driving, and allows the operator to provide driving assistance instructions as necessary. For this reason, the driving assistance control device 12 obtains real-time road information from the real-time map management system 24 via the network 22.

The real-time map management system 24 has the function of aggregating information from infrastructure such as cameras installed on roads, etc., and road information transmission devices installed in vehicles 10, etc., and analyzing the current road conditions (including road restrictions, on-street parking, etc.).

Here, while the autonomous vehicle 10 is autonomously driving, a factor that prevents autonomous driving may occur that hinders the vehicle's driving, making it difficult to continue driving thereafter.

An example of a factor that makes autonomous driving impossible is when a parked vehicle 26 (see FIG. 4, etc.) is present ahead in the direction of travel, and the autonomous vehicle 10 needs to veer into the oncoming lane to avoid the parked vehicle, making it impossible for the autonomous vehicle 10 to travel on its own judgment.

When a factor that makes autonomous driving impossible occurs, conventional driving assistance control is limited to providing driving assistance by superimposing the driving trajectory during autonomous driving, indicating the destination, on an image (a driving front image) captured by the autonomous vehicle 10. This is insufficient as driving assistance to safely continue driving the autonomous vehicle 10, taking into account the risk of accidents that may be lurking in the surrounding area.

In other words, when a factor that makes autonomous driving impossible occurs, in addition to normal driving assistance, it is preferable to provide driving assistance information that includes appropriate warning information that makes it easier to consider the risk of an accident in the surrounding area when resuming autonomous driving.

However, until now, there has not been enough material (information) available to create driving assistance information that includes appropriate warning information.

In the first embodiment, the driving assistance control device 12 analyzes the similarity with pre-stored past accident patterns based on surrounding traffic conditions such as vehicle data including the behavior of the vehicle (vehicle 10) and, if there is another vehicle, the type and behavior of the other vehicle (e.g., the parked vehicle 26 shown in FIG. 4), in addition to images of the area around the point where the vehicle 10 has difficulty in autonomous driving.

In addition, the analysis results can be used to generate driving assistance auxiliary information (such as a speed warning area image to be superimposed on the driving image) to efficiently determine safe driving assistance through remote control, and this information can be managed by the operator.

Figure 2 is a functional block diagram for executing control to provide driving assistance information executed by the driving assistance control device 12 in response to the occurrence of an autonomous driving impossibility factor from the automatic driving control device 20 according to the first embodiment. Note that each block in Figure 2 is classified by function and does not limit the hardware configuration of the driving assistance control device 12.

When a preset factor that makes autonomous driving impossible occurs while the autonomous vehicle 10 is autonomously driving, the autonomous driving control device 20 (see FIG. 1) transmits information indicating the occurrence of the factor that makes autonomous driving impossible and vehicle information including a forward image at the time when the factor that makes autonomous driving impossible occurred to the driving assistance control device 12.

The driving assistance control device 12 is equipped with a vehicle information receiving unit 150, and receives vehicle information transmitted from the autonomous driving control device 20 of the autonomous vehicle 10.

The vehicle information receiving unit 150 is connected to the accident pattern analysis unit 152, and sends the received vehicle information to the accident pattern analysis unit 152.

The accident pattern analysis unit 152 sends search information to the accident pattern information search instruction unit 154 to obtain related accident pattern information based on the vehicle information.

The accident pattern information search instruction unit 154 is connected to the accident pattern database 156 and notifies the accident pattern database 156 of the search information received from the accident pattern analysis unit 152.

The accident pattern database 156 extracts past accident pattern data associated with the search information and sends it to the accident pattern analysis unit 152.

(An example of the structure of the accident pattern database 156)

Figure 3 shows an example of a structural diagram that visualizes the storage state of cases classified by traffic conditions in the accident pattern database 156.

Traffic conditions are classified into road structure, vehicle behavior, and other vehicle information.

In addition, road structure, vehicle behavior, and other vehicle information are each subdivided.

(Road structure)

Road structure items include intersections, unsignalized intersections, store entrances, and pedestrian crossings.

(Vehicle behavior)

Vehicle behavior items include not stopping temporarily, going straight, turning right, and turning left.

(opponent information)

Other party information items include accident party, entry from the left, and entry from the right.

The pattern ID is created by assigning a tag to each item of traffic conditions (road structure, vehicle behavior, and other party information) to identify whether it applies or not (a code of "1" is assigned if it applies, and a code of "0" is assigned if it does not apply) and classifying it.

As an exception, the other parties involved in accidents are classified as follows: "1": vehicles, "2": pedestrians, "3": bicycles, etc.

Each pattern ID is assigned and stored as an example [1], [2], [3], etc.

As shown in FIG. 2, the accident pattern analysis unit 152 aggregates vehicle information received from the vehicle 10 and past accident pattern data received from the accident pattern database 156, and analyzes these using AI techniques such as machine learning and deep learning to create driving support auxiliary information that makes it easier for the operator to consider the risk of an accident when deciding to resume autonomous driving of the vehicle 10. Note that analysis using AI techniques such as machine learning and deep learning is just one example and is not required.

The accident pattern analysis unit 152 is connected to the driving support auxiliary information presentation unit 164, and sends the analysis result, that is, the driving support auxiliary information, to the driving support auxiliary information presentation unit 164.

The driving support auxiliary information presentation unit 164 is connected to the update unit 166 and the output device 168A of the user interface 168, and sends the driving support auxiliary information to the update unit 166 and outputs it to the output device 168A. The output device 168A may be a monitor or a printer, but in the first embodiment, it includes at least a monitor.

The driving support auxiliary information output to the output device 168A can be confirmed (visually confirmed) by the operator, and after the operator confirms the driving support auxiliary information, the update unit 166 uses the input device 168B (keyboard, mouse, etc.) of the user interface 168 to update the driving support auxiliary information received from the driving support auxiliary information presentation unit 164. Updating refers to editing such as deleting, changing, or adding images on the monitor, and the driving support auxiliary information becomes driving support information by updating the information in the update unit 166, and is sent to the driving support information transmission unit 170.

The driving assistance information transmission unit 170 transmits the driving assistance information to the vehicle 10.

Figure 4 (A) shows the driving support information image 28 output to the output device 168A (monitor), and Figure 4 (B) shows the driving support information image 30 after the information has been updated by the operator.

In the example of Figure 4, a parked vehicle 26 (truck) is stopped (parked) in front of an intersection, and the driving support auxiliary information image 28 in Figure 4 (A) displays a warning area 32 in part of the intersection, as well as a vehicle speed vs. accident frequency characteristic diagram (graph information 34). In addition, text information 36 (for example, "Watch out for pedestrians rushing out into the intersection, 10km/h↓") is displayed to warn the driver when passing through the intersection.

The operator visually checks this driving assistance supplementary information and fine-tunes it to enlarge the attention-warning area 32 at the intersection (see the updated driving assistance information in Figure 4 (B)).

The accident examples in the accident pattern database are collected when an accident actually occurs, so the collected data may be biased, such as an insufficient number of accident examples depending on the pattern. Driving support auxiliary information generated from such data may not be reliable enough to be sent to the vehicle as is, but by prompting the operator to update as shown in Figure 4, the driving support information is more in line with the operator's senses than the driving support auxiliary information, and the driving safety of the autonomous vehicle 10 (see Figure 1) is improved.

The operation of the first embodiment is explained below.

Figure 5 is a control flowchart for generating driving assistance information in the driving assistance control device 12 according to the first embodiment. Note that the flowchart in Figure 5 is one of various controls that are executed in a sequential loop in the driving assistance control device 12.

In step 100, it is determined whether or not a call has been made from the autonomous vehicle (vehicle 10). If the determination in step 100 is negative, the routine ends.

If the result of step 100 is affirmative, the process proceeds to step 102, where the position information of the vehicle 10 is received, and current position traffic information is received, for example, from the real-time map management system 24, and the process proceeds to step 104.

In step 104, past accident pattern examples (e.g., accident examples [1] to [13] shown in FIG. 3) for traffic conditions similar to the current traffic information are obtained from the accident pattern database 156, and the process proceeds to step 106.

In step 106, the accident pattern analysis unit 152 performs an analysis (for example, machine learning or deep learning using AI), and then proceeds to step 108, where driving support auxiliary information is presented in which the speed caution area (area 32 shown in FIG. 4(A) is superimposed on the forward image 28 of the vehicle 10, and output to the output device 168A. This allows the operator to check the output result (display) on the output device 168A (monitor).

In the next step 110, the input device 168B performs an update to increase the attention generated by the operator (updating the driving support auxiliary information to driving support information). In the first embodiment, the area 32 shown in FIG. 4(A) is expanded and edited to the area 32 shown in FIG. 4(B).

In the next step 112, the updated driving assistance information (image information) is sent to the vehicle 10 from which it was read in step 100, and this routine ends.

In this way, in the first embodiment, when an autonomous vehicle (vehicle 10) becomes unable to continue autonomous driving due to various obstacles, it receives driving assistance information from a remote driving assistance center (driving assistance control device 12) to continue autonomous driving. In particular, the location of the obstacle and an image of the road ahead are received from vehicle 10, and driving assistance auxiliary information is presented based on past accident pattern examples (big data) stored in accident pattern database 156 (for example, by learning using AI), and, if necessary, driving assistance information that has been further fine-tuned appropriately by the operator is transmitted to vehicle 10. This makes it possible to provide the remote assistance party with appropriate information that allows them to safely pass through the driving site by utilizing past accident data.

The following describes examples (Examples 1 to 11) of the image presented by the driving support auxiliary information presentation unit 164 in the first embodiment, the emphasis when presenting the presented image, the procedure for generating information to be superimposed on the presented image, and the adjustment of the presented image (driving support auxiliary information) to the driving support information operated by the operator and transmitted to the vehicle 10. Note that Examples 1 to 5 describe examples of information presented to the operator, and Examples 6 to 11 describe examples of operator operation.

(Example 1)

Figure 6 shows an example of the display of the image presented by the driving support auxiliary information presentation unit 164 (Example 1).

In Example 1, traffic condition tag information is obtained from the accident pattern database 156 on the image 40 of the area ahead in the vehicle's driving direction, and the content assumed from the obtained traffic condition tag information is superimposed as text information 36a as an assumed accident pattern.

The text information 36a of the assumed accident pattern in FIG. 6(A) ("At an intersection without traffic lights, a vehicle enters from the left while the vehicle is going straight") is of one type and is superimposed on the bottom of the image 40.

On the other hand, there are two types of text information 36a ("At an intersection without traffic lights, while the vehicle is going straight, a vehicle enters from the left") and text information 36b ("Near a parked vehicle, while the vehicle is going straight, a pedestrian jumps out from the shadow of the parked vehicle") for the assumed accident pattern in FIG. 6(B), which are superimposed on the bottom of the image 40.

In this case, in FIG. 6B, for example, if text information 36a is more important than text information 36b, the color of text information 36a can be changed to a color that visually attracts more attention (e.g., red), the font can be changed, or the size (point) of the text can be changed, to present the operator with a higher level of attention. Note that FIG. 6B shows an example in which the font is changed. While viewing the image, the operator may refer to predetermined selection requirement information (including the number of accidents, the degree of fault in the accidents, and the amount of damages and insurance money incurred in relation to the accidents) to input (i.e., select the display direction) whether or not to display text information 36a and 36b (see display form 1 to display form 3 shown in Example 3 below).

(Example 2)

Figure 7 shows an example of the display of the image presented by the driving support auxiliary information presentation unit 164 (Example 2).

In Example 2, information analyzed by the accident pattern analysis unit 152 is superimposed on the image 40 of the area ahead in the vehicle's traveling direction.

In FIG. 7(A), area 42 (intersection) and textual information 36c ("Watch out for pedestrians rushing out at intersection, 10km/h↓") for warning purposes and graph information 34 ("Vehicle speed vs. accident frequency characteristic diagram") are superimposed on image 40.

(Area Generation Method)
Region 42 may be determined, for example, by identifying and superimposing a relevant portion of the image based on tags relating to road structure in the accident pattern data (e.g., an intersection without traffic lights) (using AI technology such as deep learning, for example).

(Velocity distribution; method of generating graph information)
The graph information 34 may be generated, for example, using vehicle speed data of accident examples in the accident pattern data, and superimposed on the image.

(How to generate 10km/h in the message "Watch out for pedestrians rushing out at the intersection, 10km/h↓")
The vehicle speed information in the text information 36c may be, for example, a lower limit value of a vehicle speed distribution generated from accident examples in the accident pattern data. Alternatively, a value obtained by subtracting a value that is experimentally or empirically set in advance from the lower limit value may be used.

In FIG. 7B, area 44 (crosswalk) and warning text information 36e ("Watch out for pedestrians, 5km/h↓") are superimposed on image 40.

In FIG. 7C, area 46 (range in which other vehicle 48 may jump out), text information 36f ("Watch out for other vehicle jumping out, 5km/h↓"), and text information 36g ("Parking lot entrance") are superimposed on image 40. Text information 36g may be a reason why another vehicle may jump out.

In FIG. 7(D), area 46 (range in which another vehicle 48 may jump out), text information 36h ("Watch out for another vehicle jumping out, 5km/h↓"), and text information 36i ("side road") are superimposed on image 40. Text information 36i may be a reason why another vehicle may jump out.

(Example 3)

Figure 8 is a display example (Example 3) of a presentation image in the driving support auxiliary information presentation unit 164. That is, the presentation image shown in Figure 8 is displayed on the output device 168A (monitor), and the operator selects the display content (attention alert information), etc.

In Example 3, images of an area 42 (intersection) where an accident may occur and an area 50 (blind spot caused by parked vehicles 26) analyzed by the accident pattern analysis unit 152 are superimposed on an image 40 in the forward direction of vehicle travel.

Areas 42 and 50 are classified according to the difference in visual display form such as color or pattern according to the following conditions. In FIG. 8, area 42 (intersection) with a relatively high degree of emphasis is shown with a diagonal checkered pattern, and area 50 (blind spot caused by parked vehicles 26) with a relatively low degree of emphasis is shown with a vertical and horizontal checkered pattern. While looking at the image, the operator may refer to predetermined selection requirement information (including the number of accidents, the degree of fault in the accidents, and the amount of damages and insurance money incurred in relation to the accidents) and input (i.e., select the display method), for example, whether or not areas 42 and 50 are to be displayed (see Display Forms 1 to 3 shown below).

"Display form 1" The number of accidents is displayed together, or the number of accidents is highlighted according to the degree of emphasis (it is also possible to display only the accident with the greatest number of accidents, as in Example 2).

"Display format 2" Displays the degree of fault in the accident, or displays with a degree of emphasis according to the degree of fault in the accident (for example, when a child runs out into the road, the other party's share of fault is high, so more care is required).

"Display format 3" Displays according to the magnitude of the cost (amount of damage, insurance amount), or displays with a degree of emphasis according to the cost (the cost is also registered when past accident examples are registered in the accident pattern database 156).

(Example 4)

Example 4 of the first embodiment will be explained with reference to Figures 9 and 10.

Example 4 is an example for generating graph information 34 ([vehicle speed-accident frequency characteristic diagram]) superimposed on the driving support auxiliary information image 28 of Figure 4, for example.

When the graph information 34 directly utilizes the statistical values of cases estimated from the traffic condition tag information read from the accident pattern database 156, there may be cases where peak values of the accident frequency exist at different speeds, as shown in Figure 9 (A).

This FIG. 9(A) may be directly applied as the attention-calling graph information 34A used in FIG. 4, etc.

On the other hand, in Example 4, a clustering analysis is performed on accident examples according to the situation using graph information 34. As shown in FIG. 9(B), when each accident example is plotted on a feature space with three axes of speed, visibility, and traffic volume, the situation is classified into several clusters. FIG. 9(B) shows an example where the accident examples are classified into cluster 52A, cluster 52B, and cluster 52C. Note that the situation may be replaced with other situations such as time of day, or may be increased.

Figure 9 (C) is graph information that extracts the distribution of clusters to which the host vehicle (vehicle 10) is stopped, among the clusters analyzed by clustering in Figure 9 (B), and corresponds to, for example, graph information 34 used in Figure 4.

Figure 10 is a control flowchart showing the processing steps (A) to (C) of Figure 9, which are executed by the accident pattern analysis unit 152.

In step 54, accident examples are extracted from the accident pattern database 156, and a vehicle speed vs. accident occurrence frequency characteristic diagram is created (see Figure 9 (A)).

In the next step 56, a clustering process is performed for each accident example in the feature space (see Figure 9 (B)), and the process proceeds to step 58.

In step 58, the cluster to which the current situation of the vehicle (the calling vehicle 10) belongs (e.g., stopped), is calculated, and then the process proceeds to step 60, where graph information of the speed distribution characteristics (see FIG. 9(C)) based on the calculated cluster is calculated, and the routine ends.

(Example 5)

The fifth example of the first embodiment will be explained with reference to Figure 11.

In Example 5, a variation of graph information is shown that is superimposed on the driving support auxiliary information image 28 (see, for example, FIG. 4) for the vehicle ahead in the driving direction.

As mentioned above, the graph information may be graph information 34 in which a single characteristic curve is drawn as shown in FIG. 4, but the following variations are possible:

"Variation 1"

In contrast, variation 1 is an example in which three types of graph information are drawn side by side, with different line types or colors, as shown in Figure 11 (A). In Figure 11 (A), graph information 34B, which has a peak in accident frequency at 30 km/h, is shown by a dotted line, graph information 34C, which has a peak in accident frequency at 40 km/h, is shown by a solid line, and graph information 34D, which has a peak in accident frequency at 50 km/h, is shown by a dashed and dotted line.

In addition, all the lines may be solid, with graph information 34B having a peak in accident frequency at 30 km/h shown in yellow, graph information 34C having a peak in accident frequency at 40 km/h shown in red, and graph information 34D having a peak in accident frequency at 50 km/h shown in brown.

"Variation 2"

In variation 2, as shown in FIG. 11(B), in addition to the graph information 34 (see FIG. 4), graph information 34E of an example without an accident is added. In this case, traffic information (non-accident data) at each point is obtained, for example, from the real-time map management system 24 (it may be stored as statistical information for each point separately from the accident pattern database).

Graph information 34 and graph information 34E may be differentiated by line type or color, as in variation 1.

"Variation 3"

In variation 3, as shown in Figure 11 (C), in addition to the graph information 34 (see Figure 4), the distribution of deceleration is added.

In other words, even when an accident occurs, the speed distribution (graph information 34) can be displayed with the deceleration distribution when driving is possible, including not only the occurrence of an accident but also so-called "near misses," so that a more detailed situation can be grasped. In this case, traffic information (non-accident data, in particular near miss data that does not lead to an accident) at each point is obtained, for example, from the real-time map management system 24, etc. (this may be stored as statistical information for each point separately from the accident pattern database).

To show the distribution of deceleration, for example, box and whisker plots 62g1 to g7 are superimposed on seven speeds in the graph information 34. As shown in FIG. 11(D), the box and whisker plot 62 is a distribution range of a certain number of plots from the top to the bottom of the box, and the line inside the box indicates the peak value. By superimposing it on the graph information 34 in FIG. 11(C), the frequent deceleration at each speed can be visually clearly expressed. In this case, traffic information (non-accident data) at each point is obtained, for example, from the real-time map management system 24 (it may be held as statistical information for each point separately from the accident pattern database).

Consideration is given to Figure 11(C).

Looking at the vehicle speed distribution (graph information d1) of the accident examples, the frequency is high, but looking at the distribution of deceleration using the box plot 62, it can be seen that sudden deceleration did not occur.

The vehicle speed distribution in the accident examples (graph information d1) alone can only suggest that the vehicle speed should be significantly reduced, but by superimposing the distribution of deceleration when the vehicle is able to travel without problems (box and whisker plot 62), the vehicle speed can be set a little higher.

(Example 6)

The sixth example of the first embodiment will be explained with reference to Figure 12.

Figure 12 shows a driving support information image 30 that is the result of input operations by the operator using the input device 168B based on the driving support information received from the driving support information presentation unit 164 and displayed on the output device 168A (monitor) of the user interface 168, and corresponds to the driving support information image 30 in Figure 4 (B).

In FIG. 12(A), an area 32 (here, an intersection) in which the vehicle 10 is desired to travel at a slow speed is drawn on the image ahead in the direction of travel of the vehicle 10 by operating the input device 168B.

In addition, in FIG. 12(B), an area 32 (here, an intersection) in which the vehicle 10 is desired to travel at a low speed is drawn on a planar map image (which may be a bird's-eye view) in which the vehicle 10 is viewed from above by operating the input device 168B.

(Example 7)

The seventh example of the first embodiment will be explained with reference to Figure 13.

Figure 13 shows a driving support information image 30 that is the result of input operations by the operator using the input device 168B based on the driving support information received from the driving support information presentation unit 164 and displayed on the output device 168A (monitor) of the user interface 168, and corresponds to the driving support information image 30 in Figure 4 (B).

In FIG. 13(A), an area 50 in which the driver is requested to stop (in this case, in front of the parked vehicle 26) is drawn on the image ahead in the traveling direction of the vehicle 10 by operating the input device 168B.

In addition, in FIG. 13(B), an area 50 in which the driver is requested to stop (here, in front of the parked vehicle 26) is drawn on a planar map image 28A (which may be a bird's-eye view) showing the vehicle 10 from above by operating the input device 168B.

(Example 8)

With reference to Figure 14, we will explain Example 8 of the first embodiment.

In Example 8, an example of editing the driving support auxiliary information image 64 displayed on the output device 168A (monitor) by operating the input device 168B is shown.

Figure 14 (A) shows a driving support information image 64 that is output from the driving support information presentation unit 164 and displayed on the output device 168A (monitor) of the user interface 168. The operator operates the input device 168B to edit the image to enlarge the area 32 (here, an intersection) in which the driver is desired to travel at a low speed, generating the driving support information image 66 shown in Figure 14 (B).

(Example 9)

The ninth example of the first embodiment will be explained with reference to Figure 15.

In Example 9, an example of editing the driving support auxiliary information image 64 displayed on the output device 168A (monitor) by operating the input device 168B is shown.

Figure 15 (A) shows a driving support auxiliary information image 64 that is output from the driving support auxiliary information presentation unit 164 and displayed on the output device 168A (monitor) of the user interface 168. The operator operates the input device 168B to edit the text information 36 ("Watch out for pedestrians jumping out at the intersection, 1km/h↓") requesting driving at a slower speed ("Watch out for pedestrians jumping out at the intersection, 10km/h↓") to a lower speed ("Watch out for pedestrians jumping out at the intersection, 10km/h↓"), generating the driving support information image 66 shown in Figure 15 (B).

(Example 10)

With reference to Figure 16, we will explain Example 10 of the first embodiment.

In Example 10, an example of editing the driving support auxiliary information image 64 displayed on the output device 168A (monitor) by operating the input device 168B is shown.

Figure 16 (A) shows a driving support information image 64 output from the driving support information presentation unit 164 and displayed on the output device 168A (monitor) of the user interface 168. The operator operates the input device 168B to edit the image to enlarge the area 32 (here, an intersection) in which the driver is desired to drive slowly, and to delete the area 50 (blind spot caused by the parked vehicle 26), generating the driving support information image 66 shown in Figure 16 (B). The reason for deleting the area 50 is that there is a fence or the like between the sidewalk and the roadway in front of the parked vehicle 26, making the frequency of accidents extremely low.

(Example 11)

With reference to Figure 17, we will explain Example 11 of the first embodiment.

In Example 11, an example of editing the driving support auxiliary information image 64 displayed on the output device 168A (monitor) by operating the input device 168B is shown.

Figure 17 (A) shows a driving support auxiliary information image 64 output from the driving support auxiliary information presentation unit 164 and displayed on the output device 168A (monitor) of the user interface 168. The operator operates the input device 168B to delete the accident-free data depicted in the vehicle speed-accident frequency characteristic diagram as shown in Figure 11 (B), and edit the text information 36 ("Watch out for pedestrians jumping out at the intersection, 30km/h↓") requesting driving at a slower speed to make the speed specification stricter ("Watch out for pedestrians jumping out at the intersection, 10km/h↓").

{Second embodiment}

The second embodiment of the present invention will be described below. Note that in the second embodiment, the same components as those in the first embodiment are given the same reference numerals and the description of the components will be omitted.

The second embodiment is characterized in that it uses a driving position based on predetermined driving route information as an input source, and generates driving support information for each necessary base on the driving route based on surrounding images at each driving position from, for example, a real-time map management system 24, and creates a driving profile. The surrounding images at each driving point may be generated by running a simulation along the driving route.

Figure 18 is a functional block diagram for executing control to provide driving assistance information executed by the driving assistance control device 12 upon receiving driving route information according to the second embodiment. Note that the blocks in Figure 18 are classified by function and do not limit the hardware configuration of the driving assistance control device 12.

The driving assistance control device 12 is equipped with a driving route information receiving unit 172, which receives pre-set driving route information.

The driving route information receiving unit 172 is connected to the accident pattern analysis unit 152, and sends the received vehicle information to the accident pattern analysis unit 152.

The accident pattern analysis unit 152 sends search information to the accident pattern information search instruction unit 154 to obtain related accident pattern information based on the vehicle information.

The accident pattern information search instruction unit 154 is connected to the accident pattern database 156 and notifies the accident pattern database 156 of the search information received from the accident pattern analysis unit 152.

The accident pattern database 156 extracts past accident pattern data associated with the search information and sends it to the accident pattern analysis unit 152. The structure of the accident pattern database 156 is the same as in the first embodiment, so a detailed description is omitted here.

The accident pattern analysis unit 152 aggregates vehicle information received from the vehicle 10 and past accident pattern data received from the accident pattern database 156, and by analyzing these, creates driving support auxiliary information that makes it easier for the operator to consider the risk of an accident expected in a given situation when deciding to resume autonomous driving of the vehicle 10. As an example of the analysis, it is effective to use AI technologies such as machine learning and deep learning.

The accident pattern analysis unit 152 is connected to the driving support auxiliary information presentation unit 164, and sends the analysis result, that is, the driving support auxiliary information, to the driving support auxiliary information presentation unit 164.

The driving support auxiliary information presentation unit 164 is connected to the update unit 166 and the output device 168A of the user interface 168, and sends the driving support auxiliary information to the update unit 166 and outputs it to the output device 168A. The output device 168A may be a monitor or a printer, but in the first embodiment, it includes at least a monitor.

The driving support auxiliary information output to the output device 168A can be confirmed (visually confirmed) by the operator, and after the operator confirms the driving support auxiliary information, the update unit 166 uses the input device 168B (keyboard, mouse, etc.) of the user interface 168 to update the driving support auxiliary information received from the driving support auxiliary information presentation unit 164. Updating refers to editing such as deleting, changing, or adding an image on the monitor, and the driving support auxiliary information becomes driving support information by updating the information in the update unit 166, and is sent to the driving profile generation unit 174.

The driving profile generation unit 174 is connected to the driving profile database 176 and sequentially stores the generated driving profiles in the driving profile database 176.

The operation of the second embodiment is explained below.

Figure 19 is a control flowchart for generating driving assistance information in the driving assistance control device 12 according to the second embodiment. Note that the flowchart in Figure 19 is one of various controls that are executed in a sequential loop in the driving assistance control device 12.

In step 180, it is determined whether or not driving route information has been received. If the determination in step 180 is negative, the routine ends.

If the determination in step 180 is affirmative, step 102A → step 104A → step 106A → step 108A → step 110A is repeated according to the number of caution points present in the received driving route information, and when the repeated process is completed, the process proceeds to step 182.

(Repetitive processing)

In step 102A, the position information of the vehicle 10 is received, and current position traffic information is received, for example, from the real-time map management system 24, and the process proceeds to step 104A.

In step 104A, examples of past accident patterns in traffic conditions similar to the current traffic information (e.g., cases [1], [2], [3], etc., as shown in FIG. 3) are obtained from the accident pattern database 156, and the process proceeds to step 106A.

In step 106A, the accident pattern analysis unit 152 performs an analysis (e.g., machine learning or deep learning using AI), and then proceeds to step 108A, where a driving support auxiliary information image 28 is presented in which the speed caution area (area 32 shown in FIG. 4(A) is superimposed on the forward image of the vehicle 10, and output to the output device 168A. This allows the operator to check the output result (display) on the output device 168A (monitor).

In the next step 110A, the input device 168B performs an update to increase the attention generated by the operator (updating the driving support auxiliary information to driving support information). In the first embodiment, the area 32 shown in FIG. 4(A) is expanded and edited to the area 32 shown in FIG. 4(B).

In step 182, which is reached after repeated processing equivalent to the number of warning points present in the driving route information, a driving profile is created and the process proceeds to step 184.

In step 184, the driving profile created in step 182 is stored in the driving profile database 176, and the routine ends.

According to the second embodiment, by creating a driving profile based on driving assistance information for each driving position based on driving route information in advance, not only when driving difficulties occur while the autonomous vehicle (vehicle 10) is actually driving, the risk of an accident while driving along the driving route information can be avoided in advance.

10 vehicle, 12 driving assistance control device, 14 vehicle control device, 20 automatic driving control device, 16 camera group, 16A forward camera, 16B left forward camera, 16C left rearward camera, 16D right forward camera, 16E right rearward camera, 18 radar group, 22 network, 22A wireless communication device, 24 real-time map management system, 26 parked vehicle, 150 vehicle information receiving unit, 152 accident pattern analysis unit, 154 accident pattern information search instruction unit, 156 accident pattern database, other party type/behavior, 164 driving assistance auxiliary information presentation unit, 166 update unit, 168 user interface, 168A output device, 168B input device, 170 driving assistance information transmission unit, 28 driving assistance auxiliary information image, 30 driving assistance information image, 32 area, 34, 34A, 34B, 34C, 34D Graph information, 36, 36a, 36b, 36c, 36e, 36f, 36g, 36h, 36i Text information, 40 Image, 42 Area, 44 Area, 46 Area, 48 Other vehicle, 50 Area, 62 Box plot, 64 Driving support auxiliary information image, 66 Driving support information image

Claims (10)

A driving assistance control device (12) that remotely assists the driving of a vehicle performing autonomous driving and provides driving assistance information when an autonomous driving impossible factor occurs in the vehicle,
a receiving unit (150) that receives vehicle information including at least position information indicating a driving position transmitted by a vehicle in which the autonomous driving impossibility factor occurs and a peripheral image of the driving position when the autonomous driving impossibility factor occurs;
an accident pattern database (156) storing accident pattern data in which a plurality of past accident cases are associated with traffic conditions including at least one of road structure, vehicle behavior, and other party information;
an analysis unit (152, 154) that reads out accident pattern data related to the vehicle information from the accident pattern database based on the vehicle information received by the receiving unit, and superimposes an analysis result obtained by analyzing the read out accident pattern data on the surrounding image included in the vehicle information received by the receiving unit to generate driving support auxiliary information;
a providing unit that provides the driving support auxiliary information generated by the analysis unit to the vehicle;
A driving assistance control device having the above configuration. The driving assistance control device according to claim 1, wherein the receiving unit receives the vehicle information in real time from a vehicle performing autonomous driving. The driving assistance control device according to claim 1, wherein the receiving unit receives vehicle information including a driving position based on driving route information and a surrounding image at each driving position. The providing unit,
A driving support information presentation unit that outputs the driving support information generated by the analysis unit to an output device (168A) that can be visually recognized by an operator monitoring the driving support;
An update unit that updates the driving support auxiliary information generated by the analysis unit based on input information input from an input device (168B);
a driving support information transmission unit that transmits the driving support auxiliary information updated by the update unit to a transmission source of the vehicle information received by the reception unit;
The driving assistance control device according to any one of claims 1 to 3, comprising: The driving assistance control device according to claim 4, wherein the driving assistance auxiliary information includes warning information. The driving assistance control device according to claim 5, wherein the update performed by the update unit is to select and discard the attention-calling information included in the driving assistance auxiliary information. As the warning information, information regarding a vehicle speed warning region is generated based on an accident pattern;
The driving assistance control device according to claim 5 , wherein the updating performed by the update unit includes selecting and discarding information indicating the speed caution region, and editing the speed caution region. The analysis unit, as the warning information,
8. The driving assistance control device according to claim 5, further comprising: a speed-accident frequency characteristic curve indicating a relationship between speed and accident frequency, based on the accident pattern data. The driving assistance control device according to any one of claims 5 to 7, wherein the analysis unit performs clustering of the accident pattern data based on a clustering factor including at least one of speed, visibility, traffic volume, and time of day, selects an accident example belonging to a cluster that best suits the current situation, and generates a speed-accident frequency characteristic curve showing the relationship between speed and accident frequency as the warning information based on the accident example belonging to the selected cluster that best suits the current situation. Computer,
Operate as each part of the driving assistance control device according to any one of claims 1 to 9.
Driver assistance control program.