DE102021107543A1 - driving support device - Google Patents

Abstract

The invention relates to a driving support device (110), comprising: a sign determination unit (112) which is set up to determine at least one specific type of road sign (146a, 146b, 146c), the sign determination unit (112) is arranged to base the determination on data recorded using a camera unit (120), and/or wherein the sign determination unit (112) is arranged to base the determination on data obtained from map data a traffic behavior monitoring unit (114) which is set up to monitor the behavior of at least one road user (A, B, C) who is moving on the same road (140) and in the same direction (t) as the ego - Vehicle (100), a comparison unit (116) which is set up to compare the determination result with the behavior of the at least one road user (A, B, C). hen, and a determination adjustment unit (118) which is set up to adjust the determination result output by the sign determination unit (112) in a case that the comparison unit (116) detects a discrepancy between the determination result and the behavior of the at least one road user (A, B, C) determined. The invention also relates to a vehicle and a method.

Description

This invention relates to a driving support device configured to be installed in a ego vehicle and configured to support a driver of the ego vehicle to correctly observe an environment of a road on which the vehicle is traveling is currently traveling, a vehicle including such a driving support apparatus, and a method for assisting a driver of the vehicle in correctly observing an environment of a road on which the ego vehicle is currently traveling.

Vehicles that are equipped with a driving support device that is set up to detect a road sign are known from the prior art. It is the aim of such a driving support device to determine a road sign as correctly as possible. Typically, driving assistance devices are configured to determine a specific type of road sign, such as a specific speed limit, displayed by the road sign using images captured by a camera.

Particularly high error rates have been encountered when road signs displayed in a traffic control and information system need to be determined (also referred to as "overhead service bridge signs") because they are harder to see. In addition, road signs displayed in a traffic control and information system can suddenly switch between different road sign types and can also display several different specific types of road sign or even different road signs for adjacent lanes.

Other errors that can occur are recognition of a sign on other objects, such as a sticker on a truck, and misassigning a recognized road sign to a lane or road.

Automated vehicles in particular require a high recognition quality for driving in accordance with the law. However, the recognition quality of prior art driving support devices is not sufficient for use in automated vehicles due to high error rates when determining a specific type of road sign.

In view of the above, it is an object of the present invention to overcome these disadvantages. In particular, it is an object of the present invention to provide a driving support device with a reduced error rate in determining a road sign.

According to a first aspect of the invention, this object is achieved by a driving support device configured to be installed in a ego vehicle and configured to support a driver of the ego vehicle to recognize an environment of a road correctly to observe on which the ego vehicle is currently moving, the driving support device comprising a sign determination unit configured to determine at least one specific type of road sign, the sign determination unit configured to make the determination Based on data captured using a camera unit that is installed in the ego vehicle and that is configured to capture images from the vicinity of the road on which the ego vehicle is currently traveling and the images to the pass on shield determination unit and/or wherein the shield determination unit is set up to have the determination based on data obtained from map data relating to the road on which the ego vehicle is currently moving, a traffic behavior monitoring unit which is set up to monitor the behavior of at least one road user, which is in the vicinity of the ego vehicle and which is traveling on the same road and in the same direction as the ego vehicle, a comparison unit configured to compare the determination result output from the sign determination unit with the monitored one to compare behavior of the at least one road user, which is output from the traffic behavior monitoring unit, and a determination adjustment unit, which is configured to adjust the determination result output from the sign determination unit in a case in which the comparison unit detects a deviation between the vo determined by the determination result output by the sign determination unit and the monitored behavior of the at least one road user output by the traffic behavior monitoring unit.

It should already be noted at this point that, for the sake of simplicity, the invention will be discussed below in relation to a traffic system with right-hand traffic, such as is practiced in continental Europe or the United States of America. It is obvious that the right-hand traffic disku ted situations can also be applicable to a traffic system with left-hand traffic, as practiced in Japan and the United Kingdom, for example, by simply mirroring a situation regarding the direction of travel of the ego vehicle. Furthermore, for a better understanding, the invention will be described essentially by means of a road sign indicating a speed limit. Further, it should be noted that although the present invention will be explained below with respect to two or three traffic moving on two or three lanes, the present invention is applicable to a road having at least one lane for traffic moving move in the same direction as the ego vehicle, ie explicitly also for a road with a plurality of such lanes, and it is applicable to at least one road user who moves in the vicinity of the ego vehicle, ie explicitly also for a plurality of such road users .

The present invention thus provides a solution to discrepancies between map data and camera-based detection, for example due to the map data being outdated or incorrect and/or a sign being displayed in the map data but not recognized by the camera, and/or a sign is recognized by the camera but not displayed in the map data, and/or the map data shows a different sign or specific type of road sign than the sign recognized by the camera, and /or that the sign is displayed in a traffic control and information system, as these signs are dynamic road signs which cannot be stored in a static map and therefore map data cannot support dynamic road sign recognition. In case of a corresponding discrepancy between map data and camera-based recognition, the present invention enables the correct specific type of road sign to be determined with a higher accuracy.

At least every time a road sign is determined by the sign determination unit, the driving support device according to the present invention causes the traffic behavior monitoring unit to monitor the behavior of at least one road user who is in the vicinity of the ego vehicle. The monitored behavior of the at least one road user, which is output by the traffic behavior monitoring unit, is then compared by the comparison unit with the determination result, e.g. a specific type of road sign, which is output by the sign determination unit. In this way, it can be determined whether or not the traffic surrounding the ego vehicle behaves in a manner that conforms to the regulation indicated by the particular specific type of road sign. In the case that the monitored behavior of the surrounding traffic does not match the determined specific type of road sign, the determination result, i.e. the determined specific type of road sign, can be changed by the determination adjustment unit to a determination result that matches the behavior of the surrounding traffic Traffic more appropriate. To give a simple example, if the sign determination unit determines a specific type of road sign to display a speed limit of 120 km/h, but all of the surrounding road users adjust their speed to 100 km/h, it can be assumed that the actual specific type of road sign actually indicates a speed limit of 100 km/h and has been erroneously determined to indicate 120 km/h, so the output of the driving support device indicating a speed limit for the driver of the ego vehicle and /or which sets a locomotion speed of the ego vehicle can be changed to a value of 100 km/h. Of course, not only braking of surrounding traffic can be monitored, but also acceleration and/or essentially maintaining a specific speed.

By taking the behavior of traffic surrounding the ego vehicle into consideration, the determination accuracy of the driving support device installed in the ego vehicle can be significantly improved since determination accuracies from plural different units can be combined. In other words, a kind of redundancy of the driving support device installed in the ego vehicle and sign determination units installed in other vehicles (even if it is the driver of another vehicle himself) can be used in determining a specific type of a road sign.

The driving support device according to the present invention may not only be configured to monitor the speed of the at least one road user at a specific point in time, but may monitor the behavior over a predetermined period of time, including for example acceleration, deceleration or lane change maneuvers. In this way, additional data can be recorded to check the determination result, which is issued by the sign determination unit. For example, if the sign determination unit determines a specific type of road sign to display the same or even slower speed limit than a previously determined road sign, but the at least one road user still accelerates, a deviation can be determined by the comparison unit. Thus, the determination adaptation unit can adapt the determined road sign type to a specific type of road sign that indicates a higher speed limit. As a result, recognition errors can be further reduced.

The driving support device according to the present invention may also be configured to compare the images captured by the camera unit with data obtained from map data. Sometimes road signs included in the map data may not be recognized by the camera unit, for example due to other vehicles blocking a line of sight between the camera unit and the road sign. Such an event can in particular cause the traffic behavior monitoring unit to monitor the behavior of at least one road user, so that at least the specific type of road sign, which is displayed in the map data, can be compared with the behavior of the traffic, which the ego vehicle surrounds. The map data can be stored in the ego vehicle and/or can be received from an external source, such as from a server via V2X and/or cellular communication and/or GPS and/or via the Internet.

The road sign can be any type of road sign, not just specific types of road sign displaying a specific speed limit. For example, the road sign may be a road sign indicating no overtaking, an additional road sign to specify when an associated road sign applies, e.g. at a certain time period, over a certain distance, for a certain vehicle weight or when a certain environmental condition occurs (e.g. for wet roads), lane guidance signs, e.g. for informing of a merging of several lanes, a narrowing of lanes, or road signs which identify a type of road, e.g. a motorway, a country road or an expressway, and their respective regulations, to name but a few important ones to name examples.

Furthermore, the invention is not limited to street signs that are permanently installed on the side of the road. It is also applicable to road markings, road signs displayed in a traffic control and information system, road signs displayed on specific vehicles such as a road maintenance vehicle, a police vehicle, or a heavy-duty transport vehicle, and the like. In particular, but not limited to, in the case where a road sign displayed in a traffic control and information system is displayed in the map data, then traffic sign detection thresholds in the expected range can be reduced to focus the detection. Since a road sign displayed in a traffic control and information system can be assumed to be placed essentially above the ego vehicle, the recognition algorithm of the sign determination unit can focus on this corresponding upper area.

It can be assumed that the behavior of the at least one road user is in accordance with traffic regulations and laws. However, in order to further improve the recognition result, the comparison unit can be further arranged to consider the case that at least one road user generally adapts its behavior to the speed limit indicated by the present road sign(s) that the road user however, may decide to drive faster than the displayed speed limit. The comparison unit can therefore be set up to compare the determination result output by the sign determination unit with the monitored behavior of the at least one road user, which is output by the traffic behavior monitoring unit, a predetermined acceptance range being taken into account, for example of up to 30 km/h. h, preferably up to about 20 km/h, above the actual speed limit.

In order to monitor the behavior of the at least one road user, the traffic behavior monitoring unit can be connected to at least one sensor. This at least one sensor can, for example, comprise at least one of a lidar system, a camera, a radar system and the like.

Based on this invention, the recognition quality achieved can be increased enough to integrate the driving support device into an automated vehicle. Nevertheless, the invention as an independent before be carried out direction, which is set up, for example, as a retrofit solution for conventional vehicles.

A further improvement in road sign determination can be achieved if the sign determination unit can be arranged to determine a plurality of possible specific types for a single road sign, each specific type indicating different traffic regulation information, in particular different speed limits for the road and/or the lane in which the ego vehicle is currently traveling, wherein each specific type can be assigned a different recognition probability based on a determination accuracy of the sign determination unit for a corresponding specific type of the individual road sign. In particular, determining a street sign type based on a captured image is prone to error. Typical errors that can occur are a blurred image or a reduction in contrast due to environmental influences, for example poor visibility or dirt on the road sign, or a damaged road sign. Assigning a probability of recognition to each of a plurality of possible different specific types for a single road sign can allow the comparison unit to compare the monitored behavior of the at least one road user with the possible road sign types according to their probability of recognition, e.g. starting with the highest probability of recognition first. Referring to the example given above, if the sign determination unit determines a specific type of road sign, a speed limit of 120 km/h with a detection probability of 27%, and a specific type of road sign, a speed limit of 100 km/h with a recognition probability of 24%, but the surrounding road users adjust their speed to 100 km/h, then the driving support device discards the determination result indicating 120 km/h even if it has a higher recognition probability, and the determination result adjustment unit can change the determination result to the value of 100 km/h.

The comparison process can stop when a match is detected. Consequently, a determination accuracy can be increased and/or an adjustment process of the determination result can be accelerated.

The check with the monitored behavior of the at least one road user can further increase the recognition quality, in particular in the case that the recognition probability of at least two possible different specific types for a road sign are almost the same. For example, the determination adjustment unit may adjust the determination result output from the sign determination unit to the specific type of road sign with the second highest (or third highest, fourth highest, etc.) probability when the monitored behavior deviates from the specific type of road sign with the highest probability , but matches the specific type of street sign with the second highest (or third highest, fourth highest, etc.) probability.

In this regard, the determination adjustment unit may be configured to adjust the determination result output by the sign determination unit only in the case where a difference in recognition probability between the highest recognition probability of a specific type for the individual road sign or from a first entire set of several specific types of street signs and a lower probability of recognition of a specific type for the individual street sign or for a second entire set of several specific types of street signs is less than a predetermined threshold value, in particular is less than 25%, preferably is less than about 10%. In other words, the difference of less than 25% (or less than 10%) can not only be used for the recognition probabilities of two individual specific types of street signs, but can also be used for an accumulated recognition probability of a first entire set of several street signs, e.g. comprising a first specific type of left road sign with a recognition probability of 33% and a first specific type of right road sign with a recognition probability of 31%, such that an accumulated recognition probability of the first total set is 64%, and an accumulated recognition probability of a second total Set of several road signs, e.g. comprising a second specific type of left road sign with a recognition probability of 28% and a second specific type of right road sign with a recognition probability of 27%, so that an accumulated probability of recognition of the second entire sentence is 55%.

• - ein linkes Straßenschild durch die Schild-Bestimmungseinheit so bestimmt, dass es mit einer Erkennungswahrscheinlichkeit von 33% eine Geschwindigkeitsbeschränkung von 120 km/h anzeigt, mit einer Erkennungswahrscheinlichkeit von 28% eine Geschwindigkeitsbeschränkung von 100 km/h anzeigt, mit einer Erkennungswahrscheinlichkeit von 21% eine Geschwindigkeitsbeschränkung von 130 km/h anzeigt und mit einer Erkennungswahrscheinlichkeit von 10% eine Geschwindigkeitsbeschränkung von 80 km/h anzeigt;
• - ein rechtes Straßenschild durch die Schild-Bestimmungseinheit so bestimmt, dass es mit einer Erkennungswahrscheinlichkeit von 31% eine Geschwindigkeitsbeschränkung von 130 km/h anzeigt, mit einer Erkennungswahrscheinlichkeit von 27% eine Geschwindigkeitsbeschränkung von 100 km/h anzeigt, mit einer Erkennungswahrscheinlichkeit von 25% eine Geschwindigkeitsbeschränkung von 120 km/h anzeigt und mit einer Erkennungswahrscheinlichkeit von 11% eine Geschwindigkeitsbeschränkung von 80 km/h anzeigt.

A corresponding "accumulated recognition probability" shall be understood as a combination of all the recognition probabilities of the street signs that form a corresponding "total set". An example using entire sets of street signs is given below for a better understanding. In the example becomes:

• - a left-hand road sign is determined by the sign determination unit to indicate a speed limit of 120 km/h with a probability of recognition of 33%, indicate a speed limit of 100 km/h with a probability of recognition of 28%, with a probability of recognition of 21% indicates a speed limit of 130 km/h and indicates a speed limit of 80 km/h with a detection probability of 10%;
• - a right-hand road sign determined by the sign determination unit to indicate a speed limit of 130 km/h with a probability of recognition of 31%, indicate a speed limit of 100 km/h with a probability of recognition of 27%, with a probability of recognition of 25% indicates a speed limit of 120 km/h and indicates a speed limit of 80 km/h with a detection probability of 11%.

A first complete set is now formed by the specific types of left and right road signs which have the highest probability of recognition, i.e. for the left road sign: 120 km/h (33%); for the right road sign: 130 km/h (31%). The accumulated recognition probability of the first entire sentence is 64% here. Assuming that both street signs have to be identical due to local traffic regulations, a deviation from the driving support device is consequently detected. Then the road sign recognition result is adjusted to a second entire set with the next highest accumulated recognition probability, which in the present example is a combination “left road sign: 120 km/h (33%); right road sign: 100 km/h (27%)” which has an accumulated detection probability of 60%. The accumulated recognition probability of the second full sentence (60%) is within the allowed difference from the first full sentence (64%) (difference of 4% is within the 10% range). However, a deviation will still be determined. Thus, the road sign recognition result is adjusted to a third entire set with the next highest accumulated recognition probability, which is a combination of “left road sign: 100 km/h (28%); right road sign: 130 km/h (31%)” which has an accumulated detection probability of 59%. The accumulated recognition probability of the third full sentence (59%) is within the allowed difference from the first full sentence (64%) (difference of 5% is within 10% range). However, a deviation will still be determined. Thus, the road sign recognition result is adjusted to a fourth entire set having the next highest accumulated recognition probability, which is a combination of “left road sign: 100 km/h (28%); right road sign: 100 km/h (27%)” which has an accumulated detection probability of 55%. The accumulated recognition probability of the second full set (55%) is within the allowed difference from the first full set (64%) (difference of 9% is within 10% range).

As can be seen from the example above, the fourth entire sentence satisfies the plausibility check of not violating local traffic regulations. If the fourth entire sentence further matches the behavior of other road users, it can be determined that there is a high probability that the traffic sign recognition according to the fourth entire sentence is correct.

Therefore, the determination result output from the sign determination unit can be prevented from being adjusted to an unlikely road sign type. Furthermore, comparing the monitored behavior to the specific type of road sign with the highest probability and, if applicable, to a limited number of other determined specific types of road sign that are within the predetermined threshold can speed up an adjustment process of the determination result. As a result, the behavior of the ego vehicle can be adapted to a roadsign-compliant behavior earlier and/or the result can be displayed to a driver of the ego vehicle earlier.

It is conceivable that even a specific type of street sign with the second-highest (third-highest, etc.) detection probability also does not match the monitored traffic behavior or does not pass the plausibility check. Then the next more likely specific type of street sign can be selected. Advantageously, this can only be done if that next most likely specific type of street sign is still within of the predetermined threshold value, in particular a difference to the specific type of road sign with the highest probability of recognition is less than 25%, preferably less than about 10%.

Advantageously, the sign determination unit can be set up to determine a plurality of street signs at essentially the same time and to determine at least one specific type for each street sign, wherein the driving support device can further comprise a plausibility control unit which is set up to to re-check the at least one particular specific type for each road sign with regard to existing road traffic regulations. Implementing a plausibility check unit in the driving support apparatus according to the present invention makes it possible to exclude determination results of specific types of road sign which have obviously been incorrect, for example violating local traffic regulations. This can also lead to a more correct determination of a specific street sign.

In one embodiment, the plausibility control unit can be set up to indicate a possible error in the determination of the plurality of road signs in the event that different specific road sign types are determined for a road sign that is fixed to one side of a section of the road , and for a road sign fixedly placed on the other side of the same section of road. This means that in many traffic systems it is defined that road signs permanently installed on both sides of the same road segment must display the same traffic regulation information, e.g. the same speed limit. Therefore, if the sign determination unit determines different specific types of a road sign for the road sign arranged on one side and for the road sign arranged on the other side of the same section of the road, this may indicate that at least one of the determination results are not correct.

Alternatively or additionally, the plausibility control unit can be set up to indicate a possible error in the determination of the plurality of street signs in the event that two different specific types of a street sign can be determined in a traffic control and information system for traffic systems with right-hand traffic, the specific type of road sign associated with a left lane indicates a speed regulation value that is less than the speed regulation value of the specific type of road sign associated with a right lane (seen in the direction of travel of the ego vehicle). For right-hand traffic systems, it is mainly defined that when a plurality of road signs displaying different speed limits are displayed in a traffic control and information system, the different speed limits are arranged in ascending order from the right road sign to the left road sign. Consequently, a determination result for a right road sign indicating a higher speed limit than a left road sign may indicate a recognition error. This can further improve the accuracy of the determination result.

According to a further embodiment of the invention, in the event that a road sign is detected in a traffic control and information system and a certain specific type of road sign for the lane on which the ego vehicle is currently moving is different from certain specific types of road signs for other lanes, the traffic behavior monitoring unit may be configured to monitor only the behavior of traffic surrounding the ego vehicle on the same lane of the road on which the ego vehicle is currently traveling and/or may be configured to do so to monitor the behavior of at least one road user placed near the ego vehicle in at least one lane different from the lane in which the ego vehicle is currently traveling using a multiplication factor which is the same as the other specific type of road sign for the ent corresponding other lane, and which represents a difference from corresponding road regulation information. By focusing on monitoring the behavior of at least one road user in the same lane as the ego vehicle, detection errors based on considering a behavior of at least one road user driving in a different lane than the ego vehicle can be prevented , which is provided with a different type of street sign. For example, in a traffic system with right-hand traffic, the ego vehicle can move in a lane with an assigned speed limit of 100 km/h, while a speed limit of 120 km/h is assigned to a lane adjacent to the left. Now, if the behavior of the road user is monitored, which is moving to the left of the ego vehicle, a multiplication factor of 1.2 can be used to correctly determine traffic behavior. In other words, by using a multiplication factor when monitoring a road user traveling in a lane associated with a speed limit that is different from the speed limit for the ego vehicle's lane, wrong conclusions can be drawn regarding a relationship between the traffic behavior and a specific determination result can be prevented. Of course, other multiplication factors are possible for other speed limits, eg a multiplication factor of 1.25 for a lane with an assigned speed limit of 80 km/h and an adjacent lane with an assigned speed limit of 100 km/h. In this context, the multiplication factor is not strictly limited to a factor by which a speed limit of a first lane is to be multiplied in order to obtain the speed limit of another lane, but the multiplication factor can also indicate an offset value with regard to different speed limits of at least two lanes . For example, in some traffic systems speed limits displayed in service bridges usually have a fixed speed difference of 20 km/h between adjacent lanes and steps 60, 80, 100, 120 are usually used. Therefore, an offset value of 20 km/h can be considered when monitoring traffic in an adjacent lane that has a corresponding different speed limit.

In addition, different road signs assigned to different lanes can be stored by the driving support device and taken into account when deciding whether or not adjustment of the determination result by the determination adjustment unit is necessary. Based on the fact that a decreasing amount and/or an increasing amount of a speed limit from a first street sign to a second street sign, which follows the first street sign in the direction of travel of the ego vehicle, can often be the same for all lanes, this can Considering a previously stored speed limit of a first road sign increase a recognition accuracy when determining the specific type of the second road sign. Additionally, storing a road sign assigned to a lane adjacent to the lane the ego vehicle is currently traveling in may simplify a lane changing maneuver since the speed limit applicable to the target lane is already known.

The traffic behavior monitoring unit may further be configured to monitor a behavior of at least two road users, to detect a distance between at least two road users in a case that a first road user drives ahead of a second road user on the same lane, at least one road user as an independent to determine road users, with an independence of a road user being determined in a case that the distance between the at least two road users essentially exceeds a predetermined distance, in particular a distance value in meters which is half the speed value in km/h of the second road user, and/or in a case that the second traffic user accelerates or decelerates and the first traffic user substantially maintains its speed for a predetermined period of time, and only the monitored behavior en output the at least one independent road user to the comparison unit. This means that the behavior of one road user can depend strongly on the behavior of another road user, as is the case when a road user follows a preceding road user, in particular when the following road user uses an adaptive cruise control unit. Therefore, the behavior of the following road user can be categorized as non-independent, ie the following road user only decelerates/accelerates in response to the preceding road user. However, in a case that the following road user brakes/accelerates independently of the preceding road user, this may indicate that the following road user is trying to adapt to changing traffic regulations, e.g. a reduced/increased speed limit, with the preceding road user still making a corresponding adjustment not performed. In addition, in a case where the distance between the two road users is greater than a predetermined distance, it can also be determined that both road users are acting independently of one another. Of course, the predetermined distance is different for different speeds of the road users, since at low speeds a moderate distance can already bring about independence, while at high speeds it does not. Therefore, also a time interval of t = d/v (time interval t = distance d divided by speed v), which represents a distance between road users in relation to the speed of the Road users indicates used to determine a dependency or independence of road users. For example, the distance value in meters mentioned above, which is half the speed value in km/h of the second road user, essentially results in a time interval of 1.8 seconds. A determination accuracy of a sign determination can be improved by only considering the behavior of independent road users, since road users who only react to other road users but do not react directly to street signs on the road can be excluded from traffic monitoring.

It should also be noted here that an independent behavior of a traffic participant must be able to be monitored for a minimum period of time, e.g. 1.5 seconds or 2 seconds, in order to prevent incorrect classification due to a late reaction of the second traffic participant following the first traffic participant. However, it can additionally be advantageous to keep this minimum time period relatively short in order to actually detect an intention of a road user. For example, assume that there are two vehicles A and B that are classified as "independent" with B driving behind A. If B tries to adjust to a higher speed limit but A does not, then B will accelerate and shorten the distance to A, but after a short period of time B will have to decelerate to avoid colliding with A. Thus the acceleration behavior of B can only be observed for a short period of time. By correctly setting the time period, B can be prevented from being incorrectly classified as "non-independent".

In this context, the traffic behavior monitoring unit may further be configured to monitor a behavior of at least three independent road users, wherein the comparison unit may further be configured to compare the monitored behavior of the at least three independent road users and only consider that behavior , which is similar for a majority of the monitored road users. With an increasing number of unrelated road users exhibiting similar behavior, the likelihood that a certain specific type of road sign matching that behavior is identical to the actual road sign increases.

Traffic changes, e.g. due to an impending traffic jam or a closure due to an accident, can also influence the behavior of at least one road user. Taking into account the behavior of at least one road user who is influenced in this way can lead to wrong conclusions and thus to wrong determination results. In order to improve the determination accuracy, the driving support device may further include a traffic information receiving unit configured to receive traffic information regarding a traffic situation, particularly a traffic jam, on a portion of the road ahead of a current position of the ego vehicle, and a Traffic information consideration unit which is set up to determine a predetermined traffic situation-compliant behavior of road users on the basis of the traffic information which is output by the traffic information receiving unit, wherein the comparison unit can also be set up to determine the monitored behavior of the at least one road user, which is output from the traffic behavior monitoring unit, to compare with the predetermined traffic situation-compliant behavior which is output from the traffic information consideration unit, and nd wherein the determination adaptation unit can also be set up to discard a displayed deviation between the determination result output by the sign determination unit and the monitored behavior of the at least one road user output by the traffic behavior monitoring unit in the event that the monitored behavior of the at least of a road user matches the predetermined traffic situation-compliant behavior. For example, the traffic information receiving unit may be configured to receive traffic information from a global positioning system, a wireless device, a cellular network, an external server, and the like. As an example, when a road user reduces his/her speed, even if the sign determination unit indicates a much higher speed limit, but the road user's speed reduction matches a traffic situation-compliant behavior due to a traffic jam ahead, then the determination adjustment unit adjusts the determination result based on the Does not take into account that the road user adapts his behavior to a surrounding traffic situation instead of a speed limit, which is indicated by road signs. Traffic information could also be received using V2X. By doing this, vehicles in a jam could notify following vehicles of the jam.

In addition to receiving traffic information from an external source, the traffic information receiving unit can also be set up to collect traffic information, eg to detect a traffic jam, solely by using on-board sensors. As an example, a congestion can be detected when all vehicles suddenly decelerate at a rate that is higher than is normally observed when adapting to a slower speed limit.

• ein Bestimmungsergebnis eines spezifischen Typs eines Straßenschilds, welches von der Schild-Bestimmungseinheit ausgegeben wird und/oder von der Bestimmung-Anpassungseinheit ausgegeben wird;
• eine eindeutige Kennung des entsprechenden Straßenschilds, insbesondere eine eindeutige Position an der Straße; und
• vorzugsweise ein Datum und eine Zeit des übertragenen Bestimmungsergebnisses;

wobei die Server-Verbindungseinheit ferner dazu eingerichtet sein kann, von dem Server zu empfangen:

• Straßenschild-Informationen für ein eindeutiges Straßenschild, welche einen spezifischen Typ des Straßenschilds anzeigen, welcher zuvor von einer Mehrzahl von anderen Verkehrsteilnehmern bestimmt worden ist;
• wobei die Bestimmung-Anpassungseinheit ferner dazu eingerichtet sein kann, das von der Schild-Bestimmungseinheit ausgegebene Bestimmungsergebnis in einem Fall einer Abweichung zwischen dem von der Schild-Bestimmungseinheit ausgegebenen Bestimmungsergebnis und den von dem Server empfangenen Straßenschild-Informationen anzupassen. Beispielsweise können Fahrer, welche ein Straßenschild passieren, das von der entsprechenden, in dem jeweiligen Fahrzeug installierten Schild-Bestimmungseinheit ausgegebene Bestimmungsergebnis bestätigen oder ablehnen. Bestätigte Bestimmungsergebnisse können dann zu einem Server als Straßenschild-Informationen hochgeladen werden. Die Straßenschild-Informationen können dann zu anderen Verkehrsteilnehmern übertragen werden, z.B. als aktualisierte Kartendaten, und/oder können zu Verkehrsteilnehmern übertragen werden, wenn sie bei dem entsprechenden Straßenschild eintreffen. Das von der Schild-Bestimmungseinheit ausgegebene Bestimmungsergebnis kann dann mit den Straßenschild-Informationen verglichen werden, welche von dem Server empfangen werden. Im Fall einer Abweichung kann die Bestimmung-Anpassungseinheit das Bestimmungsergebnis anpassen.

Advantageously, the driving support device may further include a server connection unit configured to transmit to a server:

• a determination result of a specific type of road sign, which is output from the sign determination unit and/or is output from the determination adjustment unit;
• a unique identifier of the corresponding street sign, in particular a unique position on the street; and
• preferably a date and time of the transmitted determination result;

wherein the server connection unit may be further configured to receive from the server:

• road sign information for a unique road sign indicating a specific type of road sign previously determined by a plurality of other road users;
• wherein the determination adjustment unit may be further configured to adjust the determination result output from the sign determination unit in a case of discrepancy between the determination result output from the sign determination unit and the road sign information received from the server. For example, drivers who pass a road sign can confirm or reject the determination result issued by the corresponding sign determination unit installed in each vehicle. Confirmed determination results can then be uploaded to a server as street sign information. The road sign information can then be transmitted to other road users, eg as updated map data, and/or can be transmitted to road users when they arrive at the corresponding road sign. The determination result output from the sign determination unit can then be compared with the road sign information received from the server. In case of deviation, the determination adjustment unit can adjust the determination result.

It may be advantageous in this respect to take into account the number of certain specific types of road signs, for example in the form of a threshold amount of confirmed certain specific types of road signs, which must be exceeded before the road sign information is taken into account. This allows ignoring road sign information uploaded to the server by only a very small number of road users. Furthermore, a traffic sign can change over time, for example when it is displayed in a traffic control and information system. To determine such a street sign, it may also be advantageous to consider the date and time of the street sign information. With an increasing period of time after specific street sign information has been uploaded to the server, the probability that the particular street sign type is no longer valid increases. In order to prevent outdated street sign information received from the server from having a negative impact on the recognition result, the comparison unit can be set up to only take into account street sign information received from the server that is within a specific threshold value , e.g. 15 minutes on the same day. However, since some service bridges frequently (e.g. within a few minutes) change the displayed speed limit based on the current traffic flow, the server can also use an intelligent algorithm for a robust implementation. For example, if the server determines that a speed limit is from a service bridge and a specific number of road users, e.g. three road users, reporting the same new speed limit in succession within a certain short period of time, e.g. less than 3 minutes, the server will then change itself adjust accordingly. Conversely, if no vehicle reports a traffic sign at the position of a service bridge for a certain period of time, then the service bridge is assumed to be "off", i.e. showing no speed restrictions.

At this point, it should be noted that the driving support device according to the present invention may be partially or entirely implemented by a processor, as in FIG for example a central processor (CPU) or the like, which executes a program (software) stored in a memory. All or some of the components may be implemented in hardware, such as Large Scale Integration (LSI), an Application Specific Integrated Circuit (ASIC), or a Field Programmable Gate Array (FGPA), and/or may be implemented through software and hardware interactions. The previously stored program may be stored in a storage device such as a hard disk drive (HDD) or a flash memory, and may be stored in a removable storage medium such as a DVD or a CD-ROM and may be installed on a storage device when the storage device is mounted on a vehicle.

According to a second aspect, the invention relates to a vehicle comprising a driving support device according to the first aspect of the invention. The vehicle can adapt its behavior according to the determined specific type of road sign. Consequently, the vehicle can run with improved precision.

In addition, it should be understood that a ego vehicle according to the present invention can be a car, truck, motorcycle, bus or any other, particularly automated, vehicle that is configured to travel on a road.

• einen Schild-Bestimmungsschritt, in welchem wenigstens ein spezifischer Typ eines Straßenschilds bestimmt wird, wobei in dem Schild-Bestimmungsschritt die Bestimmung auf Daten basiert, welche unter Verwendung einer Kameraeinheit aufgenommen werden, welche in dem Ego-Fahrzeug installiert ist und welche Bilder von der Umgebung der Straße aufnimmt, auf welcher sich das Ego-Fahrzeug momentan fortbewegt und/oder wobei in dem Schild-Bestimmungsschritt die Bestimmung auf Daten basiert, welche aus Kartendaten bezüglich der Straße erhalten werden, auf welcher sich das Ego-Fahrzeug momentan fortbewegt,
• einen Verkehrsverhalten-Überwachungsschritt, in welchem das Verhalten von wenigstens einem Verkehrsteilnehmer überwacht wird, welcher sich in der Nähe des Ego-Fahrzeugs befindet und welcher sich auf derselben Straße und in dieselbe Richtung wie das Ego-Fahrzeug fortbewegt,
• einen Vergleichsschritt, in welchem das in dem Schild-Bestimmungsschritt ausgegebene Bestimmungsergebnis mit dem überwachten Verhalten des wenigstens einen Verkehrsteilnehmers verglichen wird, welches in dem Verkehrsverhalten-Überwachungsschritt ausgegeben wird, und
• einen Bestimmung-Anpassungsschritt, in welchem das in dem Schild-Bestimmungsschritt ausgegebene Bestimmungsergebnis in einem Fall angepasst wird, dass in dem Vergleichsschritt eine Abweichung zwischen dem in dem Schild-Bestimmungsschritt ausgegebenen Bestimmungsergebnis und dem in dem Verkehrsverhalten-Überwachungsschritt ausgegebenen überwachten Verhalten des wenigstens einen Verkehrsteilnehmers bestimmt wird.

According to a third aspect, the present invention relates to a method for supporting a driver of an ego vehicle in correctly observing surroundings of a road on which the ego vehicle is currently moving, the method comprising the following steps:

• a sign determining step in which at least one specific type of road sign is determined, wherein in the sign determining step the determination is based on data captured using a camera unit installed in the ego vehicle and images of the surroundings the road on which the ego vehicle is currently traveling and/or wherein in the sign determining step the determination is based on data obtained from map data on the road on which the ego vehicle is currently traveling,
• a traffic behavior monitoring step in which the behavior of at least one road user who is in the vicinity of the ego vehicle and who is moving on the same road and in the same direction as the ego vehicle is monitored,
• a comparison step in which the determination result output in the sign determination step is compared with the monitored behavior of the at least one road user, which is output in the traffic behavior monitoring step, and
• a determination adjustment step in which the determination result output in the sign determination step is adjusted in a case that in the comparison step there is a deviation between the determination result output in the sign determination step and the monitored behavior of the at least one road user output in the traffic behavior monitoring step is determined.

It should already be stated at this point in time that all features, advantages and functions which are described in relation to the driving support device according to the present invention are also applicable to the method according to the invention and vice versa. A similar statement applies to a vehicle according to the second aspect of the invention.

As already described with regard to the driving support device according to the present invention, comparing a determination result of a specific type of road sign with a monitored behavior of road users moving around the ego vehicle can significantly increase an accuracy of the determination result , since on the basis of the comparison result a deviation and in turn an incorrectly determined specific type of road sign can be identified.

The method according to the present invention can be further improved if in the sign determining step also a plurality of possible different specific types can be determined for a single road sign, each specific type indicating different traffic regulation information, each specific type having a different probability based on a determination accuracy of the sign determining step for a corresponding specific type for the individual road sign. Thus, in a case that a deviation between the determined specific type of road sign and the behavior of the surrounding traffic is detected, a determination result can be sent determination result can be adjusted, which is associated with a second highest (or third highest, fourth highest) probability of recognition, assuming that the corresponding specific type of road sign matches the behavior of the surrounding traffic.

Advantageously, in the determination adjustment step, the determination result output in the sign determination step can be adjusted only in the case that a difference in recognition probability between the highest recognition probability of a specific type for the single road sign or of a first entire set of plural specific types of road signs and a low probability of recognition of a specific type for the single street sign or a second entire set of several specific types of street signs is less than a predetermined threshold value, in particular is less than 25%, preferably less than about 10%. This can prevent a determination result from being adjusted to a determination result associated with a lower detection probability in the event that this lower detection probability is "not likely enough", i.e. a determination result cannot be adjusted in a case in which the lower detection probability for a specific type of road sign differs from the highest detection probability for a specific type of road sign by more than 25% or 10%.

• 1 eine beispielhafte schematische Gestaltung eines Ego-Fahrzeugs zeigt, welches mit einer Fahr-Unterstützungsvorrichtung gemäß einem ersten Aspekt der Erfindung ausgerüstet ist;
• 2a bis 2g die Funktionalität der Fahr-Unterstützungsvorrichtung bezüglich unterschiedlichen beispielhaften Verkehrssituationen zeigen;
• 3 ein Flussdiagramm zeigt, welches eine Ausführungsform eines Betriebs der Fahr-Unterstützungsvorrichtung erklärt;
• 4 ein Flussdiagramm einer Subroutine zeigt, welches eine Ausführungsform eines Betriebs einer Verkehrsverhalten-Überwachungseinheit der Fahr-Unterstützungsvorrichtung erklärt;
• 5 ein Flussdiagramm einer anderen Subroutine des Flussdiagramms aus 3 zeigt; und
• 6 ein beispielhaftes Flussdiagramm eines Verfahrens zum Unterstützen eines Fahrers gemäß einem dritten Aspekt der Erfindung zeigt.

The invention will be described in more detail in terms of specific embodiments, reference being made to the accompanying drawings, in which:

• 1 12 shows an exemplary schematic configuration of an ego vehicle equipped with a driving support device according to a first aspect of the invention;
• 2a until 2g show the functionality of the driving support device with regard to different exemplary traffic situations;
• 3 12 shows a flowchart explaining an embodiment of an operation of the driving support device;
• 4 FIG. 14 is a flowchart of a subroutine explaining an embodiment of an operation of a traffic behavior monitoring unit of the driving support apparatus; FIG.
• 5 a flowchart of another subroutine of the flowchart 3 indicates; and
• 6 12 shows an exemplary flowchart of a method for supporting a driver according to a third aspect of the invention.

In 1 1, an ego vehicle is denoted generally by reference numeral 100, but is not shown in more detail. The ego vehicle 100 comprises a brake 102, an accelerator device 104, a vehicle speed control device 106 which is configured to control the speed of the vehicle by selectively activating the brake 102 and/or the accelerator device 104, and a driving support device 110 according to the present invention. In order to display information collected by the driving support device 110 to the driver of the ego vehicle 100 , the driving support device 110 may be connected to a human-machine interface unit 108 . This information may be collected by a shield determination unit 112, such that the shield determination unit 112 may also be directly connected to the man-machine interface unit 108.

In the example shown, the driving support device 110 comprises a sign determination unit 112, a traffic behavior monitoring unit 114, a comparison unit 116 and a determination adjustment unit 118. The driving support device 110 is connected to a camera unit 120 and a map data unit 122. The camera unit 120 is installed in the ego vehicle 100 and configured to capture images of the surroundings of the road on which the ego vehicle 100 is currently traveling and relay the images to the sign determination unit 112 . In this example, the map data unit 122 is set up to determine a position of the ego vehicle 100, e.g. from a global positioning system using a GPS antenna 124, in map data which is stored in a memory (not shown) which is of the driving support device 110 is connected. Based on the position of the ego vehicle, the map data unit 122 transmits corresponding map data on the road on which the ego vehicle 100 is currently traveling to the sign determining unit 122, the corresponding map data specifically indicating traffic regulation information, such as Street signs (also called "traffic signs") or specific types of street signs, which indicate, for example, a specific speed limit.

Based on the data received from map data unit 122 and/or the images received from camera unit 120, sign determination unit 112 determines at least one specific type of road sign, for example a speed limit for the road and/or lane on which the Ego vehicle 100 currently moving. For this purpose, the sign determination unit 112 can access a plurality of previously stored specific types of road signs, which are stored, for example, in a memory which is connected to the sign determination unit 112, and can compare the detected specific type of road sign with the previously stored specific compare types.

The sign determination unit 112 can determine a plurality of possible different specific types for a single road sign, each specific type indicating different traffic regulation information. Each specific type is assigned a different recognition probability based on a determination accuracy of the sign determination unit 112 for a corresponding specific type for a single road sign. For example, the sign determination unit 112 compares an image obtained from the camera unit 120 with a plurality of road sign types, and an underlying algorithm assigns a detection probability to each of the plurality of road sign types. The sign determination unit 112 outputs the determination result of the road sign to the comparison unit 116 .

The traffic behavior monitoring unit 114 monitors the behavior of at least one road user who is in the vicinity of the ego vehicle 100 and who is traveling on the same road in the same direction as the ego vehicle 100 . The monitored behavior may include, for example, acceleration, deceleration, and/or lane change maneuvers. In order to monitor the behavior of at least one road user, the traffic behavior monitoring unit 114 is connected to at least one sensor 125 in this case. This at least one sensor 125 can, for example, comprise at least one lidar system, at least one camera, at least one radar system and the like. The monitored behavior of the at least one road user is output by traffic behavior monitoring unit 114 to comparison unit 116 .

By comparing the determination result output from the sign determination unit 112 with the monitored behavior of the at least one road user, which is output from the traffic behavior monitoring unit 114, the comparison unit 116 determines whether there is a discrepancy between them or not. There is a deviation if the monitored behavior of the at least one road user differs from behavior that conforms to the type of street sign. Such a road sign-type compliant behavior can be determined by the comparison unit 116 based on the determination result that is output from the sign determination unit 112 .

The comparison result is output to the determination adjustment unit 118 . In the case that the comparison unit 116 determines the deviation described above, the determination adjustment unit 118 is configured to adjust the determination result. The determination adjustment unit 118 transmits an adjusted road sign type to the man-machine interface 108 and/or the speed control unit 106 to display the adjusted determination result and/or adjust the behavior (speed) of the ego vehicle.

In the case that the sign determination unit 112 outputs a plurality of possible specific types for a single road sign with a respective associated recognition probability, the determination adaptation unit 118 can also determine a difference between each of the corresponding recognition probabilities corresponding to the possible different specific types of a Street signs are assigned. In an exemplary case, the determination adjustment unit 118 may only adjust the determination result output from the shield determination unit 112 if this difference between the highest probability of detection and a lower probability of detection is less than a predetermined threshold, specifically less than 25%, preferably less than about 10% This can prevent a situation in which the determination result is adjusted to a specific type of road sign having too low a probability even if the surrounding road users do not behave according to the determined specific type of road sign having the highest recognition probability.

In order to determine the specific type of the road sign more correctly, the driving support device 110 of FIG 1 shown example also a plausibility control unit 126. The plausibility control unit 126 checks the at least one specific type of road sign determined by the sign determining unit 112 in view of existing road traffic regulations and/or local laws/regulations. Furthermore, the plausibility control unit 126 indicates a possible error in determining a road sign in the event that a certain specific type of road sign violates existing road traffic regulations and/or local laws/regulations. As an example, when two street signs are permanently installed on either side of a street, it is often defined by local regulations that both street signs must be identical. Thus, in the event that the determination results for the two street signs are different from each other, the plausibility check unit 126 may indicate that at least one of the determination results is likely to be incorrect. Then, in particular, these two determination results for the two street signs can be compared with the monitored behavior of the at least one road user.

To receive traffic information, the in 1 The driving support device 110 shown also has a traffic information receiving unit 130. The traffic information receiving unit 130 receives traffic information regarding a traffic situation, in particular a traffic jam, on a section of the road in front of the current position of the ego vehicle 100. In addition, the driving support device 110 includes here a traffic information consideration unit 132 which is set up to determine a predetermined traffic situation-compliant behavior of the at least one road user on the basis of the traffic information which is output by the traffic information receiving unit 130 . The driving support device 110, particularly the traffic information receiving unit 130, may be connected to a cellular antenna 134, V2X communication, and the like to receive traffic information.

The driving support device 110 may further include a server connection unit 136 . The server connection unit 136 receives a determination result for a specific type of road sign from the sign determination unit 112 and/or from the determination adjustment unit 118. The server connection unit 136 transmits the determination result for a specific type of road sign and a unique identifier of the corresponding one road sign to a server (not shown).

The unique identifier includes, for example, a unique position of the corresponding street sign on the street and preferably a date and a time of the transmitted determination result. The unique identification allows the server to associate the determination results with a specific street sign, for example on a map. Further, when the date and time of the transmitted determination result is transmitted, the server is able to specify the age of the determination result. In particular, in the case where the road sign is arranged in a traffic control and information system, the specific type of the road sign changes regularly, and the age of the determination result can indicate how reliable the determination result is.

The server connection unit 136 is further configured to receive street sign information for a unique street sign from the server. This road sign information indicates a specific type of road sign which has previously been determined by a plurality of other road users. The determination adjustment unit 118 may adjust the determination result output from the signage determination unit 112 in a case of discrepancy between the determination result output from the signage determination unit 112 and the road signage information received from the server. Whether or not there is a deviation can be determined by the comparison unit 116 . For this reason, the comparison unit 116 can be connected to the server connection unit 136 and can receive the road sign information for a unique road sign from the server via the server connection unit 136 .

The functionality of the driving support device 110 is explained in more detail regarding specific situations in a right-hand traffic system with reference to FIG 2a until 2g to be discribed. The in the 2a until 2g The traffic situations shown have some things in common. Therefore, analogous parts are given the same reference numerals, and descriptions from one of these figures may be transposed to another of these figures, unless otherwise noted.

2a 12 shows the ego vehicle 100 traveling on a road 140 having a left lane I, a middle lane II, and a right lane III. in the in 2a the situation shown, the ego vehicle 100 is moving on lane II in a direction of travel t. At a portion 144 of the road 140, a road sign 146a is fixed to the right side 148a (also indicated by the arrow R) of the road 140, and a road Sign 146b is fixedly positioned on the left side 148b (also indicated by arrow L) of the road 140.

When the ego vehicle 100 approaches the road signs 146a and 146b, the camera unit 120 captures at least one image of each road sign 146a and 146b and/or the map data unit 122 provides map data regarding the road signs 146a and 146b. Based on the captured images and/or the map data, the sign determination unit 112 determines at least one specific type for each road sign 146a and 146b, for example a speed limit of 100 km/h. In order to prevent incorrect determination of the road signs 146a and 146b, e.g. incorrect determination of the specific type of the road sign 146a and 146b, the sign determination unit 112 and/or the comparison unit 116 can compare the at least one captured image with the map data. However, since discrepancies between map data and a camera-based detection can occur, for example due to the fact that the map data is outdated or incorrect and/or the fact that the map data shows a different sign or a different specific type of road sign than the sign recognized by the camera, the present invention proposes to monitor the behavior of at least one road user to determine which of the specific types of road sign is correct.

in the in 2a In the situation shown, there are three other road users, labeled A, B, C, who are moving along lane I of road 140, and who are driving so close to each other that their actions could influence each other (as before regarding dependent/independent behavior described by road users). The road users A, B, C are monitored by the traffic behavior monitoring unit 114 . It is then determined whether the behavior of road users A, B, C matches the determined specific type of road sign or not, so that an incorrect determination of the specific type of road sign by the sign determination unit can be identified, as already described above is.

Here, the plausibility control unit 126 can check the determined specific types of the road signs 146a and 146b with regard to the fact that they must be identical according to the regulations of the specific traffic system.

In 2 B a situation is shown, which is essentially based on the situation 2a based, in which the preceding road user A brakes first, that the road user B brakes and finally the road user C brakes. Consequently, traffic participants B and C could decelerate solely because of the behavior of traffic participant A and not because they intend to adapt their speed to the speed limit shown on the road signs 146a, 146b. Therefore, only road user A acts independently of other road users, i.e. only one road user reduces its speed, so that a deviation between the monitored behavior of road users and the determined specific type of road sign is less likely if the behavior of road user A does not correspond to the determined one specific type of road sign 146a, 146b which has been determined by the sign determining unit 112.

In an alternative situation, which also re 2 B can be described, road user C brakes first, then road user B brakes, and then road user A brakes. Consequently, all three vehicles brake independently. Therefore, it is highly likely that the specific type of road sign 146a, 146b indicates a reduced speed limit (compared to a previous speed limit). This conclusion can be used to verify the specific type of road sign 146a, 146b determined by the sign determination unit 112.

In 2c the three road users A, B, C drive at essentially the same speed as the ego vehicle 100. The ego vehicle 100 moves on in the middle lane II, road users B and C move on in the left lane I and the Road user A moves on the right lane III. Then road user A begins to accelerate, and road user C begins to rear up behind road user B, ie road user C also accelerates. Consequently, two of the three road users are accelerating (or at least trying to accelerate), indicating that it is highly probable that the specific type of road sign 146a, 146b is displaying an increased speed limit (compare to a previous speed limit).

In an alternative exemplary situation, which is 2d is shown, which is essentially based on the situation of 2c is based, the road user A keeps his speed even after the street signs 146a, 146b. However, the road user B starts a Accelerate and road user C starts accelerating behind road user B. Consequently, road user C could merely follow road user B without respecting the actual speed limit. Therefore, only a single road user is truly accelerating independently, namely B. This may indicate that an increased speed limit is less likely.

2e is intended to visualize that a road user B following a road user A can be assumed to brake independently of road user A, even if B brakes behind A, when a predetermined distance between road user A and the Road user B is present or even exceeded. This predetermined distance can depend on an instantaneous speed, for example a distance value in meters which is half the speed value in km/h of road user B, or to put it another way, a distance which is necessary so that a point on the road 140, which is passed by traffic participant A, is not passed by traffic participant B within at least about 2 seconds. It must also be taken into account that road user B must eventually brake after a certain time in order not to collide with road user A, even if A and B are “far away”. In one example, if the distance between road users A and B is large enough, e.g. the distance value in meters is more than the speed value in km/h of road user B, then it can be assumed that a (moderate) braking of road user A will Speed of surrounding road user B is not affected within the next 1.5 seconds, ie a reaction from B within 1.5 seconds (approximately 1 second reaction time + 0.5 seconds to perform the deceleration) after A's reaction could indicate that both reactions have occurred independently.

A street sign can also be included in a traffic control and information system 150 ( 2f) are displayed. There may be a street sign for each lane, such as in 2f a lane III road sign 146a, a lane II road sign 146b, and a lane I road sign 146c. In other cases, there may be multiple road signs, as in FIG 2g is shown. Often the specific type of street sign displayed in a traffic control and information system 150 is dynamic and can vary over time. However, in a traffic control and information system 150, at least in traffic systems with right-hand traffic, the speed limits are arranged in a descending order from the left lane I to the right lane III, if they differ at all.

In the following, the operation of an exemplary embodiment of the driving support device 110 is explained in more detail with reference to the flowchart 3 to be discribed.

The process begins in step S100 in which the ego vehicle 100 is started. The process then proceeds to step S110 in which the behavior of at least one road user is monitored.

The process then proceeds to step S120, in which it is determined whether or not the road 140 on which the ego vehicle 100 is traveling has more than one lane. If not (step S120: NO), the process returns to step S110.

When the ego vehicle 100 is traveling on a road 140 having more than one lane (step S120: YES), the process proceeds to step S130, in which it is determined whether or not at least one road sign from the camera unit 120 and/or or is detected by the map data unit 122. If no road sign is detected (step S130: NO), the process returns to step S110.

If at least one road sign is detected (step S130: YES), the process proceeds to step S140, in which the at least one road sign is determined. In the event that a plurality of road signs are detected, all of the road signs are determined. The specific types of street signs can form at least one entire set of street signs as described above.

Next, in a step S150, the process determines whether or not the at least one street sign is placed in a traffic control and information system 150. If in step S150 the at least one road sign is detected in a traffic control and information system 150 (step S150: YES), the process proceeds to a sub-routine S152, which will be described later with reference to FIG 5 is described. If in step S150 the road sign is not detected in a traffic control and information system 150 (step S150: NO), the process determines in step S210 whether or not the behavior of more than one road user can be monitored. If not (step S210: NO), the process returns to step S110 back, in which the behavior of all road users is monitored.

If the behavior of more than one road user can be monitored (step S210: YES), the process continues with step S220, in which it is determined whether only one road sign is detected and/or whether a deviation between at least two road signs 146a, 146b can be identified and/or whether a discrepancy between at least one of the recognized street signs 146a, 146b and the map data can be identified. If not (step S220: NO), the process returns to step S110.

When the determination result of step S220 is affirmative (step S220: YES), the process proceeds to a subroutine S180 (described below). As a result, data D190 is obtained from the subroutine S180.

In the following step S200, it is determined whether or not the at least one specific road sign is different from the presumed results of the subroutine S180, based on the data D190. In particular, a first overall analysis result (in the case that step S200 is performed for the first time; otherwise it is a newly adjusted overall analysis result) based on the data concerning the behavior of the road users and based on a certain specific type for all detected street signs, including the corresponding detection probabilities. The first (or current) overall analysis result is examined in step S200 for deviations and a plausibility check is carried out as described above. In the negative case (S200: NO), i.e. the case that no abnormality is detected in the current overall analysis result, the process proceeds to a step S202 in which the current overall analysis result is accepted, i.e. a previous result is discarded or will be overwritten. Then the process returns to step S110 to be ready to perform a new process iteration in case, for example, new road signs are detected.

In the affirmative case of step S200 (S200: YES), the process proceeds to step S204. In this step S204 the first/current total analysis result is adjusted, for example by selecting a second total set of road signs with the next higher accumulated recognition probability, as described above.

Then the process goes to a step S170. In this step S170 it is determined whether or not the new total analysis result and/or the new total set of road signs by more than 10% (in this example (a value of e.g. 25% is also conceivable)) from the first total - analysis result and/or the first complete set of road signs differs (in particular the overall analysis result and/or the complete set of road signs with the highest overall recognition probability). When this determination result is negative (step S170: NO), the process returns to step S110. In the affirmative case of step S170 (step S170: YES), the process returns to and repeats step S200 described above.

The loop S200->S204->S170->S200 is repeated as many times as necessary, including assuming different variables of a current overall analysis result and/or an entire set of road signs, until no deviation is determined in step S200, then if the current overall analysis result is accepted by means of step S202, i.e. the previous overall analysis result is overwritten, or until the difference of 10%/25% is exceeded, then the process returns to step S110 with the result that a deviation in the current overall analysis result, but it is not possible to resolve it at this time.

An embodiment of the subroutine S180 will now be described with reference to FIG 4 be described, which is designed to classify the behavior of at least one road user.

The subroutine S180 starts with a step S270 and goes to a step 272, in which all road users in the vicinity of the ego vehicle 100 who are moving on the same road 140 and in the same direction t as the ego vehicle 100 are detected , where, in particular, speed change information is collected about all road users in a sensor area, for example by continuously monitoring their behavior.

Next, the sub-routine S180 goes to a step S274 in which acceleration counter data D276, deceleration counter data D278, constant speed data D280 and block counter data D282 are set to zero. The sum of all data D276, D278, D280 and D282 is equal to the number of road users detected at the end of subroutine S180.

The sub-routine S180 then proceeds to a step S284 in which a first road user A is analyzed. Of course, this and the following steps can be iterated for all detected road users, but they are only shown below for the first road user A to provide an example.

The sub-routine S180 then proceeds to step S286, in which it is determined whether or not the first road user A whose behavior is to be categorized can be detected. This iteration is carried out until all detected road users have been dealt with. In the negative case of step S286 (step S286: NO), the subroutine S180 is ended in a step S288.

In the affirmative case of step S286 (step S286: YES), the subroutine S180 goes to a step S290. In this step S290 it is determined whether or not the first road user A is significantly slower or faster than a previous speed limit, since road users which are significantly slower (e.g. trucks) or faster (e.g. speeding vehicles) than a previous speed limit are ignored as they are not an indicator of the subsequent speed limit. In the affirmative case of step S290 (step S290: YES), the sub-routine S180 proceeds to a step S292 in which a next road user B is selected, and the sub-routine S180 proceeds to step S286 and performs another iteration, as has been described above.

If the first road user A is not significantly slower or faster than a previous speed limit (step S290: NO), the subroutine S180 proceeds to a step S294, in which it is determined whether the first road user A is accelerating or starting to tailgate. If so (step S294: YES), the subroutine S180 goes to a step S296, where the acceleration counter data D276 is incremented. The sub-routine S180 then proceeds to step S292.

When the first road user A neither accelerates nor starts to tail (step S294: NO), the sub-routine S180 goes to a step S298. In the step S298, it is determined whether or not a distance between the first vehicle A and a vehicle preceding the first vehicle A exceeds a predetermined distance. If this is the case, road user A can be assumed to act independently.

In the affirmative case of step S298 (step S298: YES), the subroutine S180 proceeds to step S300 in which it is determined whether or not the first road user A is decelerating. If the first road user A is decelerating (step S300: YES), then it is assumed that the analyzed first road user A is decelerating regardless of a preceding road user, and the subroutine S180 first increments the deceleration counter data D278 in step S302 and then goes to step S292 about. If the first road user A is not decelerating (step S300: NO), then it is assumed that the analyzed first road user A maintains its current speed without being obstructed, and the subroutine S180 first increments the constant speed counter data D280 in step 304 and goes then to step S292.

When the distance between the first road user A and a preceding road user does not exceed the predetermined distance (step S298: NO), the subroutine S180 proceeds to step S306, where it is determined whether the first road user A is decelerating. If this is not the case (step 306: NO), it is assumed that the analyzed road user is traveling at approximately the same speed due to the preceding road user (which is blocking the analyzed first road user A), and the subroutine S180 first goes to step S308, in which the block counter data D282 is incremented, and then proceeds to step S292.

When the first road user A is decelerating (step S306: YES), the behavior of the first road user needs to be analyzed, and the sub-routine S180 proceeds to a step S310. In this step S310, it is determined whether or not the road user ahead of the first road user A starts decelerating earlier (considering a time interval as described above, A starts decelerating e.g. no later than 2 seconds after its predecessor) than the first road user A. If this is not the case (step S310: NO), it is assumed that the analyzed first road user A is decelerating independently of its predecessor, and the sub-routine S180 proceeds to step S302.

If step S310 returns a positive result (step S310: YES), it is assumed that the analyzed first road user A is decelerating due to this preceding road user and not due to a lower speed limit. Therefore, the sub-routine S180 proceeds to step S308, where the block counter data D282 is incremented. Next, the subroutine S180 goes to step S292.

It should be noted that the sub-routine S180 described above can be designed to run continuously so that the output, i.e. data D276, D278, D280 and D282, from multiple passes can be filtered. For example, multiple acceleration data values D276 can be averaged to generate hysteresis. This way the output values will be more robust.

A higher speed limit can be assumed when a value of the acceleration counter data D276 is larger than the sum of a value of the constant speed counter data D280 and a value of the deceleration counter data D278. A lower speed limit can be assumed when a value of the deceleration counter data D278 is greater than the sum of a value of the constant speed counter data D280 and a value of the acceleration counter data D276.

Based on the acceptance result output from the subroutine S180 and a road sign determined according to the process, as referred to above with reference to FIG 3 described, the determination adjustment unit 118 adjusts a determination result for a specific type of road sign indicating a specific speed limit.

An embodiment of the subroutine S152 will now be described with reference to FIG 5 which is designed to handle signs displayed at a service bridge (traffic control and information system 150).

The subroutine S152 starts at a step S154.

Next, the sub-routine S152 proceeds to a sub-routine S180, which is identical to the sub-routine S180 described above, so that the above description is referred to.

Then, the subroutine S152 goes to a step S160. In this step S160 it is determined whether or not road sign dependencies of the at least one road sign are violated, i.e. whether existing road laws and/or regulations are violated, e.g. if a road sign in the traffic control and information system 150 is assigned to a right lane indicates a higher speed limit than a road sign in traffic control and information system 150 assigned to a left lane (which is a violation in most right-hand traffic systems). If this is the case (step S160: YES), in a step S204 the first/current entire set of road signs is adjusted, for example by selecting a next entire set with the next higher accumulated recognition probability (similar to step S204 described above).

Thereafter, the process proceeds to step S170, which is identical to step S170 described above. Thus, if a current total set of road signs with an associated specific probability of recognition and/or a current total analysis result does not differ from the first (and/or the highest) total set of road signs/total analysis result by more than 10% or 25 % discriminates (step S170: YES), the sub-routine S152 again enters the step S160. In the negative case of step S170 (step S170: NO), the subroutine ends in a step S156 and the process proceeds as described with reference to FIG 3 has been described. How re 3 described above, a negative result of step S170 may indicate that there is a deviation, but that there is no (better) solution at this point in time.

If step S160 returns a negative result (step S160: NO), the process proceeds to step S200a, which is almost identical to step S200 described above, except that step S160 is shown as a step separate from Step S200a is off while step S200 3 also includes the plausibility check of step S160. Thus, where applicable, reference is made to the above description.

It should be mentioned here that even if it is stated here that a certain step, in particular the subroutine S152, can be similar/identical to another step, i.e. its general logic can be similar/identical, the steps completely different data can process, such as the plausibility check included in step S160 can identify a lack of plausibility when a plurality of road signs installed in a service bridge differ from each other in the correct order, wherein the plausibility check included in step S200 can identify a lack of plausibility , when two street signs installed on both sides of a street are different from each other.

The data D190, which is derived from the subroutine S180 (see the above description regarding the data D190), is entered in step S200a as input data.

Now, when step S200a returns an affirmative result (step S200a: YES), the process proceeds to step S204 described above. If step S200a returns a negative result (step S200a: NO), the current overall analysis result is accepted, ie a previous result is discarded or overwritten in step S202 (just like step S202 described with reference to FIG 3 was described).

After step S202, the subroutine S152 ends in step S156, and the process proceeds as with reference to FIG 3 described further.

Referring to the flow chart below 6 An embodiment of a method for assisting a driver of a ego vehicle 100 in correctly monitoring an environment of a road 140 on which the ego vehicle 100 is currently traveling will be described in more detail below.

The method starts with a sign determination step S400, in which at least one specific type of road sign is determined. The determination in the shield determination step S400 is based on data captured using a camera unit 120 and/or based on data obtained from map data on the road 140 on which the ego vehicle 100 is currently traveling. The camera unit 120 is installed in the ego vehicle 100 and captures images around the road 140 on which the ego vehicle 100 is currently traveling.

In addition, a plurality of possible different specific types can also be determined for a single road sign in the sign determination step S400. Each specific type indicates different traffic regulation information. Further, each specific type may be assigned a different recognition probability based on a determination accuracy of the sign determining step S400 for a corresponding specific type for the individual road sign.

The method then proceeds to a traffic behavior monitoring step S410, in which the behavior of at least one road user A is monitored. The road user A is placed near the ego vehicle 100 and is traveling on the same road and in the same direction as the ego vehicle 100. According to another embodiment, the traffic behavior monitoring step S410 can be continuous and independent of the sign determining step S400 be performed.

The result of the sign determining step S400 and the traffic behavior monitoring step S410 is output to a comparison step S420. In the comparison step S420, the determination result output from the sign determination step S400 is compared with the monitored behavior of the at least one road user A, which is output from the traffic behavior monitoring step S410.

The method may then proceed to a determination adjustment step S430, in which the determination result output from the shield determination step S400 is adjusted in the case that in the comparison step S420 there is a discrepancy between the determination result output from the shield determination step S400 and the monitored behavior of the at least one road user A is determined, which is output from the traffic behavior monitoring step S410.

In addition, in the determination adjustment step S430, the determination result output in the sign determination step S400 could be adjusted only in the case that a difference in recognition probability between a highest recognition probability of a specific type for a recognized road sign and a second highest recognition probability of a specific type for the same road sign is less than 25%, preferably less than 10%. Finally, the method ends in step S440.

Claims (15)

Driving support device (110) which is adapted to be installed in a ego vehicle (100) and is adapted to assist a driver of the ego vehicle (100) in correctly understanding an environment of a road (140). to observe on which the ego vehicle (100) is currently moving, the driving support device (110) comprising: a sign determination unit (112) which is set up to determine at least one specific type of road sign (146a, 146b, 146c), wherein the shield determination unit (112) is configured to base the determination on data recorded using a camera unit (120) installed in the ego vehicle (100). and which is set up to capture images of the surroundings of the road (140) on which the ego vehicle (100) is currently traveling and to forward the images to the sign determination unit (112) and/or wherein the sign determination unit (112) is set up to base the determination on data obtained from map data relating to the road (140) on which the ego vehicle (100) is currently traveling, a traffic behavior monitoring unit (114) which to is set up to monitor the behavior of at least one road user (A, B, C) who is in the vicinity of the ego vehicle (100) and is on the same road (140) and in the same direction (t) as the ego - vehicle (100) moves, a comparison unit (116) which is set up to compare the determination result output by the sign determination unit (112) with the monitored behavior of the at least one road user (A, B, C ) output from the traffic behavior monitoring unit (114), and a determination adjustment unit (118) adapted to adjust the determination result output from the sign determination unit (112) in the case that the comparison unit ( 116) determines a deviation between the determination result output by the sign determination unit (112) and the monitored behavior of the at least one road user (A, B, C) which is output by the traffic behavior monitoring unit (114). Driving support device (110) after claim 1 , characterized in that the sign determination unit (112) is set up to determine a plurality of possible specific types for a single road sign (146a, 146b, 146c), each specific type indicating different traffic regulation information, in particular different speed limits, for the road (140) and/or lane (I, II, III) on which the ego vehicle (100) is currently traveling, each specific type having a different recognition probability based on a determination accuracy of the sign determination unit (112) for a corresponding specific type for the individual street sign (146a, 146b, 146c). Driving support device (110) after claim 2 , characterized in that the determination adjustment unit (118) is arranged to adjust the determination result output from the sign determination unit (112) only in the case that a difference in recognition probability between the highest recognition probability of a specific type for the individual road sign ( 146a, 146b, 146c) or a first complete set of several specific types of street signs (146a, 146b, 146c) and a lower recognition probability of a specific type for the single street sign (146a, 146b, 146c) or a second complete set of several specific ones types of street signs (146a, 146b, 146c) is less than a predetermined threshold value, in particular less than 25%, preferably less than about 10%. Driving support device (110) according to one of Claims 1 until 3 , characterized in that the sign determining unit (112) is arranged to determine a plurality of road signs (146a, 146b, 146c) at substantially the same time and to assign at least one specific type to each road sign (146a, 146b, 146c). determine, wherein the driving support device (110) further comprises: a plausibility control unit (126), which is set up to check the at least one specific type determined for each road sign (146a, 146b, 146c) with regard to existing road traffic regulations. Driving support device (110) after claim 4 , characterized in that the plausibility control unit (126) is set up to indicate a possible error in the determination of the plurality of street signs (146a, 146b, 146c) in the event that different specific types for a street sign (146a, 146b, 146c) fixed to one side (148a) of a section (144) of the road (140), and for a road sign (146a, 146b, 146c) fixed to the other side (148b) of the same section (144) of the road (140) can be determined. Driving support device (110) after claim 4 or 5 , characterized in that the plausibility control unit (126) is set up to indicate a possible error in the determination of the plurality of street signs (146a, 146b, 146c) in the event that for traffic systems with right-hand traffic for a first street sign (146a, 146b, 146c) displayed in a traffic control and information system (150) to the left of a second street sign (146a, 146b, 146c) displayed in the same traffic control and information system (150), a specific type of street sign (146a , 146b, 146c) indicating a speed regulation value lower than the speed regulation value of the second street sign (146a, 146b, 146c). Driving support device (110) according to one of Claims 1 until 6 , characterized in that in the event that a street sign (146a, 146b, 146c) is detected in a traffic control and information system (150) and a certain specific type of street sign (146a, 146b, 146c) for the lane (I, II, III) on which the ego vehicle (100) is currently moving is different from certain specific types of road signs (146a, 146b, 146c) for other lanes (I, II, III), the traffic behavior monitoring unit (114 ) is set up to monitor only the behavior of traffic surrounding the ego vehicle (100) in the same lane (I, II, III) of the road (140) on which the ego vehicle (100) is currently traveling , And / or is set up to monitor the behavior of at least one road user (A, B, C), which is placed in the vicinity of the ego vehicle (100) on at least one lane (I, II, III), which differs from the track (I, II, III) on which the ego vehicle (100) is currently moving using a multiplication factor which is associated with the other specific type of road sign (146a, 146b, 146c) for the corresponding other lane (I, II, III) and which represents a difference from corresponding road regulation information . Driving support device (100) according to one of Claims 1 until 7 , characterized in that the traffic behavior monitoring unit (114) is further set up to: monitor a behavior of at least two road users (A, B); to detect a distance between at least two road users (A, B) in the case that a first road user (A) drives ahead of a second road user (B) in the same lane (I, II, III); to determine at least one road user (A, B) as an independent road user (A, B), wherein an independence of a road user (A, B) is determined in a case that the distance between the at least two road users (A, B) in Substantially exceeds a predetermined distance, in particular at least one distance value in meters, which is half the speed value in km/h of the second road user (B), and/or in a case that the second road user (B) accelerates or brakes and the first road user (A) substantially maintains his speed for a predetermined period of time; and to output only the monitored behavior of the at least one independent road user (A, B) to the comparison unit (116). Driving support device (110) after claim 8 , characterized in that the traffic behavior monitoring unit (114) is also set up to monitor a behavior of at least three independent road users (A, B, C), wherein the comparison unit (116) is also set up to compare the monitored behavior of the at least to compare three independent road users (A, B, C) and only to consider the behavior which is similar for a majority of the monitored road users (A, B, C). Driving support device (110) according to one of Claims 1 until 9 , characterized in that the driving support device (110) further comprises: a traffic information receiving unit (130) which is set up to receive traffic information regarding a traffic situation, in particular a traffic jam, on a section of the road (140) in front of a current position of the ego vehicle (100); and a traffic information consideration unit (132) which is set up to determine a predetermined traffic situation-compliant behavior of road users (A, B, C) on the basis of the traffic information which is output by the traffic information receiving unit (130); wherein the comparison unit (116) is further set up to compare the monitored behavior of the at least one road user (A, B, C) output by the traffic behavior monitoring unit (114) with the predetermined traffic situation-compliant behavior which is determined by the traffic information consideration unit (132) is issued; and wherein the determination adaptation unit (118) is further set up to calculate a displayed deviation between the determination result output by the sign determination unit (112) and the monitored behavior of the at least one road user (A, B, C), which is derived from the traffic behavior Monitoring unit (114) is issued to discard in the event that the monitored behavior of the at least one road user (A, B, C) matches the predetermined traffic situation-compliant behavior. Driving support device (110) according to one of Claims 1 until 10 , characterized in that the driving support device (110) further comprises a server connection unit (136) configured to transmit to a server: a determination result of a specific type a road sign (146a, 146b, 146c) which is output from the sign determination unit (112) and/or is output from the determination adjustment unit (118); a unique identifier of the corresponding street sign (146a, 146b, 146c), in particular a unique position on the street (140); and preferably a date and time of the transmitted determination result; wherein the server connection unit (136) is further adapted to receive from the server: road sign information for a unique road sign (146a, 146b, 146c) indicating a specific type of road sign (146a, 146b, 146c), which has previously been determined by a plurality of other road users (A, B, C); wherein the determination adjustment unit (118) is further configured to adjust the determination result output from the sign determination unit (112) in a case of discrepancy between the determination result output from the sign determination unit (112) and the road sign information received from the server to adjust. Vehicle (110), comprising a driving support device (110) according to any one of Claims 1 until 11 . Method for supporting a driver of an ego vehicle (100) in correctly observing the surroundings of a road (140) on which the ego vehicle (100) is currently moving, the method comprising the following steps: a sign determining step (S400) in which at least one specific type of road sign (146a, 146b, 146c) is determined, wherein in the sign determining step (S400) the determination is based on data recorded using a camera unit (120). be installed in the ego vehicle (100) and which captures images of the surroundings of the road (140) on which the ego vehicle (100) is currently traveling, and/or wherein in the sign determination step (S400) the determination is based on data, which are obtained from map data regarding the road (140) on which the ego vehicle (100) is currently traveling, a traffic behavior monitoring step (S410) in which the behavior of at least one traffic user (A, B, C) which is in the vicinity of the ego vehicle (100) and is on the same road (140) and into the same is monitored direction (t) as the ego vehicle (100) moves, a comparison step (S420) in which the determination result output in the sign determination step (S400) is compared with the monitored behavior of the at least one road user (A, B, C) which is output in the traffic behavior monitoring step (S410), and a determination adjustment step (S430) in which the determination result output in the shield determination step (S400) is adjusted in a case where in the comparison step (S420) there is a deviation between the determination result output in the shield determination step (S400) and the monitored behavior of the at least one road user (A, B, C), which is output in the traffic behavior monitoring step (S410). procedure after Claim 13 , characterized in that in the sign determining step (S400) a plurality of possible specific types are also determined for a single road sign (146a, 146b, 146c), each specific type indicating different traffic regulation information, each specific type of a different recognition probability is assigned based on a determination accuracy of the sign determining step (S400) for a corresponding specific type of the individual road sign (146a, 146b, 146c). procedure after Claim 14 , characterized in that in the determination adjusting step (S430), the determination result outputted in the sign determining step (S400) is adjusted only in the case that a difference in recognition probability between the highest recognition probability of a specific type for the single road sign (146a, 146b, 146c) or a first complete set of several specific types of street signs (146a, 146b, 146c) and a lower recognition probability of a specific type for the single street sign (146a, 146b, 146c) or a second complete set of several specific types of street signs (146a, 146b, 146c) is less than a predetermined threshold value, in particular less than 25%, preferably less than about 10%.

Images