DE102021206634A1 - Method and warning device for warning a user of a vehicle of a potentially dangerous situation - Google Patents

Abstract

A method for warning a user of a first vehicle (1) of a potentially hazardous situation at at least one destination of the vehicle (1), comprising:- detecting optical information about a roadway section (8) and optionally also its surroundings in a defined route section (8) by an image recording device (20b) of a second vehicle (2),- calculation of information about a potential development of a critical road condition by an evaluation device (10b) of the second vehicle (2) on the basis of the information recorded by the image recording device (20b) and current and /or predicted meteorological data, - transmission of the information about a possible critical road condition and the position data of the route section (8) for which the critical road condition is precalculated to a storage device (10a, 14) arranged outside the second vehicle, - transmission of information about the possible Chen critical road condition and the position data of the road section (8), for which the critical road condition is precalculated, from the storage device (10a, 14) to an output device (6a) of the first vehicle (1).

Description

The present invention relates to a method and a warning device for warning a user of a vehicle of an, in particular potential and/or expected, dangerous situation at at least one destination of the vehicle. In particular, the present invention relates to a method for warning the driver using optical and/or acoustic methods in good time before using a route section with an increased risk potential.

Many modern motor vehicles have sensors such. B. a front camera that can observe the road and its immediate surroundings. Information that is recorded by the vehicle using such optical devices is used in this vehicle to avoid accidents. For example, a person can be detected who is moving towards the roadway and the speed can be reduced in good time in order to avoid a collision between the vehicle and the person.

Some vehicles are not equipped with particularly powerful systems for recording environmental data or, due to their design, cannot be equipped with such systems. An example of such vehicles are motorcycles. Especially in autumn, dirt on the road, for example dirt from agricultural machinery, leaves, or the like, and bad weather lead to critical road conditions. In many cases, these cannot be seen before driving on the affected section of road, for example entering a curve. Accordingly, such a situation may surprise the driver. Motorcyclists are particularly at risk from road pollution.

From the U.S. 8,831,869 B2 a road condition monitoring network is known in which vehicles equipped with appropriate sensors and a wireless communication system continuously monitor their surroundings using the sensors and wirelessly transmit this information to other vehicles via vehicle-to-vehicle communication. It is envisaged that a number of vehicles will detect the same dangerous road condition with their individual sensors and the reports relating to a road condition from the various vehicles will be aggregated in order to obtain an aggregated probability of the presence of the detected road condition. This is then transmitted to vehicles approaching the hazard. The information packets can be transmitted vehicle-to-vehicle (V2V) or vehicle-to-infrastructure-to-vehicle (V2l2V).

the U.S. 9,475,500 B2 also describes using crowd-sourced data from vehicles to determine road friction conditions. In particular, it is envisaged that road surface friction data from multiple vehicles will be collected and analyzed at a central server to identify friction conditions by lane and location, and provide indications of road friction conditions to vehicles on the road.

These systems have the disadvantage that the specific danger must be present and must have been detected by a vehicle traveling on this section of road before a warning can be issued to vehicles that intend to use this section of road in the future.

The object of the present invention is to overcome the disadvantages known from the prior art and to provide a method and a warning device by means of which a hazard can be detected at an early stage, if possible before a hazardous state occurs, and transmitted to other vehicles.

According to the invention, this object is achieved by the subject matter of the independent claims. Advantageous embodiments and developments of the invention are the subject matter of the dependent claims.

In a method according to the invention for warning a user of a first vehicle of a potentially dangerous situation at at least one destination of the vehicle, it is provided that optical information about a roadway section and optionally also its surroundings in a defined route section are recorded by an image recording device of a second vehicle. Information about a potential development of a critical road condition is then calculated by an evaluation device of the second vehicle on the basis of the information recorded by the image recording device and current and/or predicted meteorological data. The information about a possible critical road condition and the position data of the road section for which the critical road condition is precalculated are then transmitted to a memory device arranged outside the second vehicle. From here, the information about the possible critical road condition and the position data of the route section for which the critical road condition is precalculated is transmitted to an output device of the first vehicle.

This method is particularly easy to implement, since it preferably only requires an optical sensor, namely the image recording device. For example, to determine potentially dangerous roadway conditions, such as reduced friction, it is not necessary to determine the actual coefficient of friction and, for example, to extrapolate the development of the coefficient of friction data recorded from a number of vehicles over a long period of time.

Rather, the present method allows a prediction to be made from the analysis of the optical properties of the roadway, including current or predicted meteorological data, as to whether and when a potentially dangerous situation is likely to arise in the defined section of the route.

The image recording device is preferably selected from a group which includes a (color) camera, a front camera, a rear camera and the like, as well as combinations thereof.

The image recording device preferably generates locally resolved (in particular 2D and/or 3D) image data.

The image data are preferably generated and/or determined while the vehicle is driving.

The image data can either only be stored locally in the second vehicle or optionally also be loaded into a cloud or a vehicle-independent memory.

The information about a potential development of a critical road condition is preferably calculated using a processor device and/or data processing device, using at least one (computer-implemented) computer vision method in which (computer-implemented) perception and/or detection tasks are carried out , for example (computer-implemented) 2D and/or 3D object recognition methods and/or (computer-implemented) methods for semantic segmentation and/or (computer-implemented) object classification (“image classification”) and/or (computer-implemented) object localization.

The data recorded by the image recording device are preferably stored in a (temporary and/or local) storage device of the vehicle. The memory device can be a ring memory. Preferably only the image data of a specified driving route, a specified driving length and/or driving time (preferably in the internal memory device of the vehicle) are kept ready. The image data are particularly preferably overwritten and/or deleted after the specified driving distance and/or driving time has been exceeded.

The image data is evaluated in the evaluation device, preferably using an artificial neural network.

The method or individual method steps can be carried out (at least in part) before or when the journey (of the first vehicle) starts. However, it is also conceivable for the method to be initiated and/or triggered if the vehicle detects a (new and/or changed) destination and/or route. A travel destination and/or a travel route can be recorded by a (voice-based and/or gesture-based and/or physical and/or manual) input by a user or by (automatic) determination by the vehicle.

A travel destination is understood to mean, in particular, any location along a (intended) travel route or at a predetermined spatial distance from a (intended) travel route of the user. In particular (although it may not be) the destination does not have to be a navigation destination present in a navigation system for the vehicle or a location derived therefrom. It can be an intermediate destination or the final destination of a (intended) driving route. However, it can also be, in particular, any point along the travel route (taken and/or to be taken).

The destination can be entered (by the user) on an input device of the vehicle or for the vehicle. It is also conceivable, however, for the vehicle to set a destination and/or a route, for example based on at least one thing based on frequency and/or regularity and/or a time and/or a day of the week and/or a number of occupants and/or the user and/or or at least one previous journey and/or several previous journeys based on a criterion (in particular as a suggestion) and/or selects.

If, for example, a change in color and/or a pattern on the roadway is detected in a section of road, this can be attributed to fallen leaves or other dirt on the roadway, such as soil from construction vehicles or agricultural traffic, for example. Such dirt on the road alone usually does not represent a dangerous situation about which the following vehicles should be informed. Nevertheless, such a Pollution is a potential hazard that could become acute, for example, after precipitation in the area of the respective route section.

The objects that trigger a (potential) dangerous situation and/or lead to a (potential) dangerous situation can preferably be situations and/or objects that have arisen due to the weather (such as a puddle of rain) and/or which have been caused by the effects of meteorological factors Influences (such as wind and/or storm and/or precipitation) only became such (potential) dangerous situations triggering and/or objects leading to a (potential) dangerous situation (such as the ground becoming slippery due to rain or leaves blown onto the road).

The objects that trigger a (potential) dangerous situation and/or lead to a (potential) dangerous situation are preferably objects that are (permanently) connected and/or linked to the roadway and lead to a potentially dangerous situation (such as moisture, leaves). .

The relevance and level of the potentially dangerous situation can also depend on the type of the first vehicle. For example, reduced coefficients of friction due to damp leaves or damp soil on the roadway are significantly more dangerous for single-track vehicles than for two-track vehicles. Accordingly, the potential of the hazard could be determined depending on the type of the first vehicle.

If, for example, an increased reflection of the road surface is detected in a section of road, this can be attributed to a damp road. Depending on the height of the film of water on the road, this can also pose a risk (aquaplaning). However, this danger usually only exists during heavy rainfall. On the other hand, if ground frost sets in (or is predicted) even if the humidity is low, ice formation could begin, which in turn can pose a significant hazard to following vehicles. Here, too, there is the possibility of recognizing the relevance of the potential danger as a function of meteorological data and/or weather forecasts at an early stage and before the increased danger arises, and of warning vehicles and/or their drivers against using the endangered route section.

In a variant of the method, the information about a possible critical road condition and the position data of the road section for which the critical road condition is precalculated is transmitted from a (second) vehicle directly to another (first) vehicle (“vehicle to vehicle”). "V2V"). Such a procedure is particularly easy to implement because it does not require central storage of the data, but the information can be sent directly from the second vehicle to all vehicles in the area that are equipped and accessible for the transmission of the data. This is usually sufficient, as it is particularly likely that these vehicles will use the section of road in question.

In a preferred variant of the method, however, it is alternatively or additionally provided that the storage device is an external storage device, in particular a cloud-based and/or a central storage device. This has the advantage that the computing capacity can be adapted to changed requirements in terms of computing power in a particularly simple manner and preferably independently of the vehicle. In particular, the consideration of high-resolution meteorological data and the prediction of meteorological data for a specific route section can involve very large amounts of data and be computationally intensive, so that a vehicle-bound memory device is not able to do this, or at least not in an acceptable time. Furthermore, a cloud-based and/or a central storage device has the advantage that any faulty data of an individual vehicle can be identified as such and accordingly (not) taken into account when determining a risk potential.

A method is preferred in which the vehicle-bound evaluation device for calculating the information about a potential development of a critical road condition includes at least one piece of optical information about a road, which is selected from a group consisting of the color of the road surface, the reflectivity of the road surface, the structure of the road surface, cast shadows on the road surface, brightness of the surroundings, structure of the surroundings, cloud formation, visibility and fog. This visual information has proven to be particularly suitable for being an indication of a potential development of a dangerous situation. For example, as already explained above, dirt can be identified from the color and/or a pattern on the roadway and the type of dirt can be determined, for example, taking the pattern into account. With a stripe pattern in the longitudinal direction of the roadway, soiling from soil, for example tire tracks from agricultural vehicles, is likely to occur while an irregular pattern can be an indication of fallen leaves on the roadway. Reflectivity can be an indication of road surface moisture.

Likewise, it can be advantageous that information from the recorded surroundings is also included in the evaluation. For example, it can be determined in this way whether the route section is located in a forest where increased foliage is to be expected as contamination of the roadway. The brightness of the surroundings can be an indication of the cloud cover and thus of the precipitation to be expected. Likewise, the visibility and/or the increased refraction of light towards diffuse scattering (e.g. due to airborne droplets (rain, fog) or particles (snow, dust) can be included in the risk assessment).

If the inclusion of environmental data is not possible, for example for data protection reasons, it has been shown that environmental parameters can also be indirectly determined by solely analyzing the optical properties of the road surface. For example, it can be determined from the frequency of regularly occurring shadows (if necessary and taking into account the vehicle speed and/or position data of the vehicle in connection with a stored map) whether the vehicle is moving through a forest, a city, an unobstructed open landscape or over one of trees flanked street moves.

The information about the possible critical road condition and the position data of the route section for which the critical road condition is precalculated is preferably output optically and/or acoustically to a driver of the first vehicle by an output device. This is particularly preferred because most of the information is given to the driver via these communication channels and he is therefore particularly familiar with this type of communication. In addition, most vehicles have corresponding communication interfaces such as displays or loudspeakers, so that such communication can be implemented easily.

The information about the possible critical road condition and the position data of the route section for which the critical road condition is precalculated is preferably output to the driver in a driver information display.

In the case of single-track vehicles such as motorcycles, mopeds or bicycles in particular, it has been shown to be particularly advantageous for the driver to be given information about the possible critical road condition and the position data of the route section for which the critical road condition is precalculated by a helmet warning unit. This is due in particular to the fact that these vehicles usually have little or even no driver information display. For most of these vehicles, however, a helmet is mandatory or at least recommended. Accordingly, it is advantageous to send the relevant information to the driver via the helmet or a warning unit of the helmet.

In a further preferred method, the warning signal is displayed using a (in particular augmented reality) head-up display (HUD) as an output device and/or as a virtual image component.

A virtual image component which is characteristic of the potential dangerous situation is preferably output by means of the output device, in particular by means of the (augmented reality) head-up display.

For example, the potentially dangerous situation could be shown and/or visualized to the user by means of an AR-HUD. For example, a contact-analogous symbol for the warning point, for example a leaf on the street, could be displayed.

The virtual image component and/or the contact-analogous symbol is preferably projected onto a vehicle interior surface, preferably onto a vehicle window and particularly preferably onto a windshield, but appears to the user to be displayed at a certain distance in front of the vehicle. Such a display and/or projection is preferably based on the user's viewing direction and/or eye position (determined in particular by a viewing direction detection device of the vehicle).

Alternatively or additionally--particularly preferred in the case of single-track vehicles--the visual display can be carried out in glasses or a visor.

The method also includes determining a warning variable on the basis of the possible critical road condition determined for the respective route section, as a function of which a warning signal can be provided for output to the user by means of an output device. The warning variable is preferably a measure of the level of the potential hazard and/or the probability of the hazard occurring.

It is not mandatory that the information about the possible critical lane condition and the position data of the route section for which the critical road condition has been precalculated is output to the driver. Particularly in the area of autonomous operation of the vehicle, it is conceivable and preferred that the information about the possible critical road condition and the position data of the route section for which the critical road condition is precalculated and/or the warning variable to the vehicle and/or to a control device of the vehicle is transmitted.

The control device is preferably a navigation system of the vehicle.

The vehicle control device preferably takes into account the information about the possible critical road condition and the position data of the road section for which the critical road condition is precalculated and/or the warning variable when controlling at least one vehicle function of the vehicle.

The vehicle function can be, for example, a driving function of the vehicle, such as route planning and/or a route change and/or (autonomous and/or semi-autonomous) operation of the vehicle and/or a vehicle assistance function.

The vehicle function is preferably a function of a vehicle component of the vehicle. The vehicle component is preferably selected from a group which includes a navigation system, a vehicle safety system (e.g. a braking system), a system for guiding the vehicle, a locking system for a vehicle door and/or a vehicle window, a windscreen wiper mechanism, a vehicle locking system, a roof removal mechanism, a sunroof mechanism Includes infotainment system, an entertainment system and / or a comfort system to increase the driving comfort of an occupant and combinations thereof.

In particular, a variant of the method is preferred in which the information recorded by the image recording device is transmitted to an evaluation device arranged outside of the second vehicle, preferably a vehicle-independent evaluation device. It is preferred that this transmission of the information (image data) recorded by the image recording device to an evaluation device arranged outside the second vehicle takes place in addition to the transmission of the information about a possible critical road condition. The recipient of these two transmissions may be different or the same. For example, it is conceivable and preferred in one variant that the information about a possible critical road condition is sent from the second vehicle to a first vehicle, whereas the information recorded by the image recording device of the second vehicle is sent to a vehicle-independent evaluation device, for example a cloud-based and/or or a central evaluation device.

The vehicle-independent evaluation device preferably has a memory device assigned to it, which it can access.

The evaluation of the image data is preferably carried out by the vehicle-independent evaluation device using an artificial neural network.

A vehicle-independent storage device is to be understood in particular as a storage medium that is external to the vehicle, for example a server, in particular a backend server. The external server is, for example, a backend of a vehicle manufacturer or a service provider, which is set up to manage and optionally evaluate image data relating to potential dangerous situations relating to a large number of vehicles. The functions of the backend or the external server can be performed on (external) server farms. The (external) server can be a distributed system. The external server and/or the backend can be cloud-based.

The dangerous situations can thus advantageously be exchanged via the cloud or via a central server. This offers the advantage that an individual vehicle does not have to drive past the danger spot, such as the puddle, in order to locate an ice situation there after a drop in temperature.

Such a vehicle-independent evaluation device preferably determines a change in the condition of the roadway from a large number of optical items of information available for this route section or transmitted to the storage device. If a potentially critical road condition is determined by the vehicle-independent evaluation device, this information is also made available to vehicles that are passing or intend to pass this section of road.

Such an additional evaluation of the optical information about a road section, which may have been recorded by a large number of vehicles at different times, by the vehicle-independent evaluation device makes it possible to eliminate errors in the calculation by individual to avoid on-board evaluation devices and, in addition, to determine a long-term course of the change in the potential hazard on a specific route section.

Determining the long-term course of the change in the potential hazard on a specific route section makes it possible to identify recurring potential hazards on a specific route section periodically (for example following the course of the seasons or a day) and to recognize them as such. In this way, suitable preventive measures can be taken at an early stage to prevent an increased risk from occurring again on particularly endangered sections of road.

Furthermore, the determination of the long-term course of the change in the potential hazard on a certain stretch of road enables the detection of long-term changes in the condition of the road. For example, the formation of a pothole can be recognized and possibly recognized before it is torn open over a large area, and road users can be warned accordingly. Likewise, an order to remove the expected damage to the road could be given. This also applies analogously to other types of deformation of the roadway, such as the sinking of sections of roadway (e.g. due to terrain shifts (land slide)) or the occurrence of blow-ups.

In particular, it is preferred that the data that is determined when determining the long-term course of the change in the potential hazard on a specific route section is made available to at least one third party. This third party is preferably selected from a group that includes the owner of the road, the responsible administration, a (long-distance) road authority, a leaseholder, a toll station operator, a road maintenance department and a company commissioned with maintenance.

Optionally, such a third party can also be granted access to further data that is determined as part of a method as described above or with a warning device as described below. For example, a road maintenance department (or another third party) could also receive warnings regarding dangers to be expected at short notice in order to be able to eliminate this danger, prevent it from occurring or block the section of road.

In a further preferred variant, in addition to the information about the expected danger, the third party is also provided with (raw) data that was recorded directly by an image recording device, for example image data of the affected route section. This can simplify the reaction to the expected danger. A specialist or an evaluation device set up for this purpose can preferably view and/or evaluate the image material available for the section of road and accordingly take possible measures to eliminate the potential danger spot or to prevent the emergence of a danger.

In particular, it is preferred that the change in the condition of the road determined by the vehicle-independent evaluation device includes a short-term change (such as dirt or moisture) and/or a long-term change (such as the formation of a pothole or the lowering of a section of the road).

The method is preferably carried out within a predetermined period of time, which extends from the time a second vehicle travels a section of road until a first vehicle drives on this section of road. In some specific embodiments, however, the method can also be ended before the first vehicle has passed the route section. For example, if the potential danger for the first vehicle is estimated to be so high that the use of this endangered section of the route appears unreasonable and this vehicle is therefore assigned an alternative route. In this case, the procedure ends with the suggestion of the alternative route.

If necessary, the procedure can be carried out again for the individual route sections of the alternative route.

In a preferred variant, the information on potential dangers in a route section and/or the image data stored for this route section are kept for a predetermined period of time and, if necessary, used to calculate information on potential dangers.

This period can be specified by a memory size, which deletes the oldest data after reaching the maximum memory capacity (rolling memory).

In particular, the specified period is a period between one day and one month, preferably a period between one day and two weeks, preferably a period between one day and one week, preferably a period between one day and 5 days and particularly preferably a period between one day and lie two days. It is also conceivable that a period of time is selected as the predetermined time period which is a maximum of 24 hours, preferably a maximum of 20 hours, preferably a maximum of 15 hours, preferably a maximum of 10 hours, preferably a maximum of 5 hours and particularly preferably a maximum of 2 hours.

The vehicle, in particular an evaluation device of the vehicle, preferably compares the (current and/or intended) destination of the vehicle with available information about a possible critical road condition of this destination. The vehicle and in particular an evaluation device (of the vehicle and/or an external server, in particular a backend) preferably determines a route with a low risk potential and/or informs the driver about potential risks on the selected route.

The evaluation device can (in particular by means of a computer program and/or using modern algorithms such as AI, machine learning and/or deep learning or the like) classify a dangerous situation into predefined levels.

In a further preferred method, the data transmitted from the second vehicle to the first vehicle (via V2V) includes processed information on at least one potential hazardous situation determined by the second vehicle. The information relating to the (raw) image data is preferably essentially reduced to data which is relevant to the at least one potential dangerous situation. This offers the advantage that only small amounts of data need to be transmitted and stored in the first vehicle.

In a further preferred method, meteorological data relating to the vehicle and/or the destination of the vehicle are ascertained and the warning variable is ascertained as a function of the meteorological data. This offers the advantage that, based on the meteorological data, the image data can be evaluated with regard to a predictive prediction of a current state of a potentially hazardous situation in the route section on which the image data was recorded.

Meteorological data can be understood in particular as data selected from a group that includes a wind direction, wind speed, (ambient) temperature (of the vehicle), humidity, air pressure, stability, cloud cover, precipitation and the like and combinations thereof.

The meteorological data are preferably obtained from external sources. The meteorological data are preferably queried (at least in part), in particular via a wireless communication link, from an external server and/or via an online service (such as a weather service).

In a further preferred method, meteorological data assigned to the image data, in particular (outside) temperature data, are stored on the storage device. Meteorological data are preferably recorded and/or determined and/or assigned to this location essentially at the same time as the recording and/or determination of the image data, in particular at the image recording location (at which the image data are recorded).

In particular, information about the current outside temperature or the outside temperature to be expected within a defined time interval is preferably also stored with the image data.

The present invention is also directed to a method and/or system for operating a vehicle in a potentially occurring and/or expected dangerous situation at at least one destination of the vehicle, comprising capturing optical information about a road section and optionally also its surroundings in one defined route section by an image recording device of a second vehicle, calculation of information about a potential development of a critical road condition by a vehicle-bound evaluation device based on the information recorded by the image recording device and current and/or predicted meteorological data, transmission of the information about a possible critical road condition and the Position data of the route section for which the critical road condition is precalculated is to a storage device arranged outside of the second vehicle, and transmission of a Information about the possible critical road condition and the position data of the road section for which the critical road condition is precalculated from the storage device to an output device of the first vehicle.

The method can have one or more method steps described in the context of the above method individually or in combination with one another and vice versa.

The meteorological data is preferably a date or a plurality of data selected from a group that includes a temperature, an (expected or measured) amount of precipitation, an air humidity, a wind strength, a wind direction and a dew point.

When planning a route for the first vehicle, the method and/or the system preferably takes into account the information transmitted by the second vehicle and/or a vehicle-independent evaluation device about potentially occurring and/or expected dangerous situations.

The present invention is also aimed at a warning device for warning a user of a first vehicle of a potentially dangerous situation at at least one destination of the first vehicle, comprising an output device of the first vehicle by means of which information about a possible critical road condition and position data of the route section for the critical road condition can be received by a storage device, the storage device being arranged outside a second vehicle and the storage device having a receiving device for receiving information about a possible critical road condition and the position data of the route section for which the critical road condition has been determined from a second vehicle . The second vehicle has an image recording device for capturing optical information about a section of road and optionally also its surroundings in a defined section of road, as well as a vehicle-bound evaluation device, which is set up and provided for information about a potential development of a critical road condition by a Calculate based on the information recorded by the image recording device and current and/or predicted meteorological data.

The warning device is preferably set up, suitable and/or intended to carry out the method described above and all method steps described in connection with the method individually or in combination with one another or individual method steps using it. Conversely, the method for warning a user of a vehicle can be carried out individually or in combination with all of the features described in the context of the warning device. In particular, the warning device can be used to carry out the method described above.

Preferably, the warning device also has a vehicle-independent evaluation device, which is set up and provided to determine a change in the condition of the roadway from a large number of optical information items recorded for this stretch of road by an image recording device of one or more second vehicles and transmitted to the vehicle-independent evaluation device and, in the event the determination of a potentially critical road condition to provide this information for at least a first vehicle, which is currently in this route section or is intended to use this route section in a defined future time interval.

The image recording device is preferably selected from a group which includes a (color) camera, a front camera, a rear camera and the like, as well as combinations thereof.

Locally resolved (in particular 2D and/or 3D) image data can preferably be generated by the image recording device, optionally in conjunction with the evaluation device.

The storage device is preferably arranged in the second vehicle. Alternatively or additionally, it is preferred that the storage device is an external storage device, in particular a cloud-based and/or a central storage device.

The warning device preferably includes an output device by means of which the information about the possible critical road condition and the position data of the route section for which the critical road condition is precalculated can be output optically and/or acoustically to a driver of the first vehicle. This is preferably a (in particular augmented reality) head-up display (HUD). A virtual image component and/or a symbol can preferably be projected by the output device onto a vehicle interior surface, preferably onto a vehicle window and particularly preferably onto a windshield. In the case of single-track vehicles, an output device is particularly preferred which displays/projects a virtual image component and/or a symbol in glasses or a visor.

The warning device preferably includes a vehicle-independent evaluation device. This evaluation device preferably includes a memory device in which a large number of optical information items are stored for a route section. This visual information may have been provided and/or transmitted by different vehicles and/or at different times. The vehicle-independent evaluation device is preferably set up to determine a change in the condition of the roadway from the multitude of optical information. Preferably, the vehicle-independent gige evaluation device furthermore a transmission device, by means of which the information about changes in the condition of the roadway can be transmitted to a vehicle which is arranged in this route section or for which a route planning system foresees the use of this route section. The transmission to such a vehicle preferably takes place only if a potentially critical road condition is determined.

The vehicle-independent evaluation device preferably includes a large memory which is designed to store at least 5, preferably at least 10, particularly preferably at least 20, more preferably at least 30, at least 40 or at least 50 data sets with optical information for a large number of route sections. Due to the fact that this evaluation device is independent of the vehicle, a sufficiently large storage capacity can be reserved and supplied with energy.

The data records for the respective route sections preferably include data records from a period of at least one day, preferably at least 3 days, particularly preferably at least one week, more preferably at least one month, at least 3 months, at least 6 months and particularly preferably at least one year. This makes it possible to identify and predict long-term and periodically recurring dangers at an early stage.

The output device (for a vehicle) can be an element and/or a (fixed) component of the vehicle.

The present invention is also directed to a warning system comprising at least one warning device for a vehicle as described above, according to one embodiment, and comprising an external server for managing a central database with image data and preferably location data associated with it and/or meteorological data associated with it (as described above) . The warning system is preferably configured in such a way that a large number of image capturing devices from different second vehicles can transmit image data and/or location data to the external server and/or retrieve them from it (in particular based on a comparison with a location of the vehicle). Meteorological data are obtained from the external server, preferably from a third party, and are saved together with the image data.

The present invention is also aimed at a vehicle, in particular a motor vehicle, comprising an output device for a vehicle as described above, which is part of an embodiment of a warning device as described above.

The output device is preferably a (fixed, in particular a non-destructively detachable) component of the vehicle or an item of equipment carried by the vehicle driver, such as a protective helmet.

The second vehicle can in particular be a (motorized) road vehicle. A vehicle can be a motor vehicle which, in particular, is a motor vehicle controlled by the driver himself (“driver only”), a semi-autonomous, autonomous (e.g. autonomy level 3 or 4 or 5 (of the SAE J3016 standard)) or self-driving motor vehicle is. Autonomy Level 5 designates fully automated vehicles. The vehicle can also be a driverless transport system. The vehicle can be controlled by a driver or drive autonomously. In addition to a road vehicle, the vehicle can also be an air taxi, an airplane and another means of transportation or another type of vehicle, for example an air vehicle, water vehicle or rail vehicle.

The first vehicle can be the same vehicles as listed above for the second vehicle. However, the first vehicle within the meaning of this invention can also be vehicles that are not also a second vehicle. In the case of single-track vehicles, non-motorized vehicles, air taxis and (light) aircraft in particular, there is often a design-related compulsion to save weight or space, which means that such vehicles may not have all the equipment (e.g. an image recording device) required for the function as a second vehicle is necessary. Nevertheless, these vehicles (regardless of their degree of autonomy) can also be the first vehicles to benefit from the present invention.

The present invention is also directed towards an external server, in particular a backend (preferably described above), which is part of a warning device described above according to one embodiment. In this case, the warning device and/or the external server is suitable and intended for determining at least one possible critical road condition.

The present invention is further directed to a computer program or computer program product, comprising program means, in particular a program code which represents or encodes at least one or more method steps of the method according to the invention individually or in combination with one another and preferably one of the preferred embodiments described and is designed to be executed by a processor device.

The present invention is also aimed at a data memory on which at least one embodiment of the computer program according to the invention or a preferred embodiment of the computer program is stored.

• Darin zeigen:
  • 1 eine schematische Darstellung eines Ausführungsbeispiels einer Warneinrichtung; und
  • 2 eine schematische Darstellung der Verfahrensschritte eines Ausführungsbeispiels eines Verfahrens zur Warnung eines Nutzers eines ersten Fahrzeugs.

Further advantages and embodiments result from the attached drawings:

• Show in it:
  • 1 a schematic representation of an embodiment of a warning device; and
  • 2 a schematic representation of the method steps of an embodiment of a method for warning a user of a first vehicle.

1 shows a schematic representation of an embodiment of a warning device. A second vehicle 2 drives on a section of road 8. An image recording device 20b of the second vehicle takes at least one, preferably several, pictures of the roadway 8. The recorded images are transmitted to an evaluation device 10b. The image recording device 20b can be an optical sensor device for detecting the surroundings of the vehicle and in particular the roadway, for example a front camera of the second vehicle 2.

Furthermore, the second vehicle 2 includes a transmission device 16b, which is preferably capable of bidirectional communication. The images recorded by the image recording device 20b of the second vehicle 2 are transmitted to the transmission device 16b and sent from there to an external storage device or cloud 14 . From there or from another provider, the transmission device 16b receives meteorological data which describe the future weather conditions in the route section 8 or from which the future weather conditions for the route section 8 can be derived. This data (possibly modified) is forwarded to the evaluation device 10b.

The optical information about the roadway and the meteorological data provided by the image recording device 20b are processed in the evaluation device 10b. If, for example, soiling 4 of the roadway 8 can be seen on the data provided by the image recording device 20b, the evaluation device 10b calculates whether a potential hazardous situation could develop from this soiling given the predicted future weather conditions. If a risk of a critical road condition developing is determined, information about the potentially critical road condition and the position data of the route section for which the critical road condition is precalculated is sent to first vehicle 1 via transmission device 16b.

In the first vehicle 1, which in the example shown is also a passenger car, which also has an image recording device 20a, but could also be a single-track vehicle such as a motorcycle or bicycle, which due to its design does not have an image recording device 20a, the data received from the second vehicle 2 received by a transmission device 16a and stored in a storage device. This can be part of the evaluation device 10a. The evaluation device 10a checks whether the data received from the second vehicle 2 are relevant for the route currently being driven by the first vehicle 1 . If the first vehicle 1 drives, for example, as in 1 shown the same route section 8 as the second vehicle 2, but a few seconds or minutes later, this is recognized and the warning about the potentially critical road condition is forwarded to the driver by means of an output device 6a.

Alternatively or in addition to this, it is also conceivable for the information about the potentially critical road condition to be transmitted to a driver assistance system without providing the driver with information, and for this to initiate measures independently, even without interaction with the driver, to correct the potentially critical road condition reduce risk. This could be achieved, for example, by reducing the speed.

It is noted in this respect that a road condition that was detected by the second vehicle 2 can be a potentially critical road condition for a first vehicle 1 even if the same conditions do not represent an increased risk for the second vehicle 2 . This could be caused, for example, by design-related differences between the vehicles. In this regard, particular attention is drawn to the increased risk of dirt on the road for single-track vehicles. But other differences such as the tires (summer tires, winter tires, snow chains), vehicle weight, a possibly existing trailers or the like can be included in the individual assessment of the risk arising from a detected road surface condition for a specific vehicle.

In addition, the in 1 illustrated example, the determination of a potential critical road condition by the vehicle-independent evaluation device 14. As described above, the images recorded by the image recording device 20b of the second vehicle 2 are sent to the external storage device or cloud 14. If the second vehicle 2 drives a section of the route several times or if several second vehicles send data recorded by their image recording device 20b, the vehicle-independent evaluation device 14 has a large amount of visual information for this section of the route, from which a change in the condition of the roadway can be determined. The period over which changes in the condition of the road can be documented can be significantly longer than with a vehicle-bound evaluation device. This enables longer-term analyzes of the road surface and the detection of periodically recurring risks.

In addition, a vehicle-independent evaluation device 14 offers the advantage that communication between the first vehicle 1 and the vehicle-independent evaluation device 14 is already possible before the start of the journey and thus the determination of a suitable route. The range of the direct communication between two vehicles (V2V) described above is usually limited, so that meaningful communication between these two vehicles usually makes sense when both vehicles are actually already using the same route section 8 .

However, the data provided by the vehicle-independent evaluation device 14 is preferably already available before the first vehicle 1 starts driving, so that the route planning can include the potential hazardous situations determined for a specific route.

In addition, the long-term storage of the image data in the vehicle-independent evaluation device 14 offers the advantage that long-term roadway changes can also be detected and countermeasures can thus be taken, possibly even before a critical roadway condition arises. For example, the formation of a pothole before it breaks up is indicated by a deformation of the road surface. If such a deformation is detected, countermeasures can be taken before the road surface breaks open and the pothole is formed.

In 2 a schematic sequence of the method steps of an exemplary embodiment of a method for warning a user of a first vehicle is shown.

Reference number 101 designates the method step of capturing optical information about the condition of the roadway using the image recording device of the second vehicle.

Reference number 102 designates the procedural step of providing meteorological data. This data can be provided by an external service provider such as a weather service.

The meteorological data obtained by method step 102 are transmitted to the second vehicle and used there together with the optical data on the roadway in the second vehicle obtained from method step 101 for further calculations. These method steps, which use the data processing capacity of the second vehicle, are in 2 along the left path and comprises the method steps with the reference numerals 103, 105, 107 and 110.

Both the meteorological data provided by method step 102 and the optical information about the condition of the road recorded by the image recording device of the second vehicle according to method step 101 are also transmitted to a vehicle-independent evaluation device and used there for further calculations. These method steps, which use the data processing capacity of a vehicle-independent evaluation device, are in 2 along the right path and comprises the method steps with the reference numerals 104, 106, 108, 109 and 111.

When determining a potential source of danger in the second vehicle, the data material received from the image recording device is first evaluated in method step 103 and the local road condition is determined. This can be done using an artificial neural network.

Then, in step 105, deviations from the desired state of the roadway are detected, such as can be caused by contamination, for example. It is further analyzed in step 105 to determine what type of soiling there is with high probability act. Leaves and/or soil on the roadway are thus detected, in particular using a neural network.

Once the type of soiling has been determined, in step 107 the degree of soiling can be classified. This classification is preferably carried out taking into account the meteorological data provided in step 102 . Such a classification can be used to reduce the subsequent computational effort. In addition, such a classification allows an assessment of the type of vehicle for which the determined contamination of the roadway potentially poses a risk.

In step 110, the risk classification determined in step 107 is passed on to other vehicles in the area. This transmission preferably takes place directly from vehicle to vehicle (V2V), possibly using the local infrastructure for data transmission.

In addition, the classification of the risk for a specific route section determined in step 107 in the second vehicle is also stored in a database arranged outside of the second vehicle. The local road condition is stored in such a database, also referred to as an online road book, in step 111.

Parallel to the local calculation of a hazard potential for a specific route section in the second vehicle, a potential source of danger for this route section is also determined outside the second vehicle.

For this purpose, the data provided in 101 and 102 are transmitted in step 104 to such a vehicle-independent evaluation device. The transmitted data thus includes the optical information about the roadway determined by the second vehicle in step 101 .

Data from multiple vehicles and/or multiple uses of the same route section by a single vehicle arrive at different times in the vehicle-independent evaluation device. This image data is stored for a defined period of time and is therefore available for comparison with current image data. In step 106, the currently transmitted image data is compared with the image data of this route section already stored in the database.

If changes are detected, the historical development of the roadway can be evaluated in the vehicle-independent evaluation device due to the longer retention time of older image data.

Depending on the result of the comparison of the current image data with image data of this route section stored in the database, a classification into long-term changes of this route section takes place (step 108), whereby, for example, the formation of potholes can be recognized and short-term changes (step 109), to which, for example Contamination by leaves or soil are to be attributed.

The type of soiling detected by the vehicle-independent evaluation device and the risk potential of a specific route section that can be derived from this is then stored in step 111 in the database located outside the second vehicle, which, as mentioned above, is also referred to as an online roadbook.

If a first vehicle is in the vicinity of the second vehicle, so that V2V communication is possible, the first vehicle receives the information from the second vehicle about a possible danger in the route section traveled by the second vehicle. This danger is communicated to the driver of the first vehicle in step 112 .

In addition, the first vehicle receives information about a possible hazard in a section of road that the first vehicle is likely to travel in the future, also from the online roadbook, in which in step 111 both information about a possible hazard from a second vehicle and from the vehicle-independent evaluation device are stored . This makes it possible, for example, to include potential hazards in the calculation of the most favorable route before the start of the journey.

A potential hazard is displayed in a route section according to step 112, preferably regardless of which of the paths described above was used to determine the potential hazard.

The applicant reserves the right to claim all features disclosed in the application documents as essential to the invention, provided they are new compared to the prior art, either individually or in combination. It is also pointed out that the individual figures also describe features which can be advantageous in and of themselves. The person skilled in the art recognizes immediately that a specific feature described in a figure is advantageous even without adopting further features from this figure can be liable. Furthermore, the person skilled in the art recognizes that advantages can also result from a combination of several features shown in individual figures or in different figures.

Reference List

1

(First) vehicle

2

(second) vehicle

4

pollution

6

Output device, driver information display

8th

Roadway (section), route section

10a, 10b

Evaluation device (with memory device)

14

external storage device, cloud, vehicle-independent evaluation device

16a, 16b

transmission facility

20a, 20b

image capture device

101 - 112

process step

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• US 8831869 B2 [0004]
• US9475500B2 [0005]

Claims (10)

Method for warning a user of a first vehicle (1) of a potentially dangerous situation at at least one destination of the vehicle (1), comprising: - Recording of optical information about a roadway section (8) and optionally also its surroundings in a defined road section (8) by an image recording device (20b) of a second vehicle (2), - Calculation of information about a potential development of a critical road condition by an evaluation device (10b) of the second vehicle (2) on the basis of the information recorded by the image recording device (20b) and current and/or predicted meteorological data, - Transmission of the information about a possible critical road condition and the position data of the route section (8) for which the critical road condition is precalculated to a storage device (10a, 14) arranged outside of the second vehicle, - Transmission of information about the possible critical road condition and the position data of the road section (8) for which the critical road condition is precalculated from the storage device (10a, 14) to an output device (6a) of the first vehicle (1). procedure after claim 1 , characterized in that the storage device (10a, 14) is an external storage device (14), in particular a cloud-based and/or a central storage device (14). Method according to one of the preceding claims, characterized in that the vehicle-bound evaluation device (10b) for calculating the information about a potential development of a critical road condition includes at least one piece of optical information about a road, which is selected from a group, the color of the road surface, reflectivity of the road surface, structure of the road surface, shadows cast on the road surface, brightness of the surroundings, structure of the surroundings, clouds, visibility and fog. Method according to one of the preceding claims, characterized in that the information about the possible critical road condition and the position data of the route section (8) for which the critical road condition has been precalculated is transmitted optically and/or acoustically to a driver of the vehicle by an output device (6a). first vehicle, preferably in a driver information display (6a) and/or by a warning unit of a helmet. Method according to one of the preceding claims, characterized in that the information recorded by the image recording device (20b) is transmitted to an evaluation device (10a, 14) arranged outside the second vehicle, preferably a vehicle-independent evaluation device (14). procedure after claim 5 , characterized in that a vehicle-independent evaluation device (14) determines a change in the condition of the road from a multitude of optical information available for this road section (8) and, if a potentially critical road condition is determined, makes this information available to vehicles (1, 2) which pass or intend to pass this section (8). procedure after claim 6 , characterized in that the change in the condition of the road determined by the vehicle-independent evaluation device (14) includes a short-term change such as dirt (4) or wetness and/or a long-term change such as the formation of a pothole or the lowering of a section of the road. Warning device for warning a user of a first vehicle (1) of a potentially dangerous situation at at least one destination of the first vehicle (1), comprising an output device (6a) of the first vehicle (1) by means of which information about a possible critical road condition and position data of the Section (8) for which the critical road condition can be received from a storage device (10a, 14), the storage device (10a, 14) being arranged outside a second vehicle and the storage device (10a, 14) having a receiving device for receiving information about a possible critical road condition and the position data of the route section (8) for which the critical road condition has from a second vehicle (2), characterized in that the second vehicle (2) has an image recording device (20b) for capturing optical information about a road section (8 ) and optionally additionally dere n Environment in a defined section (8) and a vehicle-bound evaluation device (10b), which is set up and provided for this purpose, information about a potential development of a critical road condition by means of information recorded by the image recording device (20b). and calculate current and/or forecast meteorological data. warning device claim 8 , characterized in that it also has a vehicle-independent evaluation device (14), which is set up and provided for this purpose, from a large number of images recorded for this route section (8) by an image recording device (20a, 20b) of one or more second vehicles and to the vehicle-independent evaluation device (14) to determine a change in the condition of the roadway and, if a potentially critical roadway condition is determined, to provide this information for at least a first vehicle (1) which is currently in this section (8) or in a defined future time interval is intended for use of this section (8). Vehicle (1, 2), in particular a motor vehicle, comprising a warning device according to one of Claims 8 or 9 .

Images