DE102021207258B3 - Method for automatically controlling at least one vehicle function of a vehicle and notification system for a vehicle - Google Patents

Abstract

Method for automatically controlling at least one vehicle function of a vehicle on the basis of at least one visual notification signal in relation to a traffic situation in a vehicle environment of the vehicle, the visual notification signal being output by an optical output device in a vehicle environment detected by at least one sensor device for detecting the vehicle environment of the vehicle is, comprising:- receiving environmental data determined by the sensor device in relation to the detected vehicle environment;- processing characteristic data for the environment data by means of an evaluation device using a machine learning model, the model comprising a set of parameters which are set to values , which were learned as a result of a training process, and thereby determining at least one notification size, which for the optical notification signal, in particular for a Bede interpretation of the visual notification signal in relation to the traffic situation, is characteristic;- providing a message evaluation signal for controlling the vehicle function as a function of the at least one notification variable.

Description

The present invention relates to a method for, in particular automatic, control of at least one vehicle function of a vehicle, in particular on the basis of at least one optical notification signal, in particular a warning signal, in relation to a traffic situation in a vehicle environment of the vehicle, the visual notification signal from an optical output device is or will be output in a vehicle environment detected by at least one sensor device for detecting the vehicle environment of the vehicle, in particular as a spatially resolved representation. The present invention further relates to a notification system for a vehicle.

With increasing digitization and the advent of projection systems and displays on vehicles, the density of information in traffic is increasing. These systems present road users with a large number of symbols and/or warning and information graphics that people can understand “quickly” and “unambiguously”.

These representations are not standardized at the present time and are highly dependent on the respective manufacturer's design and the application market. In human perception, slight differences, deviating from what has been learned, can be abstracted in the appropriate context (e.g. known language, but unknown dialect). Slight differences in the graphics/patterns/symbols can lead to incorrect results for machine recording and, in particular, an evaluation.

The display of traffic signs (e.g. danger and regulation signs) by means of projection or via stationary or mobile vehicle displays and infrastructure displays is not permitted according to the current status of the law - this is reserved solely for state bodies.

The design of graphics, which are displayed on the vehicle with the help of display systems and are intended to provide information about traffic situations or point out dangers, is in the hands of the vehicle manufacturer. Each designed graphic must be approved separately by the testing authorities, which documents a clear graphic differentiation from the official traffic signs. Since there is not (yet) a uniform design guideline for this type of graphics and every vehicle manufacturer uses its own design language, there are differences in the graphics. For example, the same meaning is translated into a different graphic expression.

From the U.S. 2020/0329103 A1 a method for communication between two vehicles is known, in which light-based electronic messages, such as static or animated text or a set of predefined symbols, can be displayed on an external vehicle surface. The symbols cannot be changed or designed.

From the U.S. 8,064,643 B2 a method for recognizing and identifying a traffic sign, which is arranged in the vehicle environment of a vehicle, in a system with a camera, which is arranged on the moving vehicle, is known. An image recorded by the camera is divided into a first image section for detecting the traffic sign and a second image section for other assistance functions. A camera parameter for detecting the traffic sign is set as a function of a recognized potential traffic sign. The camera parameters usually have different values for traffic sign recognition and for other driver assistance functions.

In the U.S. 9,053,637 B2 a broadcast notification system for providing alert notification to vehicle operators is described. The system selects an alarm notification based on a predefined set of vehicle inputs. Alarm notifications relate, for example, to vehicle accidents, driving conditions or the presence of stopped vehicles.

In the U.S. 8,223,033 B1 describes a vehicle communication system with a display mounted in or on a vehicle, which can be viewed by persons in other vehicles or pedestrians in the vicinity of the vehicle. The display can be used, for example, to show a "STOPPING" message when the vehicle brakes. The messages can be pre-programmed or entered by a user.

In the U.S. 9,919,648 B1 describes a warning and control system for a vehicle in which the activities of the driver and/or the passengers are monitored using vehicle sensors. This includes activities related to the use of cell phones and/or wireless communication devices. The sensor network monitors hazardous driving situations, weather conditions, road conditions, and the like to provide an alert and/or control signal for alerting the driver or controlling the use of the cellular phone or other wireless communication device.

From the DE 10 2019 005 543 A1 a vehicle with a projection unit is known, with provision being made for projecting a warning symbol behind the vehicle and in front of the following vehicle onto a surface of the lane.

At the one from the U.S. 10,134,280 B1 known system is provided that a critical driving situation is detected by a first vehicle (source vehicle) and then a warning signal is projected onto the roadway. This warning signal can be detected by a second vehicle (target vehicle) and the critical driving situation can be reacted to.

the DE 10 2019 209 560 A1 and the DE 10 2017 105 764 A1 show methods for recognizing traffic signs by a vehicle with a neural network, the traffic signs being objects in the vehicle environment that are detected by a vehicle sensor system, such as a camera.

The present invention is based on the object of overcoming the disadvantages known from the prior art and providing a method for automatically controlling at least one vehicle function and a reliable notification system which reliably detects and evaluates notification signals, in particular warning signals, transmitted in a vehicle environment .

The object is achieved according to the invention by the subject matter of the independent claims. Advantageous embodiments and developments of the invention are the subject matter of the dependent claims.

In a (preferably computer-implemented) method according to the invention for the, in particular automatic, control of at least one vehicle function of a vehicle and/or for the transmission and/or for the detection of at least one notification based on at least one optical notification signal with regard to a traffic situation in a vehicle environment of the vehicle, the optical notification signal is or will be output by an optical output device in a vehicle environment detected by at least one sensor device for detecting the vehicle environment of the vehicle, in particular as a spatially resolved representation.

The proposed method is preferably an AI-based symbol recognition and/or recognition of warning and/or information graphics, in particular in projections and on displays.

The traffic situation is preferably a current traffic situation. However, it is also possible that a predicted (future) traffic situation is involved.

The optical notification signal is preferably a warning signal, in particular with regard to road safety. The optical notification signal is preferably a notification signal relating to a potentially dangerous situation, for example for the vehicle and/or an occupant or user of the vehicle and/or another road user and/or another vehicle. However, it is also conceivable that this is information relating to a destination and/or a traffic situation, such as information relating to leaving a parking space and/or information relating to a search for a parking space.

According to the invention, environmental data determined, in particular locally resolved, environmental data (in particular within the framework of a computer implemented process step). In this context, “receive” can be understood to mean a data transmission, for example via a digital data exchange and/or via a (at least partially and preferably in its entirety) wired and/or wireless network. Furthermore, “receiving” can in particular also be understood as forwarding and/or retrieving environment data determined and/or recorded by the sensor device, in particular within the vehicle.

Furthermore, the method includes (in particular in a computer-implemented method step) processing of data characteristic of the environmental data by means of an (in particular processor-based) evaluation device, preferably using a machine learning model, in particular a trainable one. The model preferably includes a set of parameters, in particular trainable ones, which are set to values that were learned as a result of a training process. Furthermore, at least one notification variable is determined (by this means or by the processing), which is characteristic of the visual notification signal, in particular of a meaning of the visual notification signal in relation to the traffic situation. The data characteristic of the surroundings data is preferably the surroundings data and in particular data determined and/or generated by the at least one sensor device (Raw) sensor data and/or data derived therefrom.

At least one message evaluation signal for controlling the vehicle function (and/or for issuing warnings) is preferably provided as a function of the at least one notification variable (in particular in a computer-implemented method step). It is conceivable that the message evaluation signal is a signal to be transmitted to a road user, for example an occupant of the vehicle and/or another road user and/or another vehicle other than the vehicle.

At least one vehicle function is preferably controlled, in particular automatically, and/or executed, in particular automatically, as a function of the message evaluation signal.

The vehicle preferably has the at least one sensor device for detecting the surroundings of the vehicle. In other words, the at least one sensor device for detecting the vehicle surroundings is preferably a part or (integral) component of the vehicle and/or (in particular not detachable non-destructively) in and/or on the vehicle. However, it is also conceivable that the at least one sensor device is an external sensor device in relation to the vehicle, which is not arranged on the vehicle and transmits environmental data recorded and/or determined in particular in relation to the vehicle's surroundings to the vehicle and in particular to the notification system and/or to an evaluation device (preferably of the vehicle and/or the notification system).

In other words, an optical, in particular spatially resolved, notification signal is preferably output in an (in particular external) vehicle environment of the vehicle, preferably by the optical output device. The optical output preferably takes place in such a way that the optical notification signal can be perceived and/or detected by the sensor device for detecting the vehicle environment of the vehicle.

Preferably, the sensor device (at least temporarily) detects at least one area of the vehicle's surroundings and in particular an area of the vehicle's surroundings in which the visual notification signal is output and/or represented and/or displayed. The sensor device preferably ascertains and/or generates surroundings data from or in relation to the detected vehicle surroundings (in which the visual notification signal is output and/or represented and/or displayed). The surroundings data preferably include data relating to the (issued), in particular spatially resolved, optical notification signal. This means in particular that the environmental data are (at least partially) characteristic of the optical notification signal.

The optical notification signal is or is preferably output in such a way that it can be (visually) perceived (at least temporarily and at least partially) by a user, in particular a driver, of the vehicle. For example, the optical notification signal can be output and/or displayed on a roadway and/or on a display area of another vehicle area located in the vehicle environment.

In particular, it is proposed that the notification signal is transmitted to the vehicle and in particular to the evaluation device via an optical communication path. The proposed evaluation via a trainable machine learning model enables a reliable evaluation of different optical notification signals of equal importance. For example, different optically output notification signals, such as symbols, which represent the same notification, such as the same warning, can have a different design depending on the manufacturer or brand. By using a trained or trainable machine learning model, it is possible to recognize a large variety of different designs and/or symbols which are, however, of the same meaning in relation to the traffic situation.

In a preferred method, the optical output device is an optical output device, in particular a projection module and/or a display device, of another vehicle (hereinafter also referred to as “surrounding vehicle”) from the vehicle environment of the vehicle, which is preferably used to output the notification signal in the vehicle environment, in particular in an area that can be perceived from the outside and/or on a vehicle exterior surface (e.g. a display or a display unit of the other vehicle) and/or a floor surface. This advantageously enables external communication between two vehicles via the output (and detection) of an optical notification signal. This makes it possible for notifications to be able to be transmitted via a non-digital communication path, so that preferably no communication protocol, such as with network-based communication paths, is required for transmission.

The sensor device preferably records a locally resolved and preferably graphical (and/or two-dimensional) representation (caused by the optical output device) of the optical notification signal in the detected vehicle environment.

The sensor device preferably detects a projected representation of the optical notification signal, which was or is projected onto a projection surface and preferably onto a floor area and/or onto a roadway area. The evaluation device (in particular with the aid of the machine learning model) for recognizing symbols in the projections and/or displayed (image) elements can preferably be recognized.

A graphic (image) element and in particular a symbol or, in the graphic configuration, a symbol or a symbol is preferably represented by the optical output device. This offers the advantage that these can be detected intuitively and quickly and reliably both by road users, for example by occupants of the vehicle, and by machines.

A (geometric) size and/or an alignment and/or a distortion and/or a position and/or a shape and/or a brightness and/or a color of the output (projected and/or displayed) optical notification signal ( in particular the representation in the projection surface or on the display device of the optical output device (in particular the surrounding vehicle) adjustable and/or (in particular automatically) changeable and/or (in particular automatically) specifiable. Preference is given to the setting and/or (automatic) change and /or (automatic) specification depending on a (preferably relative) position of the vehicle (in particular to the surrounding vehicle and/or to the optical output device) and/or lane occupancy and/or type of vehicle and/or distance of the vehicle to the surrounding vehicle and in particular made to the optical output device.

Preference is given to the optical notification signal and particularly preferably several (preferably all) optical notification signals in the form of in particular statutory, prescribed and/or standardized signs, preferably traffic signs, and/or (abstract and/or high-contrast) symbols by the output (in particular via a Projection and / or display) shown.

The at least one optical notification signal and preferably a plurality of optical notification signals (preferably each) can be output (in particular projected and/or displayed) in the form of geometric graphics, such as traffic signs, and/or schematic drawings.

In principle, all types of symbols and/or graphics that can also be understood by a person in the respective context can be used. The optical notification signal is preferably designed in such a way or is output in a form that it is output (by projection and/or display) as a symbol and/or graphic and/or symbol, which is used in particular by a large number and preferably by a majority of people (In particular a given area) is understood. This offers the advantage that a user of the vehicle who directly (visually) perceives the issued notification signal can thereby immediately understand the meaning of the issued notification signal and, if necessary, could react immediately accordingly.

A form of representation (in which the optical notification signal is preferably output, in particular projected onto a projection surface and/or displayed by means of a display device) is preferably selected from a group that includes symbols, traffic signs, landmarks, (general, special) danger signs, regulatory signs, signs or Symbols for waiting bids, stopping bids, directions of travel, driving past, hard shoulders as lanes, bus stops, taxi ranks, special routes, traffic bans, speed limits, overtaking bans, stopping and parking bans and the like, as well as combinations thereof. Such a form of representation offers the advantage that these warnings can be directly detected not only by the sensor device (in particular the vehicle) but also by a road user such as a user of the vehicle (without requiring an evaluation by an evaluation device).

The form of representation can preferably be selected from a large number of forms of representation that are particularly preferably stored in a memory device of the surrounding vehicle.

The visual notification signal (in particular projected) (in particular as a graphic and/or symbol) can preferably be characteristic of information about a traffic situation ahead. In particular, the optical notification signal can be characteristic of an object and/or a surrounding area of interest (in relation to a traffic situation), which or which of the particular as high / wide vehicle (e.g. SUV, truck, bus and the like) configured surrounding vehicle is covered.

As a further preferred example, the optical notification signal can be characteristic of the presence of other road users and/or of a reason for (in particular current) driving behavior of the surrounding vehicle. For example, the large vehicle (representing the surrounding vehicle in this example) stops in front of a zebra crossing to allow pedestrians to cross and projects the information backwards onto the road. Vehicles following behind recognize the graphic (which, for example, represents a person crossing a street) and are thus informed that the (surrounding) vehicle has stopped.

The (in particular projected) visual notification signal (in particular as a graphic and/or symbol) can preferably be characteristic of a (preferably currently carried out and/or intended) driving maneuver (of the surrounding vehicle and/or another vehicle participating in a traffic situation in the vehicle surroundings). . For example, the optical notification signal can be output (in particular projected and/or displayed) in the form of at least one (in particular schematic) arrow (or arrows) and/or line (or lines) and the like. This offers the advantage that driving maneuvers (of the surrounding vehicle) can be communicated in the form of arrows, lines and the like.

As a further preferred example, the optical notification signal can be characteristic of a message to be exchanged between the vehicle and the surrounding vehicle. For example, the optical notification signal in autonomous vehicles can serve as a substitute for direct communication between two drivers. For example, the optical notification signal (which is output by the output device of the surrounding vehicle) can be characteristic of a notification from which it can be derived that a (or the) particularly (at least partially and preferably completely) autonomously driving (surrounding) vehicle signals that it has detected or seen another vehicle and/or has recognized a planned driving maneuver and/or has given way, for example.

As a further preferred example, the optical notification signal can be characteristic of at least one planned driving maneuver (in particular of the surrounding vehicle). This offers the advantage that communication of planned driving maneuvers can take place. For example, when looking for a parking space, the searching (surrounding) vehicle communicates this information to the traffic area or to the vehicle environment of the vehicle in order to provide preferential information as to why it is perhaps moving a little more slowly. For example, when leaving a parking space, an optical notification signal can be sent, from which the information can be derived that this (parking) space will soon be free.

The optical notification signal (in particular projected) that is output (in particular as a graphic and/or symbol) can preferably be characteristic of one or more support request(s) (in particular from the surrounding vehicle) to other road users. For example, a symbol could be output or displayed from which an indication can be derived (in particular from a user of the vehicle and/or an evaluation device of the vehicle) that a vehicle needs support, for example because of a breakdown and/or the emergency call should be dialled (or an emergency call signal is to be sent, in particular by the vehicle) and/or whether everything is in order and, in particular, no support is required.

It is possible for the displayed and/or projected visual notification signal to be a static or dynamic object or image element whose shape changes, in particular as a function of time.

In a further preferred method, the training process is based on a training data set, which, in particular by law (or by regulation), prescribed and/or standardized signs, preferably traffic signs, and/or symbols and/or symbols, and their meaning in relation to a Traffic situation characteristic size includes. The set of trainable parameters is preferably determined on the basis of this training data set and/or values are defined for it. Learning of known symbols (by the model) preferably takes place (thereby). This training process is preferably carried out before the trained model is used to process the environmental data or data characteristic of this and is particularly preferably completed.

A basic (training) data record is preferably provided which includes known and/or generally valid symbols and/or in particular legally (or by regulation), prescribed and/or standardized signs, preferably traffic signs and/or symbols and/or symbols and (respectively) their meaning or one for their meaning in relation to a traffic si tuation characteristic size includes. The training process and/or the determination of the parameters (or the determination of values for the parameters of the model) is preferably based on the basic (training) data record. A basic data set consisting of known/generally valid symbols and their meanings can therefore advantageously be learned.

The (basic) data record is preferably enriched by, for example, symbols designed specifically for a brand, and the (basic) data set particularly preferably includes symbols designed specifically for a brand. This offers the advantage that manufacturer-specific deviations or different representations and characteristics of the symbols for notification signals with the same meaning and/or similar meanings (which is understood in particular as symbols designed specifically for a brand) can be learned and taken into account in later use. For example, two brand-specific symbols can differ in their font, line thickness, color selection.

The (basic) data set is preferably enriched with design-specific graphic elements or symbols and the (basic) data set particularly preferably comprises design-specific symbols and/or graphic elements. Design-specific symbols or graphic elements are understood to mean, in particular, two different symbols or graphic configurations, by means of which the same meaning is to be conveyed. A data set can be learned from design-specific graphic elements. For example, the danger signs used in Ireland are diamond-shaped ("diamond") with a yellow background and thus differ significantly from the danger signs of the other European countries, in which the danger signs chosen as traffic signs are a red-bordered triangle on a white background with the tip pointing upwards as the basic shape is chosen.

The (basic) data record and/or the training data record can be (purely) photorealistic training data and/or (purely) synthetic training data.

The machine learning model is preferably suitable for executing a (computer-implemented) computer vision method and determines in which (computer-implemented) perception and/or detection tasks are carried out, for example (computer-implemented) 2D and/or 3D Object recognition methods and/or (computer-implemented) methods for semantic segmentation and/or (computer-implemented) object classification (“image classification”) and/or (computer-implemented) object localization.

During the object classification, an object detected and/or represented in the environmental data (or in the data characteristic of the environmental data) is assigned to a (previously learned and/or predetermined) class. In the case of an object localization, in particular in addition to an object classification, a location of an object recorded and/or represented in the environmental data (in particular in relation to the environmental data or data characteristic of this) is determined or ascertained, which is marked and/or highlighted in particular by a so-called bounding box becomes. In the case of semantic segmentation, in particular each pixel of the environmental data (or data characteristic of this) is assigned a class (for classifying an object) (in particular from a particularly predetermined multiplicity of classes) (class annotation).

The (detected and/or displayed) object is preferably an object which is characteristic of the visual notification signal (or its display and/or image in the environment data). The (detected and/or displayed) object preferably corresponds to the environmental data assigned to the optical notification signal.

The classes can (among other things) be a meaning (in particular with regard to a traffic situation) of a detected object and/or a notification variable that is characteristic of a meaning of the detected object or a meaning of the (detected and in particular displayed) optical trade notification signal.

For example, the classes can be the respective meaning of different types of hazard signs and/or warning signs, or an identification or a classification of the detected object as a specific predefined hazard sign.

A detected object is preferably classified on the basis of a geometric shape of the object (for example circular shape, triangular shape, octagon, rectangular shape).

When processing the environmental data, the machine learning model preferably separates design, symbol (or graphic sign or element and/or symbol) and meaning (or notification size) (of the detected object). A notification size is preferably determined as a function of the detected meaning, which is characteristic of the meaning of the optical notification signal and in particular the captured spatially resolved representation of the optical notification signal.

The machine learning model is preferably based on an (artificial) neural network (AI—Artificial Intelligence). The environment data (or data derived therefrom) are preferably fed to the artificial neural network as input variables. The artificial neural network preferably maps the input variables to output variables as a function of a processing chain that can be parameterized or parameterized (by the trainable or trained parameters), with at least one notification variable that is particularly characteristic of the (detected) optical notification signal being selected as the output variable .

Such a neural network can be designed, for example, as a deep neural network (DNN), in which the parameterizable processing chain has a plurality of processing layers, and/or a so-called convolutional neural network (CNN) and/or a recurrent Neural network (RNN, English Recurrent Neural Network) be designed. The parameterizable processing chain is preferably parameterized by the training. Data sets relating to the above-described (basic) data sets and/or training data sets are preferably used as training data. The training preferably takes place by means of supervised learning. However, it would also be possible to train the artificial neural network using unsupervised learning, reinforcement learning or stochastic learning.

In a further preferred method, a checking device plausibilizes and/or verifies the determined notification variable on the basis of a further notification signal that is different from the detected optical notification signal and is received by at least one communication device of the vehicle.

A plausibility check and/or a verification of the recognized symbols is preferably carried out. This offers the advantage of a redundant second transmission possibility, which advantageously causes an increase in reliability.

The further notification signal is preferably a non-optical notification signal, i.e. in particular a signal that is not (at least partially) transmitted via an optical communication path. This advantageously achieves the possibility of checking the detection and/or evaluation of the optical notification signal and increases the overall process reliability.

In a further preferred method, (at least) one further training step of the machine learning model is carried out on the basis of the received environment data and the plausibility check and/or verification of the determined notification variable. In particular, new symbols and their meaning are learned and recognized by comparison with symbols and/or shared information data received in particular (substantially) simultaneously via another communication path, for example via Car2Car and/or Car 2 Infrastructure (C2X). This offers the advantage that the machine learning model already deployed is adaptable and capable of improvement. In particular, the detection and evaluation accuracy can be further improved and new symbols and/or symbols and/or characters and/or graphic elements (and their meaning) can be included and integrated.

For the at least one further training step, at least one (in particular further) training data set based on the environmental data (detected and/or determined by the sensor device) and/or based on the further notification signal (and in particular depending on the result of the plausibility check and/or or verification). This training data record (thereby) preferably contains photorealistic training data.

The (already) trained machine learning model is preferably subjected to a further training step, in particular using sensor data recorded (during its use) and/or further notification signals, and in particular the values set for the parameters of the model after a completed training process (which as a result of the training process were learned) changed.

It is also conceivable that, instead of a further training step, a machine learning model which, in particular, contains the basic training data set and which the training dataset is retrained.

A plausibility check of the recognized symbols and graphics is preferably carried out by verification with other information channels (for example Car2X, DAB+, mobile online services). This advantageously increases the data quality (such as a position-accurate capability) and/or reliability achieved. Independence from hardware interfaces and communication standards is also advantageously achieved.

In a further preferred method, the further notification signal is sent via a digital communication connection and/or via a wireless (in particular private and/or public) network, preferably via car2car communication, C2C communication and/or car2x communication and/or car2i communication, receive. The network can be a local network, such as a WLAN network.

The detected symbols can preferably be passed on to other road users and/or vehicles (by projection and/or digitally via C2X).

In a further preferred method, an output signal for output (and/or transmission) to at least one other road user and/or another vehicle is provided by an, in particular optical, output device of the vehicle on the basis of the determined notification variable.

The information (for example end of traffic jam, hazardous area) is preferably transmitted optically and/or digitally to other road users, for example via a projection/display module on the vehicle or via Car2X.

The vehicle can be at least indirectly and preferably directly in communication with at least one other vehicle (e.g. via Car2Car communication, C2C communication and/or Car2x communication and/or Car2I communication). The notification system and/or the evaluation device can transmit and/or transfer the determined environment data and/or the detected objects and/or the determined notification variables at least indirectly (via other road users or infrastructure) and preferably directly to at least one other vehicle.

The at least one determined notification variable and/or the message evaluation signal can either only be stored locally in the vehicle and can also optionally be uploaded to a cloud.

The recognized information/warning message (and/or the determined notification size and/or the determined message evaluation signal) is preferably displayed in at least one display and preferably in displays in the interior (of the vehicle), particularly preferably via light-emitting diodes (LEDs) and/or by Audio signal/message.

In a further preferred method, based on the determined notification variable, an output signal is provided for output to at least one occupant of the vehicle by at least one, in particular optical and/or acoustic, output device of the vehicle. The recognized symbols can thus be displayed advantageously for occupants of the vehicle.

The message evaluation signal to be output in particular to an occupant of the vehicle is preferably selected from a group that includes silent notification signals, haptic notification signals, optical notification signals, acoustic notification signals and combinations thereof. It is possible for the message evaluation signal to be output via a contact element of the vehicle, in particular a seat and/or a belt tensioner, via a mobile terminal device and/or via an optical display device, such as a display.

The vehicle and in particular the evaluation device (of the vehicle and/or the external server) preferably determines whether the notification variable relates to the lane selected and/or to be used by the vehicle and/or is relevant to the traffic safety of the vehicle.

The sensor device for detecting a vehicle environment (of a vehicle) is preferably selected from a group of sensors which include a (color) camera, a front camera, a rear camera, an infrared camera, LIDAR (abbreviation for light detection and ranging or light imaging, detection and ranging), radar, ultrasonic sensors and the like, and combinations thereof. The sensor device preferably generates locally resolved (in particular 2D and/or 3D) sensor data (from a vehicle environment of the respective vehicle) for detecting a vehicle environment.

In a preferred method, the evaluation device is an external evaluation device, in particular a cloud-based and/or an external server. An external server is to be understood in particular as a server that is external to the vehicle, in particular a backend server. The external server is, for example, a backend of a vehicle manufacturer or a service provider, which is set up to store environmental data and/or data derived therefrom (in particular in relation to traffic situations with regard to a large number of vehicles) to evaluate and/or to manage and/or store (determined) notification size(s). The functions of the backend or the external server can be performed on (external) server farms. The (external) server can be a distributed system. The external server and/or the backend can be cloud-based.

The method also includes determining a control variable based on the environment data and/or based on the message evaluation signal. The control variable is preferably a variable for controlling the acceleration and/or braking behavior of the vehicle.

The vehicle function is preferably a function of a vehicle component of the vehicle. The vehicle component is preferably selected from a group which includes a vehicle safety system (e.g. a braking system), a system for guiding the vehicle, in particular automatic vehicle lateral control, a lane keeping system, a lane change assistance system, a navigation system, a locking system for a vehicle door and/or a vehicle window, a windscreen wiper mechanism, a vehicle closing system, a roof opening mechanism, a sunroof mechanism, an infotainment system, an entertainment system and/or a comfort system for increasing the driving comfort of an occupant and combinations thereof.

The present invention is also directed to a notification system for a vehicle for automatically controlling at least one vehicle function of a vehicle based on at least one optical notification signal, in particular a warning signal, in relation to a traffic situation in a vehicle environment of the vehicle, in particular in relation to a potentially dangerous situation.

The optical notification signal is output and/or can be output by an optical output device in a vehicle environment detected by at least one sensor device for detecting the vehicle environment of the vehicle, in particular as a spatially resolved representation. In this case, the notification system is suitable and intended to receive environmental data, in particular spatially resolved, determined by the sensor device in relation to the detected vehicle environment.

According to the invention, the notification system is suitable and intended for processing data characteristic of the surroundings data by means of an evaluation device, preferably using a machine learning model, in particular a machine learning model that can be trained.

The model preferably includes a set of parameters, in particular trainable ones, which are set to values that were learned as a result of a training process, and in this way at least one notification variable can be determined and/or is determined, which for the optical notification signal, in particular for a Significance of the optical notification signal in relation to the traffic situation is characteristic. The notification system is preferably suitable and intended to provide (preferably a control device of the vehicle) a message evaluation signal for controlling the vehicle function as a function of the at least one notification variable.

It is therefore also proposed within the scope of the notification system according to the invention that visual notification signals (e.g. a warning triangle projected onto the roadway) emitted (e.g. by a vehicle in the vicinity) are recorded and the notification to be transmitted (e.g. a warning) is recognized (by the evaluation device).

The sensor device is preferably part of the notification system and/or (integral) part of the vehicle.

An in particular high-resolution projection module (HDP=high definition projector) is preferably provided as the optical output device. Each light source of this high-resolution projection system can be designed as a laser light source and/or an LED light source, for example.

An optical display device and in particular a display is preferably provided as the optical output device, which is preferably arranged on an outer surface of the vehicle. This is preferably a rear outer surface of the vehicle (with respect to a forward direction of the vehicle). However, it is also conceivable that this is a lateral outer surface and/or a front surface of the vehicle. Furthermore, the rear and/or side windows (of the vehicle) can also serve as projection surfaces. The projectors required for this are then preferably located in the interior of the vehicle and are suitable and intended for projecting onto the pane.

The notification system is preferably set up, suitable and/or intended for the method described above and all of those already mentioned above carry out the method steps described in connection with the method individually or in combination with one another. Conversely, the method can be equipped with all of the features described in the context of the notification system, either individually or in combination with one another.

The present invention is also aimed at a vehicle, in particular a motor vehicle, comprising a notification system for a vehicle as described above, according to an embodiment. The vehicle can in particular be a (motorized) road vehicle.

A vehicle can be a motor vehicle which, in particular, is a motor vehicle controlled by the driver himself (“driver only”), a semi-autonomous, autonomous (e.g. autonomy level 3 or 4 or 5 (of the SAE J3016 standard)) or self-driving motor vehicle is. Autonomy Level 5 designates fully automated vehicles. The vehicle is preferably a vehicle from the transport sector. The vehicle can also be a driverless transport system. The vehicle can be controlled by a driver or drive autonomously. In addition to a road vehicle, the vehicle can also be an air taxi, an airplane and another means of transportation or another type of vehicle, for example an air vehicle, water vehicle or rail vehicle.

The present invention is also aimed at a computer program or computer program product, comprising program means, in particular a program code, which represents or encodes at least some and preferably all method steps of the method according to the invention and preferably one of the preferred embodiments described and is designed to be executed by a processor device.

The present invention is also aimed at a data memory on which at least one embodiment of the computer program according to the invention or a preferred embodiment of the computer program is stored.

• Darin zeigen:
  • 1 eine Darstellung einer Verkehrssituation zum Einsatz eines erfindungsgemäßen Verfahrens gemäß einer Ausführungsform; und
  • 2 eine schematische Darstellung eines erfindungsgemäßen Systems gemäß einer Ausführungsform.

Further advantages and embodiments result from the attached drawings:

• Show in it:
  • 1 a representation of a traffic situation for the use of a method according to the invention according to an embodiment; and
  • 2 a schematic representation of a system according to the invention according to one embodiment.

1 shows a representation of a traffic situation in which a proposed method and a proposed notification system 1 are used. A (surrounding vehicle 40 in a dangerous situation, such as on a hard shoulder L3 (as in 1 illustrated) stalled vehicle projected, in particular by means of an optical output device and here a projection device 48, a warning, here a warning triangle 52 and a warning symbol 50, which warns of a danger point, onto the lane L2, on which a (subsequent) (consecutive )Vehicle 10 is located, as well as on the hard shoulder L3. In addition, a hazard warning system 42 of the vehicle 40 can also be activated.

The reference number 46 denotes an illustration of the projection of the optical notification signal 50 by an optical output device of the surrounding vehicle 48 onto the roadway L2, on which it is displayed as a warning 50.

The (following) vehicle 10 can use the sensor device 12 to detect the projected warning notice 50 and the projected warning triangle, which are located in the field of view B1 and the field of view B2 of a sensor device 12 embodied here as a camera. Sensor device 12 embodied here as a camera for detecting a vehicle environment of vehicle 10 can be suitable and intended for recording and/or detecting the vehicle environment in front, which in particular includes or contains notification signals 50 and/or 52 .

With the help of a Kl-based evaluation device 30, the following vehicle recognizes the symbol, the warning triangle and the hazard warning lights. A projection with an alternative display of the information or warning is also recognized. In particular, a recognition and plausibility check of symbols/warnings is possible using Kl.

Reference numerals 14 and 44 identify communication devices of the two vehicles, with the aid of which Car2Car communication for verifying the data is possible.

2 shows a schematic representation of a system 1 according to the invention according to one embodiment. According to one embodiment, an AI is trained on the basis of a (vehicle data record 2, which contains the traffic signs and their respective meaning.

Optionally, the (vehicle) data record 2 can be enriched by, for example, manufacturer-specific graphics and their meanings.

The system 1 has in particular an optical sensor system 12 (or an (optical) sensor device for recording spatially resolved sensor data), for example a camera, for recording the environment and projections 52, 50 and displays contained therein.

The graphics (projected or displayed in the traffic space) can be detected in the images or sensor data of the optical (environment) sensors 12 of the vehicle.

Furthermore, the system 1 preferably has an AI or an evaluation device 18 based on artificial intelligence for recognizing symbols and graphics, in particular warning and/or information graphics. The AI-based evaluation device can be on the vehicle ECU 16 (ECU Electronic Control Unit) or a vehicle control device or in a cloud backend 20 .

By means of AI extraction of the information and/or notes and/or warnings (meanings) contained and communicated in the graphic, the graphic can be segmented and classified.

Furthermore, the system 1 preferably has communication interfaces to other road users and to the traffic environment (e.g. via a Car2X receiver 19, a radio receiver 15, in particular DAB+) for the verification of evaluated information and/or for learning new symbols and graphics (optional).

In particular, a checking device or a plausibility check module 17 can be used to check the recognized symbols and graphics for plausibility by verifying them with other information channels, e.g. Car2X, DAB+. This results in the advantage of an increase in data quality, such as positional accuracy and reliability. Independence from hardware interfaces and communication standards also advantageously results.

The learning of new symbols and their meaning in the traffic area is preferably carried out by comparison with simultaneously shared and plausible information data from e.g. Car2X.

The learned graphics and their meanings can be shared via Car2X and/or Cloud.

The recognized symbols and/or graphics 50, 52 can be output to other road users 6 by means of an environment output device 22, such as a projection module (or a projection device), an HD headlight and/or display and/or a rear light will.

Furthermore, an output device for optical output, for example via a display 24 in the instrument cluster, and/or for audio output 26 can be output to the user of the vehicle such as a driver.

The applicant reserves the right to claim all features disclosed in the application documents as essential to the invention, provided they are new compared to the prior art, either individually or in combination. It is also pointed out that the individual figures also describe features which can be advantageous in and of themselves. The person skilled in the art recognizes immediately that a specific feature described in a figure can also be advantageous without adopting further features from this figure. Furthermore, the person skilled in the art recognizes that advantages can also result from a combination of several features shown in individual figures or in different figures.

Reference List

1

notification system

2

vehicle record

4

user

6

road users

10

vehicle

12

sensor device

14

communication facility

15

radio receiver

16

control device

17

plausibility check device

18

Evaluation device, symbol graphic recognition

19

Communication device, Car2X receiver

20

cloud

22

environment output device

24

output device

26

Acoustic output device

30

evaluation device

40

Surrounding vehicle in dangerous situation

42

hazard lights

44

communication facility

46

Illustration projection

48

Optical output device, projection module

50, 52

Optical notification signal

B1, B2

Illustration of the area around the vehicle, the field of view of the camera

L1, L2

roadway

L3

side stripes

Claims (10)

Method for automatically controlling at least one vehicle function of a vehicle (10) on the basis of at least one optical notification signal (50, 52), in particular a warning signal, in relation to a traffic situation in a vehicle environment (L1 - L3) of the vehicle (10), in particular in relation to a potentially dangerous situation, the optical notification signal (50, 52) being transmitted by an optical output device (48) in a vehicle environment (B1, B2) detected by at least one sensor device (12) for detecting the vehicle environment (L1 - L3) of the vehicle (10). ), in particular as a spatially resolved representation, comprising: - receiving by the sensor device (12) in relation to the detected vehicle environment (B1, B2) determined, in particular spatially resolved, environment data; - Processing of data characteristic of the environment data by means of an evaluation device (18) using a machine learning model, in particular a trainable one, the model comprising a set of parameters, in particular trainable ones, which are set to values that were learned as a result of a training process and thereby determining at least one notification variable which is characteristic of the optical notification signal, in particular of a meaning of the optical notification signal in relation to the traffic situation; - Providing a message evaluation signal to control the vehicle function depending on the at least one notification size. procedure after claim 1 , characterized in that the optical output device (48) is an optical output device (48), in particular a projection module and/or a display device, of a further vehicle (40) from the vehicle environment (L1 - L3) of the vehicle (10) is, which is suitable and intended for the output of the notification signal in the vehicle environment (L1 - L3). Method according to one of the preceding claims, characterized in that the training process is based on a training data set which, in particular by law, prescribed and/or standardized signs, preferably traffic signs, and/or symbols and a variable that is characteristic of their significance in relation to a traffic situation . Method according to one of the preceding claims, characterized in that a checking device (17) checks the plausibility of the determined notification variable on the basis of a further notification signal which is different from the detected optical notification signal (50, 52) and received by at least one communication device (14) of the vehicle (10). and/or verified. Method according to one of the preceding claims, characterized in that a further training step of the machine learning model is carried out on the basis of the received environment data and the plausibility check and/or verification of the notification variable determined. Method according to the preceding claim, characterized in that the further notification signal is received via a digital communication link and/or via a wireless network, preferably via Car2Car communication, C2C communication and/or Car2x communication and/or Car2I communication . Method according to one of the preceding claims, characterized in that based on the determined notification variable, an output signal for output to at least one other road user and/or another vehicle is provided by an, in particular optical, output device of the vehicle (10). Method according to one of the preceding claims, characterized in that an output signal for output to at least one occupant of the vehicle (10) is provided by an, in particular optical, output device of the vehicle (10) on the basis of the notification variable determined. Notification system (1) for a vehicle (10) for the automatic control of at least one vehicle function of a vehicle (10). Position of at least one optical notification signal (50, 52), in particular a warning signal, in relation to a traffic situation in a vehicle area (L1 - L3) of the vehicle (10), in particular in relation to a potentially dangerous situation, the optical notification signal (50, 52 ) by an optical output device (48) in a vehicle environment (B1, B2) detected by at least one sensor device (12) for detecting the vehicle environment (L1 - L3) of the vehicle (10), in particular as a spatially resolved representation, and/or can be output is, wherein the notification system (1) is suitable and intended to receive, in relation to the detected vehicle environment (B1, B2) determined by the sensor device (12), in particular spatially resolved, environment data, characterized in that the notification system (1) is suitable and intended for the environment data characteristic data by means of an evaluation device (18) using a s, in particular trainable, model of machine learning, the model comprising a set of, in particular trainable, parameters which are set to values that were learned as a result of a training process, and in this way at least one notification variable can be determined and/or is determined , which is characteristic of the optical notification signal, in particular for a meaning of the optical notification signal in relation to the traffic situation, the notification system being suitable and intended to provide a message evaluation signal for controlling the vehicle function depending on the at least one notification variable. Vehicle (10), in particular a motor vehicle, comprising a notification system (1) according to the preceding claim.

Images