DE102017213694B4 - Method, device, mobile user device, computer program for machine learning for a sensor system of a vehicle - Google Patents

Abstract

Method for machine learning for a sensor system of a vehicle, comprising:- determining a trajectory pattern of a first vehicle when driving along a route section,- determining a parameter representing a sensing artifact in the first vehicle in connection with the determined trajectory pattern, and- assigning the determined parameter of the sensing artifact to the trajectory pattern and storing the assigned data,- upon detection of at least one trajectory pattern, in particular when repeatedly driving along the route section, controlling at least one sensor and/or adapting at least one environment detection measured value based on one or more sensors and/or a processed measured value of the sensor system of at least the first and/or a second vehicle.

Description

The invention relates to a method, device and a corresponding computer program for machine learning for a sensor system of a vehicle. Furthermore, the scope of the invention also includes a correspondingly configured mobile user device.

Modern motor vehicles have a large number of vehicle functions that are based on the evaluation of sensor data that capture the immediate or partially distant surroundings of the vehicle. Areas in front of, behind or to the side of the vehicle can typically be captured. Examples of such vehicle functions include parking distance control, a parking assistant, a rear view camera, a so-called surround view system, etc., cruise control systems, warnings for vehicles in a blind spot area or when other objects approach from the side. Furthermore, at least partially automated, highly automated or automatic driving also relies on such sensor systems.

Sensor systems can include sensors that are based on different physical principles. Sensors used include, for example, ultrasonic sensors, radar sensors, camera systems and LIDAR sensors. In order to provide a vehicle function, in particular a driver assistance function, the measured values provided by the various sensors can be partially merged for situation interpretation.

A basic problem is that every type of sensor has weaknesses of a greater or lesser degree. These include, for example, incorrect measured values or values that deviate from an objective value for a certain nature of the objects to be measured. Some sensors are susceptible to interference, in particular mutual interference from other sensors. Errors can occur, for example, in ultrasonic sensors due to frequent echoes, external ultrasound or object surfaces that absorb ultrasound. In a camera system, certain objects, such as road curbs or median strips, cannot be correctly detected and measured under certain conditions, such as rain, fog or drifting snow. The problem with radar sensors is that carbon bodies are not visible to the sensors. Depending on their nature or the aforementioned conditions, certain objects may not be recognized or may not be correctly recognized as a certain type of object.

Deviations in the measured values of just one sensor can make it difficult to merge several measured values and lead to unsatisfactory results. In particular, non-linear effects in the case of deviations in measured values can be different in certain sensing situations when using different sensor types. Since many driver assistance functions depend on the measured values of the sensor system, there is a risk of processing incorrect measured values from the sensors and a corresponding malfunction of a vehicle function, which may be associated with a considerable risk to road safety.

EN 10 2013 012 324 A1 discloses a method for automatic route finding for an ego vehicle, comprising the steps of: a) receiving position information from at least one preceding vehicle; b) constructing a route based on the received position information.

EN 10 2013 226 631 A1 discloses a method and a device for determining local weather conditions, local precipitation and/or local road conditions in the immediate vicinity of a vehicle. The determination is made by evaluating signals from at least one vehicle environment sensor. At least one interference signal from the at least one vehicle environment sensor is taken into account to determine the local weather conditions, the local precipitation and/or the local road conditions.

EN 10 2015 014 651 A1 discloses a method for providing lane information of a lane of a motor vehicle, in which, while the motor vehicle is moving in the lane, a travel trajectory of the motor vehicle is determined with respect to a reference system external to the vehicle, wherein the determined travel trajectory is transmitted to a central lane information collection device external to the vehicle, and the lane information determined at least on the basis of the transmitted travel trajectory is provided by the lane information collection device to at least one further motor vehicle.

EN 10 2015 218 830 A1 discloses a method for a motor vehicle for detecting poor road conditions, the method comprising: detecting a road area using at least one sensor of the motor vehicle. Examining the detected road area for an impairing feature of the road area. The impairing feature is indicative of an impairment of a journey of a motor vehicle. And in the event that the impairing feature is detected: Geographically localizing a location of the impairing feature. The location is geographically localized with a precision that is higher than the precision of a geographical location using a satellite-based localization method according to a public Global Positioning System standard. And the storage of road surface condition information for location information. The location information contains the location of the impairing feature. And the road surface condition information contains information about the impairing feature.

It is an object of the present invention to provide a method and a device which lead to an improved operation or an improved design of a sensor system with regard to the data of the sensor system which can be used in various vehicle functions.

The problem is solved by the features of the independent patent claim. Advantageous embodiments are characterized in the subclaims.

The invention is characterized by a method for machine learning for a sensor system of a vehicle. The invention is also characterized by a device that is designed to carry out the method. The method comprises determining a trajectory pattern of a first vehicle when driving along a section of the route. A parameter is determined that represents a sensing artifact in the first vehicle in connection with the determined trajectory pattern, and the determined parameter of the sensing artifact is assigned to the trajectory pattern and associated data is stored. When at least one trajectory pattern is recognized, in particular when the section of the route is repeatedly driven, at least one sensor is controlled and/or at least one environmental detection measurement value is adjusted based on one or more sensors and/or a processed measurement value of the sensor system of at least the first and/or a second vehicle.

The absolute position is not necessarily decisive for the trajectory pattern in question. This means that a deliberate decoupling from the geographical position can even be achieved.

What is particularly important is the relationship between position values and/or relative positions, in particular the shape of a curve in which the vehicle approaches an object to be sensed, for example, and the like.

The relative positions can, for example, be positions relative to immobile conditions and/or moving objects that are related to the occurrence of the sensing artifact.

Particularly preferably, the relative positions in relation to road markings, posts, sidewalks, tunnels, traffic lights, traffic light poles, zebra crossings, etc. can be taken into account.

In particular, the sensing artifact is a sensing error or a section of the sensor data with an unsatisfactory, low quality or plausibility. A sensing artifact or a corresponding section of the sensor data is caused in particular by, for example, excessive measurement deviations, fluctuations in the values, or insufficient agreement between the sensor data relating to a part of the vehicle's surroundings, which were recorded with at least two different sensors or on the basis of at least two different physical principles. For example, a sensing artifact can correspond to a deviation that results in a significant, impermissible effect on a vehicle functionality based on the sensor or sensor system.

A classification of the sensing artifact can be used as a parameter representing a sensing artifact. This is advantageous for later adaptation. Examples of this are a drop in performance in a section of the sensor data and/or an error characteristic of the physical sensing principle, e.g. a wave reflection and/or an implausible or insufficiently plausible value within the data of the sensor system and/or a short-term failure or unavailability of the sensor.

According to one embodiment of the invention, the sensor system of the first vehicle or of another (second) vehicle can be controlled.

Alternatively or additionally, depending on the assigned data, the method can be used to change the priority or qualifier of the data of at least one sensor, in particular predictively, before or during the reaching of a certain position information and/or to adapt at least one vehicle function of the first and/or second vehicle related to environmental detection.

When the respective vehicle travels along the route section, depending on the assigned data, control information can be provided to control the sensor system and/or of a vehicle function based on the sensor data, e.g. a driver assistance system, and provided wirelessly to the on-board network of the respective vehicle. The control information in question can be based on the recognition (recognition), preferably on the prediction of the trajectory pattern.

Alternatively or additionally, the method may perform scene recognition with respect to a respective environment detection situation in which the sensing artifact occurred, and take the scene into account when determining the associated data, as described below.

According to an optional embodiment, an environmental detection situation, in particular a scene that is related to the occurrence of the sensing artifact, is recognized, in particular classified and processed when determining the associated data.

The scene is classified in particular using pattern recognition. For example, image-based classification and pattern recognition can be carried out in images or image sequences in relation to the images or image sequences captured in the surroundings of the vehicle. This is preferably a scene that has at least a certain (spatial, temporal) connection with at least one sensing artifact.

A sensing artifact can be associated with a problematic, particularly statistically problematic, environment detection situation, or an environment detection situation of a certain type. This can be additionally taken into account in the method by recognizing the scene for the environment detection situation and expressing it using qualitative and/or quantitative values and/or preferably classifying and/or coding it.

The said environmental detection situation and/or scene can be determined and/or recognized as certain parameters characterizing them, in particular information about certain (suitably predefined) types or classes.

For example, the associated data can also represent, take into account or include a class of the corresponding scene. The scene for the environment detection situation can, for example, be represented as a type and/or parameter range of a situation, e.g. a so-called use case, which is related to the occurrence of the sensing artifact.

Such scenes for the environment detection situation are, for example, a standard, typical garage, a garage with an incline such as a duplex garage, a curve between 30 - 70 km/h with bushes and the like. Such a scene can be recorded, recognized and taken into account using a situation vector, for example.

The information recorded on the said environment detection situation and/or scene may be information that does not in itself allow any direct conclusions to be drawn about at least one specific sensing artifact or characteristics of a sensing artifact. In other words, the method does not necessarily have to determine a logical, for example mathematically comprehensible, connection between an environment detection situation or scene in general and the occurrence of the said sensing artifact.

Information about the environment detection situation can be executed, for example, in the first vehicle or in a computing unit arranged at a distance, e.g. in a backend. Preferably, the information, in particular the class of the environment detection situation, is taken into account when determining the associated data.

Preferably, a sensing artifact in the method can be associated with a specific use case that (e.g. repeatedly) leads to sensor errors or related functional errors, particularly in certain sections or positions within the route. The method can thus be used very efficiently primarily to close functional gaps in certain situations or certain types of sensing situations.

The environment detection situation can be characterized by one or more objects that are located in the detection range of the sensor system and have a predetermined nature. The environment detection situation can be characterized by a specific constellation of moving and/or immobile objects that are located in the detection range of the sensor system. The environment detection situation can also be characterized by certain boundary conditions that are relevant for detecting the environment using the sensor system, e.g. using one or more specific physical principles. Such boundary conditions can be, for example, the brightness prevailing in the area surrounding the vehicle (too bright or too dark), snow, fog, icing of one or more sensors and the like.

Particularly preferably, a number of quantitative criteria of the environment detection situation are determined and taken into account. The recognition of the environment detection situation can in particular be a recognition of a class of environment detection situation, for example depending on the number of quantitative criteria. Preferably, the recognition of the environment detection situation can be at least the recognition of one of several (specific) classes (eg as belonging to one of several possible classes or category values) of environment detection situations.

The environment detection situation can be recognized based on a large number of predetermined factors. A given environment detection situation can be recognized using the means of the first vehicle. A specific (special) environment detection situation can be determined (and recognized as such) in the method using several method variants described below. For example, a specific type of environment detection situation can be determined by a recognized situation (use case) and/or based on specific position information (coordinates).

According to a further optional embodiment, control information is determined which depends on the stored data, in particular on learning data, and which causes a change in the properties of a sensor and/or the sensing method used, in particular to a certain quantitative extent, in particular the emission of light or waves and/or interpretation of the received light reflection or wave echo by the sensor system of the first and/or second vehicle.

The control information can be determined using means of the vehicle, in particular within the sensor or a sensor fusion unit.

The control information is preferably quantitative information and/or a large number of different parameters. A large number of the parameters of the sensor system (which are used, for example, to determine different measured variables) can also be controlled in order to improve the detection of the specific environmental detection situation at hand.

The interpretation of the received light reflection or of the wave echo by the sensor system of the first vehicle can be carried out, for example, by means of an activation or control of one or more alternative or at least partially complementary activatable modes, states, mode transitions or state transitions of the sensor system or a functionality of the respective vehicle connected downstream of the sensor system.

Preferably, the said change or adaptation takes place essentially for the duration and/or in a quantitative dependence on the environment detection situation or the determined scene of the environment detection, in particular on at least one qualitative and/or quantitative parameter of the environment detection situation or the scene.

• - eine oder mehreren Wellenlängen in der eine Sensierung durchgeführt wird, und/oder
• - eine Pulsfolge oder Pulssequenz, insbesondere eine Kombination und/oder zeitliche Abhängigkeit von bzw. zwischen zumindest zwei Messvorgängen, und/oder
• - eine Veränderung der Phase eines oder mehrerer Messimpulse oder Berücksichtigung der jeweils veränderten Phase der empfangenen Impulse.

According to a further optional embodiment, the sensor system is controlled with the determined control information in such a way that the adaptation concerns:

• - one or more wavelengths in which sensing is carried out, and/or
• - a pulse sequence or pulse order, in particular a combination and/or temporal dependence of or between at least two measuring processes, and/or
• - a change in the phase of one or more measuring pulses or consideration of the changed phase of the received pulses.

The sensors (ultrasonic sensors, various radars, light-emitting sensors such as laser scanners, LED sensors) in the sensor system can be controlled in different ways. A sensing mode (wavelength, phases, pulses) or even the sensing method can be changed.

In other words, depending on the determination of at least one data set, the sensor system changes its properties (at the points or in accordance with the points within the route where at least one sensor of the first vehicle has caused or delivered a sensing artifact and/or to the extent necessary to avoid or mitigate the sensing artifact) in such a way that correct measurement data or at least measurement data that correspond better to reality or the (downstream) functionality of the vehicle that depends on the measured values result. In particular, specific (e.g. frequently occurring in the specific situation) sensor-related errors (special echoes, crosstalk between the sensors, special reflections) can be avoided or their consequences for the corresponding sections of the sensor data or a vehicle functionality that depends on them can be significantly reduced.

For example, depending on the spatially related data determined, the measurement sequence of ultrasonic sensors can be changed so that interference in a cross-echo measurement or other artifacts or physical effects that are typical for the particular environment detection situation or certain types of environment detection situations and/or in connection with certain position information.

Depending on the assigned data or the control information, a spatial sensing area, e.g. an elevation of radar systems, can be varied, in particular adjusted. This allows a more targeted detection of certain spatial areas, in particular to avoid certain (specific) interference.

According to a further optional embodiment, the control information causes an alternating, in particular temporally alternating change in the properties of a sensor and/or the sensing method used.

A changed mode of a sensing method can also have its own particular weaknesses. It is therefore advantageous, particularly within the same environment detection situation, to alternate between two or more sensing methods of the sensor system or to repeatedly change the properties of at least one sensor in relation to the same or similar scene when detecting the environment.

The method can also include a further step in which the resulting measured values (data of the sensor system), which were recorded in two different sensing phases (each with at least two properties of a sensor and/or the sensing method used), are compared with each other (and, if necessary, merged).

Such a comparison of the data can be particularly effective depending on the data that represents information about the performance of at least one sensor or the sensor system.

In other words, at a problematic location (e.g. a complex constellation of objects, e.g. in a garage that is not tidy), where the vehicle's sensor system does not provide satisfactory data, for example the position information and (in this case about the vehicle's own garage, etc.), the particular type of scene in connection with the environment detection situation in question is recognized (e.g. a certain type of garage, in particular from a certain position or direction). The sensor system consisting of a side radar system with 4 side radars and an ultrasound system with 8-12 sensors is switched to a mode that delivers better results for this environment detection situation. The sensor system is switched to at least two different modes approximately every second, each of which has different strengths and weaknesses. In a further step of the process, the resulting measurement data are compared with each other, in particular fused, in such a way that the strengths of both modes are emphasized to the maximum and/or the weaknesses (susceptibility to errors, non-linearities and others) are at least partially compensated or weakened. In this case, the information representing a performance of at least one sensor or sensor system of the vehicle and/or the associated data, in particular the learning data, can be taken into account.

• - Erfassen eines Trajektorienmusters, welches zumindest einer lokalen Position in Relation zu zumindest einer Fahrbahnmarkierung, Fahrbahnbegrenzung oder Leitplanke der Fahrbahn entspricht, im Zusammenhang mit dem besagten Sensierungsartefakt,

bevorzugt auch umfassend:

• - Erfassen einer globalen Position, die mit dem Auftritt des Sensierungsartefakts zusammenhängt durch das erste Fahrzeug,
• - Ermitteln von Zuordnungsdaten, die repräsentativ sind für ein Abgleichen der erfassten lokalen Position zu zumindest einer Spurenmarkierung, Leitplanke oder Fahrbahnrand zu einer globalen Position im Zusammenhang mit dem Auftritt des besagten Sensierungsartefakts.

According to a further optional embodiment, the method additionally comprises

• - detecting a trajectory pattern corresponding to at least one local position in relation to at least one lane marking, lane boundary or guard rail of the roadway in connection with said sensing artifact,

preferably also comprehensive:

• - Detecting a global position associated with the occurrence of the sensing artifact by the first vehicle,
• - Determining mapping data representative of a comparison of the detected local position of at least one lane marking, guardrail or road edge to a global position associated with the occurrence of said sensing artifact.

The method preferably comprises determining at least one lane marking, guard rail and/or lane boundary by means of a sensor system of the first vehicle. The method comprises determining at least one expected local position that is related to the occurrence of the sensing artifact, at least in the transverse direction relative to at least one lane marking, guard rail and/or lane boundary determined by means of the sensor system of the first vehicle and/or determining at least one parameter of the sensing artifact, in particular the type of sensing artifact (type, class, e.g. low data quality, failure, breakdown, unavailability, etc.) depending on the information provided on the basis of the assignment data.

• - Berücksichtigung der Zuordnungsdaten beim Ermitteln der zugeordneten Daten, und/oder
• - Bereitstellen, insbesondere Übermitteln einer von den Zuordnungsdaten abhängigen Information an eine Recheneinheit zur Nutzung in dem ersten/und oder zweiten Fahrzeug.

The following is then preferably done:

• - Taking the assignment data into account when determining the assigned data, and/or
• - Providing, in particular transmitting, information dependent on the assignment data tion to a computing unit for use in the first and/or second vehicle.

According to a further optional embodiment, the trajectory pattern is determined on the basis of a sequence of vehicle coordinates, e.g. global coordinates of the movement of the vehicle in connection with the occurrence of a sensing artifact, in particular the sensing artifact of a certain type.

In particular, the control information is determined depending on two or more position information, particularly preferably depending on a position sequence.

Particularly preferably, the scene is determined during the environment detection from a sequence of positions of the first vehicle, which includes absolute and/or relative time information, e.g. time interval information for the positions.

For example, a situation when approaching a narrow passage from two different angles (and position sequences) is treated differently, as described below as an example.

For example, in a first position sequence (first approach trajectory), a first control information is determined which, for example, reduces the influence of an ultrasonic sensor in the environment detection, which would cause errors in such an approach trajectory. In a second position sequence (second approach trajectory), a second control information is determined which causes a change in the mode of the ultrasonic system and at the same time fuses or confirms the resulting values with other sensor data.

When determining the control information, it is also possible to take into account the relative position (e.g. distance, offset, angle information) of the first and/or second vehicle in relation to a certain constellation of objects and/or how quickly it is approaching the situation.

According to a further optional embodiment, the assigned data are linked to one or more specific application conditions, wherein the one or more application conditions are taken into account when the road section is repeatedly driven over by at least the first and/or second vehicle.

The one or more application conditions can be determined depending on the conditions prevailing in the first vehicles when the sensing artifact occurs.

In this case, a corresponding execution of the control of the sensor of the first and/or second vehicle can take place depending on the fulfillment of at least one application condition.

For example, depending on the application condition, one of several control measures can be selected, each of which relates to an application condition, e.g. to a speed range, e.g. 10-20 km/h, 20-30 km/h, ... 70-80 km/h, ... 180-210 km/h. These can then be applied depending on whether the application condition is met, e.g. at the relevant time of the current speed of the first and/or second vehicle when repeatedly driving the said section of the route. To put it simply, the stored data can include an "instruction leaflet" with one or more application conditions.

Optionally, the method can determine data representing a specific environment detection situation, in particular when the first and/or second vehicle approaches the corresponding location (or object), in particular at < 20, 15, 10, 5, 3, 1 meters.

According to a further optional embodiment, the sensor system also comprises means or methods for carrying out a sensor fusion, wherein the fusion of the sensor data from at least two sensors is varied depending on a determined environmental sensing situation and/or on a determined control information.

• - Zuordnung von Messwerten eines oder unterschiedlicher Sensoren zueinander und/oder zu bestimmten Objekten oder ausgewählten Raumbereichen (Areas of Interest), und/oder
• - Gewichtung von Messwerten zumindest zweier Sensoren beim Ermitteln zumindest eines fusionierten Messwerts, und/oder
• - Zeitversatzes zwischen den zu fusionierenden Messwerten zumindest zweier Sensoren.

Preferably, the fusion of the sensor data is varied with respect to one or more of the following points:

• - Assignment of measured values from one or different sensors to each other and/or to specific objects or selected spatial areas (areas of interest), and/or
• - Weighting of measured values of at least two sensors when determining at least one fused measured value, and/or
• - Time offset between the measured values of at least two sensors to be merged.

The performance of a static or dynamic environment modeling can be subsumed under the term “sensor system”.

According to a further optional embodiment, a prediction of the occurrence of a certain scene is determined during the environment detection for the near future, and a, in particular predictive planning for controlling the sensor system and/or the vehicle function based on it.

Also, a wireless unilateral or mutual coordination of the change in the properties of a sensor and/or the applied sensing method can be carried out between at least two vehicles.

According to a further optional embodiment, a weighting between map information and environment detection data for operating at least one vehicle function, in particular a driver assistance function, takes place in the first and/or second vehicle depending on the stored data.

In a simplified example, in the case of an expected sensing artifact, e.g. incorrect environmental sensing or lower quality of sensor information, a customer function can be switched from an operation that is essentially dependent on environmental sensing to an operation that is essentially based on map information, in particular for a suitable time interval.

This can also be carried out within (as part of) the relevant vehicle function operated on the basis of the sensor system, e.g. driver assistance function.

• - Erkennen, insbesondere prädizieren eines Trajektorienmusters,
• - Prädizieren eines Sensierungsartefakts, insbesondere eines Sensierungsartefakts mit einem bestimmten Parameter, in einer nahen Zukunft abhängig von dem Trajektorienmuster,
• - Ermitteln einer Steuerungsinformation zum Ansteuern des Sensoriksystems des ersten und/oder zweiten Fahrzeugs in Abhängigkeit von der erkannten bestimmten Umfelderfasungssituation,

ferner bevorzugt umfassend:

• - Ermitteln zumindest einer Zeitinformation und/oder Positionsinformation auf die sich die Steuerungsinformation bezieht,
• - Ansteuern des Sensoriksystems des ersten und/oder zweiten Fahrzeugs oder einer mit dem Sensoriksystem zusammenhängende Fahrzeugfunktion abhängig von der Steuerungsinformation, bevorzugt abgängig von dem Trajektorienmuster.

According to a further optional embodiment, the method additionally comprises:

• - Recognizing, in particular predicting a trajectory pattern,
• - predicting a sensing artifact, in particular a sensing artifact with a certain parameter, in a near future depending on the trajectory pattern,
• - Determining control information for controlling the sensor system of the first and/or second vehicle depending on the detected specific environment detection situation,

further preferably comprising:

• - Determining at least one time information and/or position information to which the control information refers,
• - Controlling the sensor system of the first and/or second vehicle or a vehicle function associated with the sensor system depending on the control information, preferably depending on the trajectory pattern.

The said prediction of a specific sensing artifact, in particular a sensing artifact with a specific parameter, in the near future can also be carried out using any (other) means. As part of the said prediction, only a rudimentary parameter can be determined, which in itself would not be sufficient to avoid or compensate for the sensing artifact. The method can then determine control information (which is much more precise, in particular tailored to the expected sensing artifact), which in turn can be used to control at least one sensor and/or the sensor system of the vehicle.

In this case, the control can relate in particular to at least one time interval and/or travel distance interval, in particular a specific time interval and/or travel distance interval. In this way, a particularly targeted adaptation can be achieved.

• - abhängig von den Daten über eine aktuelle und/oder für die nahe Zukunft prädizierte Positionsinformation des ersten und/oder zweiten Fahrzeugs, und/oder
• - abhängig von der Information über eine bestimmte Szene bei der fraglichen Umfelderfassung, insbesondere im Zusammenhang mit einem bestimmten Manöver, durchgeführt oder beabsichtigt wird.

According to a further optional embodiment, a prediction of a specific sensing artifact, in particular a sensing artifact with a specific parameter, is determined in a near future,

• - depending on the data on current and/or near-future predicted position information of the first and/or second vehicle, and/or
• - is carried out or intended depending on the information relating to a particular scene during the environment detection in question, in particular in connection with a particular manoeuvre.

In particular, a position at which a sensing artifact is to be expected can be determined from the global coordinates (e.g. in the vicinity of a garage, parking garage, a bottleneck, a special piece of land), and/or from the detection of a specific maneuver (e.g. a parking or shunting maneuver, parking, parking out, shunting maneuver, turning maneuver, driving through a bottleneck).

Preferably, a sensing artifact (expected in the near future) from a specific (currently executed or predicted) maneuver at a specific position (e.g. road section of +/- 10, 5, 2, 1 meters) can be predicted depending on the position information.

A specific maneuver can also refer to a specific type of maneuver. The type of maneuver can also depend on a driving vehicle operation by the driver (reverse gear, steering angle, operating sequence, etc.). For example, a certain maneuver or a certain type of maneuver can be recognized by recognizing a certain (typical, characteristic) pattern, e.g. combination of signals.

The sensor system of the first vehicle is prepared to carry out a change in the properties of a sensor and/or the sensing method used (according to a specific plan, in particular comprising time information, e.g. sequence). The information representing such a plan can be communicated to at least one vehicle in the environment, and/or the plan (the vehicle's own plan) can be changed depending on information transmitted by another vehicle.

• - der zugeordneten Daten auf Basis eines mehrmaligen Befahren des Fahrstreckenabschnitts, und/oder
• - der zugeordneten Daten von mehreren ersten Fahrzeugen ausgeführt wird.

According to a further optional embodiment, control information for controlling at least one sensor of a further vehicle is determined on the basis of the one or more associated data of the first vehicle, wherein in particular a comparison:

• - the associated data based on multiple travel along the route section, and/or
• - the associated data is executed by several first vehicles.

The comparison of the associated data can be carried out in a computing unit of the first vehicle and/or in the computing unit of the second vehicle and/or in the computing unit of a mobile user device carried (with the first vehicle or with a second vehicle).

The comparison of the assigned data can include an aggregation of said assigned data. Particularly preferably, the comparison determines improved, optimized control information for controlling the sensor system, comprising at least one sensor of the second vehicle.

According to a further aspect, the invention is characterized by a mobile user device configured to carry out a method for machine learning for a sensor system of a vehicle together with a first vehicle and/or with a second vehicle, wherein the mobile user device is configured to carry out at least part of the method for determining or providing the data based on the assignment and/or part of the method for determining the information for controlling the sensor system of at least one second vehicle.

The mobile user device can be configured to carry out at least part of the method according to the first aspect. For example, the mobile user device can also cause certain features of the method to be carried out in the (first or second) vehicle in which it is carried.

In the context of this description, the mobile application device can be a smartphone, tablet, smart watch, smart closes, in particular a piece of clothing with an electronic device integrated into the piece of clothing, a motorcycle helmet with a human-machine interface, data glasses for the driver of the vehicle, a vehicle computer that can be carried with the vehicle and connected to the vehicle with a data interface, an external navigation device, or an appropriate further development of such devices.

According to a further aspect, the invention is characterized by a computer program, in particular a computer program product comprising a computer program, wherein the computer program is designed to carry out the method according to one of claims 1-14 when executed on a computing unit of the first vehicle, the second vehicle or a mobile user device carried in a first vehicle or in a second vehicle.

A second vehicle is preferably a different vehicle than the first vehicle. However, in exceptional cases, a second vehicle can (temporarily, when the route section is repeatedly driven) correspond to a first vehicle.

• 1 Ein Ablaufdiagramm zum maschinellen Lernen für ein Sensorsystem eines Fahrzeugs und
• 2 eine Abbildung des maschinellen Lernens für das Sensorsystem des Fahrzeugs.

Embodiments of the invention are explained in more detail below with reference to the schematic drawings. They show:

• 1 A machine learning flowchart for a vehicle sensor system and
• 2 a machine learning map for the vehicle's sensor system.

The 1 shows a machine learning flowchart for a vehicle sensor system.

The program can be processed by one device or distributed by several devices. The device comprises in particular a computing unit, a program and Data storage, as well as one or more communication interfaces.

The program is started in a step S1, in which variables can be initialized if necessary.

In a step S3, a trajectory pattern of a first vehicle is determined when driving along a route section.

In a step S5, a parameter is determined representing a sensing artifact in the first vehicle in connection with the determined trajectory pattern.

In a step S7, the determined parameter of the sensing artifact is assigned to the trajectory pattern and the assigned data is stored.

In a step S9, when at least one trajectory pattern is recognized, in particular when the route section is repeatedly driven, at least one sensor is activated and/or at least one environment detection measurement value is adjusted on the basis of one or more sensors and/or a processed measurement value of the sensor system of at least the first and/or a second vehicle.

In a step S11 the program is terminated and can be started again in step S1 if necessary.

• Einen Schritt, bei dem eine Umfelderfassungssituation, insbesondere eine Szene, die im Zusammenhang mit dem Auftritt des Sensierungsartefakts steht, erkannt, insbesondere klassifiziert wird und beim Ermitteln der zugeordneten Daten verarbeitet wird.

The method may optionally comprise one or more of the following steps:

• A step in which an environmental detection situation, in particular a scene that is associated with the occurrence of the sensing artifact, is recognized, in particular classified, and processed when determining the associated data.

A step in which control information dependent on the stored data, in particular learning data, is determined, which causes a change in the properties of a sensor and/or the sensing method used, in particular to a certain quantitative extent, in particular the emission of light or waves and/or interpretation of the received light reflection or wave echo by the sensor system of the first and/or second vehicle.

• - eine oder mehreren Wellenlängen in der eine Sensierung durchgeführt wird, und/oder
• - eine Pulsfolge oder Pulssequenz, insbesondere eine Kombination und/oder zeitliche Abhängigkeit von bzw. zwischen zumindest zwei Messvorgängen, und/oder
• - eine Veränderung der Phase eines oder mehrerer Messimpulse oder Berücksichtigung der jeweils veränderten Phase der empfangenen Impulse.

A step in which the sensor system is controlled with the determined control information in such a way that the adjustment affects:

• - one or more wavelengths in which sensing is carried out, and/or
• - a pulse sequence or pulse order, in particular a combination and/or temporal dependence of or between at least two measuring processes, and/or
• - a change in the phase of one or more measuring pulses or consideration of the changed phase of the received pulses.

A step in which the control information causes an alternating, in particular temporally alternating, change in the properties of a sensor and/or the sensing method used.

• - Erfassen eines Trajektorienmusters, welches zumindest einer lokalen Position in Relation zu zumindest einer Fahrbahnmarkierung, Fahrbahnbegrenzung oder Leitplanke der Fahrbahn entspricht, im Zusammenhang mit dem besagten Sensierungsartefakt,

bevorzugt auch umfassend:

• - Erfassen einer globalen Position, die mit dem Auftritt des Sensierungsartefakts zusammenhängt durch das erstes Fahrzeug,
• - Ermitteln von Zuordnungsdaten, die repräsentativ sind für ein Abgleichen der erfassten lokalen Position zu zumindest einer Spurenmarkierung, Leitplanke oder Fahrbahnrand zu einer globalen Position im Zusammenhang mit dem Auftritt des besagten Sensierungsartefakts.

One step comprehensive:

• - detecting a trajectory pattern corresponding to at least one local position in relation to at least one lane marking, lane boundary or guard rail of the lane in connection with said sensing artifact,

preferably also comprehensive:

• - Detecting a global position associated with the occurrence of the sensing artifact by the first vehicle,
• - Determining mapping data representative of a comparison of the detected local position of at least one lane marking, guardrail or road edge to a global position associated with the occurrence of said sensing artifact.

A step in which the trajectory pattern is determined based on a sequence of vehicle coordinates, in particular global coordinates of the movement of the vehicle in connection with the occurrence of a sensing artifact, in particular the sensing artifact of a certain type.

A step of linking the associated data to one or more specific application conditions, wherein the one or more application conditions are taken into account when at least the first and/or second vehicle repeatedly travels along the roadway section.

A step in which the sensor system also comprises means or methods for carrying out a sensor fusion, whereby depending on a determined environmental sensing situation and/or on a determined control information, the Fusion of sensor data from at least two sensors is varied.

A step in which a prediction is made about the occurrence of a certain scene in the near future when detecting the environment, and a plan, in particular a forward-looking one, is determined for controlling the sensor system and/or the vehicle function based on it.

A step in which, in the first and/or second vehicle, a weighting is carried out between map information and environment detection data depending on the stored data for operating at least one vehicle function, in particular a driver assistance function.

• - Erkennen, insbesondere prädizieren eines Trajektorienmusters,
• - Prädizieren eines Sensierungsartefakts, insbesondere eines Sensierungsartefakts mit einem bestimmten Parameter, in einer nahen Zukunft abhängig von dem Trajektorienmuster,
• - Ermitteln einer Steuerungsinformation zum Ansteuern des Sensoriksystems des ersten und/oder zweiten Fahrzeugs in Abhängigkeit von der erkannten bestimmten Umfelderfasungssituation, ferner bevorzugt umfassend:
• - Ermitteln zumindest einer Zeitinformation und/oder Positionsinformation auf die sich die Steuerungsinformation bezieht,
• - Ansteuern des Sensoriksystems des ersten und/oder zweiten Fahrzeugs oder einer mit dem Sensoriksystem zusammenhängende Fahrzeugfunktion abhängig von der Steuerungsinformation, bevorzugt abgängig von dem Trajektorienmuster.

One step comprehensive:

• - Recognizing, in particular predicting a trajectory pattern,
• - predicting a sensing artifact, in particular a sensing artifact with a certain parameter, in a near future depending on the trajectory pattern,
• - Determining control information for controlling the sensor system of the first and/or second vehicle depending on the detected specific environment detection situation, further preferably comprising:
• - Determining at least one time information and/or position information to which the control information refers,
• - Controlling the sensor system of the first and/or second vehicle or a vehicle function associated with the sensor system depending on the control information, preferably depending on the trajectory pattern.

• - abhängig von den Daten über eine aktuelle und/oder für die nahe Zukunft prädizierte Positionsinformation des ersten und/oder zweiten Fahrzeugs, und/oder
• - abhängig von der Information über eine bestimmte Szene bei der fraglichen Umfelderfassung, insbesondere im Zusammenhang mit einem bestimmten Manöver, durchgeführt oder beabsichtigt wird.

A step in which a prediction of a particular sensing artifact, in particular a sensing artifact with a particular parameter, is determined in a near future,

• - depending on the data on current and/or near-future predicted position information of the first and/or second vehicle, and/or
• - is carried out or intended depending on the information relating to a particular scene during the environment detection in question, in particular in connection with a particular manoeuvre.

• - der zugeordneten Daten auf Basis eines mehrmaligen Befahren des Fahrstreckenabschnitts, und/oder
• - der zugeordneten Daten von mehreren ersten Fahrzeugen ausgeführt wird.

A step in which control information for controlling at least one sensor of another vehicle is determined on the basis of the one or more associated data of the first vehicle, in particular a comparison:

• - the associated data based on multiple travel along the route section, and/or
• - the associated data is executed by several first vehicles.

The method can optionally be carried out together with a method for adapting a sensor system, which is described below.

• Erkennen einer vorgegebenen Umfelderfassungssituation mit Mitteln des Fahrzeugs;
• Ermitteln einer Steuerungsinformation zum Ansteuern des Sensorsystems des Fahrzeugs in Abhängigkeit von der erkannten, vorgegebenen Umfelderfassungssituation; und Ansteuern des Sensorsystems des Fahrzeugs derart, dass die im Zusammenhang mit der erkannten Umfelderfassungssituation erfassten oder zu erfassenden Messwerte zumindest eines der Sensoren und/oder aus den Messwerten eines oder mehrerer Sensoren des Sensorsystems ermittelte Daten zum Betreiben zumindest einer Fahrzeugfunktion zumindest teilweise korrigiert werden.

The method for adapting a sensor system may include the following steps:

• Recognition of a given environmental detection situation using vehicle resources;
• Determining control information for controlling the sensor system of the vehicle depending on the detected, predetermined environment detection situation; and controlling the sensor system of the vehicle in such a way that the measured values of at least one of the sensors recorded or to be recorded in connection with the detected environment detection situation and/or data determined from the measured values of one or more sensors of the sensor system for operating at least one vehicle function are at least partially corrected.

The sensor system of the vehicle is a system that, among other things, provides data for a vehicle function, in particular a driver assistance function, a driver information system, a driving dynamics function or a vertical dynamics function of the vehicle.

The given environment detection situation is in particular a situation that represents an increased complexity or an increased degree of difficulty or a significantly pronounced special feature for a sensory detection of the environment with the aid of the sensor system.

A given environment detection situation also exists in particular when there is an increased complexity or an increased level of difficulty for the evaluation of the data provided by the sensor system. A given environment detection situation can therefore also be a specific application case that has repeatedly led to errors in the sensor system and related functional errors in the past.

The method aims in particular to close functional gaps associated with given (difficult) environment detection situations (temporary unavailability or faulty functionality of one or more vehicle functions) and thus also to provide the vehicle user with the most consistent experience possible. The method can include the detection of (possible) faulty, poor-quality measured values of the sensor system and/or the functionality of the at least one vehicle function. The execution of further steps of the method can then take place depending on the detection of the faulty, poor-quality measured values of the sensor system and/or the functionality of the at least one vehicle function.

The data determined from the measured values of one or more sensors of the sensor system for operating at least one vehicle function can be so-called environment model data, which, for example, describe parts of the environment of the vehicle and represent a data model that refers to at least one point in time.

Alternatively or additionally, the data determined from the measured values of one or more sensors of the sensor system for operating at least one vehicle function can be one or more target variables and/or a control variable of a vehicle function.

The environment detection situation can be characterized by one or more objects that are located in the detection range of the sensor system and have a predetermined nature. The environment detection situation can be characterized by a specific constellation of moving and/or immobile objects that are located in the detection range of the sensor system. The environment detection situation can also be characterized by certain boundary conditions that are relevant for detecting the environment using the sensor system, e.g. using one or more specific physical principles. Such boundary conditions can be, for example, the brightness prevailing in the area surrounding the vehicle (too bright or too dark), snow, fog, icing of one or more sensors and the like.

Particularly preferably, a number of quantitative criteria of the environment detection situation are determined and taken into account. The recognition of the environment detection situation can in particular be a recognition of a class of environment detection situation, for example depending on the number of quantitative criteria. Preferably, the recognition of the environment detection situation can be at least the recognition of one of several (specific) classes (e.g. as belonging to one of several possible classes or category values) of environment detection situations.

The environment detection situation can be recognized based on a large number of predetermined factors. The recognition of a given environment detection situation can be carried out using the vehicle's resources. The recognition of a given environment detection situation can be carried out in the method depending on the measured values of at least one of the sensors and/or depending on data determined from the measured values of one or more sensors of the sensor system.

The predefined environment detection situation can be recognized based on the measured values of at least one first sensor and/or based on data determined from the measured values of one or more first sensors of the sensor system, and the sensor system of the vehicle can then be controlled in such a way that the measured values of at least one second sensor recorded or to be recorded in connection with the recognized environment detection situation and/or the data determined from the measured values of one or more second sensors of the sensor system for operating at least one vehicle function are at least partially corrected. The at least one first sensor and the at least one second sensor can be different and/or at least partially based on different physical principles.

For example, in the method, depending on the measured values of a radar sensor, the control of an ultrasonic sensor and/or a camera system of the vehicle can take place, wherein the resulting values of the ultrasonic sensor and/or the camera system are at least partially corrected, e.g. improved according to predetermined criteria. For example, in the method, depending on the measured values of a camera system, the control of an ultrasonic sensor and/or a radar sensor of the vehicle can take place, wherein the resulting values of the ultrasonic sensor and/or the radar sensor are at least partially corrected, e.g. improved according to predetermined criteria.

Alternatively or additionally, the specified environment detection situation can be recognized using information about a specific driving maneuver performed by the vehicle. For example, a recognized situation that corresponds to a specific application case can also be included in this.

The recognition of the environment detection situation can also be determined based on certain relative position information of the vehicle to an object constellation. Alternatively or additionally, depending on the recognition of the environment detection situation, a result and/or a consequence from a specific (possible, selectable) control or change of the sensor system or control information can also be determined or estimated in the method. Depending on the determined result and/or a consequence of a specific control, the method can decide, for example, to carry out a specific control or not to carry it out and/or to select one of several possible control information for controlling at least one sensor of the sensor system.

The control information can be determined in the process by a computing unit in the vehicle. The computing unit can be the measured values of a sensor or the computing unit that processes the measured values of all the different sensor systems in the vehicle.

The control information is quantitative information and/or parameters with which, for example, a measuring method of one or more sensors can be influenced. Preferably, a sequence of a measuring method relating to at least one sensor and/or quantitative dimensions associated with the measuring method can be adapted. The control can be a large number of parameters of the sensor system that are used to determine different measured variables and/or to improve the detection of the specified environmental detection situation.

• - Ermitteln der Umfelderfassungssituation, zumindest als eine Klasse der Umfelderfassungssituation;
• - Bestimmen der zu optimierenden Kriterien zu einem Messverfahren;
• - Erzeugen der Steuerungsinformation;
• - Durchführen zumindest eines Messvorgangs und/oder der Verwendung der Sensorwerte unter Berücksichtigung der ermittelten Steuerungsinformation.

Preferably, control information can be determined that influences the measurement method in favor of a desired result for the environment detection situation. Such a variant of the method can include:

• - Determining the environmental detection situation, at least as a class of the environmental detection situation;
• - Determining the criteria to be optimized for a measurement procedure;
• - Generating the control information;
• - Carrying out at least one measuring process and/or using the sensor values taking into account the determined control information.

The method is based on the idea that in certain situations, e.g. with certain problematic objects or a certain object constellation (arrangement of objects) in the environment, the sensor system is changed by control in such a way that improved (largely error-free and precise) environmental detection is possible. For example, the parameters of an infrared light emitted by a sensor (in the case of a lidar or time-of-light sensor), electromagnetic waves (in the case of radar sensors) or ultrasonic waves (in the case of ultrasonic sensors) can be adapted to suit the situation of the sensor detection. Such an adaptation takes place depending on the nature of certain objects, a certain object constellation or prevailing boundary conditions. As described, an adaptive correction of quantitative parameter values and/or the evaluation or processing of the measured values can also be carried out. An adaptation can also be carried out by a different weighting of the data from different sensors in a sensor fusion as well as the changed control of the sensors or when evaluating measured values or phase information.

The procedure starts where a certain situation in the vehicle's environment has proven to be problematic for its sensor system with intended (i.e. factory-set) parameters in the past. Such situations include, for example, objects that are detected too late on a specially designed road, a construction site, a car wash, etc. These situations are taken into account by adapting the sensor system.

The method can also be made adaptable by means of an application that runs on the vehicle and is configurable, for example, with the help of an app (= application, i.e. an application program) running on a user's mobile device (terminal device).

In a first variant, the method can be implemented, for example, in the following way: recognizing a predetermined environment detection situation using means of the vehicle; determining a first control information for controlling the sensor system of the vehicle depending on the recognized, predetermined environment detection situation; and controlling the sensor system of the vehicle in such a way that the measured values of at least one of the sensors recorded or to be recorded in connection with the recognized environment detection situation are at least partially corrected.

The method can be implemented in a second variant, for example, in the following way: Recognising a given environment detection situation using the vehicle’s means; Determining a second control information for controlling the sensor system of the vehicle depending on the recognized, predetermined environment detection situation; and controlling the sensor system of the vehicle in such a way that the data determined in connection with the recognized environment detection situation from the measured values of one or more sensors of the sensor system are at least partially corrected for operating at least one vehicle function.

The method can preferably be implemented in a third variant, for example, in the following way: recognizing a predetermined environment detection situation using means of the vehicle; determining a second control information for controlling the sensor system to correct data determined from the measured values of one or more sensors of the sensor system for operating at least one vehicle function of the vehicle depending on the recognized, predetermined environment detection situation; controlling the sensor system of the vehicle in such a way that the data determined in connection with the recognized environment detection situation from the measured values of one or more sensors of the sensor system for operating at least one vehicle function are at least partially corrected; determining a first control information for controlling the sensor system of the vehicle to at least partially correct the measured values to be recorded by at least one of the sensors depending on the recognized, predetermined environment detection situation; and controlling the sensor system of the vehicle in such a way that the measured values to be recorded by at least one of the sensors in connection with the recognized environment detection situation are at least partially corrected.

In other words, in the third variant, the second variant is first carried out, in which the measured values are at least partially corrected for operating at least one vehicle function. A quick corrective reaction can be applied to the sensor system so that the vehicle function can be operated with at least partially corrected measured values. The first variant of the method is then carried out, in which at least one sensor is controlled using the first control data. This control can, for example, also include an adjustment of the measurement signals emitted by the sensor, etc. Along with an execution of the first variant, the execution of the second variant can be reduced or stopped.

The third variant has the advantage of a very fast reaction and a subsequent correction (improvement) of the underlying sensor data, in particular the raw data of at least one sensor.

• Einen Schritt bei dem durch die Steuerungsinformation Eigenschaften eines oder mehrerer der Sensoren des Sensorsystems und/oder ein angewandtes Messverfahren des Sensorsystems verändert werden.

The procedure for adapting a sensor system can optionally include the following step:

• A step in which the control information changes properties of one or more of the sensors of the sensor system and/or an applied measurement method of the sensor system.

The change in the properties occurs in particular with regard to the duration of a measurement carried out by a sensor and/or in a quantitative manner. The change in an applied measurement method of the sensor system can be realized, for example, by changing the sequence of measurements of one or more sensors.

• Einen Schritt bei dem zur Veränderung der Eigenschaften des oder der Sensoren ein jeweiliger Sensor derart angesteuert wird, dass er bei einer Messung ein Signal mit einer oder mehreren Wellenlängen abgibt.

The procedure for adapting a sensor system can optionally include the following step:

• A step in which, in order to change the properties of the sensor(s), a respective sensor is controlled in such a way that it emits a signal with one or more wavelengths during a measurement.

For example, all sensors can emit a signal of a given wavelength at the same time. Alternatively, one sensor can emit a signal of one wavelength and at least one other sensor can emit a signal of a different wavelength. Likewise, different wavelengths can be emitted by a sensor one after the other. Alternatively or additionally, it can be provided that in order to change the properties of the sensor(s), a respective sensor is controlled in such a way that it emits a pulse train or pulse sequence during a measurement. A pulse sequence can be, for example, a combination of pulses or pulse trains. There can be a temporal dependency between the two measuring processes. Such a temporal dependency is not mandatory.

In particular, the method can also be used to determine the influence of the (in a certain way) changed wavelength of the emitted signal on a first measured value detected by the sensor(s), in particular a first reflection value resulting from the changed wavelength, wherein the subsequent control of the sensor system is varied depending on the first measured value.

• Einen Schritt bei dem zur Veränderung der Eigenschaften des oder der Sensoren ein jeweiliger Sensor derart angesteuert wird, dass er bei einer Messung eine bestimmte, insbesondere eine veränderte, Pulsfolge oder Pulssequenz abgibt.

The procedure for adapting a sensor system can optionally include the following step:

• A step in which the properties of the sensor(s) are changed by a sensor is controlled in such a way that it emits a specific, in particular a modified, pulse train or pulse sequence during a measurement.

• Einen Schritt bei dem zur Veränderung der Eigenschaften des oder der Sensoren ein jeweiliger Sensor derart angesteuert wird, dass er eine Phase eines oder mehrerer Messimpulse verändert, und optional, die jeweils veränderten Phasen empfangener Messimpulse verarbeitet.

The procedure for adapting a sensor system can optionally include the following step:

• A step in which, in order to change the properties of the sensor(s), a respective sensor is controlled in such a way that it changes a phase of one or more measuring pulses and, optionally, processes the respectively changed phases of received measuring pulses.

These designs can provide for different types of sensors (e.g. ultrasonic sensors, different radars, light-emitting sensors such as laser scanners or LED sensors) to be controlled differently.

A measurement mode can be changed in terms of wavelength, phase, pulse sequence or even the measurement method. This means that, depending on the detected environmental detection situation, one or more sensors change their properties so that correct measurement data or at least more realistic ones result. This makes it possible to avoid specific errors that occur in a certain situation and are caused by the measurement. Such errors include, for example, specific echoes, crosstalk between the sensors or reflections.

Depending on spatially determined data, the measurement sequence of ultrasonic sensors can be changed to reduce interference in a cross-echo measurement or other artifacts or physical effects that are typical for certain sensing situations or types of sensing situations.

The spatial detection range of one or more sensors, e.g. the elevation of radar sensors, can also be varied. This allows for more targeted detection of specific spatial areas, particularly to avoid interference.

• Einen Schritt, bei dem durch die Steuerungsinformation eine abwechselnde, insbesondere zeitlich alternierende, Veränderung der Eigenschaften des oder der Sensoren und/oder des Messverfahrens initiiert wird.

The procedure for adapting a sensor system can optionally include the following step:

• A step in which the control information initiates an alternating, in particular temporally alternating, change in the properties of the sensor(s) and/or the measuring method.

In order to further improve the result and processing of the measured values of the sensors of the sensor system, it is expedient to switch between two or more different measuring methods of the sensor system or a repeated change in the properties of at least one sensor within the same environment detection situation. This change preferably takes place automatically, for example when a special type of environment detection situation occurs. For example, a sensor system that includes four side radar sensors and eight to twelve ultrasonic sensors can be switched to a mode that delivers better results for the environment detection situation.

The sensor system can, for example, switch between two different measurement modes every second, with each measurement mode having different strengths and weaknesses. The resulting measurement data from the at least two different measurement modes are compared and/or merged in a processing step in such a way that the strengths of both measurement modes are highlighted and/or the weaknesses in terms of susceptibility to errors, non-linearity and the like are at least partially compensated or at least mitigated. For example, the performance of a sensor or sensor type of the vehicle can be taken into account. It is also possible to take learning data into account.

According to a further expedient embodiment, it can be provided that the resulting measured values of the sensor system, which were recorded in different measuring phases or different measuring modes, are compared and combined with one another.

• Einen Schritt, bei dem abhängig von den Messwerten und/oder von aus den Messwerten eines oder mehrerer Sensoren des Sensorsystems ermittelten Daten ein bestimmtes Muster erkannt, und
• abhängig von dem erkannten Muster ein Erkennen der vorgegebenen Umfelderfassungssituation, insbesondere zumindest eine Klasse der Umfelderfassungssituation, erfolgt.

The procedure for adapting a sensor system can optionally include the following step:

• A step in which a specific pattern is recognized depending on the measured values and/or data determined from the measured values of one or more sensors of the sensor system, and
• Depending on the recognized pattern, the given environmental detection situation, in particular at least one class of the environmental detection situation, is recognized.

Then, in particular depending on the at least one conclusion about the predetermined environment detection situation, control information for controlling the at least one first sensor and/or at least one second sensor of the sensor system of the vehicle can be determined. The first sensor(s) and the second sensor(s) can be the same or different sensors or sensor types.

Such a pattern can be recognized based on an amplitude curve, a phase curve, a signal shape, a dependence on a temporal behavior of at least one measured value and/or interruptions in the measured values of at least one of the sensors and/or data determined from the measured values of one or more sensors of the sensor system.

Then, depending on the recognized pattern, the conclusion about the given environment detection situation can be determined. Such a conclusion can in particular be a classification of the environment detection situation and/or an exclusion of one or more specific environment detection situations, preferably an assignment to a specific environment detection situation, or a given class of the environment detection situation.

In this case, a pattern can be recognized depending on the measured values of the first sensor(s), which indicates a certain sensing inaccuracy, and then a second sensor can be controlled in such a way that the data determined at least as a function of the second sensor are at least partially corrected for operating at least one vehicle function.

Alternatively or additionally, an object constellation when approaching a narrow passage from two different angles, which is determined by respective sequences of positions of the vehicle relative to the object constellation, can be considered as two different environment detection situations or can be recognized in the process as different environment detection situations and processed differently accordingly.

In one example of the method, when the vehicle approaches the object constellation with a first approach trajectory, a first control information can be determined which, for example, reduces the influence of one or more sensors, e.g. ultrasonic sensors in the environment detection, which would cause errors with such an approach trajectory. When approaching according to a second approach trajectory, a second control information can be determined which causes a change in the control of the ultrasonic sensor(s) and optionally merges the resulting values with further sensor data in order to achieve or confirm the improved measurement results.

According to a further expedient embodiment, information about the mobility of at least one object measured by the sensor system is processed. In particular, an environment detection situation can be recognized based on the approach trajectory. At the same time, the mobility of at least one of the objects to be sensed can be detected. An approach trajectory can refer to a relative trajectory between the vehicle or its sensor(s) and an object constellation. The mobility of an object can be determined in the method based on the object class (determined elsewhere, known a priori or assigned).

• Einen Schritt, bei dem die Steuerungsinformation abhängig von Daten eines energieversorgungsfreien Datenträgers, welcher in dem sensorisch erfassten Umfeld des Fahrzeugs angebracht und/oder eingelernt ist, ermittelt wird.

The procedure for adapting a sensor system can optionally include the following step:

• A step in which the control information is determined depending on data from a data carrier without a power supply, which is installed and/or taught in the sensor-detected environment of the vehicle.

A power-free data carrier can be a passive chip for wireless near-field communication, in particular an RFID chip or a machine-readable code.

An RFID chip can be programmed in any desired way. In particular, it can be assigned to one of several environment detection situations available, such as there is a narrow passage, there is a parking situation, the parking space has no boundaries. The RFID chip can also carry the information that at the location of the data carrier without power supply, detection with a sensor of the vehicle system is not possible or may be associated with difficulties.

For example, the machine-readable code can be designed as a QR code. A QR code can be read using standardized software, for example, as soon as the QR code is recognized by a camera in the vehicle. Such a code can also be read and checked using a user's device. A machine-readable code that has contrasts in an infrared spectrum that depend on the information stored is advantageous because it has better resistance to contamination compared to a code in visible light. It can also be better recognized in poor lighting conditions.

Likewise, the energy-free data carrier can consist of a combination of a passive chip for wireless near-field communication and a machine-readable code.

For example, such a power-free data carrier can be used in different places placed where a difficult environment detection situation exists, as described above. With the help of an identifier, a data carrier without a power supply can be assigned to a specific location. The assignment of the identifier of the data carrier without a power supply and the location can be stored in a data set. The data in the data set can be read out by the sensor system's processing unit. For this purpose, the database can be taught by the vehicle through a learning process. The database can also be stored outside the vehicle, so that a corresponding exchange of data takes place through communication between the vehicle's processing unit and the database.

Using a typical reading distance of commercially available passive chips for wireless near-field communication, the approach of the vehicle to the location or object marked with a non-powered data carrier can be determined to be less than 20 m, 15 m, 10 m, 5 m, 3 m, 1 m.

• Einen Schritt bei dem das Sensorsystem Mittel zur Ausführung einer Sensorfusion umfasst, durch das abhängig von dem sensorisch erfassten Umfeld des Fahrzeugs und/oder von der ermittelten Steuerungsinformation die Fusion der Messwerte von zumindest zwei Sensoren des Sensorsystems variiert wird.

The procedure for adapting a sensor system can optionally include the following step:

• A step in which the sensor system comprises means for carrying out a sensor fusion, by means of which the fusion of the measured values of at least two sensors of the sensor system is varied depending on the sensor-detected environment of the vehicle and/or on the determined control information.

In particular, measurement values from one or more sensors can be assigned to one another and/or to specific objects or selected spatial areas of the environment. Alternatively or additionally, measurement values from at least two sensors can be weighted when determining at least one fused measurement value. Further alternatively or additionally, a time offset between the measurement values to be fused between at least two sensors can be processed. Logical and/or quantitative relationships of the fusion process can be adapted.

• Einen Schritt, bei dem eine Voraussage zum Eintritt der vorgegebenen Umfelderfassungssituation für die nahe Zukunft ermittelt wird, und eine, insbesondere vorausschauende,
• Vorschrift oder Planung zum Ansteuern des Sensorsystems des Fahrzeugs ermittelt wird.

The procedure for adapting a sensor system can optionally include the following step:

• A step in which a prediction is made about the occurrence of the given environmental detection situation in the near future and a, in particular, forward-looking,
• Regulation or planning for controlling the vehicle's sensor system is determined.

• Einen Schritt, bei dem folgende weitere Schritte durchgeführt werden:
  • - Ermitteln, dass die vorgegebene Umfelderfassungssituation innerhalb eines vorgegebenen Zeitraums, insbesondere im Sekundenbereich, eintreten wird;
  • - Ermitteln der Steuerungsinformation zum Ansteuern des Sensorsystems des Fahrzeugs in Abhängigkeit von der erkannten, vorgegebenen Umfelderfassungssituation und zumindest einer Zeitinformation, auf die sich diese bezieht;
  • - Variieren des Sensorsystems des Fahrzeugs oder einer mit dem Sensorsystem zusammenhängenden Fahrzeugfunktion abhängig von der Steuerungsinformation und der Zeitinformation.

The procedure for adapting a sensor system can optionally include the following step:

• A step in which the following further steps are carried out:
  • - Determine that the specified environmental detection situation will occur within a specified period of time, in particular in the range of seconds;
  • - Determining the control information for controlling the sensor system of the vehicle depending on the detected, predetermined environment detection situation and at least one time information to which this refers;
  • - Varying the sensor system of the vehicle or a vehicle function related to the sensor system depending on the control information and the time information.

The determination that the specified environment detection situation is within a specified period of time can be made from information about a specific maneuver of the vehicle that is currently being carried out or is intended to be carried out. A specific maneuver can be recognized by the driver's vehicle operation, such as engaging reverse gear, steering angle, an operating sequence, etc.

This design allows the control parameters for controlling the sensor system to be determined before the environment detection situation occurs. The measurement method used can be carried out according to the planning, which includes time information or a time sequence.

The 2 illustrates the above machine learning procedure again.

The trajectory pattern TM of the first vehicle 10 is determined when driving along a section of the route. In addition, the parameter P, representing a sensing artifact in the first vehicle 10 in connection with the determined trajectory pattern TM, is determined.

The determined parameter P of the sensing artifact is assigned to the trajectory pattern TM and associated data ZD are stored.

Subsequently, when at least one trajectory pattern TM is detected, in particular when the route section is repeatedly driven, at least one sensor is activated and/or at least one environment detection measurement value is adjusted on the basis of one or more several sensors and/or a processed measured value of the sensor system of at least the first and/or a second vehicle 10, 20.

The steps may be performed by a device of the first vehicle 10 and/or the second vehicle 20 and/or by a mobile user device 14, 14'.

Furthermore, the associated data ZD can be transmitted to a backend 15 and/or to the second vehicle 20, so that the control and/or adaptation can take place in the respective vehicle 10, 20.

There may also be two or more than two first vehicles 10 present and used for the procedure.

Claims (16)

Method for machine learning for a sensor system of a vehicle, comprising: - determining a trajectory pattern of a first vehicle when driving along a section of the route, - determining a parameter representing a sensing artifact in the first vehicle in connection with the determined trajectory pattern, and - assigning the determined parameter of the sensing artifact to the trajectory pattern and storing the assigned data, - upon detection of at least one trajectory pattern, in particular when repeatedly driving along the section of the route, controlling at least one sensor and/or adapting at least one environmental detection measurement value based on one or more sensors and/or a processed measurement value of the sensor system of at least the first and/or a second vehicle. Procedure according to Claim 1 , in which an environmental detection situation, in particular a scene that is related to the occurrence of the sensing artifact, is recognized, in particular classified, and processed when determining the associated data. Method according to one of the preceding claims, in which control information dependent on the stored data, in particular on learning data, is determined, which causes a change in the properties of a sensor and/or the sensing method used, in particular to a certain quantitative extent, in particular the emission of light or waves and/or interpretation of the received light reflection or wave echo by the sensor system of the first and/or second vehicle. Procedure according to Claim 3 , in which the sensor system is controlled with the determined control information in such a way that the adaptation affects: - one or more wavelengths in which sensing is carried out, and/or - a pulse train or pulse sequence, in particular a combination and/or temporal dependence of or between at least two measuring processes, and/or - a change in the phase of one or more measuring pulses or consideration of the respectively changed phase of the received pulses. Procedure according to Claim 3 or 4 , in which the control information causes an alternating, in particular temporally alternating change in the properties of a sensor and/or the sensing method used. Method according to one of the preceding claims, also comprising: - detecting a trajectory pattern which corresponds to at least one local position in relation to at least one lane marking, lane boundary or guard rail of the roadway, in connection with said sensing artifact, preferably also comprising: - detecting a global position which is related to the occurrence of the sensing artifact by the first vehicle, - determining assignment data which are representative for comparing the detected local position of at least one lane marking, guard rail or road edge to a global position in connection with the occurrence of said sensing artifact. Method according to one of the preceding claims, in which the trajectory pattern is determined on the basis of a sequence of vehicle coordinates, in particular global coordinates of the movement of the vehicle in connection with the occurrence of a sensing artifact, in particular the sensing artifact of a certain type. Method according to one of the preceding claims, in which the assigned data are linked to one or more specific application conditions, wherein the one or more application conditions are taken into account when the road section is repeatedly driven over by at least the first and/or second vehicle. Method according to one of the preceding claims, in which the sensor system also comprises means or methods for carrying out a sensor fusion, wherein the fusion of the sensor data from at least two sensors is varied depending on a determined environmental sensing situation and/or on a determined control information. Method according to one of the preceding claims, in which a prediction of the occurrence of a certain scene in the surroundings detection for the near future is determined, and a, in particular predictive, plan for controlling the sensor system and/or the vehicle function based thereon is determined. Method according to one of the preceding claims, in which in the first and/or second vehicle, depending on the stored data, a weighting is carried out between map information and environment detection data for operating at least one vehicle function, in particular a driver assistance function. Method according to one of the preceding claims, comprising: - recognizing, in particular predicting a trajectory pattern, - predicting a sensing artifact, in particular a sensing artifact with a specific parameter, in a near future depending on the trajectory pattern, - determining control information for controlling the sensor system of the first and/or second vehicle depending on the recognized specific environment detection situation, further preferably comprising: - determining at least one time information and/or position information to which the control information relates, - controlling the sensor system of the first and/or second vehicle or a vehicle function associated with the sensor system depending on the control information, preferably dependent on the trajectory pattern. Method according to one of the preceding claims, in which a prediction of a specific sensing artifact, in particular a sensing artifact with a specific parameter, is determined in the near future, - depending on the data about current and/or near-future predicted position information of the first and/or second vehicle, and/or - depending on the information about a specific scene in the environment detection in question, in particular in connection with a specific maneuver, is carried out or intended. Method according to one of the preceding claims, in which control information for controlling at least one sensor of a further vehicle is determined on the basis of the one or more associated data of the first vehicle, wherein in particular a comparison is carried out: - of the associated data on the basis of driving along the route section several times, and/or - of the associated data of several first vehicles. Mobile user device, configured to implement a method for machine learning for a sensor system of a vehicle together with a first vehicle and/or with a second vehicle according to one of the Claims 1 until 14 to carry out, wherein the mobile user device is configured to carry out at least part of the method for determining or providing the data based on the assignment and/or part of the method for determining the information for controlling the sensor system of at least one second vehicle. Computer program, in particular a computer program product comprising a computer program, wherein the computer program is designed, when executed on a computing unit of the first vehicle, the second vehicle or a mobile user device carried in a first vehicle or in a second vehicle, to carry out the method according to one of the Claims 1 - 14 to carry out.

Images