DE102022211631A1 - Method for determining a spatially resolved evaluation of a detection condition - Google Patents

Abstract

A method (100) for determining a spatially resolved assessment of a detection condition in an environment of a sensor system (200) is described, comprising: providing at least one parameter (112, 114, 116) for a spatial assessment of the detection condition in the environment of the sensor (200); analyzing a spatial dependency (120) of the at least one parameter (112, 114, 116) for a spatial assessment of the detection condition; determining at least one influencing factor type (140) for the spatial dependency of the at least one parameter based on the spatial dependency of the at least one parameter; and evaluating the at least one detection condition (150) in the environment of the sensor based on the at least one influencing factor type and the spatial dependency of the at least one parameter to determine the spatially resolved assessment.

Description

State of the art

The automation of driving goes hand in hand with the equipping of vehicles with increasingly comprehensive and powerful sensor systems for detecting the environment. In some cases, vehicle sensors cover 360° of the environment and different ranges redundantly, using several sensors and sensor modalities. Sensor modalities used include video, radar, lidar, ultrasound and microphone sensors.

Sensor data is merged to form a secure environment model to represent the environment of the vehicle. The requirements for the scope and quality of the environment model depend on the driving functions implemented on it. In the driverless vehicle, for example, comprehensive driving decisions are made on the basis of the environment model and the actuators are controlled accordingly.

The draft standard ISO/PAS 21448 (Road vehicles - Safety of the Intended Functionality (SOTIF)) addresses a problem that the safety concept for environmental perception in autonomous systems (ADAS, AD) must take into account the performance or the capabilities and deficiencies of the exteroceptive sensors used. Safety-critical effects of such sensor deficiencies that must be avoided include incorrect measurements, false positives (FP) or false negatives (FN), which in turn can lead to "triggering conditions". ISO/PAS 21448 deals with processes with which triggers can be identified during development and then mitigated in the product.

Disclosure of the invention

On the other hand, it would be advantageous to have a method with which, during the operation of the at least partially automated vehicle, triggers of deficiencies, which depend in particular on the detection conditions, can be reliably detected and, in particular spatially resolved, taken into account in the safety concept.

The invention is based on determining detected information about a plurality of currently prevailing detection conditions affecting sensor systems in the environment of the sensor system and/or an at least partially automated vehicle during operation in a structured and easily usable format and providing it for a representation of the environment and/or a fusion of sensor data.

This can increase the security integrity (reliability, SOTIF) of the representation of the environment and/or the fusion result and avoid negative effects of individual deficiencies, which can result in an increase in the robustness of the environment detection and a faster generation of hypotheses about the environment.

According to aspects of the invention, a method for determining a spatially resolved evaluation of a detection condition in an environment of a sensor, a method for checking the plausibility of data from a sensor system, a method for fusing data from a first sensor system and a second sensor system, an evaluation device, a computer program, and a machine-readable storage medium are proposed according to the features of the independent claims. Advantageous embodiments are the subject of the dependent claims and the following description.

Throughout this description of the invention, the sequence of method steps is presented in such a way that the method is easy to understand. However, the person skilled in the art will recognize that many of the method steps can also be carried out in a different order and lead to the same or a corresponding result. In this sense, the order of the method steps can be changed accordingly. Some features are provided with counting words to improve readability or to make the assignment clearer, but this does not imply the presence of certain features.

According to one aspect of the invention, a method for determining a spatially resolved evaluation of a detection condition in an environment of a sensor system is proposed. In a first step of the method, at least one parameter for a spatial evaluation of the detection condition in the environment of the sensor can be provided. In a further step, a spatial dependency of the at least one parameter for a spatial evaluation of the detection condition can be analyzed. In a further step, at least one influencing factor type for the spatial dependency of the at least one parameter can be determined based on the spatial dependency of the at least one parameter. In a further step, the at least one detection condition in the environment of the sensor can be evaluated based on the at least one influencing factor type and the spatial dependency of the at least one parameter, in order in particular to evaluate the at least one detection condition in a spatially resolved manner.

Detection conditions can be conditions that affect the environment of a mobile platform and/or sensors or sensor systems, in particular including an analysis of data from the sensors or sensor systems, for representing an environment of the mobile platform with regard to a representation of the environment. These can, for example, alternatively or additionally be caused by weather conditions, lighting conditions, temperature, reflections of emitted signals, light irradiation, nature of the environment, disturbing images, disturbing surfaces, other road users, dynamic occlusions and/or disturbing radiation from static sources and/or from dynamic sources such as road users and/or a complexity of the environment.

By means of the spatially resolved evaluation of a detection condition, triggers of deficiencies in data from different sensor systems that are provided to an at least partially automated mobile platform can be reliably detected and, in particular, taken into account across sensors for a security concept.

A parameter for a spatial evaluation of the detection condition can be derived from data from at least one internal or external sensor and/or from geographical data, such as map data and/or infrastructure data and/or from data provided by other road users (V2V data).

• Globale Detektionsbedingungen kann die von Sensorsystemen bereitgestellten Daten beeinträchtigen, wie beispielsweise:
  • - Wetterverhältnisse, z.B. Regen bzw. Regendichte und/oder Schnee bzw.
• Schneedichte und/oder Nebel bzw. Nebel-dichte, etc.;
  • - Lichtverhältnisse, die abhängig von einer Tageszeit sein können und/oder abhängig von einer Bedeckung des Himmels, etc.;
  • - Nähe zu einer störenden Radar-Strahlungsquelle, wie beispielsweise z.B.
• Flughafen, Militärgelände); und
  • - Außentemperaturen.

A variety of detection conditions can affect the data provided by sensor systems:

• Global detection conditions can affect the data provided by sensor systems, such as:
  • - Weather conditions, e.g. rain or rainfall and/or snow or
• Snow density and/or fog or fog density, etc.;
  • - lighting conditions, which may depend on the time of day and/or on sky coverage, etc.;
  • - Proximity to an interfering radar radiation source, such as
• airport, military site); and
  • - Outside temperatures.

• - Zonen mit Tendenz zu Reflektionen, wie z.B. Tunnelwände und/oder Brücken und/oder Leitplanken und/oder Metallzäune und/oder Fensterfronten, etc.;
• - Straße in Abhängigkeit von Wasserbelag und/oder Regenmengen; und
• - Schatten durch dynamische oder statische Objekte, z.B. Gebäude je nach Sonnenstand.

Zone-related detection conditions can affect the data provided by sensor systems, such as:

• - Zones with a tendency to reflections, such as tunnel walls and/or bridges and/or guard rails and/or metal fences and/or window fronts, etc.;
• - road depending on water surface and/or rainfall; and
• - Shadows caused by dynamic or static objects, e.g. buildings depending on the position of the sun.

• - Verdeckungen durch Objekte, z.B. durch Gebäude; Bäume; Stromkästen, etc.;
• - Verdeckungen durch Straßentopografie, z.B. Kurven; Hügelkuppen;
• - Reflektierende Objekte, wie z.B. Verkehrsschilder, die Lidar-Systeme beeinflussen können, Stahlplatten, die Radar-Systeme beeinflussen können und beispielsweise auf einem Untergrund platziert sind;
• - Straßenbeschaffenheit, wie z.B. Schlaglöcher; Kanten, etc.;
• - Blendungen durch tiefstehende Sonne oder deren Reflektionen;
• - Bilder auf Litfaßsäulen, Werbetafeln, Schildern; und
• - Material/Textur von anderen Verkehrsteilnehmern.

Relative and/or local detection conditions can affect the data provided by sensor systems, such as:

• - Obscurations by objects, e.g. buildings, trees, electrical boxes, etc.;
• - Occlusions due to road topography, e.g. curves; hilltops;
• - Reflective objects, such as traffic signs that can affect lidar systems, steel plates that can affect radar systems and are placed on a substrate, for example;
• - Road conditions, such as potholes, edges, etc.;
• - Glare from low sun or its reflections;
• - Images on advertising columns, billboards, signs; and
• - Material/texture of other road users.

• - Dynamische Verdeckungen, wie z.B. durch andere Verkehrsteilnehmer;
• - Störungen durch Radar und/oder Lidar anderer Verkehrsteilnehmer; und
• - eine herrschende Verkehrsdichte, die entsprechend der resultierenden Komplexität Sensor-Messung und/oder deren Interpretation bei vielfältigen

und/oder sich „überschneidenden“ Objekten beeinträchtigen kann, etc.Furthermore, dynamic detection conditions can affect the data provided by sensor systems, such as:

• - Dynamic obscurations, e.g. by other road users;
• - interference from radar and/or lidar of other road users; and
• - a prevailing traffic density, which, according to the resulting complexity of sensor measurements and/or their interpretation in a variety of

and/or “overlapping” objects, etc.

The detection conditions can each be traced back to different types of influencing factors and relate to an environment of a sensor system and/or an environment of a platform, in particular a mobile platform.

• Sensordaten können eine Kenngröße für eine Bewertung von Detektionsbedingung sein:
  • - Dedizierte Sensoren für die Bestimmung der jeweiligen Detektionsbedingung, wie z.B. ein Regensensor oder ein Helligkeitssensor;
  • - explizite Mustererkennung mittels einzelner exterozeptiven Sensoren, wie z.B. typische Regen-Reflektionsmuster die mit Lidar-System bestimmt werden können;
  • - Mustererkennung und -interpretation durch Auswertung von Daten eines oder mehrerer Sensoren, beispielsweise mittels künstliche Intelligenz bzw. MLM; und
  • - Evidenzen, die implizit aus der Bewertung von Ergebnissen von Sensordatenverarbeitung, wie bei Pro-Processing oder Fusion ermittelt werden, wie z.B. hohes Rauschen.
• Kartendaten können eine Kenngröße für die Bewertung von Detektionsbedingung sein:
  • - statische und/oder dynamische Kartendaten;
  • - Numerische und/oder analytische Auswertung von Karten-Topographie, wie z.B. eine Bestimmung von Schattenflächen, abhängig vom Sonnenstand und/oder Sichtbereichen hinter einem Hügelkamm, etc.;
  • - Explizit markierte Zonen oder Objekte, d. h. „Points-of-interest“, wie z.B. Tunnel, Fensterfronten; andere reflektierende Objekte; Werbetafeln mit Bildern; Verkehrsschilder mit Ausrichtungswinkel; etc.; und
  • - Crowdsourcing (typische Effekte, Auslöser).
• Infrastruktur-Daten können eine Kenngröße für eine Bewertung von Detektionsbedingung sein:
  • Infrastruktur-Daten, wie z.B. Leitsysteme; Road Side Units; V2I;
    • - Sensordaten, wie oben aufgeführt, die aus Infrastruktur-Sensorsystemen bereitgestellt werden;
    • - Lokale Informationen, z.B. eine Reflektivität eines lokalen Straßenabschnitts bei Nässe; eine Position von reflektierenden Objekten; etc.;
    • - Informationen über Verkehrsdichte; Bestückung von Fahrzeugen im Umfeld mit und konkrete Ausrichtung von Lidar-Systemen und/oder Radar Systemen;
    • - Informationen anderer Verkehrsteilnehmer (V2V);
    • - Informationen über Fahrzeuge: Bestückung mit und konkrete Ausrichtung von Lidar-Systemen und/oder Radar Systemen; und
    • - Andere Verkehrsteilnehmer, insbesondere ohne AD-Funktionalität: Vom AV
• lesbarer „Marker“, z.B. Piktogramm, QRC Code, Radar-reflektierendes Kleidungsstück, etc. mit Informationen.

A parameter for evaluating detection conditions can be provided either individually or by a combination of, for example, the following sources:

• Sensor data can be a parameter for evaluating detection conditions:
  • - Dedicated sensors for determining the respective detection condition, such as a rain sensor or a brightness sensor;
  • - explicit pattern recognition using individual exteroceptive sensors, such as typical rain reflection patterns that can be determined with Lidar systems;
  • - Pattern recognition and interpretation by evaluating data from one or more sensors, for example using artificial intelligence or MLM; and
  • - Evidence implicitly obtained from the evaluation of results of sensor data processing, such as pro-processing or fusion, such as high noise.
• Map data can be a parameter for evaluating detection conditions:
  • - static and/or dynamic map data;
  • - Numerical and/or analytical evaluation of map topography, such as determination of shadow areas depending on the position of the sun and/or visibility areas behind a hill ridge, etc.;
  • - Explicitly marked zones or objects, ie “points of interest”, such as tunnels, window fronts; other reflective objects; billboards with images; traffic signs with orientation angles; etc.; and
  • - Crowdsourcing (typical effects, triggers).
• Infrastructure data can be a key parameter for evaluating detection conditions:
  • Infrastructure data, such as control systems; road side units; V2I;
    • - Sensor data, as listed above, provided from infrastructure sensor systems;
    • - Local information, e.g. reflectivity of a local road section in wet conditions; position of reflective objects; etc.;
    • - Information on traffic density; equipping of vehicles in the area with and specific orientation of lidar systems and/or radar systems;
    • - Information from other road users (V2V);
    • - Information about vehicles: equipment and specific orientation of lidar systems and/or radar systems; and
    • - Other road users, especially without AD functionality: From AV
• readable “marker”, e.g. pictogram, QRC code, radar-reflective garment, etc. with information.

Here are some examples of types of influencing factors that affect detection conditions and in particular can affect the detection condition.

• - Verdeckung von einem Bereich des Umfelds eines Sensorsystems kann beispielsweise durch Objekte, wie Gebäude, Bäume, Stromkosten und Ähnliches, hervorgerufen werden. Solche Verdeckungen können den Sichtbereich (englisch:
  • Field-of-View) eines Sensors einschränken. Darüber hinaus können Verdeckungen aus einer Topographie von Straßen, beispielsweise Kurven oder Hügelkuppen, resultieren die ungünstig für einen Sichtbereich, insbesondere in Bezug auf einen Sichtwinkel, eines Sensors sind.
  • Eine Kenngröße für die räumliche Bewertung der Detektionsbedingung kann aus einer topographischen Karte und/oder einem anderen Umfeldmodell analytisch abgeleitet werden.
  • Eine solche Verdeckung kann bewirken, dass zum Beispiel Objekte nicht erkannt werden und/oder ein freier Bereich fälschlicherweise erkannt wird (englisch: False Negatives).
  • Der betroffene räumliche Bereich im Umfeld eines Sensorsystems, d. h. eine Projektion der Wirkung eines solchen Einflussfaktor-Typs auf das Umfeld des Sensorsystems kann beispielsweise in einem dreidimensionalen räumlichen System aus Gitterzellen abgebildet werden. Dabei können die Betroffenen Gitterzellen in dem System aus Gitterzellen durch Strahlverfolgung (Raytracing) bestimmt werden. Dazu können alle Gitterzellen, durch die ein den verdeckten Bereich definierender Strahl, ausgehend von dem Sensorsystem oder der mobilen Plattform, verläuft, oder die sich im verdeckten Bereich zwischen zwei solcher Strahlen befinden, identifiziert werden. Diesen betroffenen Gitterzellen kann die entsprechende räumlich aufgelöste Bewertung der Detektionsbedingung zugeordnet werden. Messwerte von Sensoren, die den betroffenen räumlich aufgelösten Gitterzellen zugeordnet sind, hinsichtlich ihrer Konfidenz heruntergewichtet oder komplett verworfen werden.
    • - Blendung in einem Bereich des Umfelds eines Sensorsystems kann beispielsweise durch eine tiefstehende Sonne und/oder Licht-/Sonnen-Reflektion auf einem Objekt und/oder stark reflektierende Objekte, wie beispielsweise ein Lastwagen in Bezug auf ein Radarsensor und/oder aktive exterozeptive Sensoren anderer Verkehrsteilnehmer, hervorgerufen werden. Solche Blendungen können den Sichtbereich (englisch: Field-of-View) eines Sensors einschränken.
  • Eine Kenngröße für die räumliche Bewertung der Detektionsbedingung kann aus einer Position des Sensorsystems und/oder einer mobilen Plattform und entsprechenden astronomischen Daten und/oder mittels einer Umgebungs-Karte und/oder einem anderen Umfeldmodell analytisch abgeleitet werden.
  • Eine solche Blendung kann bewirken, dass zum Beispiel Bereiche des Umfelds durch die entsprechend geblendet den Sensoren nicht abgedeckt werden, und unter Umständen durch Algorithmen verursacht werden, wie beispielsweise CFAR bei einem Radar-Sensor.
  • Der betroffene räumliche Bereich im Umfeld eines Sensorsystems, d. h. eine Projektion der Wirkung eines solchen Einflussfaktor-Typs auf das Umfeld des Sensorsystems kann beispielsweise in einem dreidimensionalen räumlichen System aus Gitterzellen abgebildet werden. Dabei können die betroffenen Gitterzellen in dem System aus Gitterzellen durch in einem Kegelbereich um die relative Position der Sonne und/oder des stark reflektierenden Objekts identifiziert werden. Diesen betroffenen Gitterzellen kann die entsprechende räumlich aufgelöste Bewertung der Detektionsbedingung zugeordnet werden. Messergebnisse des beeinträchtigen Sensors bzgl. des dadurch betroffenen Messattributes, die den entsprechenden Gitterzellen zugeordnet werden, können hinsichtlich ihrer Konfidenz heruntergewichtet oder komplett verworfen werden.
    • - Eine starke Reflexion in einem Bereich des Umfelds eines Sensorsystems kann beispielsweise durch ein Verkehrsschild, insbesondere für einen Lidar-Sensor und/oder eine Stahlplatte auf dem Boden, für einen Radar-Sensor und/oder Leitplanken hervorgerufen werden.
  • Solche starken Reflexionen können den Sichtbereich (englisch: Field-of-View) eines Sensors einschränken.
  • Eine Kenngröße für die räumliche Bewertung der Detektionsbedingung kann aus einer Umgebung-Karte und/oder aus den jeweiligen Messungen eines gesamten Sets von Sensoren, die für eine Repräsentation des Umfelds des Sensorsystems eingerichtet sind und/oder einem Modell des Umfelds des Sensorsystems analytisch abgeleitet werden.
  • Eine solche starke Reflexion kann bewirken, dass zum Beispiel Geisterobjekte detektiert werden können und/oder eine Überstrahlung echter Objekte des Umfelds in der Repräsentation generiert werden und/oder unscharfe Konturen echter Objekte des Umfelds generiert werden.
  • Der betroffene räumliche Bereich im Umfeld eines Sensorsystems, d. h. eine Projektion der Wirkung eines solchen Einflussfaktor-Typs auf das Umfeld des Sensorsystems kann beispielsweise in einem dreidimensionalen räumlichen System aus Gitterzellen abgebildet werden. Dabei können die Gitterzellen in dem System aus Gitterzellen durch in einem definierten Bereich um das reflektierende Objekt betroffen sein, wodurch die Detektionsbedingung in den betroffenen Gitterzellen als beeinträchtigt bewertet werden können. Messergebnisse des beeinträchtigen Sensors bzgl. des dadurch betroffenen Messattributes, die den entsprechenden Gitterzellen zugeordnet werden, können hinsichtlich ihrer Konfidenz heruntergewichtet werden.

Examples of possible influencing factors that can be projected onto the grid cell system are:

• - Occlusion of an area of the environment of a sensor system can be caused by objects such as buildings, trees, electricity costs and the like. Such occlusions can limit the field of view (English:
  • Field-of-View) of a sensor. In addition, occlusions can result from the topography of roads, such as curves or hilltops, which are unfavorable for a sensor's field of view, particularly in terms of its viewing angle.
  • A parameter for the spatial evaluation of the detection condition can be analytically derived from a topographical map and/or another environmental model.
  • Such occlusion can, for example, result in objects not being detected and/or a free area being incorrectly detected (false negatives).
  • The affected spatial area in the environment of a sensor system, i.e. a projection of the effect of such an influencing factor type on the environment of the sensor system, can be mapped, for example, in a three-dimensional spatial system of grid cells. The affected grid cells in the system of grid cells can be determined by ray tracing. For this purpose, all grid cells through which a beam defining the hidden area, originating from the sensor system or the mobile platform, passes, or which are located in the hidden area between two such beams, can be identified. The corresponding spatially resolved evaluation of the detection condition can be assigned to these affected grid cells. Measured values from sensors that correspond to the affected spatially resolved grids cells are downweighted in terms of their confidence or rejected completely.
    • - Glare in an area surrounding a sensor system can be caused, for example, by a low sun and/or light/sun reflection on an object and/or highly reflective objects, such as a truck in relation to a radar sensor and/or active exteroceptive sensors of other road users. Such glare can limit the field of view of a sensor.
  • A parameter for the spatial evaluation of the detection condition can be analytically derived from a position of the sensor system and/or a mobile platform and corresponding astronomical data and/or by means of an environmental map and/or another environmental model.
  • Such glare can, for example, result in areas of the surroundings not being covered by the correspondingly glared sensors and can, under certain circumstances, be caused by algorithms such as CFAR in a radar sensor.
  • The affected spatial area in the environment of a sensor system, i.e. a projection of the effect of such an influencing factor type on the environment of the sensor system, can be mapped, for example, in a three-dimensional spatial system of grid cells. The affected grid cells in the system of grid cells can be identified by a cone area around the relative position of the sun and/or the highly reflective object. The corresponding spatially resolved evaluation of the detection condition can be assigned to these affected grid cells. Measurement results of the affected sensor with regard to the measurement attribute affected by this, which are assigned to the corresponding grid cells, can be weighted down in terms of their confidence or rejected completely.
    • - A strong reflection in an area of the environment of a sensor system can be caused, for example, by a traffic sign, in particular for a lidar sensor and/or a steel plate on the ground, for a radar sensor and/or guard rails.
  • Such strong reflections can limit the field of view of a sensor.
  • A parameter for the spatial evaluation of the detection condition can be analytically derived from an environment map and/or from the respective measurements of an entire set of sensors that are set up to represent the environment of the sensor system and/or a model of the environment of the sensor system.
  • Such a strong reflection can cause, for example, ghost objects to be detected and/or an overexposure of real objects in the environment to be generated in the representation and/or blurred contours of real objects in the environment to be generated.
  • The affected spatial area in the environment of a sensor system, i.e. a projection of the effect of such an influencing factor type on the environment of the sensor system, can be mapped, for example, in a three-dimensional spatial system of grid cells. The grid cells in the system of grid cells can be affected by a defined area around the reflecting object, whereby the detection condition in the affected grid cells can be assessed as impaired. Measurement results of the impaired sensor with regard to the measurement attribute affected by this, which are assigned to the corresponding grid cells, can be weighted down with regard to their confidence.

• - eine Beschaffenheit einer Straße, wie z.B. Schlaglöcher und/oder Kanten, etc.;
• - Bilder auf Litfaßsäulen und/oder Werbetafeln und/oder Schildern;
• - Material und/oder eine Textur von anderen Verkehrsteilnehmern.

Other types of influencing factors may relate to local conditions, such as

• - a condition of a road, such as potholes and/or edges, etc.;
• - Images on advertising columns and/or billboards and/or signs;
• - Material and/or texture from other road users.

• - Zonen mit einer Tendenz zu Reflektionen, zum Beispiel Tunnelwände und/oder Brücke und/oder Leitplanken und/oder Metallzäune und/oder Fensterfronten, etc.;
• - Straße in Abhängigkeit von Wasserbelag und/oder Regen;
• - Schatten durch dynamische oder statische Objekte, z.B. Gebäude je nach Sonnenstand.

Other influencing factor types can be zone-related, such as

• - Zones with a tendency to reflections, for example tunnel walls and/or bridges and/or guard rails and/or metal fences and/or window fronts, etc.;
• - Road depending on water surface and/or rain;
• - Shadows caused by dynamic or static objects, e.g. buildings depending on the position of the sun.

• - Dynamische Verdeckungen, z.B. durch andere Verkehrsteilnehmer;
• - Störungen durch Radar und/oder Lidar anderer Verkehrsteilnehmer;
• - Hohe Verkehrsdichte, beispielsweise aufgrund einer Komplexität der Messung und Interpretation der jeweiligen Messung bei vielfältigen und sich „überschneidenden“ Objekten, etc.

Other influencing factor types can relate to dynamic conditions, such as

• - Dynamic obscuring, e.g. by other road users;
• - Interference from radar and/or lidar of other road users;
• - High traffic density, for example due to the complexity of the measurement and interpretation of the respective measurement in the case of diverse and “overlapping” objects, etc.

In other words, by means of a geometric assignment of the spatially resolved evaluation of the detection condition, corresponding to a projection, an effect of external influencing factors on perceptual abilities can be described using a system of grid cells.

The described, particularly geometric, mapping of the evaluation of the detection condition makes it possible to locally isolate sensor or perception deficiencies caused by external effects.

This allows individual measurements to be spatially verified very precisely, while also preventing measurements in unnecessarily large spatial areas from being unjustifiably assessed as unsafe. This increases both the safety and the performance or availability of a perception system.

Advantageously, such a system of grid cells can be used to map and/or determine safety-relevant boundary conditions for the perception of the environment of the sensor system.

According to one aspect, it is proposed that the method additionally determines a spatial origin of the spatial dependence of the at least one parameter based on the spatial dependence of the at least one parameter, wherein the determination of the at least one influencing factor type for the spatial dependence of the at least one parameter is based on the spatial dependence of the at least one parameter and the spatial origin of the spatial dependence of the at least one parameter. Then, the evaluation of the at least one detection condition for the environment of the sensor can be based on the at least one influencing factor type and the spatial dependence of the at least one parameter and the spatial origin of the at least first parameter.

• In einem Schritt wird zumindest eine weiteren Kenngröße für die räumliche Bewertung der Detektionsbedingung in dem Umfeld des Sensors bereitgestellt. In einem weiteren Schritt wird eine räumliche Abhängigkeit der weiteren Kenngröße für eine räumliche Bewertung der Detektionsbedingung analysiert. Alternativ oder zusätzlich kann ein räumlicher Ursprung der räumlichen Abhängigkeit der weiteren Kenngröße, basierend auf der räumlichen Abhängigkeit der weiteren Kenngröße bestimmt werden. In einem weiteren Schritt kann zumindest ein Einflussfaktor-Typ für die räumliche Abhängigkeit der weiteren Kenngröße, basierend auf der räumlichen Abhängigkeit der einen Kenngröße und/oder der weiteren Kenngröße und/oder basierend auf dem räumlichen Ursprung der räumlichen Abhängigkeit der weiteren Kenngröße bestimmt werden. In einem weiteren Schritt kann die zumindest eine Detektionsbedingung in dem Umfeld des Sensors, basierend auf dem zumindest einen Einflussfaktor-Typ, und der räumlichen Abhängigkeit der einen Kenngröße und der weiteren Kenngröße und/oder basierend auf dem räumlichen Ursprung der räumlichen Abhängigkeit der einen Kenngröße und/oder der weiteren Kenngröße, bewertet werden, um insbesondere die räumlich aufgelöste Bewertung zu bestimmen.

According to one aspect, it is proposed that the method additionally comprises the following steps:

• In one step, at least one further parameter is provided for the spatial evaluation of the detection condition in the environment of the sensor. In a further step, a spatial dependence of the further parameter is analyzed for a spatial evaluation of the detection condition. Alternatively or additionally, a spatial origin of the spatial dependence of the further parameter can be determined based on the spatial dependence of the further parameter. In a further step, at least one influencing factor type for the spatial dependence of the further parameter can be determined based on the spatial dependence of the one parameter and/or the further parameter and/or based on the spatial origin of the spatial dependence of the further parameter. In a further step, the at least one detection condition in the environment of the sensor can be evaluated based on the at least one influencing factor type and the spatial dependence of the one parameter and the further parameter and/or based on the spatial origin of the spatial dependence of the one parameter and/or the further parameter, in order to determine in particular the spatially resolved evaluation.

According to one aspect, it is proposed that the method comprises the following steps: projecting the spatially resolved evaluation of the detection condition into a representation of the environment of the sensor system, in order to check the plausibility of sensor data of a sensor system, wherein in particular the projection of the spatially resolved evaluation of the detection condition into a representation of the environment of the sensor system depends on the specific influencing factor type.

Data from sensor systems can be processed in various steps to represent the environment of a vehicle, with the data being further abstracted with each processing step and ultimately being combined or fused to form a secure environment model. The common algorithms for different sensor modalities for object detection, object classification, object tracking, distance calculation, etc. can be vulnerable to input data that is incorrect, particularly due to the detection conditions under which it was determined.

In these cases, typical methods for object detection and object classification can cause incorrect determinations due to false-positive and false-negative determinations of environmental determinants.

This advantageously allows a projection of the evaluation of the detection condition onto grid cells according to the specific cause, in Form of an influencing factor type. For mapping to a three-dimensional system of grid cells, the algorithm for projection can be dependent on the type of influencing factor and/or dependent on an effect of the influencing factor type. In other words, if the influencing factor type has a global effect, for example in the case of glare, a global projection algorithm can be applied. If the influencing factor type has a local effect, such as spray behind a vehicle, a local projection algorithm can be applied accordingly.

The projection can be done using a projection algorithm that depends on the type of influencing factor, for example the projection algorithm can be global or local.

The global projection algorithm first determines a geometric area in the environment of the sensor system, or in the system of grid cells, that is influenced by the influencing factor. Subsequently, the evaluation of the detection conditions that depend on the type of influencing factor can be assigned to corresponding grid cells of the system of grid cells. Such a global projection can be used, for example, for an influencing factor type of glare from sunlight and/or occlusion by objects.

A local projection algorithm is based on a local localization of the type of influencing factor and can determine a direct projection onto the grid cell of the system of grid cells. Based on this, a local influence on surrounding grid cells can be determined using a local projection algorithm. This can be done, for example, in the form of Bayes networks. Such a local projection can be used, for example, for the influencing factor of spray from a vehicle.

According to one aspect, it is proposed that the at least one influencing factor type for the spatial dependence of the at least one parameter is determined additionally or alternatively by means of an influencing factor expert system based on the specific spatial origin and the specific spatial dependence of the at least one parameter. The influencing factor expert system can be based on a correspondingly set up database.

According to one aspect, it is proposed that the sensor system is mechanically coupled to a mobile platform, and in particular the representation of the environment of the sensor system is configured to be moved along with the sensor system.

According to one aspect, it is proposed that the method additionally comprises the following steps: In one step, a spatial system of grid cells is defined around the sensor system, which is in particular set up to be moved along with the sensor system. In a further step, the respective spatially resolved evaluation of the detection condition can be assigned to a spatially correspondingly assigned grid cell of the system of grid cells.

Advantageously, a reliability of the sensor technology, which is based on the spatial evaluation of the detection condition, can be mapped to the system of grid cells and in this form spatially assigned to a representation of the environment.

Since the evaluated detection condition can be different at different locations in the environment of a sensor system and/or a mobile platform, the method and the information content provided thereby can be improved by assigning the evaluated detection condition prevailing there to spatial coordinates, so that information about the environment of a sensor system and/or a mobile platform provided by a sensor system can be evaluated in a spatially resolved manner.

For this purpose, a system of grid cells can be defined around the sensor system and/or the mobile platform, with each grid cell being assigned a respective evaluated detection condition that is specific to the location of the grid cell. In this way, such spatially resolved evaluated detection conditions can be structured in the form of a corresponding three-dimensional matrix and provided in an easy-to-use format to other applications and/or modules and/or systems, e.g. for generating an environment of the sensor system or a mobile platform. This means that weighting factors for a fusion of data from sensor systems can be easily generated using such a structured matrix with the respective spatially assigned evaluated detection conditions.

According to one aspect, it is proposed that the influencing factor type comprises a global influencing factor and/or a zone-related influencing factor and/or at least one local influencing factor and/or at least one dynamic influencing factor.

Advantageously, this makes it possible to determine which grid cells of the system of grid cells are included in the spatial evaluation of the detection conditions.

According to one aspect, it is proposed that the at least one parameter for the spatial evaluation of the detection condition comprises data from at least one sensor from the environment of the sensor system and/or data from at least one geographical map with a reference to the environment of the sensor system and/or data from an infrastructure of the environment of the sensor system.

According to one aspect, it is proposed that the at least one parameter of the environment of the sensor system for a spatial evaluation of the detection condition is based on data from at least one sensor dedicated to the detection condition and/or is based on a pattern recognition for the detection condition of data from at least one exteroceptive sensor and/or is based on a pattern recognition for the detection condition of data from at least two exteroceptive sensors and/or is based on an evaluation of results of data processing of at least one sensor and/or is based on at least one detection condition that is associated with geographical data and describes an environment of a corresponding sensor in more detail and/or is based on an evaluation of map topographies and/or is based on data provided by road users in the environment of the corresponding sensor.

A method for checking the plausibility of data from a sensor system is proposed, wherein the detection conditions are evaluated in a spatially resolved manner using one of the methods described above. In a further step, the spatially resolved evaluation of the at least one detection condition is assigned to correspondingly spatially resolved data from the sensor system in order to check the plausibility of sensor data from a sensor system in an environment of the sensor system.

The different types of influencing factors, especially external ones, can have a detrimental effect on every element of a signal chain that evaluates signals from a sensor to determine the representation of the environment. This means that depending on the type of influencing factor, both signal recording, i.e. a physical measuring principle and/or a sensor system, and signal processing, such as processing the raw signals into sensor data and/or feature extraction, especially from the sensor data, e.g. using artificial intelligence, can be affected by the type of influencing factor.

Advantageously, the spatial evaluation of the detection conditions, which can affect a detection by means of a sensor and/or an evaluation algorithm for representing the environment of the sensor system, can be projected into the environment of the sensor system or the mobile platform according to a reliability of the sensor system on grid cells of a three-dimensional system of grid cells.

According to one aspect, it is proposed that in the plausibility check method, the plausibility check of the sensor data of the sensor system is carried out by means of a perception expert system which describes how the evaluated at least one detection condition affects a perception for a representation of the environment of the sensor system.

Advantageously, with this method for plausibility checks using reliably recorded information on the influencing factors, such as a robust sensor and/or map material and/or a V2X, etc., and a perception expert system, corresponding to a knowledge base that describes how the influencing factors affect the existing perception of the sensor system and/or the algorithms affect a representation of the environment, the reliability of the sensor system can be projected onto the three-dimensional system of grid cells around the sensor system or the mobile platform. For the mapping or projection onto the three-dimensional system of grid cells, an adapted projection algorithm can be used according to the type of influencing factor.

A method for fusing data from a first sensor system and a second sensor system is proposed, which method comprises the following steps: in one step, the spatially resolved evaluation of the at least one detection condition is determined based on the at least first influencing factor type and the spatial dependence of the at least first parameter and/or the spatial origin of the at least first parameter according to one of the methods described above for the first sensor system and/or the second sensor system. In a further step, the data from the first sensor system and the second sensor system are evaluated based on the spatially resolved evaluation of the at least one detection condition for the respective sensor system and/or the spatial origin of the at least first parameter to fuse the data from the first sensor system and the second sensor system in order to determine a representation of an environment of at least one of the two sensor systems.

With the spatially resolved evaluation of the detection condition, a sensor fusion system can use information about the detection condition to combine the data from different sensors systems according to the current situation regarding the detection conditions. If, for example, a sensor system cannot detect objects in the environment or can only detect them poorly based on the evaluated detection condition, a non-detection of the object by this sensor system cannot lead to the object hypothesis established by other sensor systems after detection of this object being reduced in its probability of having detected the object correctly. This can be achieved by reducing the weighting of the poorly detecting sensor system in the fusion accordingly.

According to one aspect, it is proposed that in the method for fusing data, the data of the first sensor system and the data of the second sensor system are weighted based on the spatially resolved evaluation of the at least one detection condition for representing the environment.

A method is proposed which provides a control signal for controlling an at least partially automated vehicle by means of an evaluation of the at least one detection condition determined by one of the methods described above. Alternatively or additionally, the method can provide a warning signal for warning a vehicle occupant based on the at least one evaluation of the at least one detection condition.

The term "based on" is to be understood broadly in relation to the feature that a control signal is provided based on an evaluated detection condition. It is to be understood that the evaluated detection condition is used for any determination or calculation of a control signal, although this does not exclude the possibility that other input variables are also used for this determination of the control signal. This applies accordingly to the provision of a warning signal.

Highly automated systems can, for example, use such a control signal to initiate a transition to a safe state, for example by causing an at least partially automated vehicle to slowly stop on a hard shoulder.

An evaluation device for at least one detection condition is proposed, which is based on at least one parameter for a spatial evaluation of the detection condition in the environment of the sensor, wherein the evaluation device is set up with a calculation unit to carry out one of the methods described above for determining the spatially resolved evaluation of the detection condition.

Such an evaluation device can be used to reliably determine and provide explicit knowledge about currently prevailing detection conditions in the environment of an at least partially automated vehicle that can affect sensor systems.

The use of an evaluation device as described above for fusing data from a first sensor system and a second sensor system to determine a representation of an environment of at least one of the two sensor systems is proposed.

A computer program is proposed which comprises instructions which, when the computer program is executed by a computer, cause the computer to carry out one of the methods described above.

Such a computer program enables the use of the described procedure in different systems.

A machine-readable storage medium is proposed on which the computer program described above is stored. The computer program described above is transportable by means of such a machine-readable storage medium.

Examples of implementation

• 1 skizziert schematisch ein Verfahren zur Plausibilisierung von Daten eines Sensorsystems mit Schritten eines Verfahrens zum Bestimmen einer räumlich aufgelösten Bewertung einer Detektionsbedingung in einem Umfeld eines Sensorsystems;
• 2 skizziert schematisch ein Umfeld eines Sensorsystems wobei eine Detektionsbedingung durch einen Einflussfaktor-Typ:
  • Sonneneinstrahlung beeinflusst ist;
• 3 skizziert schematisch ein Umfeld eines Sensorsystems wobei eine Detektionsbedingung durch einen Einflussfaktor-Typ: Reflexionen beeinflusst ist; und
• 4 skizziert schematisch ein Umfeld eines Sensorsystems wobei eine Detektionsbedingung durch einen Einflussfaktor-Typ: Gischt beeinflusst ist.

Embodiments of the invention are described with reference to the 1 to 4 and explained in more detail below. They show:

• 1 schematically outlines a method for checking the plausibility of data of a sensor system with steps of a method for determining a spatially resolved evaluation of a detection condition in an environment of a sensor system;
• 2 schematically outlines an environment of a sensor system where a detection condition is determined by an influencing factor type:
  • is influenced by solar radiation;
• 3 schematically outlines an environment of a sensor system where a detection condition is influenced by an influencing factor type: reflections; and
• 4 schematically outlines an environment of a sensor system where a detection condition is influenced by an influencing factor type: spray.

1 schematically outlines a method 100 for determining a spatially resolved evaluation of a detection condition in an environment of a sensor system 200. In a first step, three parameters 112, 114, 116 are provided for a spatial evaluation of the detection condition in the environment of the sensor 200. This can, for example, relate to a parameter 112 that is based on a traffic network (V2X; English, vehicle-to-everything). Alternatively or additionally, it can relate to a parameter 114 that is based on an evaluation of map topographies. Alternatively or additionally, it can relate to a parameter that is based on at least one exteroceptive sensor of a mobile platform, wherein the sensor system 200 is mechanically coupled to the mobile platform.

In a further step 120, a spatial dependence of the at least one characteristic variable 112, 114, 116 can be analyzed for a spatial evaluation of the detection condition.

In a further step 130, a spatial origin of the spatial dependence of the at least one characteristic variable 112, 114, 116 can be determined based on the spatial dependence of the at least one characteristic variable 112, 114, 116.

In a further step 140, an influencing factor type for the spatial dependence of the at least one characteristic variable can be determined based on the spatial dependence of the at least one characteristic variable and/or the spatial origin of the spatial dependence of the at least one characteristic variable 112, 114, 116.

In a further step 150, the at least one detection condition in the environment of the sensor can be evaluated based on the at least one influencing factor type and the spatial dependence of the at least one characteristic and/or the spatial origin of the at least first characteristic 112, 114, 116 in order to evaluate the one detection condition in a spatially resolved manner.

In a further step 160, the spatially resolved evaluation of the detection condition can be projected into a representation of the environment of the sensor system in order to make sensor data of a sensor system plausible. The projection of the spatially resolved evaluation of the detection condition into a representation of the environment of the sensor system can depend on the specific influencing factor type.

In a further step 170, data from a sensor system can be checked for plausibility by assigning the spatially resolved evaluated detection condition to corresponding spatially resolved data from the sensor system. The plausibility check can be carried out based on a perception expert system, which describes how the evaluated at least one detection condition affects a perception for a representation of the environment of the sensor system.

In a further step 180, a spatial system of grid cells 210 can be defined around the sensor system 200, wherein the system of grid cells 210 can be configured to move with the sensor system 200.

The respective spatially resolved evaluation of the detection condition can then be assigned to a spatially assigned grid cell 220 of the system of grid cells 210.

The 2 schematically outlines, in a top view and two-dimensionally, a three-dimensional scene of a detection condition, where one type of influencing factor is based on the influence caused by glare caused by the sun. This glare from the sun can influence, and in particular limit, individual video-based perception functions. In the system of three-dimensional grid cells 210, which are shown here two-dimensionally, the gray hatched grid cells 240 indicate an area in which the detection condition for sensors can be assessed as limited.

The lines emanating from the sensor system 100 outline in the 2 and 3 a viewing angle of the sensor system 100. For example, the three-dimensional grid can have cells of a size of 10×10×10cm ³ .

The 3 sketches schematically, in a top view and two-dimensionally, a three-dimensional scene of a detection condition, where an influence factor type is based on the influence caused by strong reflections from a truck, with respect to a radar-based perception function, e.g. due to CFAR Threshold.

In the system of three-dimensional grid cells 210 of the 3 , which are shown here two-dimensionally, the grey hatched grid cells 240 indicate an area in which the detection condition for sensors can be assessed as restricted.

The 4 sketches schematically, in a top view and two-dimensionally, a three-dimensional scene of a detection condition, where a Influence factor type is based on the influence caused by spray behind a vehicle. In this case, only the grey marked grid cells 240 are to be considered restricted, which are directly affected by the spray.

Claims (15)

Method (100) for determining a spatially resolved assessment of a detection condition in an environment of a sensor system (200), comprising: Providing at least one parameter (112, 114, 116) for a spatial assessment of the detection condition in the environment of the sensor (200); Analyzing a spatial dependency (120) of the at least one parameter (112, 114, 116) for a spatial assessment of the detection condition; Determining at least one influencing factor type (140) for the spatial dependency of the at least one parameter based on the spatial dependency of the at least one parameter; and Evaluating the at least one detection condition (150) in the environment of the sensor based on the at least one influencing factor type and the spatial dependency of the at least one parameter to determine the spatially resolved assessment. Procedure 100 according to Claim 1 , additionally comprising: determining a spatial origin (130) of the spatial dependence of the at least one characteristic variable (112, 114, 116) based on the spatial dependence of the at least one characteristic variable (112, 114, 116); and wherein determining the at least one influencing factor type (140) for the spatial dependence of the at least one characteristic variable (112, 114, 116) is based on the spatial dependence of the at least one characteristic variable (112, 114, 116) and the spatial origin (130) of the spatial dependence of the at least one characteristic variable (112, 114, 116); and wherein the evaluation of the at least one detection condition (150) for the environment of the sensor is based on the at least one influencing factor type and the spatial dependence of the at least one characteristic variable (112, 114, 116), and the spatial origin (130) of the at least first characteristic variable (112, 114, 116). Method (100) according to one of the preceding claims, comprising: Providing at least one further characteristic (112, 114, 116) for the spatial evaluation of the detection condition in the environment of the sensor; Analyzing a spatial dependency of the further characteristic (112, 114, 116) for a spatial evaluation of the detection condition; and/or Determining a spatial origin of the spatial dependency of the further characteristic (112, 114, 116) based on the spatial dependency of the further characteristic (112, 114, 116); Determining at least one influencing factor type (140) for the spatial dependence of the further characteristic variable (112, 114, 116) based on the spatial dependence of the one characteristic variable (112, 114, 116) and/or the further characteristic variable (112, 114, 116) and/or based on the spatial origin of the spatial dependence of the further characteristic variable (112, 114, 116); and Evaluating the at least one detection condition (150) in the environment of the sensor based on the at least one influencing factor type and the spatial dependence of the one characteristic variable (112, 114, 116) and the further characteristic variable (112, 114, 116) and/or based on the spatial origin of the spatial dependence of the one characteristic variable (112, 114, 116) and/or the further characteristic variable (112, 114, 116) to determine the spatially resolved evaluation. Method (100) according to one of the preceding claims, comprising: Projecting the spatially resolved evaluation (160) of the detection condition into a representation of the environment of the sensor system, for the plausibility of sensor data of a sensor system, wherein in particular the projection of the spatially resolved evaluation of the detection condition into a representation of the environment of the sensor system depends on the specific influencing factor type. Method (100) according to one of the preceding claims, wherein the at least one influencing factor type for the spatial dependence of the at least one characteristic is additionally or alternatively determined by means of an influencing factor expert system based on the determined spatial origin and the determined spatial dependence of the at least one characteristic. Method (100) according to one of the preceding claims, wherein the influencing factor type comprises a global influencing factor and/or a zone-related influencing factor and/or at least one local influencing factor and/or at least one dynamic influencing factor. Method (100) according to one of the preceding claims, wherein the at least one parameter (112, 114, 116) for the spatial evaluation of the detection condition comprises data from at least one sensor from the environment of the sensor system (200) and/or data from at least one geographical map with a reference to the environment of the Sensor system (200) and/or data from an infrastructure of the environment of the sensor system (200). Method (100) according to one of the preceding claims, wherein the at least one characteristic (112, 114, 116) of the environment of the sensor system (200) for a spatial evaluation of the detection condition (150) is based on data from at least one sensor dedicated to the detection condition and/or is based on a pattern recognition for the detection condition of data from at least one exteroceptive sensor and/or is based on a pattern recognition for the detection condition of data from at least two exteroceptive sensors and/or is based on an evaluation of results of data processing of at least one sensor and/or is based on at least one detection condition that is associated with geographical data and describes an environment of a corresponding sensor in more detail and/or is based on an evaluation of map topographies and/or is based on data provided by road users in the environment of the corresponding sensor. Method (100) for checking the plausibility of data from a sensor system (180), wherein the detection conditions are evaluated in a spatially resolved manner according to one of the preceding claims; and assignment of the spatially resolved evaluation of the at least one detection condition to correspondingly spatially resolved data from the sensor system, for checking the plausibility of sensor data from a sensor system. Procedure (100) according to Claim 9 , wherein the plausibility of the sensor data of the sensor system is carried out by means of a perception expert system (170) which describes how the evaluated at least one detection condition affects a perception for a representation of the environment of the sensor system Method (100) for fusing data of a first sensor system and a second sensor system, comprising: determining the spatially resolved evaluation of the at least one detection condition based on the at least first influencing factor type and the spatial dependence of the at least first characteristic variable and/or the spatial origin of the at least first characteristic variable, according to one of the Claims 1 until 8th for the first sensor system and/or the second sensor system; evaluating the data of the first sensor system and the second sensor system, based on the spatially resolved evaluation of the at least one detection condition for the respective sensor system and/or the spatial origin of the at least first characteristic variable, for fusing the data of the first sensor system and the second sensor system in order to determine a representation of an environment of at least one of the two sensor systems. Procedure (100) according to Claim 11 , wherein the data of the first sensor system and the data of the second sensor system are weighted based on the spatially resolved evaluation of the at least one detection condition for representing the environment. Method (100) which, based on a method according to one of the Claims 1 until 8th certain evaluation of the at least one detection condition, provides a control signal for controlling an at least partially automated vehicle; and/or based on the at least one evaluation of the at least one detection condition, provides a warning signal for warning a vehicle occupant. Computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method according to one of the Claims 1 until 12 to execute. Machine-readable storage medium on which the computer program is Claim 14 is stored.

Images