DE102019214345B4 - Method for detecting misalignment of at least one sensor of a motor vehicle - Google Patents

Abstract

Method for misalignment detection of at least one sensor (2) of a motor vehicle (K1), which is used to detect objects in the area surrounding the motor vehicle (K1), with at least the following method steps: It is checked in which current initial situation the motor vehicle (K1) is located .• If there is an initial situation (S2, S3, Sx) which corresponds to a stored initial situation defined as suitable, the sensor (2) generates an evaluation of the objects detected during the journey at a specific time interval (ZI). and stored measured values (M(t1)-M(tn)).• Each of the detected measured values (M(t1)-M(tn)) is related to the time (t1-tn) of its detection and to a relative position of the motor vehicle (K1) relative to a measuring location (MO) causing the measured value (M(t1)-M(tn)) at this point in time (t1-tn).• At least measured values (M(t1)) of a first point in time (t1 ) with the meter th (M(t2)...M(tn)) of a second, later point in time (t2...tn).• From the comparison of the measured values (M(t1), M(t2), M(tn)) it is decided whether or not there is a misalignment (α) of the sensor (2), characterized in that• the stored measured values (M(t1)...M(tn)) are used to generate measurement images (MB1, MB2) which represent at least parts of its contour (K) and/or any pattern at different points in time (t1...tn) for each detected object, measurement images (MB2) with contours (K(t1), K(t2), K( tn)) of at least one object and/or with patterns that are apart are classified as a bad measurement and thus as maladjustment and measurement images (MB1) with contours (K(t1), K(t2), K(tn)) that are locally superimposed and/or with superimposed ones Patterns as a good measurement, where a threshold value or tolerance range is defined and contours (K(t1), K(t2), K(tn)) and/or patterns are determined to be apart if they w are further apart than the threshold.

Description

The invention relates to a method for detecting misalignment of at least one sensor of a motor vehicle.

From the DE 10 2011 084 264 A1 a calibration method for an environment sensor has become known. The environment sensor generates sensor data, which in particular includes images and/or videos. The sensor data are compared with reference sensor data from a database. The sensor is calibrated on the basis of this comparison.

In the DE 10 2016 223 581 A1 describes a method for checking the functionality of a sensor. In this case, sensor data are first provided by the sensor. Predetermined test data stored in a memory are then linked to the sensor data in order to obtain manipulated data. The manipulated data is then processed using a processing specification and the processed data is compared with predetermined reference data in order to check the functionality of the sensor.

From the EP 2 604 478 A1 discloses a method for detecting functional errors in a multi-sensor arrangement. The multi-sensor arrangement includes at least two sensors that monitor the same object located in the current vehicle environment. The data from the multi-sensor arrangement are merged and a current performance variable is determined from this. The current performance level is compared to a corresponding performance level expected for the error-free multi-sensor array. From the comparison, a decision can be made as to whether there is a functional error in the relevant sensor or sensors in the multi-sensor arrangement. The expected power level is determined using a method learned offline.

Of the DE 10 2016 223 290 A1 a calibration method for a sensor of a vehicle can be inferred, in which sensor data of the sensor to be calibrated are compared with recorded environmental data from another sensor. Calibration data for the sensor to be calibrated are generated from the comparison.

Of the DE 10 2016 221 440 A1 A diagnostic procedure for environment sensor systems in vehicles can be found. In this case, reference data from reference objects are stored on an external server. The reference data is sent to a motor vehicle via a so-called ITS system (Intelligent Transportation System), which networks the vehicles involved in traffic and the institutions responsible for the traffic infrastructure. There, the reference data are compared with data actually recorded for the reference object.

The having the features of the preamble of claim 1 applicable DE 10 2009 031 561 A1 describes a method for detecting and correcting a misalignment of a distance sensor of a vehicle which is driving past parked vehicles in a specific direction of travel. Specifically, in the method, a signal profile consisting of a large number of distance points is generated by scanning the parked vehicles moving relative to the distance sensor. Geometric data of a parking space are in turn extracted from the signal profile. The signal profile is observed over a certain period of time and evaluated statistically, with a pronounced asymmetry of the signal profile being interpreted as an indication of a misalignment of the distance sensor. Above a predefinable threshold value for the asymmetry of the signal profile, an indication is given that the distance sensor is incorrectly positioned and needs to be repaired.

From the DE 10 2005 059 902 A1 a method for detecting the sensor status of a distance sensor mounted on the side of a vehicle is known. In the method, sensor signals that correlate with the distance of the vehicle from a roadway boundary are first generated. A large number of time-dependent and/or location-dependent distance values are then determined from the generated sensor signals and stored in a memory of the vehicle. A statistical evaluation of the stored distance values then takes place, the result of which leads to the generation of a corresponding sensor status signal. The statistical evaluation also includes, among other things, a comparison of a time sequence of distance values with specified standard profiles. Furthermore, depending on the sensor status signal, a warning signal can be output via a warning signal generator.

Finally is out of the DE 10 2016 219 455 A1 another method for checking the surroundings of a motor vehicle has become known. Specifically, the sensor data recorded by means of at least one sensor are transmitted to a device in an infrastructure. The facility compares the sensor data to reference data that is correlated to a specific test scenario. The test scenario is part of a locally limited test zone, which is located in a predetermined position. The sensor data currently recorded by the vehicle are only then activated the device transmits when the vehicle is in the check zone.

Based on the prior art mentioned, it is the object of the present invention to provide a method for detecting misalignment of at least one sensor of a motor vehicle, which is already possible with a single sensor and in which reference data for an object to be detected can be dispensed with. Furthermore, it is an object of the invention to provide a method for misalignment detection, which is very flexible regardless of the type of objects detected and allows a general procedure.

The present object is achieved by a method having the features of patent claim 1. Advantageous versions or developments of the method can be found in the dependent claims.

• • Es wird zunächst überprüft, in welcher aktuellen Ausgangs- beziehungsweise Fahrsituation sich das Kraftfahrzeug befindet. Insbesondere wird geprüft, ob sich das Kraftfahrzeug bewegt oder nicht. Es hat sich nämlich gezeigt, dass lediglich bestimmte Fahrsituationen besonders geeignet sind, um eine Dejustage beziehungsweise fehlerhafte Messwerte ermitteln zu können. Hierauf wird später noch eingegangen.
• • Bei Vorliegen einer Ausgangssituation, welche mit einer abgespeicherten, als geeignet festgelegten Ausgangssituation übereinstimmt, erfolgt eine Auswertung der durch den Sensor während der Fahrt in einem bestimmten Zeitintervall durch Erfassung von Objekten generierten und abgespeicherten Messwerte. Eine Auswertung der ständig erzeugten Messwerte zwecks einer Dejustageerkennung erfolgt also lediglich bei Vorliegen einer geeigneten Ausgangssituation, insbesondere wenn sich das Kraftfahrzeug bewegt und somit Messwerte zu verschiedenen Zeitpunkten erfasst werden, an welchen sich das Kraftfahrzeug an verschiedenen Positionen befindet. Beispielsweise kann eine Geradeausfahrt oder eine Kurvenfahrt als geeignet bestimmt werden, wobei diese insbesondere Fahrsituationen insbesondere als geeignet bewertet werden, während eine statische Fahrsituation, insbesondere bei einem stehenden Fahrzeug, als nicht geeignet bewertet werden kann. Insbesondere sind solche Fahrsituationen geeignet, bei denen sich das Kraftfahrzeug bewegt; in diesem Fall verändert sich die Position des Kraftfahrzeugs an aufeinanderfolgenden Zeitpunkten. Bei den Objekten handelt es sich vorzugsweise um stationäre Objekte im Umfeld des Kraftfahrzeugs, also beispielsweise parkende Kraftfahrzeuge, feststehende Hindernisse und dergleichen. Das heißt, wenn sich das Kraftfahrzeug bewegt, verändert sich seine Position relativ zu den Objekten.
• • Jeder der erfassten Messwerte wird in eine Beziehung zum Zeitpunkt seiner Erfassung gesetzt. Des Weiteren wird jeder der erfassten Messwerte in eine Beziehung zu einer Relativposition des Kraftfahrzeugs (Abstand, Ausrichtung) relativ zu einem den Messwert verursachenden Messort zu diesem Zeitpunkt gesetzt.

According to one aspect of the invention, a method for detecting misalignment of at least one sensor of a motor vehicle is proposed, the sensor being used to detect objects in the area surrounding the motor vehicle. The method according to the invention has at least the following method steps:

• • It is first checked in which current initial or driving situation the motor vehicle is located. In particular, it is checked whether the motor vehicle is moving or not. This is because it has been shown that only certain driving situations are particularly suitable for being able to determine misalignment or erroneous measured values. This will be discussed later.
• • If there is an initial situation that matches a stored initial situation defined as suitable, the measured values generated and stored by the sensor during the journey at a specific time interval by detecting objects are evaluated. The constantly generated measured values are evaluated for misalignment detection only if there is a suitable initial situation, in particular if the motor vehicle is moving and measured values are therefore recorded at different points in time at which the motor vehicle is in different positions. For example, driving straight ahead or cornering can be determined as suitable, with these particular driving situations being evaluated as particularly suitable, while a static driving situation, particularly when the vehicle is stationary, can be evaluated as not suitable. Driving situations in which the motor vehicle is moving are particularly suitable; in this case the position of the motor vehicle changes at successive points in time. The objects are preferably stationary objects in the vicinity of the motor vehicle, for example parked motor vehicles, stationary obstacles and the like. That is, when the motor vehicle moves, its position changes relative to the objects.
• • Each of the measured values recorded is related to the time it was recorded. Furthermore, each of the measured values recorded is related to a relative position of the motor vehicle (distance, alignment) relative to a measurement location causing the measured value at this point in time.

In other words, it is determined in which position the motor vehicle is located relative to a detected object or a measurement location at a specific measurement time. In particular, it is also taken into account how the absolute position of the motor vehicle changes during its movement. For this purpose, vehicle odometry can be used, for example, to determine how the position and orientation of the motor vehicle changes while driving. Vehicle odometry describes a method of estimating the situation, i.e. the position and orientation, of a mobile system. For example, the direction and the amount of a speed of the system as well as any accelerations are taken into account in order to determine a change in the position. In particular, vehicle odometry enables the position, orientation and driving condition of a motor vehicle to be determined at specific points in time. Measured variables from the wheel sensors, the yaw rate sensors and also the steering sensors (steering angle), for example, can be used as input variables. In addition, vehicle navigation data (GPS data) can also be used.

• • Es werden wenigstens Messwerte eines ersten Zeitpunktes mit den Messwerten eines zweiten, späteren Zeitpunktes verglichen.
• • Aus dem Vergleich der Messwerte wird entschieden, ob eine Dejustage des Sensors vorliegt oder nicht. Unter Dejustage wird insbesondere ein Winkelversatz des Sensors von seiner korrekten Position verstanden.

This makes it possible to determine the spatial relationship between measured values that are recorded at different points in time and at different positions on the vehicle. It can thus be determined, for example, which displacement or rotation of the position of an object relative to the vehicle is due to the motor vehicle's own movement. With suitable compensation, the measured values recorded at different points in time can thus be compared with one another, transformed into a common coordinate system, or entered into a map.

• • At least measured values from a first point in time are compared with measured values from a second, later point in time.
• • By comparing the measured values, a decision is made as to whether the sensor is misadjusted or not. Misalignment is understood to mean, in particular, an angular displacement of the sensor from its correct position.

The above-mentioned method steps thus provide a method by which misalignment of a sensor can be determined even without using a plurality of sensors or a sensor cluster and without using reference data.

A method is conceivable in which measured values relating to objects in the area surrounding the motor vehicle are recorded and stored at a time interval while the motor vehicle is being driven. The recorded measured values are each assigned a point in time when they were recorded, and a relative position of the vehicle relative to a measurement location causing the measured value is determined for the recorded measured values. In particular, the measurement location causing the measured value is arranged on a surface of an object. It is checked whether the motor vehicle is in motion, and in this case it is determined on the basis of measured values from a first point in time and measured values from a second, later point in time whether the sensor is misaligned or not. In particular, a type of misalignment is determined, for example a position that is shifted and/or rotated in relation to a calibrated position or another change in the sensor in relation to an intended or calibrated state.

The following features can be freely combined with the various aspects of the invention.

A misalignment of the sensor can be pronounced in different ways. For example, the sensor acquires measurement values, each of which is assigned a measurement location, and the measurement values are linked to information relating to a position of the measurement location, in particular relative to the position and orientation of the sensor. For example, a sensor measures the intensity of the light impinging from a specific direction, with information about the direction of the incident light being recorded; in this way, for example, different pixels of a camera sensor detect light coming from different directions. Similarly, a laser scanner can detect reflective properties of surfaces in the environment as a function of an angle relative to the laser scanner. To evaluate the recorded measured values by the motor vehicle, the information provided by the sensor, in particular about the relative position of the measurement location, is evaluated and related to the motor vehicle. This is done in particular by means of a calibration, in which it is determined, for example, how the information about positions of a measurement location detected by the sensor can be transformed into a vehicle coordinate system. For example, a position and/or alignment of a sensor relative to the motor vehicle is determined.

A misalignment can now relate to a deviation in the position and/or orientation of the sensor in relation to the calibration, for example if a sensor is displaced and/or rotated in relation to an initial position. The misalignment can also affect a change in the imaging properties of the sensor, for example if the optics of a camera are changed compared to a calibrated state or the functionality of a lidar or radar scanner is changed in such a way that the acquisition of information about the position of the measurement locations changes, for example in the case of a distorted image of the measured values recorded.

According to a first embodiment of the method, it is proposed that the decision as to whether or not misalignment is present is based on a previously learned and stored decision logic. On the one hand, this contributes to the fact that ready-made reference data from certain objects are no longer necessary and, on the other hand, an "intelligence" is made available that allows a very flexible adaptation to the most diverse measurement situations. In particular, a machine learning method is used. The decision logic can include, for example, instructions for classifying the measured values using a trained classifier. In a further development of the inventive idea, it is conceivable that the stored decision logic is learned beforehand from measurement images, which are generated on the one hand by measurement runs of a motor vehicle with a deliberately misaligned sensor and on the other hand by measurement runs of a motor vehicle with a properly adjusted sensor. The measurement drives for learning the decision logic can preferably be carried out with one and the same motor vehicle. A “measurement image” refers in particular to a total of measured values of a sensor recorded at a point in time or at a position. During the training, measurement images are generated during the measurement runs with a misaligned sensor, which are defined or classified as bad measurements, ie their classification as bad measurements is already known during the acquisition. During the measurement runs with a properly adjusted sensor, measurement images are generated, which are defined as good measurements are defined or classified and classified accordingly. As a result, the decision logic that supports the method can be learned in a simple manner or trained using a machine learning method.

In a further development of the method, measurement images that were generated by means of a simulation can be used as an alternative or in addition to learning the decision logic. Methods known per se are used for this purpose, such as statistical methods or physical models of the sensor with which the method is to be carried out. In particular, measurement images classified as good or bad measurements are generated in this way, by simulating measurement values for a correctly or incorrectly adjusted sensor.

In particular, once the decision logic has been learned, it can do without reference data. It is no longer necessary to store and store reference data, for example data from specific objects.

According to an implementation of the method according to the invention, measurement images are generated from the measured values stored during the journey, which images represent at least parts of the contour of each detected object at different points in time. Measurement images with locally spaced contours of at least one object, in which at least one object is not only essentially assignable to a contour, are classified as bad measurements and thus as maladjustment. Measurement images with locally superimposed contours, in which only one clearly visible contour results from the measurement image for each object, are assessed as a good measurement. This means that measured images are compared in which an object was recorded at different points in time. In particular, a threshold or tolerance range is defined and the detected contours of the object are determined to be spaced apart if they are further apart than the threshold.

In particular, a movement of the motor vehicle with the sensor is taken into account, since the vehicle is in different positions at the different points in time. When determining whether a detected object was detected at different points in time with contours that are locally one above the other or apart, the motor vehicle's own movement is compensated for, which is determined, for example, by means of odometry.

As an alternative or in addition to contours of detected objects, the method can be applied to any pattern that can be determined using the detected measured values and that can be learned, for example, by training a machine learning system. In this case, it is determined in particular whether the patterns were recorded at different points in time at different positions, and it is accordingly decided, analogously to the above description, whether there is a misalignment or a good measurement. It does not matter what type of object it is and whether, for example, contours or other features of the object can be detected. This provides a very flexible and general way of doing it.

As mentioned at the beginning, the procedure first checks whether there is a suitable initial situation for the evaluation. It has been shown that it is very advantageous and with the help of odometry that the relative position of the motor vehicle to measurement objects can be determined particularly reliably during a measurement interval if driving through a straight road and/or cornering is considered a suitable starting situation.

The method can be designed in such a way that a specific misalignment value is determined and, if a threshold value of the determined misalignment value is exceeded, an autocalibration or an autocorrection of the sensor is carried out. A misalignment value is given in particular by an angular error or angular offset of a sensor from its correct installation position. This makes it possible for a misalignment of the sensor to be corrected within certain limits, for example in the sense of a “self-healing” of the sensor, without functional restrictions or a visit to the workshop being necessary for the motor vehicle.

The misalignment value can preferably be determined by transforming measured values from a first measurement point in time back to measured values from a second measurement point in time, for example measured values from a later measurement point in time to measured values from an earlier measurement point in time. In this case, too, the movement of the motor vehicle between the considered points in time of the acquisition of the measured values is taken into account and compensated for. For example, a later measurement image (a later contour of a detected object) is moved back/imaged onto an earlier measurement image (an earlier contour of the detected object). In particular, it can be determined which translational and/or rotational transformation steps are necessary in order to map the later measurement points to the earlier measurement points.

In further developments, other transformations can be used as an alternative or in addition, for example a non-linear transformation or a distortion, in order to compensate for the influence of a specific maladjustment on the measurement images and to be able to map measurement images recorded at different points in time to one another.

Generally speaking, parameters of a mapping rule can be determined in the method, by means of which measurement images recorded at different points in time can be mapped onto one another. The misalignment value can then be determined on the basis of the mapping rule and/or its parameters, for example in order to carry out an improved calibration of the sensor.

However, if it is determined when determining the misalignment value that an upper limit value is being exceeded, an error message is output. A function based on the misaligned sensor is deactivated or at least reduced in scope. In this way it can be ensured that the driver is notified only when absolutely necessary and that a function of the motor vehicle based on data from the sensor is restricted.

In order to be able to save computer capacity in the motor vehicle, it is optionally proposed that the measured values generated during the time interval be transmitted to an external computer and the computer uses the decision logic to determine the misalignment value from these measured values and transmit it back to the motor vehicle. Furthermore, individual steps of the method of the invention can be carried out by the external computer. In particular, machine learning, for example for training a decision logic, can be carried out by the external computer while the actual evaluation of the measurement data is taking place in the motor vehicle. Finally, it is proposed, for example to identify difficult environmental situations in which the correct acquisition of measurement data is not possible, for example due to external interference fields, to compare measurement values from a number of motor vehicles that were acquired in the same area. If a misalignment of the checked sensors of all these motor vehicles is found, a misalignment finding in this area by other motor vehicles should be ignored, at least temporarily. For this purpose, the motor vehicles are networked in particular, for example by means of a car2car or car2infrastructure connection or by means of a higher-level computer.

A preferred exemplary embodiment of the invention is shown in the figures and is explained in more detail in the following description with reference to the figures. As a result, further advantages of the invention also become clear. Identical reference symbols, even in different figures, refer to identical, comparable or functionally identical components. Corresponding or comparable properties and advantages are thereby achieved, even if a repeated description or reference thereto does not occur. The figures are not, or at least not always, drawn to scale. In some figures, proportions or distances can be exaggerated in order to be able to emphasize features of an exemplary embodiment more clearly.

• 1 ein Ablaufdiagramm für eine bevorzugte Ausbildung des Verfahrens,
• 2 die Darstellung einer Messsituation eines fahrenden Kraftfahrzeugs durch eine gerade Straße,
• 3 ein erzeugtes Messbild bei einem ordnungsgemäß justierten Sensor,
• 4 ein erzeugtes Messbild bei einem dejustierten Sensor,
• 5 ein zur Durchführung des Verfahrens geeignetes Kraftfahrzeug,
• 6 ein Verfahrensschritt zum Einlernen einer Entscheidungslogik in ein speziell dafür ausgebildetes Kraftfahrzeug,
• 7 eine Übertragung der zuvor erlernten Entscheidungslogik in eine Auswerte- und Steuereinrichtung eines Kraftfahrzeugs,
• 8 die Nutzung eines externen Rechners zur Entlastung des Kraftfahrzeugs,
• 9 die Rücktransformation von Messwerten eines Objektes, die zu einem späteren Zeitpunkt erfasst wurden, auf Messwerte des Objektes, die zu einem früheren Zeitpunkt erfasst wurden, zur Ermittlung eines konkreten Dejustagewertes und
• 10 die Darstellung der Nutzung von sogenannten Schwarmdaten zur Erkennung schwieriger Umfeldsituationen.

It shows, each schematically,

• 1 a flowchart for a preferred embodiment of the method,
• 2 the representation of a measurement situation of a moving motor vehicle through a straight road,
• 3 a measurement image generated with a properly adjusted sensor,
• 4 a measurement image generated with a misaligned sensor,
• 5 a motor vehicle suitable for carrying out the procedure,
• 6 a method step for teaching a decision logic in a motor vehicle specially designed for this purpose,
• 7 a transfer of the previously learned decision logic to an evaluation and control device of a motor vehicle,
• 8th the use of an external computer to relieve the motor vehicle,
• 9 the inverse transformation of measured values of an object, which were recorded at a later point in time, to measured values of the object, which were recorded at an earlier point in time, to determine a specific misalignment value and
• 10 the presentation of the use of so-called swarm data to identify difficult environmental situations.

First on the 1 referenced. A possible embodiment of the procedure according to the invention is shown in this figure. After starting the method sequence and starting from an initial state V0, a method step V1 first checks whether an initial situation or driving situation suitable for further continuation of the method exists for a motor vehicle with a sensor to be checked. The test may reveal that there is an initial situation S1 which allows the method to be continued is unsuitable. However, there can also be an initial situation S2 in which the motor vehicle is driving through a straight road. A further initial situation S3 can be given by the motor vehicle cornering.

The initial situations S2 and S3 represent initial situations that have proven to be suitable for further implementation of the method. Sx is intended to indicate further, suitable driving situations that are suitable, for example, for misalignment detection with other sensors or types of sensors that use different physical detection principles to record the measured values and for which misalignment may affect the measured values in a different way.

If an initial situation is classified as suitable, then in a method step V2 an evaluation of the measured values obtained through the object detection is carried out for a specific time interval with regard to misalignment detection.

In addition to the simple affirmation or denial that there is misalignment, the evaluation can also include the determination of a specific misalignment value in a method step V3.

How the evaluation of the measured values and the determination of the misalignment value can take place in method steps V2 and V3 will be explained in more detail later.

When determining a specific misalignment value, a query A1 asks whether the misalignment value, which can preferably correspond to a specific angle error α to the normal position of the sensor to be checked, is greater than a threshold value S. The exemplary embodiment assumes here that a misalignment in the form a rotation of a sensor 2 relative to a calibrated position can be present, for example the value of a rotation by an angular error α. In further exemplary embodiments, other types of maladjustment can be checked as an alternative or in addition, for example a displacement, a change in position or another deviation from an initial state.

If this is not the case, then the method returns to method step V1, in which the initial situation for the motor vehicle is checked again, if necessary with a specific or determinable time delay.

However, if query A1 reveals that the angle error α, i.e. the misalignment value, is greater than the threshold value S, the method arrives at a query A2, which asks whether the angle error α is greater than a specific limit value G.

If this is not the case, then in a method step V4 there is an autocalibration or autocorrection of the misaligned sensor or an adaptation of the data generated by the sensor.

However, if the angle error α is so large that it exceeds the limit value G, an autocalibration can no longer be carried out and an error must be set in a method step V5 in the control system, which leads to a warning being issued to the driver and a Function, which is based on the data from the misaligned sensor, is deactivated or at least restricted.

In further exemplary embodiments, it is determined by means of a decision logic EL, in particular by means of a machine learning method, whether misalignment is present or not. The corresponding procedure is explained in more detail below.

Based on 2 until 4 It should be explained how the sensor data recorded in a time interval can be evaluated in method step V2. At a point in time t1 it was determined that a suitable initial situation for a motor vehicle K1 for further implementation of the method exists. The motor vehicle K1 is driving through a straight road or lane F at a speed v.

At the edge of the roadway F, an object to be detected in the form of a parked motor vehicle K2 is shown merely as an example.

The determination of the existence of a suitable situation is also influenced by registering the speed v. In the exemplary embodiment, this is selected such that in the illustrated time interval ZI the motor vehicle K1 runs through so many measurement times t1 to tn that meaningful measurement values can be generated by a sensor 2 . The sensor 2 for object detection is preferably a camera, a radar, lidar or ultrasonic sensor. In the exemplary embodiment, only one sensor 2 is shown as an example, but several sensors 2 can be provided, possibly different types of sensors.

Furthermore, the assessment of whether or not the motor vehicle K1 is in a suitable initial situation is carried out with the aid of an on-board sensor system which, for example, has wheel speed sensors, a yaw rate sensor and Includes steering angle sensors (not shown).

During the time interval ZI, the sensor 2 carries out a measurement M(t1) at the time t1. At time tn, ie toward the end of time interval ZI, a measurement M(tn) is carried out by the same sensor 2. Between these measurements there is a specific number of further measurements, of which one measurement M(t2) at time t2 is explicitly emphasized merely by way of example. The motor vehicle K1 moves continuously, that is to say it is in a different position relative to the parked motor vehicle K2 for each of the measurements M(t1), M(t2), M(tn).

A specific measured value is generated by each of the measurements. A measuring point or measuring location MO of the vehicle contour of the motor vehicle K2 is shown purely as an example, which, in addition to many other measuring locations of the vehicle contour, contributes to the generation of measured values.

At a point in time t1, in which the motor vehicle K1 is in a first position relative to the motor vehicle K2 or to the measuring location MO, a first measurement M(t1) of the position of the measuring location MO relative to the motor vehicle K1 is carried out, with the distance of the motor vehicle K1 in particular being is measured to the motor vehicle K2 or to the measurement location MO. At time t2, at which motor vehicle K1 is in a different position relative to measurement location MO, a second measurement M(t2) takes place. Finally, towards the end of the time interval ZI, a last measurement M(tn) takes place at the time tn, with the motor vehicle K1 again being in a relative position that is already significantly removed from the measurement location MO.

Each recorded measured value is related to the time of its recording and to a relative position of the motor vehicle K1 at this point in time to the measurement location MO causing the measured value. The exact relative position of the motor vehicle relative to the parked motor vehicle K2 at one of the times t1 to tn can preferably be determined with the aid of the already mentioned vehicle odometry. In particular, positions measured using measurements M(t1), M(t2), M(tn) are entered on a map in a manner known per se, for example an environment map of the motor vehicle generated using measurements M(t1), M(tn). K1

Due to the large number of measuring locations or measuring points formed by the motor vehicle K2, a large number of measured values MW(t1), MW(t2), MW(tn) are generated at each point in time t1 to tn, which ultimately lead to a measurement image for a point in time , based on which, for example, a contour or at least part of a contour of the motor vehicle K2 or a detected object can be determined. A measurement image comprises, in particular, what is known as a “point cloud”, ie a large number of measured values MW(t1), MW(t2), MW(tn) recorded for a time step t1, t2, tn.

In particular, it is not important that a complete contour is detected, for example, by driving completely past the motor vehicle K2 or another object. Rather, the measurements M(t1), M(tn) are carried out in such a way that measurement images with measured values MW(t1), MW(t2) and MW(tn) are recorded at a sufficiently large spatial distance from one another; How large the distance must be can depend on various parameters, such as the measurement methods used, sensor types, detected environmental features and objects, a driving situation and/or the specific type of maladjustment present.

In 3 a measurement image MB1 is shown, which was generated by the measurement values MW(t1), MW(t2) and MW(tn) generated at the times t1, t2 and tn. In other words, the contour of a detected object, in this case the motor vehicle K2, is mapped for each point in time t1 to tn. The movement of the motor vehicle K1 was compensated for using the data recorded by means of odometry in such a way that the measured values MW(t1), MW(t2), MW(tn) were entered in a common map.

That in the 3 The measurement image MB1 shown represents a measurement image which was caused by a properly adjusted sensor 2. This means that all generated contours K(t1), K(t2) up to the contour K(tn) lie on top of each other in the measurement image MB1 or at least so close together that the logic evaluating the measurement image MB1 gives the impression of a single, defined contour . The contours mentioned are formed by generated measured values MW(t1) at time t1, generated measured values MW(t2) at time t2 and generated measured values MW(tn) at time tn.

The situation is different if the sensor 2 of the motor vehicle K1 is misaligned, for example due to an offset arrangement of the sensor 2 in relation to the intended installation position, as is shown in FIG 4 is shown. In this case, a measurement image MB2 is generated in which the contours generated for each point in time t1 to tn, ie K(t1), K(t2) and K(tn), for example, no longer lie on top of one another, but rather fall apart. All in all, only a "fuzzy" point cloud is created, which at most only gives an idea of a contour. In further exemplary embodiments, the arrangement of the sensor 2 is opposite ner intended position twisted, possibly in combination with an offset. The procedure can be applied analogously here.

the 5 shows the motor vehicle K1, which is prepared for carrying out the method according to the invention, in somewhat more detail. The motor vehicle K1 has a large number of sensors 2 for detecting objects in the area surrounding the motor vehicle K1. The sensors 2 are preferably embodied as cameras, radar, lidar or ultrasonic sensors, with several sensors 2 and possibly combinations of different types of sensors being able to be provided.

The motor vehicle K1 is equipped with an evaluation and control device 1 which has a memory unit 10, a timer 11 and a logic unit 12.

The evaluation and control device 1 with the sensors 2, with a large number of driver assistance systems 3 (e.g. emergency brake assistance, parking aid, navigation system with GPS device, etc.) is connected via a data bus 6, which can be designed as a CAN bus, for example connected to a display and operating device 5 and to an acoustic signaling device 4 .

If a suitable initial situation is detected as described by the sensor system of the motor vehicle K1, which in addition to the sensors 2 shown also contains other suitable sensors, such as wheel sensors, yaw rate sensors and steering angle sensors (not shown), the object data detected by the sensors 2 are recorded in the manner described evaluated by the logic unit 12 of the evaluation and control device 1. The timer 11 mentioned can be used to set the reference of each generated measured value at the time of its acquisition.

It should be mentioned at this point that a decision logic EL is stored in the logic unit 12, which decides whether there is a maladjustment or not. However, the decision logic EL, which is preferably stored in a program memory of the memory unit 10, was previously learned or trained. Methods of machine learning are used in the exemplary embodiment, with a classifier for detecting misalignment first being trained in the manner explained below using acquired measurement images M, it being known whether these were acquired with a correctly aligned or misaligned sensor 2 . The logic unit 12 then uses, in particular, a classifier that has been trained in this way, in order to classify measurement images, which were recorded with a sensor 2 whose adjustment state is unknown, according to whether there is misadjustment or not. In particular, the decision logic EL includes instructions for carrying out the classification.

An embodiment of the training is based on the 6 explained. 6 shows a motor vehicle K3 prepared for development purposes, which is equipped with a correctly installed sensor 2 and with a deliberately misaligned sensor 2'. Motor vehicle K3 is also equipped with an evaluation and control device 100 and with a comparable sensor system to motor vehicle K1. The misaligned sensor 2' has a specific, fixed angular error α.

In a specific number of measurement runs of the motor vehicle K3, measurements of the vehicle environment are now carried out with the correctly installed sensor 2. This results in measurement images M+, which are evaluated as good measurements from the outset.

A certain number of measurement runs then take place, during which the vehicle surroundings of the motor vehicle K3 are recorded with the deliberately misaligned sensor 2'. This results in measurement images M- which are assigned to a bad measurement from the outset. In particular, the angular error α known for the sensor 2′ can also be assigned to the measurement images M − .

Both the measurement images M+ and the measurement images M− are combined in such a way that a decision logic EL is created and stored in the evaluation and control device 100 .

Measurement images M+ and M− can be used to train the decision logic EL, which are recorded by means of one or more motor vehicles K3 and various sensors S2, S2′. In particular, sensors S2, S2' of the same or different sensor types can be used for this purpose, which use the same or different physical measurement principles, for example. Alternatively or additionally, the measurement images M+ and M- used can be generated artificially by means of simulations.

The decision logic EL learned in this way is read out of the evaluation and control device 100 of the motor vehicle K3, buffered if necessary and read into the evaluation and control device 1 of the motor vehicle K1 or in its memory unit 10 (see Fig 7 ). The decision logic EL is thus available to the motor vehicle K1 during its normal, everyday journeys. The storage of reference data or the like is not necessary. The in the 6 and 7 illustrated Process steps are summarized under process step V0 (see 1 ).

In the exemplary embodiment, the typically computationally intensive training of the machine learning method or the generation of the decision logic EL takes place outside of the motor vehicle 1, for example on an external server, so that it only has to carry out the typically less computationally intensive classification using the decision logic EL. In a further exemplary embodiment, in order to be able to save additional computing capacity in the motor vehicle K1, the decision logic EL can optionally also be relocated to an external computer 7, as in 8th is shown.

The measurements carried out in the time interval ZI at the times t1 to tn, which relate, for example, to the detection of objects in the form of motor vehicles Kx parked next to the lane F, are transmitted to the external computer 7 . On the basis of the decision logic EL now stored in it, the external computer 7 decides on the existence and extent of misalignment.

In the exemplary embodiment, the calculation of a specific misalignment value or an angular error α can be accomplished in the evaluation and control device 1 of the motor vehicle K1 or in the external computer 7 in such a way that measured values from a first measurement point in time are mapped to measured values from a second measurement point in time by means of a transformation. For example, measured values from a later measurement time can be transformed back to measured values from an earlier measurement time. An example is in 9 shown that measured values MW(tn) at a later point in time tn are transformed back to measured values MW(t1) at an earlier point in time t1. In this exemplary embodiment, a transformation T therefore takes place, in which it is calculated how the measured values MW(tn) or the measuring points of the contour K(tn) must be moved back in translation and/or rotation so that they are imaged on the contour K(t1). be able. Under certain circumstances, this procedure also requires a lot of computer capacity, so that outsourcing to the external computer 7 can be advantageous.

In further exemplary embodiments, other mapping or transformation steps take place as an alternative or in addition, for example in order to compensate for further linear or non-linear effects on the recorded measured values MW(t1), MW(t2), MW(tn) caused by misalignment. For example, a misalignment of a detection optics can lead to a distortion and a corresponding transformation can be determined. For example, registration methods, such as iterative approximation methods, in particular statistical methods, can be used to determine the transformation. Methods of this type are used in particular to map point clouds recorded at different points in time onto one another, in which case it is not necessary for all the individual points recorded in a plurality of time steps to correspond to one another. For example, an optimal transformation is determined such that the point clouds are optimally superimposed in a resulting array.

In a further exemplary embodiment, an analytical method can be used in order to determine parameters of a mapping rule with which measured values from a first measurement time can be mapped or back-transformed to measured values from a second measurement time. In particular, such an analytical method is used when measured values at several points in time correspond exactly to one another. This is the case, for example, when the same object is detected at several points in time and the measured values recorded at different points in time can be mapped onto one another.

In a further exemplary embodiment, for training the decision logic EL, measurement images M − are recorded for bad measurements to be trained with a specific misalignment, for example with a known angle error α. In this case, the decision logic EL is trained to recognize a misalignment with this angular error α. Furthermore, different angular errors α or other specific types of misalignment can be used for training, for example to train the decision logic EL to recognize and/or differentiate between different types of misalignment.

In this exemplary embodiment, the machine learning method can be used by means of the decision logic EL in order to identify the specific misalignment value. In particular, the recorded measurement images M(t1), M(tn) are classified and assigned to a specific angular error α. In particular, this is such an angle error α for which measurement images M- for bad measurements were previously recorded and used for training.

Finally is in 10 a situation is shown in which several motor vehicles K4, K5 and K6 are driving through an area 15 which is to be assessed as critical. Motor vehicles K4 to K6 are equipped in a manner comparable to motor vehicle K1. They also have the learned decision logic EL, in particular a trained classifier.

Area 15 is to be assessed as critical insofar as all motor vehicles K4, K5 and K6 report to an external computer 8 at specific time intervals ZI4, ZI5 and ZI6, in which evaluations were carried out according to the method according to the invention, that their sensors 2 are misaligned are. Motor vehicles K4 to K6 each have a GPS device and a communication device 14. GPS device 13 works in conjunction with a GPS satellite 9. Wireless data exchange with external computer 8 is possible via communication device 14.

The structure shown makes it possible for the computer 8 to receive not only the sensor data from the motor vehicles K4 to K6, but also their position.

Since all motor vehicles K4 to K6 in said area 15 now report that their sensors 2 are misaligned, which is to be rated as highly unlikely, a logic of computer 8 concludes that area 15 is to be assessed as critical. In a further exemplary embodiment, it is determined how many motor vehicles K4 to K6 or what proportion of motor vehicles K4 to K6 in area 15 consistently report a misalignment, and a source of interference is assumed if the absolute or relative number of faults exceeds a threshold value.

It is therefore assumed that a source of interference (not shown in detail) is present in region 15, which at least temporarily leads to incorrect acquisition of data by sensors 2. The computer 8 therefore transmits a corresponding signal to other motor vehicles which can also communicate with the computer 8 and are not (yet) located in the area 15, so that when measurements are taken in the critical area 15, sensors that are recognized as misaligned based on these measurements are not as be evaluated misaligned.

Alternatively or in addition to a specific area 15, a driving situation can be recorded and a source of interference can be identified. For example, analogously to the exemplary embodiment, it can be recognized that a source of interference is to be assumed for a specific course of the roadway, at a specific time or in the vicinity of a specific device. In this way, it can be avoided, for example, that systematic malfunctions of sensors 2 in certain driving situations lead to incorrect detection of misalignment.

Reference List

1

Evaluation and control device

2

sensors, ultrasonic sensors

2'

misaligned sensor

3

driver assistance systems

4

acoustic signaling device

5

Display and operating device

6

data bus

7

external computer

8th

external computer

9

GPS satellite

10

storage unit

11

timer

12

Logic unit (contains decision logic)

13

GPS setup

14

communication facility

15

critical area

100

Evaluation and control device

A1-A3

Interrogate

el

decision logic

f

roadway

G

limit

K(t1)

Contour of a detected object at time t1

K(t2)

Contour of a detected object at time t2

K(tn)

Contour of a detected object at time tn

K1

moving motor vehicle

K1(t1)

moving motor vehicle at time t1

K1(t2)

moving motor vehicle at time t2

K1(tn)

moving motor vehicle at time tn

K2

parked motor vehicle

K3

Motor vehicle for development purposes

MB1,

MB2 measurement images

MON

Measuring point generating measured value

MW(t1)

Measured values of the measurement time t1

MW(t2)

Measured values of the measurement time t2

MW(tn)

Measured values of the measurement time tn

M(t1)

Measurement at time t1

M(tn)

Measurement at time tn

M+

Measurement image of a good measurement

M-

Measurement image of a bad measurement

S

threshold

S1

Initial situation unsuitable

S2

Initial situation "passing through a straight road"

S3

Initial situation "cornering"

sx

other suitable starting points

T

transformation

t1-tn

times

V0-V5

process steps

v

speed

ZI

time interval

ZI4-ZI6

Time intervals of measurements of different vehicles

a

misalignment value, angle error

Claims (9)

Method for misalignment detection of at least one sensor (2) of a motor vehicle (K1), which is used to detect objects in the area surrounding the motor vehicle (K1), with at least the following method steps: It is checked in which current initial situation the motor vehicle (K1) is located . • If there is an initial situation (S2, S3, Sx) which corresponds to a stored initial situation defined as suitable, the sensor (2) during the journey at a specific time interval (ZI) by detecting objects is evaluated and generated stored measured values (M(t1)-M(tn)). • Each of the recorded measured values (M(t1)-M(tn)) is related to the time (t1-tn) of its recording and to a relative position of the motor vehicle (K1) relative to a measured value (M(t1)-M (tn)) causing measuring location (MO) set at this time (t1-tn). • At least measured values (M(t1)) of a first point in time (t1) are compared with the measured values (M(t2)...M(tn)) of a second, later point in time (t2...tn). • From the comparison of the measured values (M(t1), M(t2), M(tn)) it is decided whether or not there is a maladjustment (α) of the sensor (2), characterized in that • the stored measured values ( M(t1)...M(tn)) measurement images (MB1, MB2) are generated, which represent at least parts of its contour (K) and/or any pattern at different points in time (t1...tn) for each detected object, wherein measurement images (MB2) with locally spaced contours (K(t1), K(t2), K(tn)) of at least one object and/or with spaced patterns are classified as bad measurement and thus as maladjustment and measurement images (MB1) with locally one above the other Contours (K(t1), K(t2), K(tn)) and/or with superimposed patterns as a good measurement, with a threshold value or tolerance range being defined and contours (K(t1), K(t2), K(tn) ) and/or patterns are determined to be apart if they are further apart than the threshold. procedure after claim 1 , characterized in that the decision as to whether misalignment (α) is present or not is based on a previously learned and stored decision logic (EL). procedure after claim 2 , characterized in that the stored decision logic (EL) is previously learned by measuring images (M-, M+), which on the one hand by measuring runs of a motor vehicle (K3) with a deliberately misaligned sensor (2') and on the other hand by measuring runs of a motor vehicle (K3) be generated with a properly adjusted sensor (2). Method according to one of the preceding claims, characterized in that driving through a straight road (S2) and/or cornering (S3) is considered as a suitable starting situation. Method according to one of the preceding claims, characterized in that a misalignment value (α) is determined and an autocalibration of the sensor (2) is carried out when a threshold value (S) of the misalignment value (α) is exceeded. procedure after claim 5 , characterized in that the misalignment value (α) is determined by mapping measured values (M(tn)) of a first measurement time (tn) to measured values (M(t1)) of a second measurement time (t1) by means of a transformation. Method according to any of the preceding Claims 5 or 6 , characterized in that when an upper limit value (G) of the misalignment value (α) is exceeded, an error message is output and a function based on the misaligned sensor (2) is deactivated or at least reduced in scope. Method according to any of the preceding Claims 5 until 7 , characterized in that the measured values (MW(t1)... MW(tn)) generated during the time interval (ZI) are transmitted to an external computer (7) and the computer (7) uses the decision logic (EL). The maladjustment value (α) is determined from these measured values (MW(t1)...MW(tn)) and transmitted back to the motor vehicle (K1). Method according to one of the preceding claims, characterized in that measured values from a number of motor vehicles (K4-K6) which were recorded in the same area (15) are compared, with a finding of misalignment of the checked sensors of all these motor vehicles (K4-K6) a finding of misalignment in this area (15) by other motor vehicles is at least temporarily ignored.

Images