DE102021116859A1 - Method and detection device for lidar-based detection of exhaust gas clouds and motor vehicles - Google Patents

Abstract

The invention relates to a method and a detection device for the automated detection of exhaust gas clouds (20, 26) from other vehicles (16, 22, 32) using lidar. The invention further relates to a correspondingly equipped motor vehicle. In the method, detected objects are classified as a relevant obstacle (16, 22), as a possible exhaust gas cloud (26) or as an irrelevant obstacle (30) on the basis of object dimensions and highly reflective points. The relevant obstacles (16, 22) are possibly identified as foreign vehicles (32) based on the size of a detected exhaust area (34). The exhaust area (34) is defined by a set of points that is determined according to a predetermined distance criterion. For the foreign vehicles (32) identified in this way, all points (18, 20) are determined using a predetermined distance threshold value for a distance from a focus of the point set representing the exhaust area (34) and according to a predetermined direction criterion that has an exhaust gas exit direction (n̂) with respect to the exhaust area ( 34) are taken into account, classified as off-gas points (20) or as non-off-gas points (18).

Description

The present invention relates to a method and a detection device for lidar-based detection of exhaust gas clouds and a correspondingly equipped motor vehicle.

Precise and reliable detection of a particular environment and of objects in the particular environment is an important basic requirement for assistance functions or automated operation of motor vehicles. Nowadays, different sensor devices are used for this purpose, such as cameras, radar devices and lidar sensors. In principle, the latter have the advantage that they can provide rich depth information, which enables the distance determination of static and dynamic objects or obstacles in the environment independently of light or lighting conditions. However, adverse weather conditions such as rain, snow and fog can significantly affect the range and data quality of ToF lidar sensors (ToF: Time-of-Flight). While there are already approaches to meet these challenges, the problem that exhaust gas clouds emitted by other vehicles in the vicinity can also impair lidar measurements has not yet been solved.

Exhaust clouds emitted by motor vehicles powered by internal combustion engines contain water vapor and can reach temperatures in the region of several 100°C. After ejection, small water droplets may condense and form visible clouds before evaporating. These small water droplets can have sizes in the range from 1 μm to 10 μm, for example. Conventional ToF lidar sensors that are used in motor vehicles can have signal wavelengths in the range of 0.9 μm, for example, so that signals from the lidar sensor can be reflected by the water droplets. Although such exhaust clouds usually do not impair the ability of human drivers to drive a motor vehicle safely, assistance systems or automated driving functions that rely on lidar-based environment recognition can be impaired. Exhaust clouds from other vehicles can therefore produce artefacts or faulty perceptions of the surroundings, for example incorrect classifications of objects or an incorrect determination of their size or shape. This in turn can impair the performance of higher assistance or driving functions that are based thereon.

Exhaust clouds pose a particular challenge in this regard, as their density can result in localized groups or clusters of lidar point detections, allowing effective application of denoising algorithms that successfully detect sparsely distributed single points such as those caused by snowflakes or raindrops can, can filter out, can prevent.

One approach to detecting vehicle exhaust is in US Pat US 8,818,609 B1 described. In the process there, a density profile and a height profile are generated from a lidar-based point map. Based on this, regions of a scan zone shall then be classified as containing a surface of a vehicle or exhaust gas from the vehicle. However, features used to identify the exhaust gas, such as the level and density, can be dependent on or influenced by weather conditions, a temperature, a type or design of a respective internal combustion engine and/or a respective exhaust system and the like. Therefore, a large variety of corresponding models for describing different properties of the exhaust gas in different conditions, with different vehicles and the like would be needed more for reliable operation. The use of a position history of detected exhaust gas proposed there can also be problematic in practice, since new relevant obstacles, such as pedestrians, can appear in a corresponding area and are then possibly not recognized as such or taken into account.

The object of the present invention is to enable a particularly simple and efficient lidar-based automated detection of vehicle exhaust clouds.

According to the invention, this object is achieved by the subject matter of the independent patent claims. Possible refinements and developments of the present invention are disclosed in the dependent patent claims, in the description and in the figures.

The method according to the invention is used for the automated recognition or detection of exhaust gas clouds from other vehicles, in particular a third-party vehicle driving ahead of a motor vehicle set up for the method according to the invention, using lidar. Such exhaust gas clouds can thus be identified and localized as such by means of the method according to the invention. In a method step of the method according to the invention, a lidar point cloud with pointwise spatial coordinates and pointwise reflections is generated from the motor vehicle by means of a lidar device or a lidar sensor activity recorded. The lidar point cloud thus includes individual measurement points - referred to as points for short - of the lidar sensor, which are each described by their spatial coordinates and their reflectivity, i.e. their reflected signal intensity. From this lidar point cloud, for example, ground points originating from a subsoil or a roadway surface in an area surrounding the motor vehicle or in a measuring range of the lidar sensor can be removed using methods known per se. According to the invention, possibly remaining points of the lidar point cloud are grouped by spatial clustering to form object point sets, each of which represents an object in the area surrounding the motor vehicle. The object point sets thus each include one or more points of the lidar point cloud, ie reflections detected by the lidar sensor, which are spatially grouped into a cluster.

In a further method step of the method according to the invention, object dimensions, a respective center of mass and contained highly reflective points are determined from the spatial coordinates and the reflectivities for each object point set. The object dimensions can be or indicate, for example, a height, length and width of the respective object point set or a predetermined shape enveloping the points of the object point set, for example a cuboid, whereby the respective object is represented. A corresponding volume can also be determined here. The highly reflective points are points whose reflectivity corresponds at least to a predetermined reflectivity threshold value.

In a further method step of the method according to the invention, the objects or the points or sets of points respectively representing them, ie the object point sets, are classified according to predetermined volume and reflectivity criteria on the basis of the object dimensions and the highly reflective points. The objects can each be classified as a relevant obstacle, as a possible or potential exhaust gas cloud or as irrelevant for safe driving of the motor vehicle, ie for a driving task that is in particular fully or partially automated. For example, according to the specified criteria, an object can be classified as a relevant obstacle if its volume is within a specified range and the corresponding object point set includes at least a specified number of highly reflective points. On the other hand, an object can be classified as a possible exhaust gas cloud if its volume is smaller than a predetermined volume threshold value and the corresponding object point set does not include any highly reflective points. Otherwise, the object can be classified as irrelevant, e.g. as a non-relevant obstacle.

In a further method step of the method according to the invention, the relevant obstacles, i.e. the objects or object point sets for which a classification as a relevant obstacle was determined, are each recognized as an external vehicle if the width or size of an exhaust area of the respective obstacle differs from the width or size specified for the entire particular object or set of object points. The exhaust area is defined as a subset of the points of the obstacle, i.e. the points of the corresponding object point set, which are also referred to as exhaust area points. This subset, ie the points from the exhaust area, are determined according to a predetermined distance criterion related in a predetermined manner to at least a part or a reference point of the respective set of object points determined in a predetermined manner.

The distance criterion can, for example, take into account a respective distance between the points belonging to the respective obstacle and an imaginary or constructed plane that runs through a focal point of the subset of highly reflective points. In particular, this plane can be constructed so as to be at least essentially perpendicular to the subsoil or floor that is traveled over or detected. For the relevant obstacles detected or classified as such, points of the respective object point set are determined based on the predetermined distance criterion, which represent the exhaust area or originate from it. The width or size of this exhaust area can then be compared to that from all points of the respective object or object point set. If this comparison results in a difference below a predetermined threshold value, then the respective object is identified as a third-party vehicle. A variation or influencing of the detected object width by an exhaust gas cloud emitted by the other vehicle can be taken into account or modeled by the threshold value, ie the permissible amount of the deviation of the widths or sizes.

In a further method step of the method according to the invention, all points of the corresponding object point set for the foreign vehicles identified in this way, by means of which or from which the respective foreign vehicle was identified, are classified as exhaust gas points or as non-exhaust gas points based on a predefined distance threshold value and according to a predefined direction criterion.

The specified distance threshold value is specified for a distance of the respective point from a center of gravity of the set of points representing the exhaust area, ie the exhaust area points.

The specified direction criterion takes into account an exhaust gas exit or exhaust gas movement direction with respect to the exhaust area of the respective other vehicle. The direction criterion can be checked, for example, by analyzing vectors constructed in a predetermined manner from the respective set of object points, for example with regard to one of the focal points mentioned and/or the plane mentioned. In particular, all points of the respective object point set that are located outside or on this side of the exhaust area from the perspective of the motor vehicle, ie for example a rear or a rear area, of the respective other vehicle can be defined or classified as exhaust gas points. The respective exhaust gas cloud is then recognized or defined and localized by these exhaust gas points.

The method according to the invention can, for example, be carried out repeatedly or in a predetermined cycle. The processes or measures described can be carried out in each cycle or time step.

The present invention makes it possible to identify respective exhaust areas or rear sides of other vehicles particularly quickly or with particularly little effort or use of calculation resources, to determine whether there is also an exhaust gas cloud in a corresponding area and to identify any such exhaust gas clouds. This can be carried out in a predictable, comprehensible, analytical manner. A further advantage of the present invention is that extensive training data, a large number of different models and the like are no longer required. Instead of an algorithm that identifies a complete set of object points as a vehicle, a simpler and faster approach is proposed here, which still allows reliable detection of exhaust clouds.

The present invention is based on the finding that an exhaust gas cloud emitted by another vehicle that is still in the vicinity of the other vehicle is typically in the vicinity of highly reflective vehicle parts, such as a number plate, one or more rear lights, a direction indicator or the like and this circumstance can be used to identify the exhaust gas cloud. This circumstance is in particular at least essentially independent of weather, temperature or environmental conditions. The present invention thus enables a particularly reliable and robust detection of exhaust gas clouds, which can be carried out with particularly little effort, for example with the computing capacity and hardware already available today in conventional motor vehicles, and also no complex preparation for providing a broad database, no backend infrastructure or the like needed.

In one possible embodiment of the present invention, in order to determine the subset of points that define the respective exhaust area according to the distance criterion, those of the highly reflective points of the respective object point set are determined as highly reflective object or exhaust area points whose distance from a focus of the set of highly reflective points is at most one corresponds to the specified maximum distance. This makes it possible to exclude individual or sparsely distributed highly reflective points that originate from other fixed parts of the respective third-party vehicle, i.e. not to assign them to the exhaust area. By adapting or fitting to the specific, highly reflective exhaust area points, an imaginary plane, in particular at least substantially perpendicular to a surface on which the motor vehicle and/or the other vehicle is driving, is then constructed by calculation. An algorithm or a least squares method can be used for this purpose, for example. In addition, those points of the obstacle, ie of the respective set of object points, whose Euclidean distance from the constructed plane corresponds at most to a predetermined maximum distance, are included in the subset of points defining the exhaust area. The methodology proposed here enables a robust and reliable identification of the points belonging to the exhaust area of the respective third-party vehicle and thus also a correspondingly robust and precise determination of the exhaust gas points.

In a further possible embodiment of the present invention, according to the direction criterion, those points are classified as exhaust gas points for which the scalar product of the vector from the respective point to the center of gravity of the subset of points representing the respective exhaust area with one of the respective centers of gravity of the respective exhaust area representing subset of points and the respective object point sets - or corresponding vectors to these focal points in a predetermined coordinate system - formed difference vector greater than is zero. The difference vector represents or identifies the exhaust gas outlet direction with regard to the respective other vehicle. The geometric calculations or considerations proposed here can be carried out simply, efficiently and quickly, as a result of which a particularly low-delay and reliable detection of the exhaust gas clouds and thus use of the present invention in real-time applications is made possible. The exhaust gas cloud can be identified or localized particularly reliably and robustly by the direction criterion.

In a further possible embodiment of the present invention, the average reflectivity and the maximum reflectivity of the respective exhaust gas points determined therefrom, ie the exhaust gas points contained therein, are respectively determined for the object point sets. When corresponding predefined threshold values are reached or exceeded, the exhaust gas points, ie those points of the object point set which were previously or up to then classified as exhaust gas points, are reclassified as non-exhaust gas points. This reclassification is therefore carried out if or if the average reflectivity of the exhaust gas points for an object point set reaches or exceeds a predetermined average reflectivity threshold value and for the same object point set the maximum reflectivity of the exhaust gas points, i.e. the highest reflectivity determined or measured for one of the exhaust gas points, a predetermined one Reaches or exceeds maximum reflectivity threshold. On the other hand, if these specified threshold values are not reached, the exhaust gas points are retained or confirmed as such. In other words, the points previously or previously classified as exhaust points are confirmed as such if the average reflectivity of the exhaust points for an object point set does not exceed the average reflectivity threshold value specified for this purpose or falls below or is less than this and for the same object point set the maximum reflectivity of the exhaust points at most reaches or is less than the maximum reflectivity threshold value specified for this purpose. By taking into account the average and maximum reflectivities of the exhaust gas points as proposed here, the safety of fully or partially automated functions of the motor vehicle that are based on the detection of the exhaust gas clouds can be improved. This is the case because exhaust gas clouds typically have a relatively low reflectivity and given a relatively high reflectivity, ie when the specified threshold values are exceeded, it can be assumed that there is a solid object in the area of the exhaust gas cloud. This can be, for example, transported goods protruding beyond the exhaust area or the rear of the respective third-party vehicle, a bicycle carrier or the like. The corresponding reclassification as non-exhaust points can then ensure, for example, a correspondingly adapted intervention of subsequent collision avoidance measures.

In a further possible embodiment of the present invention, the method according to the invention is carried out repeatedly at a plurality of points in time, for example with a predetermined clock rate or frequency. Thus, for example, a corresponding detection device set up to carry out the method can be operated continuously. A history of obstacle detections, in particular a common one, is stored for the relevant obstacles for which at least one exhaust gas point, ie an exhaust gas cloud, was detected. These obstacle detections each indicate the associated detection time, the point mean value and the standard deviation of the corresponding object point set. The time of detection can be a time stamp, for example, which can indicate the time or cycle at which the respective obstacle was detected or classified as such. The point mean can be determined as the mean of the data tuples describing the points. A mean value can be determined for each coordinate or position of the data tuple and used as a coordinate value or entry for the point mean value. The mean point value can therefore be indicated, for example, by means of the respective average spatial x, y and z coordinates and the average reflectivity of the points of the respective object point set.

For the obstacle detections, the associated classifications, i.e. those determined or carried out in each case, as exhaust gas points or non-exhaust gas points are then checked using a predefined classifier based on the coordinates and reflectivities of the corresponding points. In this way, it is possible to detect detached exhaust gas clouds, ie exhaust gas clouds that are no longer in the immediate vicinity of the other vehicle from which they were emitted. Due to the fact that the obstacle detections that took place or were stored in the past are each checked again here, the method or a functionality based on it does not rely on a once-determined classification. Rather, there is a regular review of previous classifications of the obstacle detections and, if necessary, an adjustment or revision of the classifications. As a result, it can be taken into account, for example, that fixed objects, such as other vehicles or other road users, can move and, for example, leave behind loosened plumes or enter the area of a previously detected plume. Improved safety and robustness can be achieved by the embodiment of the present invention proposed here, since collisions are avoided in a particularly reliable manner and/or newly opening free spaces that could be traveled by the motor vehicle without a collision can be detected.

In a possible development of the present invention, a probabilistic classifier is used as the classifier. This enables a particularly simple and quick check or adjustment of the classifications, which can be implemented or realized with little effort.

In a possible development of the present invention, a naive Bayes classifier, in particular with a maximum a posteriori estimation, is used as the classifier. The data tuples mentioned from spatial coordinates and reflectivity can be used as observations or as feature vectors. In principle, different types of classifiers can be used, but the naive Bayes classifier proposed here represents a particularly simple way of realizing the. In particular, it can be assumed here that the spatial coordinates x, y, z—and possibly also the reflectivities—of the individual points are statistically independent of one another, as a result of which the corresponding computing effort can be saved.

In a possible development of the present invention, for the points defined or described by their spatial coordinates and their reflectivity, probabilities conditional on the classifier are specified or defined as specified below. For the classification as an exhaust gas point or as an exhaust gas cloud or as part of it, the conditional probability for the spatial coordinates is evaluated by a specified normal distribution evaluated there, i.e. at the value or at the point of the respective coordinate, and for the reflectivity by a specified exponential distribution are specified or defined. For example, a mean value μ and a standard deviation σ can be specified for the normal distribution. For classification as non-exhaust points or non-exhaust cloud or as not part of an exhaust cloud, the conditional probability for the spatial coordinates is specified or defined by a specified u-square distribution and for the reflectivity by a continuous uniform distribution. In this case, the constant uniform distribution can be specified or defined in particular with a minimum value of zero and a maximum value corresponding to the maximum detected reflectivity. In other words, corresponding probability density functions are specified, based on which the classifier checks the classifications as exhaust gas points or as non-emission gas points, or makes new ones—for example, using specified probability threshold values.

The specifications proposed here for the conditional probabilities or the corresponding probability density functions can take into account that an exhaust gas cloud typically drifts relatively slowly away from its point of generation or exit at the respective other vehicle and exhaust gas clouds or lidar points originating from them typically have a relatively low reflectivity exhibit. In the case of classification as an exhaust point, the normal distribution will provide a higher probability of classifying the respective object as an exhaust cloud if the object or specific point of the object is closer to one of the points or one of the detections. The exponential distribution, on the other hand, will provide a higher probability of classification as an exhaust plume or exhaust point for points with lower reflectivity. For classification as a non-non-exhaust point, the u-squared distribution will give a higher probability if a point is farther from other points or a detection, while the uniform distribution will give the same probability for any reflectivity, since low reflectivities are not only in Exhaust clouds occur, but also frequently occur at points that do not belong to exhaust clouds. The embodiment of the present invention proposed here enables a particularly reliable and robust classification or detection of exhaust gas clouds and other objects.

A further aspect of the present invention is a detection device for automatically detecting exhaust gas clouds using lidar data. In this case, the recognition device has an input interface for acquiring the lidar data and a processor device as well as a data memory coupled thereto for processing the lidar data. The detection device is set up to automatically execute at least one embodiment or variant of the method according to the invention. The detection device according to the invention can in particular be the detection device mentioned in connection with the method according to the invention, include it or be part of it. The detection device can include the named lidar sensor or lidar detector or be connected to it, for example via an on-board network of a force equipped with the detection device vehicle. In order to execute the corresponding method, a corresponding computer program can be stored on the data memory, for example, which represents, i.e. encodes or implements, the processes, measures or method steps of the method according to the invention and can be executed by the processor device in order to cause or to execute the corresponding method cause. The processor device can be or include, for example, a microchip, a microprocessor, a microcontroller, a hardware circuit or the like.

A further aspect of the present invention is a motor vehicle which has a detection device according to the invention and--as a separate device or as part of the detection device--a lidar detector or lidar sensor. The motor vehicle according to the invention can in particular be the motor vehicle mentioned in connection with the method according to the invention and/or in connection with the detection device according to the invention. In other words, the motor vehicle according to the invention can also be set up to carry out the method according to the invention, in particular automatically or semi-automatically. The motor vehicle according to the invention can in particular have an assistance system that performs its intended function of supporting a driver when driving the motor vehicle or for at least partially automated driving of the motor vehicle based, among other things, on recognition data provided by the recognition device according to the invention, which indicate objects recognized on a lidar basis or their classifications can.

Further features of the invention can result from the claims, the figures and the description of the figures. The features and feature combinations mentioned above in the description and the features and feature combinations shown below in the description of the figures and/or in the figures alone can be used not only in the combination specified in each case, but also in other combinations or on their own, without going beyond the scope of the invention to leave.

• 1 eine schematische Darstellung zur Veranschaulichung einer lidarbasierten Erkennung einer Abgaswolke im Bereich eines Kraftfahrzeugs;
• 2 eine schematische Darstellung zur Veranschaulichung einer lidarbasierten Erkennung einer losgelösten Abgaswolke; und
• 3 eine schematische Darstellung zur weiteren Veranschaulichung eines entsprechenden Erkennungsverfahrens.

The drawing shows in:

• 1 a schematic representation to illustrate a lidar-based detection of an exhaust gas cloud in the area of a motor vehicle;
• 2 a schematic representation to illustrate a lidar-based detection of a detached exhaust gas cloud; and
• 3 a schematic representation to further illustrate a corresponding detection method.

In the figures, identical and functionally identical elements are provided with the same reference symbols.

If exhaust gases from another vehicle are detected from a motor vehicle using a ToF lidar sensor, the result data can be a locally clustered set of points, which can comprise, for example, more than 1000 points or detections per exhaust cloud. If points originating from such an exhaust cloud, i.e. echoes or detections, are compared with other obstacles such as vehicles, vegetation or buildings, they can tend to have a lower intensity or reflectivity and a lower ground clearance on average. However, this tendency is not pronounced enough to enable clear and reliable separation or detection of exhaust plumes and other objects based thereon alone. Thus, a corresponding reliable detection or discrimination using point-by-point denoising algorithms or feature-based detection methods can be difficult or impossible to implement.

When detecting exhaust clouds in traffic, two different cases can occur. In the first case, an exhaust gas cloud can be in the immediate vicinity of a vehicle emitting the respective exhaust gas cloud. Otherwise, exhaust gas clouds 26 released from vehicles (see 2 ) appear. A method is described below, by means of which exhaust gas clouds can be correctly identified using lidar data in both cases.

For illustration shows 1 a schematic overview of a detection area 10 of a lidar sensor of a motor vehicle. Shown here are ground points 12 located in the detection area 10, which are determined by a respective underground 36 (see 3 ) represent reflected or originating lidar points. The ground points 12 are arranged here according to a scanning pattern of the lidar sensor in concentric ring sections around them. Furthermore, individual object points 14 of a first object 16 are shown here by way of example. The individual object points 14 can originate, for example, from a stationary, in particular switched off, vehicle. This vehicle is represented here as the first object 16 by a dashed line.

There are also vehicle points 18 here that are driven by another vehicle 32 (see 3 ) come shown. Exhaust gas points 20 are also shown here in the immediate vicinity of vehicle points 18, which represent an exhaust gas cloud emitted by other vehicle 32, which is still in its immediate vicinity.

After the ground points 12 have been removed using a corresponding filter algorithm, a clustering algorithm is applied to the remaining points. As a result, the individual object points 14 are combined to form a first cluster and the vehicle points 18 together with the exhaust gas points 20 to form a second cluster. The cluster formed by the vehicle points 18 and the exhaust gas points 20 represents a second object 22 here. The second object 22 thus represents the first case mentioned here. The boxes representing the first object 16 and the second object 22 can be constructed with a given cuboid basic shape in such a way that they enclose the individual clusters of points of the lidar measurement.

2 shows a further schematic representation of the detection area 10 in a different situation. Here, in addition to the ground points 12, the individual object points 14, which represent the first object 16 or originate from it, are again shown. In addition, however, two clusters of individual exhaust gas cloud points 24, each representing a detached exhaust gas cloud 26, are shown here. The detached exhaust gas clouds 26 are therefore not in the immediate vicinity of another vehicle here, but form their own separate clusters of points. In addition, edge area points 28 , which represent an edge object 30 , are shown here at an edge of the detection area 10 . The edge object 30 can be, for example, vegetation on the edge or buildings on the edge of a road on which the motor vehicle is driving.

In the method for detecting the exhaust gas clouds 26 represented by the exhaust gas points 20 or the individual exhaust gas cloud points 24, an initially unsorted set P _k = {(p ₁ , r ₁ ),..., (p _n , r _n )} of n points received from the lidar sensor. Each point is described or represented by spatial coordinates p _i ={x, y, z} and a reflectivity r _i . By means of the clustering mentioned, for example using a connected-components algorithm, a set 0 = {0 ₁ , ..., 0 _m } of m disjoint object point sets O _j ∈ O is formed, with each object point set 0 _j = {(p ₁ , _{r 1 )} , .

For each such object O _j , its width W, length L and height H and an object volume V resulting therefrom are calculated. For this purpose, for example, the distances between the points of the respective object point set O _j that are furthest apart in each dimension and a predetermined shape as the envelope, for example a cuboid, can be used. Based on this, for example, the 1 and 2 dashed boxes are constructed.

berechnet, wobei h einen vorgegebenen Reflektivitätsschwellenwert angibt. Dabei kann es sich um einen fest vorgegebenen Wert handeln, der beispielsweise basierend auf vorherigen Daten, Messungen oder Experimenten heuristisch bestimmt sein oder bestimmt werden kann.For each object point set O _j , its set of highly reflective points H = {(p ₁ , r _i ) ∈ O _j | r _i > h} and its centroid $X_c^H$

calculated, where h indicates a predetermined reflectivity threshold. This can be a permanently specified value that can be or can be determined heuristically, for example, based on previous data, measurements or experiments.

mit der Klassifikation bzw. dem Label I verwendet: $$l=\{\begin{array}{c}1wenn{V}_1<V<V_2\wedge \left|H\right|>N_1\\ 2wennV<V_3\wedge H=\{\varnothing \}\\ 0sonst\end{array}$$

No a priori classification is assumed here, but the following equation for classifying the objects O _j as classified objects $O_j^l$

used with the classification or label I: $$l=\{\begin{array}{c}1wenn{V}_1<V<V_2\wedge \left|H\right|>N_1\\ 2wennV<V_3\wedge H=\{\varnothing \}\\ 0sOnst\end{array}$$

I=1 means a classification as a relevant obstacle, I=2 a classification as a possible exhaust gas cloud and I=0 a classification as a non-relevant obstacle. V ₁ , V ₂ and V ₃ are predetermined volume thresholds and N ₁ is a predetermined number threshold for the minimum number of highly reflective points that must be present for classification as an obstacle of interest.

The classification scheme proposed here is particularly easy to implement, but still makes it possible to filter out objects with a relatively large volume, such as strips of vegetation or buildings, and at the same time to classify motor vehicles of different sizes as relevant obstacles.

In addition, clusters of points comprising only exhaust points 20 are also detected or retained and classified as a possible exhaust cloud since they typically have smaller volumes than other objects, such as buildings, and at the same time have lower reflectivity than vehicles or other solid objects.

tatsächlich um ein Fremdfahrzeug 32 handelt und anschließend eine möglicherweise vorhandene Abgaswolke identifiziert. Dabei wird ein Ansatz verwendet, der auf der Erkenntnis beruht, dass in diesem Fall ein Auspuffbereich 34 (siehe 3) des Fremdfahrzeugs 32 sich üblicherweise in einem Bereich befindet, in dem auch hochreflektive Fahrzeugteile des Fremdfahrzeugs 32 angeordnet sind. Indem zu derartigen hochreflektiven Fahrzeugteilen gehörende bzw. davon stammende Punkte identifiziert werden, kann in einem Schritt bestimmt werden, ob es sich bei dem jeweiligen Objekt $O_j^1$

um ein Fremdfahrzeug 32 handelt und - falls dies der Fall ist - dessen Auspuffbereich 34 identifiziert bzw. lokalisiert werden.To exhaust clouds in the in 1 in the form of the second object 22 illustrated scenario, it is first determined whether it is at an object classified as a relevant obstacle $O_j^1$

is actually a third-party vehicle 32 and then identifies a possibly present exhaust gas cloud. An approach is used that is based on the knowledge that in this case an exhaust area 34 (see 3 ) of other vehicle 32 is usually located in an area in which highly reflective vehicle parts of other vehicle 32 are also arranged. By identifying points belonging to or originating from such highly reflective vehicle parts, it can be determined in one step whether this is the respective object $O_j^1$

it is a third-party vehicle 32 and--if this is the case--its exhaust area 34 is identified or localized.

wird eine Teilmenge H̃ ⊆ H bestimmt, sodass gilt $\tilde{H}=\left\{p_i\in H|\Vert p_i-X_c^H\Vert <D_{max}\right\}.$

Dabei ist D_max ein vorgegebener Parameter, der eine maximal zulässige Distanz eines Punktes p_i von $X_c^H$

angibt, womit von anderen Teilen des Fremdfahrzeug 32 stammende hochreflektive Punkte herausgefiltert werden können. Anhand der Teilmenge H̃ wird durch einen vorgegebenen Fitting-Algorithmus eine Referenzebene P konstruiert.For a given object or set of object points $O_j^1$

a subset H̃ ⊆ H is determined such that $\tilde{H}=\left\{p_i\in H|\Vert p_i-X_c^H\Vert <D_{max}\right\}.$

In this case, D _max is a predetermined parameter that defines a maximum permissible distance of a point p _i from $X_c^H$

indicates how highly reflective points originating from other parts of other vehicle 32 can be filtered out. Using the subset H̃, a reference plane P is constructed using a specified fitting algorithm.

3 shows a sectional schematic perspective view of other vehicle 32 with associated exhaust gas points 20, the ground 36 driven on by it, and reference plane P.

bestimmten Breite W verglichen. Falls gilt |W_B - W| < ε_w ist der Auspuffbereich 34 gefunden und das Objektpunkteset $O_j^1$

bzw. das dadurch repräsentierte Objekt wird als das Fremdfahrzeug 32 bestätigt. Der vorgegebene Parameter ε_w erlaubt dabei die Modellierung einer durch die Abgaswolke verursachten Variation oder Beeinflussung der detektierten Breite des Objekts, hier also etwa des zweiten Objekts 22.The possible exhaust area 34 of the other vehicle 32 is then represented by the point set B={p _i ∈ 0 _j | d(P, p _i )<D _min }where d(P, p _i ) is the Euclidean distance of the respective point p _i from the reference plane P and D _min is a predetermined maximum permissible distance. The dimensions of the possible exhaust area 34 of the possible other vehicle 32 are then determined using the point set B. The width W _B of the possible exhaust area 34 determined in this way is then compared with the one previously set for the respective object point set $O_j^1$

certain width W compared. If |W _B - W| < ε _w the exhaust area 34 is found and the object point set $O_j^1$

or the object represented thereby is confirmed as the other vehicle 32 . The specified parameter ε _w allows the modeling of a variation or influencing of the detected width of the object caused by the exhaust gas cloud, in this case, for example, the second object 22.

Dabei geben $X_C^B$

den Schwerpunkt, also das geometrische Zentrum des Punktesets B und X_c den Schwerpunkt, also das geometrische Zentrum des Objektpunktesets $O_j^1$

an.The exhaust area 34 of the other vehicle 32 is identified by a direction in which the exhaust gas exits from the exhaust area 34, expressed here as an exhaust gas exit direction $\widehat{n}=-\left(X_C^B-X_C\right).$

give in the process $X_C^B$

the center of gravity, ie the geometric center of the point set B and X _c the center of gravity, ie the geometric center of the object point set $O_j^1$

on.

wird eine Klassifikation bzw. ein Label g bestimmt und dem jeweiligen Punkt zugeordnet. Dadurch wird der jeweilige Punkt als Abgas oder als Nicht-Abgas klassifiziert gemäß $$g=\{\begin{array}{ll}\text{Abgas}\hfill & wenn{\underset{\_}{v}}_i\cdot \widehat{n}>0\wedge d\left(X_C^B,p_i\right)>D_{r\ddot{u}ck}\hfill \\ \text{Nicht}-\text{Abgas}\hfill & sonst\hfill \end{array}$$

For every point $\left(p_i,{\mathrm{right}}_i\right)\in O_j^1$

a classification or a label g is determined and assigned to the respective point. This classifies the respective point as an exhaust gas or as a non-exhaust gas according to $$G=\{\begin{array}{ll}\text{exhaust}\hfill & wenn{\underset{\_}{v}}_i\cdot \widehat{n}>0\wedge \mathrm{i.e}\left(X_C^B,p_i\right)>D_{\mathrm{right}\ddot{\mathrm{and}}ck}\hfill \\ \text{not}-\text{exhaust}\hfill & sOnst\hfill \end{array}$$

und $d\left(X_C^B,p_i\right)$

den euklidischen Abstand zwischen den beiden Punkten $X_C^B$

und p_i und D_rück einen vorgegebenen Abstandsschwellenwert, also einen vorgegebenen minimalen zulässigen Abstand angibt. Durch diese Klassifikation mit dem Label g wird jeder Punkt des jeweiligen Objektpunktesets $O_j^1,$

der außerhalb des Auspuffbereichs 34 ist, als Abgaspunkt 20 klassifiziert und damit die entsprechende Abgaswolke zumindest vorläufig erkannt oder definiert.Where v _i is a respective point centroid vector from the point p _i to the centroid $X_C^B$

and $\mathrm{i.e}\left(X_C^B,p_i\right)$

the Euclidean distance between the two points $X_C^B$

and p _i and D _back indicate a predetermined distance threshold value, ie a predetermined minimum permissible distance. Through this classification with the label g, every point of the respective object point set becomes $O_j^1,$

which is outside of the exhaust area 34, classified as an exhaust point 20 and thus at least provisionally recognized or defined the corresponding exhaust cloud.

It happens, however, that other vehicles 32 transport goods that are arranged at the exhaust area 34 or protrude there beyond the other vehicle 34 or the exhaust area 34 . In order to avoid incorrect classification in such cases, for the respective set G _j = {(p ₁ , r _i ) ^g ∈ 0 _j | g=exhaust} of exhaust points 20 its average reflectivity right _G and maximum reflectivity r̂ _G determined and compared with corresponding predetermined threshold values r ₁ , r ₂ . If applies right _G > r ₁ and r̂ _G > r ₂ , it can be assumed that there is a solid object in the area of the exhaust gas points 20 . In this case, the exhaust points 20 are reclassified as non-exhaust or non-exhaust points, so that G _j ={∅} then applies. On the other hand, it applies right _G <r ₁ and r̂ _G <r ₂ , then the exhaust gas point set G _j is retained and the exhaust gas points 20 are thus confirmed as such or as an exhaust gas cloud.

probabilistisch verifiziert. Dazu werden die aktuellen und in der Vergangenheit als Abgaspunkte 20 klassifizierten, also mit dem Label g = Abgas belegten Punkte verwendet. Dazu wird in jedem Takt oder Zeitschritt k für jedes Objektpunkteset $O_j^1$

mit |G_j| > 0 dessen Mittelwert µ und Standardabweichung σ der Punkte (p_i,r_i)^Abgas ∈ G_j gesichert als Detektionen q = {k, (µ, σ)} ∈ Q gespeichert. Dabei ist Q eine Menge oder Liste aller im Laufe der Zeit angefallenen Abgasdetektionen q. Zum Verifizieren der Klassifikation eines Objektpunktesets $O_j^2$

wird hier ein naiver Bayes-Klassifikator (NBC) mit maximaler-a-posteriori-Schätzung (MAP) verwendet.around the in 2 To consider the illustrated case of detached exhaust clouds, the classifications as possible exhaust clouds for the object point sets $O_j^2$

probabilistically verified. For this purpose, the current and past points classified as exhaust gas points 20, ie points assigned the label g=exhaust gas, are used. To do this, in each cycle or time step, k is added for each set of object points $O_j^1$

with |G _j | > 0 its mean µ and standard deviation σ of the points (p _i, r _i ) ^{exhaust gas} ∈ G _j saved as detections q = {k, (µ, σ)} ∈ Q saved. Q is a set or list of all exhaust gas detections q that have occurred over time. To verify the classification of a feature point set $O_j^2$

a naive Bayesian classifier (NBC) with maximum a posteriori estimate (MAP) is used here.

$$\widehat{y}=argma{x}_{a\in \left\{\mathrm{1,...,}A\right\}}p\left(C_a|\underset{\_}{x}\right)$$

For a given feature vector x = (x ₁ , ..., x _ƒ ), representing f independent features - here the three spatial coordinates x, y, y and the reflectivity r - and possible classes C _a , where a = 0 a Classification as exhaust cloud and a = 1 means classification as non-exposure cloud, a MAP decision rule based on the NBC is defined as: $$p\left(C_a|\underset{\_}{x}\right)=p\left(C_a\right){\displaystyle \prod _{i=1}^{ƒ}p\left(x_i|C_a\right)}$$

$$\widehat{y}=a\mathrm{right}Gma{x}_{a\in \left\{\mathrm{1,...,}A\right\}}p\left(C_a|\underset{\_}{x}\right)$$

In this case, p(C _a ) represents the previous probability of the class C _a and p(x _i | C _a ) represents the conditional probability of the feature x _i for a given class C _a . The classification of the feature vector x is given by the class label ŷ=C _a , where the class C _a is assigned with the higher probability.

The conditional probabilities are defined here for class C ₁ as: $$\{\begin{array}{l}N\left(x_i|\mu ,{\sigma }^2\right)wenni=\mathrm{1,2,3}\hfill \\ \lambda e^{-\lambda x_i},x_i\ge 0wenni=4\hfill \end{array}$$

Where N(x _i | µ,σ ² ) is a given normal distribution with mean µ and standard deviation σ evaluated at x _i and λe ^{-λx i} specifies a given exponential distribution with rate λ.

The conditional probabilities for class C ₀ are defined as: $$\{\begin{array}{ll}a{\left(x_i-\beta \right)}^2,x_i\in \left[\mu \pm 3\sigma \right]\hfill & wenni=\mathrm{1,2,3}\hfill \\ u\left(\mathrm{0,}{H}_{max}\right)\hfill & wenni=4\hfill \end{array}$$

angebend eine vertikale Skala der Verteilung. Werte die außerhalb des angegebenen Intervalls [µ ± 3σ] liegen, können verworfen werden oder unberücksichtigt bleiben. U(0, H_mαx) ist eine vorgegebene kontinuierliche Gleichverteilung mit Minimalwert 0 und Maximalwert H_max. H_max entspricht dabei der maximalem von dem Lidarsensor detektierten Reflektivität aller Punkte.where α(x _i - β) ² is a given u-squared distribution with parameters β=μ representing a center of the distribution, and $a=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$18{\sigma }^3$}\right.,$

indicating a vertical scale of distribution. Values that are outside the specified interval [µ ± 3σ] can be discarded or not taken into account. U(0, H _mαx ) is a predetermined continuous uniform distribution with a minimum value of 0 and a maximum value of H _max . H _max corresponds to the maximum reflectivity of all points detected by the lidar sensor.

wird dann wie folgt durchgeführt. Der Mittelwert µ̃_j aller Punkte $\left(p_i,r_i\right)\in O_j^2$

wird verwendet als Merkmalsvektor x. Für jede Detektion q ∈ Q werden deren Mittelwert µ und Standardabweichung σ verwendet, um wie angegeben die bedingten Wahrscheinlichkeiten p(x_i | C_a) zu bilden oder zu bestimmen. Falls ŷ = C₁ für wenigstens eine Detektion q ∈ Q gilt, wird das Objektpunkteset $O_j^2$

als Abgaswolke mit Klassifikationswahrscheinlichkeit p(C₁ | µ̃_j) und Punkten $G_j=O_j^2$

The classification of an object or a set of object points $O_j^2$

is then carried out as follows. The mean µ̃ _j of all points $\left(p_i,{\mathrm{right}}_i\right)\in O_j^2$

is used as the feature vector x. For each detection q ∈ Q, its mean µ and standard deviation σ are used to form or determine the conditional probabilities p(x _i | C _a ) as indicated. If ŷ = C ₁ for at least one detection q ∈ Q, the object point set $O_j^2$

as an exhaust cloud with classification probability p(C ₁ | µ̃ _j ) and points $G_j=O_j^2$

auf I = 0 gesetzt, das entsprechende Objektpunkteset O_j also als $O_j^0$

und damit als nicht-relevantes Hindernis umklassifiziert. If ŷ = C ₀ for each detection q ∈ Q, the classification or the label I becomes for the corresponding previous object point set $O_j^2$

is set to I = 0, so the corresponding object point set O _j as ${\mathrm{<figure-callout\; id="0"\; label="O\; j"\; filenames="DE102021116859A1\_0035.png"\; state="\{\{state\}\}">O</figure-callout>}}_j^{}$

and thus reclassified as a non-relevant obstacle.

In order to improve the performance of the method described, detections q can be removed from the set Q after a predetermined number K of clocks or journals k after they have been included in the set. In this way, it can be taken into account that exhaust gas clouds usually evaporate after a certain time and then no longer influence the lidar measurement. For example, with a clock or sampling frequency of the lidar sensor of 10 Hz, K can be in the range of 50 to 60 time steps, corresponding to about 5 s to 6 s. Depending on the situation or requirements, other values can also be used here.

Finally, all points G _k ⊆ P _k classified as exhaust gas can be derived or determined according to $$G_k=\left\{{\left(p_i,{\mathrm{right}}_i\right)}^G\in G_j|\forall O_j\in P_k\wedge G_j\in O_j\wedge G=\text{exhaust}\right\}$$

Corresponding data or information can then be output or made available as an output of a corresponding detection device of the motor vehicle, for example for further processing steps or to further assistance devices, for example for sensor fusion, object tracking or the like.

The method described can be carried out permanently or permanently or continuously during operation of the motor vehicle. It may also be possible for the motor vehicle or the detection device to be set up to use the method only be carried out if the respective ambient conditions allow condensation of water vapor contained in the exhaust gas to form water droplets or exhaust gas clouds that are visible and detectable by means of lidar, i.e. only when an ambient temperature is lower than a predetermined temperature threshold value, for example. Since otherwise the formation of exhaust gas clouds that can be detected by means of lidar is not to be expected, in such cases, for example, the detection device can be switched off completely or partially to save operating energy or the execution of the method described can be suspended.

Overall, the examples described show how rapid and efficient detection of vehicle exhaust clouds or detections or artefacts caused by them, in particular in weather conditions that favor corresponding condensation, can be made possible and implemented on the basis of lidar data.

In summary, a lidar point cloud recorded using the lidar sensor can first be spatially clustered and each cluster classified or labeled as a relevant obstacle, possible exhaust gas cloud or non-relevant obstacle. A predefined algorithm for identifying the associated exhaust area 34 can then be applied to all clusters classified as a relevant obstacle. Thereafter, current exhaust gas detections and a history of previous exhaust gas detections can be used to probabilistically determine whether the classification of clusters as an exhaust plume is correct or incorrect.

The method described enables the correct detection of exhaust gas clouds using a conventional ToF lidar sensor. In contrast to other methods, no training data is required. In addition, no specialized or dedicated calculation hardware is required to carry out the method in order to achieve real-time performance.

Reference List

10

detection range

12

ground points

14

single object points

16

first object

18

vehicle points

20

exhaust points

22

second object

24

Single exhaust cloud points

26

detached exhaust cloud

28

border points

30

edge object

32

foreign vehicle

34

exhaust area

36

underground

P

reference plane

n̂

exhaust gas outlet direction

vi

Point centroid vector

pi

Point

QUOTES INCLUDED IN DESCRIPTION

This list of the documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• US8818609B1 [0005]

Claims (10)

Method for the automated detection of exhaust gas clouds (20, 26) from other vehicles (32) by means of lidar, in which - a lidar point cloud with point-by-point spatial coordinates and point-by-point reflectivity is detected from a motor vehicle and by spatial clustering into object point sets, each of which contains an object ( 16, 22) in an environment of the motor vehicle, is grouped, - for each set of object points, object dimensions, a respective focal point and highly reflective points whose reflectivity corresponds to at least a predetermined reflectivity threshold value are determined, - based on the object dimensions and the highly reflective points, the objects according to predetermined Volume and reflectivity criteria are classified as a relevant obstacle (14, 16) or as a possible exhaust gas cloud (26) or as irrelevant (30) for driving the motor vehicle, - the relevant obstacles (14, 16) are each recognized as another vehicle (32). , if a width of an exhaust area chs (34) of the obstacle (14, 16) deviates at most by a predetermined amount from the width determined for the entire respective object point set, the exhaust area (34) being defined by a subset of the points (p _i ) of the obstacle (14, 16), which is determined according to a predetermined distance criterion relating to at least part of the object point set, is defined, - for the other vehicles (32) thus identified, all points (p _i ) of the corresponding object point set using a predetermined distance threshold value for a distance to a center of gravity of the exhaust area ( 34) representing subset of points and classified as exhaust gas points (20) or as non-exhaust gas points (18) according to a predetermined direction criterion that takes into account an exhaust gas exit direction (n̂) with respect to the exhaust area (34). procedure after claim 1 , characterized in that in each case for determining the exhaust area (34) defining subset of points according to the distance criterion - those of the highly reflective points whose distance from a focus of the set of highly reflective points corresponds at most to a predetermined maximum distance are determined as highly reflective exhaust area points, - by adapting to the determined, highly reflective exhaust area points, a plane (P) that is in particular at least essentially perpendicular to a surface under which traffic is driven is constructed by calculation, and - those points (p _i ) of the obstacle (14, 16) whose distance from the plane ( P) corresponds at most to a predetermined maximum distance in which the exhaust area (34) defining subset of points are included. Method according to one of the preceding claims, characterized in that according to the direction criterion those points are classified as exhaust gas points (20) for which the scalar product of the vector (v _i ) from the respective point (p _i ) to the center of gravity of the exhaust area ( 34) representing subset of points with a difference vector (n̂) formed from the centers of gravity of the subset of points representing the exhaust area (34) and the object point set is greater than zero. Method according to one of the preceding claims, characterized in that for the object point sets in each case - the average reflectivity of the exhaust gas points (20) and the maximum reflectivity of the exhaust gas points (20) are determined, - if corresponding predetermined threshold values are exceeded, the exhaust gas points (20) become non- Exhaust gas points (18) are reclassified, and - the exhaust gas points (20) are retained as such if the corresponding predetermined threshold values are not reached. Method according to one of the preceding claims, characterized in that the method is carried out repeatedly at several points in time, with a history of obstacle detections for the relevant obstacles (14, 16) at which at least one exhaust gas point (20) was detected, each of which time of detection, the point mean value and the standard deviation of the corresponding object point set, is stored, and for the obstacle detections the associated classifications as exhaust gas points (20) or non-exhaust gas points (18) by means of a predefined classifier based on the coordinates and reflectivities of the corresponding points (p _i ) are checked. procedure after claim 5 , characterized in that a probabilistic classifier is used as the classifier. procedure after claim 5 or 6 , characterized in that a naive Bayesian classifier, in particular with a maximum a posteriori estimation, is used as the classifier. procedure after claim 6 or 7 , characterized in that for the points (p _i ) defined by their spatial coordinates and their reflectivity, the following probabilities are specified for the classifier: - for the classification as an exhaust gas point: for the spatial coordinates by a specified normal distribution evaluated there and for the reflectivity by a specified exponential distribution, and - for the classification as a non-exhaust point: for the spatial coordinates by a specified u-square distribution and for the reflectivity by a continuous uniform distribution, in particular with a minimum value of zero and a maximum value corresponding to the maximum detected reflectivity. Detection device for automatically detecting exhaust gas clouds (20, 26) based on lidar data, the detection device having an input interface for detecting the lidar data and a processor device and a data memory coupled thereto for processing the lidar data and for automatically executing a method is set up according to one of the preceding claims. Motor vehicle, having a lidar detector and a detection device claim 9 .

Images