DE102021117877A1 - Generating a filtered point cloud and at least partially automatically driving a vehicle - Google Patents

Abstract

According to a method for generating a filtered point cloud (10) for driving a vehicle (1), a sensor data set that represents an environment of the vehicle (1) is generated using an active optical sensor system (4) and a sensor point cloud (8) is using a Arithmetic unit (3) generated. The filtered point cloud (10) is generated as a subset of the sensor point cloud (8) depending on a reference point cloud (9) with a large number of reference points, whereby for each of a large number of points in the sensor point cloud the respective point in the sensor point cloud (8) only becomes the filtered point cloud (10) is added if a spatial area that corresponds to a predefined environment of the respective point of the sensor point cloud (8) is free of reference points.

Description

The present invention is directed to a method for generating a filtered point cloud for at least partially automatic driving of a vehicle, with a sensor data set representing the surroundings of the vehicle being generated by means of an active optical sensor system of the vehicle and a sensor point cloud by means of at least one computing unit of the vehicle is generated depending on the sensor data set. The invention is also directed to a method for at least partially automatically driving a vehicle, to an electronic vehicle guidance system, to a computer program and to a computer-readable storage medium.

Many automatic and partially automatic driving functions for vehicles, in particular motor vehicles, such as autonomous driving functions or driver assistance systems, ADAS, are based on the efficient processing of data that is generated using various sensors in the vehicle. In particular, active optical sensor systems, such as lidar systems, can provide valuable input to these functions. Such active optical sensor systems can provide sensor data that can be in the form of a two-dimensional or two-dimensional point cloud or can be converted into such a point cloud. However, a point cloud obtained via an active optical sensor system potentially contains a large number of points for each sensor frame. Consequently, processing such large point clouds requires a significant amount of computational resources in terms of memory and computation time. This is a disadvantage, particularly with regard to embedded systems that are commonly used in connection with automotive applications.

It is an object of the present invention to reduce the computational requirements for processing sensor data for at least partially automatically driving a vehicle, in particular sensor data obtained by means of an active optical sensor system.

This object is achieved by the subject matter of the independent claims. Further implementations and preferred embodiments are subject of the dependent claims.

The invention is based on the idea of filtering a sensor point cloud by comparing the sensor point cloud with a predefined reference point cloud, so that the filtered point cloud only contains points that have no corresponding reference point in a predefined environment.

According to one aspect of the invention, a method for guiding a filtered point cloud that is suitable for being used for at least partially automatically guiding a vehicle, in particular a motor vehicle, is provided. In this case, a sensor data record that represents the surroundings of the vehicle is generated using an active optical sensor system in the vehicle, and a sensor point cloud is generated using at least one computing unit in the vehicle as a function of the sensor data record. The filtered point cloud is generated by the at least one computing unit as a subset of the sensor point cloud depending on a predetermined reference point cloud that contains a large number of reference points. The sensor point cloud contains a large number of points, and for each of the points in the sensor point cloud, the respective point in the sensor point cloud is only added to the filtered point cloud using the at least one computing unit if a spatial area that corresponds to a predefined environment of the respective point in the sensor point cloud is Reference points is free, especially when the spatial area of reference points of the reference point cloud is free.

Here and in the following, "light" can be understood in such a way that it includes electromagnetic waves in the visible range, in the infrared range and/or in the ultraviolet range. Consequently, the term "optical" can be understood to refer to light as defined.

By definition, an active optical sensor system has a light source for emitting light or light pulses. The light source can be implemented, for example, as a laser, in particular as an infrared laser. Furthermore, according to definition, an active optical sensor system has at least one optical detector in order to detect reflected parts of the emitted light. In particular, the active optical sensor system is set up to generate one or more sensor signals, which represent the sensor data set, based on the detected components of the light, and to process and/or output the sensor signals.

The active optical sensor system can be implemented as a lidar system, for example. A known design of lidar systems are so-called laser scanners, in which a laser beam is deflected by means of a deflection unit, so that different deflection angles of the laser beam can be realized. The deflection unit can, for example wise contain a rotatably mounted mirror. Alternatively, the deflection unit may include a mirror element with a tiltable and/or pivotable surface. The mirror element can be configured as a microelectromechanical system, MEMS, for example. The emitted laser beams can be partially reflected in the surroundings and the reflected portions can in turn impinge on the laser scanner, in particular the deflection unit, which can direct them to a detector unit of the laser scanner that has the at least one optical detector. In particular, each optical detector of the detector unit generates an associated detector signal on the basis of the components detected by the respective optical detector. On the basis of the spatial arrangement of the respective optical detector together with the current position of the deflection unit, in particular its rotational position and/or its tilting and/or pivoting position, it is thus possible to infer the direction of incidence of the detected reflected components of the light. A processing unit or an evaluation unit of the laser scanner can, for example, carry out a transit time measurement in order to determine a radial distance of the reflecting object. Alternatively or additionally, a method according to which a phase difference between the emitted and detected light is evaluated can be used to determine the distance.

A point cloud can be thought of as a set of points, each point characterized by respective coordinates in a two-dimensional or three-dimensional coordinate system. For example, in the case of a three-dimensional point cloud, the three-dimensional coordinates may be determined by the direction of incidence of the reflected components of light and the corresponding transit time or radial distance measured for that particular point. In other words, the three-dimensional coordinate system can be a three-dimensional polar coordinate system. However, the information can also be pre-processed to provide three-dimensional Cartesian coordinates for each of the points. In general, the points of a point cloud are given in a disordered or unsorted way, as opposed to, for example, a camera image. In addition to the spatial information, namely the two-dimensional or three-dimensional coordinates, the point cloud can also store additional information or measured values for the individual points, such as an echo pulse width, EPW, of the respective sensor signal.

A computing unit can be understood in particular as a data processing device. The arithmetic unit can therefore process data in particular in order to carry out arithmetic operations. This can also include operations to perform indexed accesses to a data structure such as a lookup table, LUT.

In particular, the processing unit can contain a computer, a microcontroller or an integrated circuit, for example an application-specific integrated circuit, ASIC, a user-programmable logic array, FPGA, a system on a chip, SoC, and/or an electronic control unit, ECU. The processing unit can also contain a processor, for example a microprocessor, a central processing unit, CPU, a graphics processing unit, GPU, and/or a signal processor, in particular a digital signal processor, DSP. The processing unit can also contain a physical or a virtual cluster of computers or other of the units. In this case, the computing unit can also be referred to as a computing system, for example.

In various embodiments, the computing unit contains one or more hardware and/or software interfaces and/or one or more memory units. A memory device may be volatile data storage, such as dynamic random access memory, DRAM, static random access memory, SRAM, or non-volatile data storage, such as read only memory, ROM, programmable read only memory, PROM, erasable read only memory, EPROM, electrically erasable read only memory, EEPROM, flash memory, or flash EEPROM, ferroelectric random access memory, FRAM, magnetoresistive random access memory, MRAM, or phase change random access memory, PCRAM.

The at least one computing unit can contain, for example, a preprocessing or evaluation unit of the active optical sensor system and/or one or more computing units of the vehicle, which are arranged separately from the active optical sensor system. In particular, the sensor point cloud can be partially or completely generated on the basis of the sensor data set by means of the pre-processing or evaluation unit of the active optical sensor system. However, the sensor point cloud can also be generated partially or completely by means of the one or more computing units outside of the active optical sensor system.

The cloud of reference points is stored in particular on a storage unit of the vehicle or a storage unit of the at least one computing unit and also represents the surroundings of the vehicle. In general, the reference point cloud determined before the method according to the invention and stored on the storage unit so that it can be viewed as a map, in particular a high-resolution map, which is static and represents the environment at a previous point in time. However, in some implementations the determination and storage of the reference point cloud on the memory can also be considered as part of the method according to the invention.

The reference point cloud can cover a larger spatial area in the environment than the sensor point cloud. The reference point cloud can be determined by means of one or more other active optical sensor systems or laser scanners and so on, which are mounted on one or more other vehicles.

Since each point of the sensor point cloud as well as each reference point of the reference point cloud corresponds to a well-defined position in the real environment, the sensor point cloud and the reference point cloud or, in other words, their respective positions can be compared with each other and a distance between individual points of the sensor point cloud and individual points of the reference point cloud can be determined. In this way, the at least one computing unit can analyze for each of the points in the sensor point cloud whether the spatial area in the real world that corresponds to the area surrounding the respective point contains any of the reference points in the reference point cloud. This can be done by analyzing the respective spatial area for each of the considered points of the sensor point cloud. Alternatively, respective surroundings of the reference points of the reference point cloud can be analyzed with respect to points of the sensor point cloud. In both ways it can be ensured that only those points of the sensor point cloud are added to the filtered point cloud that have a spatial area that is free of reference points. In other words, points of the sensor point cloud that are closer to any reference point of the reference point cloud are not added to the filtered point cloud. In other words, the respective point of the sensor point cloud is discarded to generate the filtered point cloud.

In this way, the number of points in the filtered point cloud is significantly reduced compared to the number of points in the sensor point cloud. Nevertheless, the loss of information due to filtering is acceptable, since the reference point cloud contains the respective reference points in the spatial area around the rejected point of the sensor point cloud. Since measurements using active optical sensor systems are generally not absolutely exact, it is necessary to define a specific environment of the points of the sensor point cloud for analysis with regard to reference points. The environment can be defined, for example, by a respective threshold distance.

The sensor point cloud, which can also be viewed as a live point cloud or online point cloud of a respective frame during operation of the vehicle, contains points corresponding to objects that were already present in the environment at the time the reference point cloud was created. These points correspond, for example, to an infrastructure in the area, parts of a road or other objects which can be referred to below as static objects. In addition, the sensor point cloud contains points that correspond to objects that were not present in the environment at the time the reference point cloud was created. These objects may include, for example, other road users such as additional vehicles, pedestrians, etc., or infrastructure changes.

By generating the filtered point cloud in the manner described, it is achieved that for each frame the points corresponding to static objects can be removed from the sensor point cloud, which significantly reduces the amount of points to be processed and stored and therefore the required computing time and the required memory is significantly reduced. Therefore, the subsequent processing of the filtered point cloud instead of the sensor point cloud, in some implementations together with the reference point cloud, enables a reliable and robust perception of the environment with reduced computational requirements.

The filtered point cloud can then be used, in particular by means of an electronic vehicle guidance system of the vehicle, as an input for at least partially automatically guiding the vehicle. The at least one computing unit can, for example, use the filtered point cloud as input for an automatic perception algorithm, for example a computer vision algorithm or another perception algorithm.

Visual perception algorithms, also referred to as automatic visual perception algorithms, computer vision algorithms, or computer vision algorithms, can be viewed as computer algorithms for automatically performing a visual perception task. For example, a visual perception task can be understood as a task for extracting information from image data. In particular, the visual perception task can in principle be performed by a human who can visually perceive an image corresponding to image data. However, using a visual perception algorithm, the visual perception task is performed automatically without requiring human assistance. An algorithm for visual perception can be understood, for example, as an image processing algorithm or algorithm for image analysis, which can be trained using machine learning, for example, and can be based in particular on an artificial neural network, for example a convolutional neural network. Examples of visual perception algorithms include object recognition algorithms, object tracking algorithms, classification algorithms, particularly image classification algorithms, and/or segmentation algorithms, particularly semantic segmentation algorithms.

Corresponding algorithms can be implemented analogously on the basis of input data other than images that can be visually perceived by a human being. Point clouds or images from infrared cameras and so on, for example, can also be analyzed using appropriately adapted computer algorithms. Strictly speaking, however, the corresponding algorithms are not algorithms for visual perception, since the corresponding sensors can work in areas that are imperceptible to the human eye, such as the infrared range. Therefore, here and in the following such algorithms are referred to as perception or automatic perception algorithms. Perception algorithms therefore include, but are not limited to, visual perception algorithms with respect to human perception. Consequently, a perception algorithm according to this understanding can be regarded as a computer algorithm for automatically performing a perception task, for example using an algorithm for sensor data analysis or processing of sensor data, which can be, for example, using machine learning and can be based, for example, on an artificial neural network. The generalized perception algorithms may also include object recognition algorithms, object tracking algorithms, classification algorithms, and/or segmentation algorithms, such as semantic segmentation algorithms.

If an artificial neural network is used to implement a visual perception algorithm, a commonly used architecture is a convolutional neural network, CNN. In particular, a 2D CNN can be applied to respective 2D camera images. CNNs can also be used for other perception algorithms. For example, 3D-CNNs, 2D-CNNs or 1D-CNNs can be applied to point clouds depending on the spatial dimensions of the point cloud and the details of the processing.

The output of a perceptual algorithm depends on the specific underlying perceptual task. For example, an output of an object detection algorithm may include one or more bounding boxes that define a spatial location and optionally an orientation of one or more respective objects in the environment and/or corresponding object classes for the one or more objects. A semantic segmentation algorithm applied to a camera image may contain a pixel-level class for each pixel of the camera image. Similarly, a semantic segmentation algorithm applied to a point cloud may contain a corresponding point-level class for each of the points. For example, the pixel-level or point-level classes may define a type of object to which each pixel or point belongs.

According to several implementations of the method for generating a filtered point cloud, the at least one computing unit is used to identify the spatial area for each of the plurality of points in the sensor point cloud that corresponds to the predefined environment of the respective point in the sensor point cloud, and it is determined whether the spatial area of reference points is free.

According to several implementations, the spatial region is identified as a spherical region having a predefined radius, and the respective point of the sensor point cloud corresponds to a center of this spherical region.

In other words, only those points of the sensor point cloud are added to the filtered point cloud that have a distance to all reference points that is greater than the radius of the spherical region.

According to several implementations, a further spatial area, which corresponds to a predefined further environment of the respective reference point of the reference point cloud, is identified by means of the at least one computing unit for all reference points of the reference point cloud, and it is determined whether the further spatial area of points of the sensor point cloud is free.

According to a further aspect of the invention, a method for at least partially automatically driving a vehicle is provided. Thereby, a filtered point cloud is generated by performing a method for generating a filtered point cloud according to the invention. At least one control signal for at least partially automatically guiding the vehicle is generated as a function of the filtered point cloud, in particular by means of a control unit of the vehicle, for example the at least one computing unit. In particular, the vehicle is guided at least partially automatically as a function of the at least one control signal.

The vehicle can have an electronic vehicle guidance system, for example, which has the at least one computing unit, the active optical sensor system and the control unit, and the method for at least partially automatically driving a vehicle is carried out using the electronic vehicle guidance system.

An electronic vehicle guidance system can be understood as an electronic system configured to guide a vehicle in a fully automated or in a fully autonomous manner and in particular without requiring manual intervention or control by a driver or user of the vehicle. The vehicle automatically carries out all necessary functions such as steering manoeuvres, deceleration maneuvers and/or acceleration manoeuvres, as well as monitoring and recording road traffic and corresponding reactions. In particular, the electronic vehicle guidance system can implement a fully automatic or fully autonomous driving mode according to level 5 of the SAE J3016 classification. An electronic vehicle guidance system can also be implemented as a driver assistance system, ADAS, which supports a driver for partially automatic or partially autonomous driving. In particular, the electronic vehicle guidance system can implement a partially automatic or partially autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification. Here and in the following, SAE J3016 refers to the respective standard dated June 2018.

Thus, at least partially automatically driving the vehicle may include driving the vehicle in accordance with a fully automated or fully autonomous driving mode according to level 5 of the SAE J3016 classification. Driving the vehicle at least partially automatically may also include driving the vehicle in accordance with a partially automated or partially autonomous driving mode according to levels 1 through 4 of the SAE J3016 classification.

The at least one control signal can be provided to one or more actuators of the vehicle in order to correspondingly implement the automatic or partially automatic control of the vehicle.

According to several implementations of the method for at least partially automatically driving a vehicle, a perception algorithm is executed by means of the at least one computing unit, the filtered point cloud being used as input to the perception algorithm. The at least one control signal is generated depending on an output of the perception algorithm.

It is understood that the output of the perception algorithm, for example a segmented image, a segmented point cloud, object tracking information, bounding boxes and so on, can be further processed or used in further processing steps to determine one or more parameters or the like for at least partially automatically guiding the vehicle to determine, and the at least one control signal is generated depending on the one or more parameters.

According to several implementations, the reference point cloud is used as further input of the perception algorithm, in particular for generating the output of the perception algorithm.

In particular, the reference point cloud serves not only as a resource for filtering the sensor point cloud, but also as an additional source of information, particularly with regard to the static objects in the environment. By taking into account the reference point cloud and the filtered point cloud for the perception algorithm, improved functionality and/or greater reliability of the output from the perception algorithm is achieved.

According to several implementations, the at least one control signal is generated dependent on the output of the perception algorithm and dependent on the reference point cloud.

In other words, the at least one control signal is generated with explicit consideration of the reference point cloud and not just its influence on its filtered point cloud.

According to several implementations, one or more clusters of points of the gefil The third point cloud is determined by the at least one computing unit, and a bounding box and/or an object class is assigned to each of the plurality of clusters by the at least one computing unit. The at least one control signal is generated depending on the assigned bounding boxes and/or assigned object classes.

As such, determining the one or more clusters and assigning the bounding boxes and/or object classes to each of the clusters can be considered part of the perception algorithm.

In particular, the perception algorithm may include an object detection algorithm applied directly to the filtered point cloud or applied directly to the filtered point cloud and the reference point cloud.

The bounding boxes assigned to each of the clusters may correspond to two-dimensional geometric figures if the filtered point cloud is provided as a two-dimensional point cloud, or three-dimensional geometric figures if the filtered point cloud is provided as a three-dimensional point cloud. The respective bounding boxes can be implemented, for example, as rectangles or cuboids.

The object classes assigned to the clusters can correspond to the respective types of objects represented by the corresponding points of the cluster, or in other words the type of object causing the corresponding reflections of light emitted by means of the active optical sensor system will, correspond. The bounding boxes and/or the object classes can be viewed as the output of the perception algorithm.

According to several implementations, a camera image representing the surroundings of the vehicle is generated using a camera system of the vehicle. One or more clusters of points of the filtered point cloud are determined using the at least one computing unit and a region of interest of the camera image is determined using the at least one computing unit depending on the one or more clusters determined. An object recognition algorithm is applied to the area of interest by the at least one computing unit. The at least one control signal is generated depending on an output of the object detection algorithm.

In other words, the object detection algorithm is applied to the camera image, restricting the object detection algorithm to the area of interest. It is understood that the area of interest may include one or more areas of the camera image, which may or may not be connected to each other. In this way, the computational effort, computation time, and memory requirements for executing the object detection algorithm are reduced.

Again, the output of the object detection algorithm may include respective bounding boxes and/or object classes for each of the objects identified in the area of interest of the camera image. The area of interest may include one or more connected or separate areas, for example up to one area for each of the identified clusters.

According to another aspect of the invention, an electronic vehicle guidance system for a vehicle is provided. The electronic vehicle guidance system has an active optical sensor system for mounting on the vehicle, which is set up to generate a sensor data set that represents an area surrounding the vehicle, in particular when it has been mounted on the vehicle. The electronic vehicle guidance system has at least one computing unit that is set up to generate a sensor point cloud as a function of the sensor data set. The at least one computing unit is set up to generate the filtered point cloud as a subset of the sensor point cloud depending on a predetermined reference point cloud with a large number of reference points. The at least one computing unit is set up to only add the respective point in the sensor point cloud to the filtered point cloud if a spatial area that corresponds to a predefined environment of the respective point in the sensor point cloud is free of reference points in the reference point cloud.

According to several implementations of the electronic vehicle guidance system, the active optical sensor system is designed as a lidar system, in particular a laser scanner lidar system.

According to several implementations, the electronic vehicle guidance system, in particular the at least one computing unit, has a control unit that is set up to generate at least one control signal for at least partially automatically guiding the vehicle depending on the filtered point cloud.

According to several implementations, the electronic vehicle guidance system has a camera system, in particular for mounting on the vehicle, which is set up to generate a camera image that represents the surroundings of the vehicle, in particular when it has been mounted on the vehicle. The at least one computing unit is set up to determine one or more clusters of points of the filtered point cloud and to determine an area of interest of the camera image depending on the one or more specific clusters. The at least one computing unit is set up to apply an object recognition algorithm to the area of interest. The control unit is set up to generate the at least one control signal depending on an output of the object recognition algorithm.

Further implementations of the electronic vehicle guidance system according to the invention follow directly from the different implementations of the method for generating a filtered point cloud according to the invention and the implementations of the method for guiding a vehicle according to the invention and vice versa. In particular, an electronic vehicle guidance system according to the invention is set up to carry out a method according to the invention or carries out such a method.

According to a further aspect of the invention, a vehicle, in particular a motor vehicle, is provided which has an electronic vehicle guidance system according to the invention.

According to another aspect of the invention, a computer program with instructions is provided. If the computer program or the commands are executed using an electronic vehicle guidance system according to the invention, in particular using the at least one computing unit of the electronic vehicle guidance system, the commands cause the electronic vehicle guidance system to execute a method according to the invention.

According to another aspect of the invention, there is provided a computer-readable storage medium storing a computer program according to the invention.

Further features of the invention can be seen from the claims, the figures and the description of the figures. The features and feature combinations mentioned above in the description and the features and feature combinations mentioned below in the description of the figures and/or shown in the figures can be included in the invention not only in the respective specified combination, but also in other combinations. In particular, embodiments and combinations of features that do not have all the features of an originally formulated claim are also included in the invention. Moreover, embodiments and combinations of features that go beyond or deviate from the combinations of features set out in the recitations of the claims are included in the invention.

• 1 schematisch ein Fahrzeug mit einer beispielhaften Implementierung eines elektronischen Fahrzeugführungssystems gemäß der Erfindung;
• 2 ein schematisches Ablaufdiagramm einer beispielhaften Implementierung eines Verfahrens gemäß der Erfindung;
• 3 schematisch ein Kamerabild, das eine Umgebung eines Fahrzeugs darstellt;
• 4 schematisch eine Sensorpunktwolke, die eine Umgebung eines Fahrzeugs darstellt;
• 5 schematisch eine Referenzpunktwolke;
• 6 schematisch eine gefilterte Punktwolke;
• 7 schematisch Cluster von Punkten einer gefilterten Punktwolke;
• 8 schematisch ein Kamerabild und eine Referenzpunktwolke; und
• 9 schematisch ein Kamerabild und eine Referenzpunktwolke.

In the figures show:

• 1 schematically a vehicle with an exemplary implementation of an electronic vehicle guidance system according to the invention;
• 2 a schematic flow diagram of an exemplary implementation of a method according to the invention;
• 3 schematically a camera image that represents the surroundings of a vehicle;
• 4 schematically a sensor point cloud that represents an environment of a vehicle;
• 5 schematically a reference point cloud;
• 6 schematically a filtered point cloud;
• 7 schematic clusters of points of a filtered point cloud;
• 8th schematically a camera image and a reference point cloud; and
• 9 schematically a camera image and a reference point cloud.

1 1 schematically shows a motor vehicle 1 with an exemplary implementation of an electronic vehicle guidance system 2 according to the invention.

The electronic vehicle guidance system 2 has at least one computing unit 3 and a lidar system 4, which is implemented as a laser scanner, for example. The vehicle guidance system 2 optionally also has a camera 5 which is mounted on the vehicle 1 .

The vehicle guidance system 2 can carry out a method for generating a filtered point cloud according to the invention and/or a method for at least partially automatically guiding the vehicle 1 according to the invention. The function of the vehicle guidance system 2 and corresponding methods are described below with reference to the figures 2 until 7 explained in more detail.

2 shows a flowchart of an exemplary implementation of a method for Generating a filtered point cloud according to the invention. Thereby, step S1 can be considered as part of the method or can be performed before the method according to the invention, while steps S2 and S3 are included in the method according to the invention. Step S4 is an optional step of the method according to the invention.

In step S1, a reference point cloud 9, as shown schematically in 5 shown, which represents an environment of the vehicle 1, generated and stored on a memory unit (not shown) of the at least one computing unit 3. To this end, one or more further vehicles (not shown) can be equipped with respective lidar systems in order to acquire a large number of reference sampling points which represent the road and other static objects in the surroundings. The reference point cloud 9 can then be viewed as a high-resolution map, HD map, of the environment. The reference point cloud 9 can also be viewed as an offline HD map, which remains unchanged during subsequent individual individual sensor images of the lidar system 4 . The reference point cloud 9 contains points corresponding to infrastructure elements such as trees, buildings, the road, the landscape and so on. Mobile or dynamic objects, such as pedestrians, vehicles, mobile barriers and so on, are preferably not included in the cloud of reference points 9 .

The at least one computing unit 3 can determine the position of the vehicle 1 in the reference coordinate system, which corresponds to the reference point cloud 9, using known methods, for example based on the evaluation of GPS signals and various sensor data of the vehicle, including but not limited to data which are obtained by means of the lidar system 4 and/or the camera 5.

In step S2, the lidar system 4 generates a sensor data set during each individual sensor image, which represents the surroundings of the vehicle 1, and the at least one computing unit 3 generates a sensor point cloud 8 depending on the sensor data set, as is shown schematically in 4 shown. The sensor point cloud 8 contains points that correspond to the static objects that are also represented by the reference point cloud 9 . In addition, the sensor point cloud may contain 8 additional points corresponding to changes in the environment or infrastructure, or mobile or dynamic objects in the environment. Some examples of such objects are shown schematically in the camera image of 3 shown. For example, people 7a, 7b or other objects 7c, 7d can be in the scene.

In step S3, the at least one computing unit 3 can compare the sensor point cloud 8 with the reference point cloud 9 in order to generate a filtered point cloud 10, as in 6 shown, where the static objects are effectively removed from the sensor point cloud 8. To this end, the at least one computing unit 3 can analyze a spatial region that corresponds to a predefined environment of a given point in the sensor point cloud 8 and discard this point or, in other words, not add this point to the filtered point cloud 10 if a reference point in the reference point cloud 9 in the vicinity of the respective point of the sensor point cloud 8 is located. Only points of the sensor point cloud 8 that do not have a reference point of the reference point cloud 9 that is closer than a predefined threshold distance, for example 10 cm, are added to the filtered point cloud 10, for example.

6 shows respective clusters of points 11a, 11b, 11c, 11d in the filtered point cloud 10, which are assigned to the objects 7a, 7b, 7c, 7d in the camera image 6.

In step S4, the filtered point cloud 10 can then be used by the at least one computing unit 3 to execute a perception algorithm, for example an object recognition algorithm or a segmentation algorithm.

For this purpose, the at least one computing unit 3 can apply the perception algorithm directly to the filtered point cloud 10 . Alternatively, the at least one computing unit 3 can use the filtered point cloud 10 to determine a respective area of interest in the camera image 6 and apply the perception algorithm to the camera image 6 while restricting it to the area of interest.

In 7 shows schematically that one or more clusters 12a, 12b of points in the filtered point cloud 10 are identified by means of the at least one computing unit 3. Known approaches or known methods can be used to extract the clusters 12a, 12b. After the clustering, the locations of the clusters 12a, 12b in the camera image 6 can be determined. The coordinates of the parameters of the clusters 12a, 12b can be extracted, for example, in the coordinates of the local vehicle coordinate system.

For each cluster 12a, 12b, the coordinates of the cluster in the vehicle coordinate system can be transformed into pixel coordinates of the camera system using the respective calibration parameters for the camera 5. the ent corresponding pixel coordinates can be referred to as an area of interest. The at least one computing unit 3 can then apply the classification and/or recognition algorithm to the area of interest and deliver its result to the following software modules of the perception system.

In this way, a significant reduction in processing power and time consumption can be achieved compared to conventional camera recognition systems, where the entire image has to be searched for patterns to be recognized.

It is noted that a precise HD map with an infrastructure and other static obstacles regarding the reference point cloud 9 can also be used for other applications related to automated driving. Vehicle systems for automated driving can be equipped with software processing modules for estimating the possible trajectory of a surrounding object based on its past trajectories, its orientation, its motion model, other properties and so on. The possible trajectories of the driver's vehicle can also be estimated analogously and a respective trajectory that the driver's vehicle should follow can be selected. Among the trajectories that are calculated, some may be discarded due to possible collision with surrounding objects, as they may violate traffic rules and/or they may lead to collisions with infrastructure obstacles.

Assuming that a precise HD map with infrastructure obstacles, such as the reference point cloud 9, is given and that the vehicle 1 is located precisely in this HD map, it is possible that a trajectory estimation system of the vehicle 1 uses this trajectory calculation or Estimation refined because the HD map provides information about the infrastructure that can go beyond the field of view of the sensors 4, 5 of the vehicle. The passable area for the vehicle 1 can then be estimated, even if the infrastructure is not in the direct line of sight with the sensors 4, 5 or is covered.

In 8th an example of an HD point cloud map 13 and an inset showing a corresponding camera image 15 is shown. For example, the sensor perception of the vehicle is limited to approximately 150 m. However, the information contained in the point cloud map 13 enables a description of the infrastructure of several hundreds of meters including, for example, the roadsides. Box 14 indicates the extent to which the at least one computing unit 3 can estimate a safe trajectory for vehicle 1 .

9 14 shows another example of an HD point cloud map 16 and an inset showing a corresponding camera image 18 . The box 17 indicates a position of an emergency area for the vehicle 1 to stop safely, which is identifiable in the point cloud map 16 but not in the field of view of the camera 5 at the time of the snapshot given by the camera image 18 .

Claims (14)

Method for generating a filtered point cloud (10) for the at least partially automatic driving of a vehicle (1), wherein a sensor data record, which represents an area surrounding the vehicle (1), is generated by means of an active optical sensor system (4) of the vehicle (1) and a sensor point cloud (8) is generated by means of at least one computing unit (3) of the vehicle (1) depending on the sensor data set, characterized in that - the filtered point cloud (10) by means of the at least one computing unit (3) depending on a predetermined reference point cloud (9 ), which contains a large number of reference points, is generated as a subset of the sensor point cloud (8); - for each of a large number of points in the sensor point cloud (8), the respective point in the sensor point cloud (8) is only added to the filtered point cloud (10) by means of the at least one computing unit (3) if a spatial area that corresponds to a predefined environment of the corresponding point of the sensor point cloud (8) is free of reference points. procedure after claim 1 , characterized in that by means of the at least one computing unit (3) for each of the plurality of points in the sensor point cloud (8) the spatial area that corresponds to the predefined environment of the respective point in the sensor point cloud (8) is identified and it is determined whether the spatial area is free of reference points. procedure after claim 1 , characterized in that by means of the at least one computing unit (3) for all reference points of the reference point cloud (9) a further spatial area, which corresponds to a predefined further environment of the respective reference point of the reference point cloud (9), is identified and it is determined whether the further spatial area is free of points of the sensor point cloud (8). A method for at least partially automatically driving a vehicle (1), characterized in that - a filtered point cloud (10) is generated by performing a method according to any one of the preceding claims; and - at least one control signal for at least partially automatically guiding the vehicle (1) is generated as a function of the filtered point cloud (10). procedure after claim 4 , characterized in that - a perception algorithm is executed by means of the at least one computing unit (3), the filtered point cloud (10) being used as input to the perception algorithm; and - the at least one control signal is generated dependent on an output of the perception algorithm. procedure after claim 5 , characterized in that - the reference point cloud (9) is used as a further input to the perception algorithm; and/or - the at least one control signal is generated depending on the output of the perception algorithm and depending on the reference point cloud (9). Procedure according to one of Claims 4 until 6 , characterized in that - by means of the at least one computing unit (3), one or more clusters (11a, 11b, 11c, 11d, 12a, 12b) of points in the filtered point cloud (10) are determined and each of the plurality of clusters (11a , 11b, 11c, 11d, 12a, 12b) a bounding box and/or an object class are assigned; and - the at least one control signal is generated depending on the assigned bounding boxes and/or object classes. Procedure according to one of Claims 4 until 6 , characterized in that - a camera image (6) representing the surroundings of the vehicle (1) is generated by means of a camera system (5) of the vehicle (1); - one or more clusters (11a, 11b, 11c, 11d, 12a, 12b) of points in the filtered point cloud (10) are determined by means of the at least one computing unit (3); - a region of interest of the camera image (6) is determined by means of the at least one computing unit (3) depending on the one or more specific clusters (11a, 11b, 11c, 11d, 12a, 12b); - an object recognition algorithm is applied to the region of interest by means of the at least one computing unit (3); and - the at least one control signal is generated depending on an output of the object detection algorithm. Electronic vehicle guidance system (2) for a vehicle (1), the electronic vehicle guidance system (2) comprising - an active optical sensor system (4) which is set up to generate a sensor data record which represents an environment of the vehicle (1); and - at least one computing unit (3) which is set up to generate a sensor point cloud (8) depending on the sensor data set, characterized in that the at least one computing unit (3) is set up to - the filtered point cloud (10) as a subset to generate the sensor point cloud (8) depending on a predetermined reference point cloud (9) with a large number of reference points; and - only adding the respective point of the sensor point cloud (8) to the filtered point cloud (10) for each of a plurality of points of the sensor point cloud (8) if a spatial area corresponding to a predefined environment of the respective point of the sensor point cloud (8), is free of reference points. Electronic vehicle guidance system (2) after claim 9 , characterized in that the active optical sensor system (4) is designed as a laser scanner. Electronic vehicle guidance system (2) according to one of claims 9 or 10 , characterized in that the electronic vehicle guidance system (2) has a control unit which is set up to generate at least one control signal for at least partially automatically guiding the vehicle (1) depending on the filtered point cloud (10). Electronic vehicle guidance system (2) according to one of claims 9 until 11 , characterized in that - the electronic vehicle guidance system (2) has a camera system (5) for the vehicle (1), which is set up to generate a camera image (6) representing the surroundings of the vehicle (1); and - the at least one computing unit (3) is set up to - determine one or more clusters (11a, 11b, 11c, 11d, 12a, 12b) of points in the filtered point cloud (10); - to determine an area of interest of the camera image (6) depending on the one or more determined clusters (11a, 11b, 11c, 11d, 12a, 12b); and - apply an object detection algorithm to the region of interest; and - the control unit is set up to generate the at least one control signal depending on an output of the object recognition algorithm. Computer program with commands which, when executed by an electronic vehicle guidance system (2) according to one of claims 9 until 12 be executed, the electronic vehicle guidance system (2) cause a method according to one of Claims 1 until 8th to execute. Computer-readable storage medium containing a computer program Claim 13 saves.

Images