DE102021000803A1 - Method for training a neural network of an electronic computing device of a motor vehicle - Google Patents

Abstract

The invention relates to a method for training a neural network (10) of an electronic computing device (12) of a motor vehicle, in which environmental data (14) are provided with at least one object and virtual training data (16) are provided for a further object, the neural network (10) is trained at least with the environment data (14) and the virtual training data (16), the virtual training data (14) for the further object being changed as a function of a parameter (18) and as further virtual training data ( 20) are provided for yet another object and the neural network (10) is trained by means of the environment data (14), the virtual training data (16) and the further virtual training data (18).

Description

The invention relates to a method for training a neural network of an electronic computing device of a motor vehicle according to the preamble of claim 1.

It is already known from the prior art that data are required for at least partially autonomous operation of a motor vehicle or for fully autonomous operation of the motor vehicle in order to train a corresponding neural network for partially autonomous driving or fully autonomous driving. When such data are actually recorded, however, the corresponding scenes are often empty of objects, which leads to many similar-looking scenes with little variance. It is common practice to augment the data during training and also to create a catalog of the existing objects, which can then be reinserted into the data at other points. This increases the variance of the data. However, this amount of data is still very limited in terms of objects that can still only be used that occur somewhere in the data set. Some methods therefore combine this type of augmentation with simulated data or data that are generated directly from noise from neural networks. This increases the performance and is inexpensive. However, here too the generated objects are often limited in their spatial extent or are not optimally matched to the scene properties of the data set.

The DE 10 2018 210 937 A1 relates to a method for the realistic simulation of physical measurement data, which can be used, for example, to generate training data for AI systems in the field of autonomous driving.

The object of the present invention is to provide a method by means of which an improved training of a neural network of an electronic computing device is implemented.

This object is achieved by a method according to the independent patent claim. Advantageous embodiments are specified in the subclaims.

One aspect of the invention relates to a method for training a neural network of an electronic computing device of a motor vehicle, in which environment data are provided with at least one object and virtual training data are provided for a further object, the neural network at least with the environment data and the virtual training data is trained.

It is provided that the virtual training data for the further object are changed as a function of a parameter and are provided as further virtual training data for yet another object and the neural network is trained using the environment data, the virtual training data and the further virtual training data .

This allows improved training of the neural network to be implemented. The method presented is not limited to the field of autonomous driving, but can also be used in all applications in which a neural network is trained with, for example, three-dimensional scenes or objects. In particular, it is therefore proposed that an extension with an adjustable parameter be provided for the spatial expansion of objects. The analysis of the spatial expansion properties of three-dimensional objects in road traffic is carried out. Furthermore, a simple and inexpensive generation of various large-scale 3D objects for data augmentation can be implemented. In particular, this means that less real data may be required. Better augmentation can be achieved than with previous variants, for example by sampling real objects from the data record or inserting synthetic objects. The objects created are much more diverse than with other approaches and can also be parameterized directly by the user. Furthermore, essentially an infinite number of objects with a very large number of different appearances can be generated, which leads to a better generalization of perception networks. These perception networks can be, for example, object detection, semantic segmentation, object tracking or the like.

In particular, the invention thus relates to a method for the generative augmentation of real and simulated image data for training a neural network for object recognition by an autonomous motor vehicle. The neural networks that generate point clouds of objects are trained. In particular, it is proposed that, in the augmentation method, the spatial extent of an object is determined by a variable parameter and the neural network architecture is expanded accordingly. This parameter is continuous and not discrete, with which “style information” is calculated directly from the “shape”.

According to an advantageous embodiment, the neural network is provided as a generating, opposing network.

Furthermore, it has proven to be advantageous if the environment data are generated on the basis of a real environment detection.

It is also advantageous if the virtual training data are generated on the basis of the environment data, the virtual training data describing a further object which has a low presence below a presence threshold value in the environment data.

In a further advantageous embodiment, a size of the further object is virtually changed by means of the parameter, so that the virtual further object is generated again on the basis of the size change.

It is also advantageous if the neural network is provided for an evaluation of an environment by means of a lidar sensor device as a detection device.

Further advantages, features and details of the invention emerge from the following description of a preferred exemplary embodiment and with reference to the drawing. The features and combinations of features mentioned above in the description as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the single figure can be used not only in the specified combination, but also in other combinations or on their own, without the frame to leave the invention.

The single FIGURE shows a schematic flow chart for training a neural network.

In the figure, elements that are the same or have the same function are provided with the same reference symbols.

The figure shows a schematic flow diagram for training a neural network 10 an electronic computing device 12th for a motor vehicle, not shown. There are environmental data 14th provided with at least one object. Furthermore, virtual training data 16 provided for another property. The neural network 10 is at least with the environment data 14th and the virtual training data 16 trained. In particular, it is provided that the virtual training data 16 for the further object depending on a parameter 18th can be changed and as further virtual training data 20th for yet another object and the neural network 10 by means of the environmental data 14th , the virtual training data 16 and the other virtual training data 20th is trained.

In particular, it can be provided that the neural network 10 is provided as a generating, opposing network. The neural network 10 can in particular be designed as a so-called GAN (Generative Adversarial Network).

In particular, it can be provided that the environmental data 14th based on a real environment detection 22nd be generated. The virtual training data 16 are based in particular on the environmental data 14th generated, the virtual training data 16 describe another object which has a low presence below a presence threshold value in the environmental data 14th having.

Furthermore, it can be provided that by means of the parameter 18th a size of the second object is changed virtually, so that the virtual further object is generated again on the basis of the change in size.

In particular, a network architecture is therefore proposed that is around the parameter 18th is expanded. This parameter 18th regulates the spatial expansion of the objects. For example, the neural network can thus use the aspect ratios 10 be urged to generate, for example, a so-called pick-up truck as a passenger car. This means that not only the objects recorded by the sensors are used for training, but also their calculated aspect ratios. By also entering aspect ratios for inference that are not included in the training, the neural network is 10 able to generate novel object variations. This increases the variance in the data.

A simple and inexpensive generation of additional objects is therefore proposed in order to carry out a large-scale augmentation of the real sensor data. For this purpose, the object clouds are inserted into, for example, lidar sensor recordings and thus not only increase the number of objects of a previously underrepresented class, but also their diversity. In the present case, a lidar sensor device is used in particular. However, the data can also be used for a radar sensor device, an ultrasonic sensor device or a camera. In particular, the performance of the object recognition or semantic segmentation can thus be increased. In addition, the motor vehicle that is equipped with this technology is also significantly more robust against the occurrence of unexpected objects or objects that otherwise occur very rarely.

For example, more data from animals, such as dogs, can be generated so that the motor vehicle also correctly recognizes the dog, although there are probably very few dogs in the real sensor data. In addition, the generated data have partly different statistical properties than the recorded data, which helps the neural network 10 to memorize less and understand more. As a result, the motor vehicle is therefore also less susceptible to domain changes or unexpected effects.

Overall, the invention shows a method for the generative augmentation of real or simulated lidar data.

List of reference symbols

10

neural network

12th

electronic computing device

14th

Environmental data

16

virtual training data

18th

parameter

20th

further virtual training data

22nd

Environment detection

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102018210937 A1 [0003]

Claims (6)

Method for training a neural network (10) of an electronic computing device (12) of a motor vehicle, in which environmental data (14) are provided with at least one object and virtual training data (16) are provided for a further object, the neural network (10 ) is trained at least with the environment data (14) and the virtual training data (16), characterized in that the virtual training data (14) for the further object are changed as a function of a parameter (18) and as further virtual training data (20 ) are provided for yet another object and the neural network (10) is trained by means of the environment data (14), the virtual training data (16) and the further virtual training data (18). Procedure according to Claim 1 , characterized in that the neural network (10) is provided as a generating, opposing network. Procedure according to Claim 1 or 2 , characterized in that the environment data (14) are generated on the basis of a real environment detection (22). Method according to one of the preceding claims, characterized in that the virtual training data (16) are generated on the basis of the environment data (14), the virtual training data (16) describing a further object which indicates a low presence below a presence threshold value in the environment data ( 14). Method according to one of the preceding claims, characterized in that a size of the further object is virtually changed by means of the parameter (18), so that the virtual further object is generated again on the basis of the size change. Method according to one of the preceding claims, characterized in that the neural network (10) is provided for an evaluation of an environment by means of a lidar sensor device as a detection device.

Images