DE102018216172A1 - Method for automatically generating a label for training a self-learning system and motor vehicle - Google Patents

Abstract

The invention relates to a method for automatically generating a label (10) for training a self-learning system (12), in which a sensor signal (S, S) is generated with the help of at least two sensor units (6a, 6b) for the same real situation and different information from the two sensor units (6a, 6b) is used for the label (10) generated.

Description

The invention relates to a method for automatically generating a label for training a self-learning system and a motor vehicle.

So-called labels are used to train self-learning systems, in particular self-learning systems based on artificial intelligence. The labels are preferably understood to mean data pairs which contain a known input signal and a known output signal. Label is therefore to be understood as an assignment of input information to output information. In order to train the self-learning system, the known input signal and the known output signal are predefined for it, for example, so that the self-learning system learns, for example with regard to signal processing, in such a way that when the input signal is input, the output signal is generated due to the signal processing.

Especially against the background of satisfying a user of the self-learning system, ever increasing demands are placed on self-learning systems and therefore also on labels. The creation of the labels, especially high-quality labels, is therefore costly and time-consuming.

When creating the labels, there is the method of generating them using a simulation. The disadvantage here is that this creates a so-called simulation-reality gap. The simulation-reality gap is understood here to mean that the simulated labels are sometimes not always applicable in reality and / or do not always reflect reality, so that the self-learning system could be trained in a realistic manner.

A second known method is the manual creation of the labels. Manual generation is understood to mean that these labels are generated manually, for example by people. However, this second method also has the disadvantage that it is costly and time-consuming.

Usually, a third method is used to generate labels by companies that specialize in the creation of labels. Existing data is transferred from the client to the company, which is then "processed" into labels. However, the transmission of the data to the company is sometimes complex and involves a considerable amount of time and money.

Proceeding from this, the object of the invention is to implement the generation of labels for training a self-learning system simply and inexpensively.

The object is achieved according to the invention by a method for automatically generating a label with the features of claim 1. The label generated is used for training a self-learning system. Advantageous refinements, developments and variants are the subject of the dependent claims.

The self-learning system is preferably a system based on an artificial intelligence (K I)

In a first step of the method, at least two sensor units are used to generate a sensor signal relating to the same real situation. Under the real situation, e.g. understood an everyday situation in which the self-learning system is active and collects information. “Equal” here means that the two sensor units generate the respective sensor signal at one and the same real situation, for example at the same time. In other words, there are two different sensor signals for the same real situation.

Furthermore, several modules are provided for processing the sensor signals, at least one of the modules being designed as a self-learning system. All modules are preferably designed as self-learning systems. Furthermore, the self-learning systems are preferably the self-learning systems to be trained.

In a second method step, at least one of the sensor signals is then transmitted to an module as an input signal.

Output signals are then generated by the at least one module. The output signals contain processed information on the real situation and are generated on the basis of the sensor signal transmitted in each case. The processed information is understood to mean specifically information about the real situation that has been extracted from the information contained in the sensor signals. These are, for example, physical quantities of objects, such as dimensions, which are detected by the sensor units, or e.g. Ambient conditions, such as temperature or brightness.

• Im Fall a) erfolgt die Labelerzeugung derart, dass die Sensorsignale von zumindest zwei Sensoreinheiten an ein Modul übermittelt werden und auf Basis des Ausgangssignals dieses Moduls das Label erzeugt wird.
• Im Fall b) erfolgt die Erzeugung des Labels derart, dass die aufbereiteten Informationen von zumindest zwei Modulen, die jeweils Sensorsignale von zwei unterschiedlichen Sensoreinheiten erhalten haben, zusammengeführt werden..Auf Basis der zusammengeführten Informationen wird anschließend das Label erzeugt. Unter den zusammengeführten Informationen werden hierbei die aufbereiteten Informationen von zumindest zwei Modulen verstanden, die zusammen für die Erzeugung des Labels herangezogen werden. Unter unterschiedliche Sensoreinheiten werden Sensoreinheiten verstanden, welche auf unterschiedlichen Messprinzipien beruhen und somit bei einer identischen realen Situation einen unterschiedlichen Informationsgehalt aufweisen, also beispielsweise unterschiedliche Arten von Informationen.

A label is then generated in a next process step. There are two possible cases for this:

• In case a), the label is generated in such a way that the sensor signals are transmitted from at least two sensor units to a module and the label is generated on the basis of the output signal of this module.
• In case b) the label is generated in such a way that the processed information from at least two modules, each of which has received sensor signals from two different sensor units, is combined. The label is then generated on the basis of the combined information. The merged information here means the processed information from at least two modules, which are used together to generate the label. Different sensor units are understood to mean sensor units which are based on different measurement principles and thus have a different information content in an identical real situation, for example different types of information.

Different information from two sensor units is thus used to generate the label for the label generated in particular according to case b).
The processed information is preferably transmitted as output signals from the modules to a label generator, which then generates the label from these. The label generator thus combines the merged information into a known input signal and a known output signal.

In a last process step, the labels generated are provided for training the self-learning system. For this purpose, these are stored, for example, on an internal memory of the self-learning system. As an alternative to this, the labels are stored on a network-based storage, for example a cloud.

By generating the labels on the basis of the output signals generated by the modules, a particularly simple and inexpensive generation of labels for training the self-learning system is achieved. That is, there is no need to involve an (external) company to generate the labels. This also means that the self-learning system generates “its” labels itself, which means that a cost and time factor is advantageously optimized. In other words, the actual operation of the self-learning system is used to collect data from this for the generation of the labels and then to use this for the generation of the labels. I.e. the self-learning system trains itself based on self-generated labels.

According to a preferred embodiment, in the case of a) label generation, the module for generating the processed information is designed as a conventional data processing program. The advantage here is that standard programs can be used to generate the processed information. In particular, the conventional data processing program is not a self-learning system. Alternatively or additionally, a conventional data processing program can also be used in case b) of the generation of the labels.

The method according to the invention is a computer-implemented method in which all method steps except for the generation of the sensor signals and the transmission of these to the respective modules run on a data processing unit.

According to a particularly preferred development, the self-learning system is a neural network. That is, thus at least one of the modules is also designed as a neural network. In particular, all modules are designed as such a self-learning system, in particular as a neural network. In general, an artificial neural network (KNN) for data and information processing, e.g. Understood.

The generated label is used as a training file for the modules and thus the neural networks. The advantage here, as already mentioned, is that the neural networks thus generate the labels in actual operation and there is no need for complex methods for generating labels. The training file is understood here to mean a file (in particular the label) which is fed to the self-learning system, for example during a training mode, and which the self-learning system adapts its data processing algorithm based on the training file and thus learns.

The at least two sensor units are expediently different sensors for generating environmental data. For example, one of the at least two sensor units is designed as a camera for generating image data. The other of the at least two sensor units is, for example, a radar sensor unit or a LiDAR sensor (Light Detection And Ranging Sensor). Alternatively one of the at least two sensor units is an ultrasonic sensor.

The advantage here is that by means of the sensor units designed as sensors for generating environmental data, different sensor signals, in the present case, for example, different image signals (specifically with regard to geometric structures) can be generated for the one real situation. Such environmental data are particularly easy to evaluate using the modules already mentioned, in particular using the self-learning systems.

According to a preferred embodiment, in case a) the label is generated in such a way that, in a first step, a first sensor signal from the one sensor unit, which is designed, for example, as a camera for generating a 2D image, is first evaluated by the module with regard to first information. The first piece of information is in particular one of the previously prepared information. The first piece of information is, in particular, a width and / or a height of an object located within the image signal generated by the camera. In this context, the object is understood to mean, for example, a person or a vehicle.

In a second step, the module evaluates a second sensor signal from the second sensor unit with regard to second information. The second sensor unit is a sensor unit designed as an image sensor for generating 3D image data. Such a sensor unit is, for example, the already mentioned LiDAR sensor. The second piece of information is also part of the previously prepared information and, for example, a depth of the object already detected by the one sensor unit. In other words, in two steps, for example, a motor unit is initially detected by the one sensor unit, which is designed as a camera, and the width and height are determined and evaluated as first information. A depth (geometrically, the length of the motor vehicle) of the object is then determined and evaluated by the second sensor unit, which is designed as an image sensor.

The two evaluated information (on the one hand width and height as information and on the other hand the depth of the object) are then combined to create the label. The label thus has these two pieces of information in the manner of a pair of data, so that the self-learning system can be trained on the basis of this information. On the basis of the labels, the self-learning system can thus recognize the (usually) dimensions of such an object, especially the motor vehicle, and subsequently identify such objects as motor vehicles.

With this method, in particular the creation of labels with multiple information and thus multiple issues is achieved. In other words, new information (for example the depth information of the second sensor unit) can be added to the first information (height and width of the object) so that the label contains more information overall and thus leads to better training of the self-learning system.

Examples of neural networks that generate such labels are Mask R-CNN or Faster R-CNN.

• Zunächst werden zwei unterschiedliche Sensoreinheiten bereitgestellt, wobei jeder Sensoreinheit ein Modul zugeordnet ist, welches als ein selbstlernendes System ausgebildet ist.

Alternatively or in addition, the labels are generated as follows and used for training the self-learning system:

• First, two different sensor units are provided, with each sensor unit being assigned a module which is designed as a self-learning system.

In a first method step, the two sensor units each generate a sensor signal relating to the same real situation. The same real situation is understood here to mean, for example, the real situation already mentioned above.

In a further method step, at least one of the sensor signals is transmitted to a module. The module or the modules already mentioned are designed to process the sensor signals.

A respective output signal is then generated by the modules, the output signals containing processed information about the real situation. The processed information is understood here to mean, for example, a length and or a width of an object, for example a vehicle.

On the basis of the output signals generated by the modules, at least two labels are then generated, one label containing information from the first sensor unit and the other life containing information from the second sensor unit. The label generated by one module is then used as a training file for the other module. In other words, the two modules train each other, particularly using the labels they create. For example, a label created on the basis of a 2D image recorded by a camera is used as a training file for a module of a LiDAR sensor used and vice versa. This type of training is also known as "tandem learning". The respective generation of the labels and that subsequent mutual training of the respective modules takes place according to a preferred further development in each case after a journey section or after a defined journey time.

For this purpose, the self-learning system that is to be trained is preferably used in a motor vehicle. The travel section is understood here to mean, for example, a journey from a standstill (with the engine switched off) to a renewed switch-off of the engine and thus a standstill of the vehicle. For example, a trip from an apartment to a place of work. The defined journey time is understood here to mean that a period of time, for example 2 hours, is specified in which the labels are generated and then the training takes place using the generated labels.

This alternative or additional configuration has the advantage that a continuous improvement process is guaranteed. That is, During the travel section already described or the defined travel duration, data for generating the labels is collected and the labels are generated. Subsequent to the defined journey section or the defined journey duration, the self-learning system is trained, i. H. of the respective modules with each other. This means that an "improved" self-learning system is available before the start of a new journey section or a defined journey duration, since this has been trained with the previously generated labels and thus "updated".

The self-learning system is preferably part of an assistance system of a motor vehicle. The assistance system is, for example, a unit for camera-based object recognition in an at least partially autonomous and preferably (fully) autonomous motor vehicle. The method described above and the use of the self-learning system as part of the assistance system achieve a significant and continuous improvement in this assistance system, thereby increasing driving safety of the at least partially autonomous motor vehicle. For example, objects such as other motor vehicles or people in the vicinity of the motor vehicle are (continuously) improved by the self-learning system.

In addition, as already mentioned, manual and thus costly and time-consuming generation of the labels can be dispensed with, since these are now generated by the self-learning system itself and are made available for the training.

The object is further achieved according to the invention by a motor vehicle with a control unit having the features of claim 9. The motor vehicle with the control unit is in this case in particular designed to carry out the method described above. The advantages and preferred configurations listed with regard to the method are to be applied analogously to the motor vehicle and vice versa.

Basically, the method described is not limited to use in an assistance system of a motor vehicle. Rather, a usability extends to almost all devices that have and / or use self-learning systems that are (should) be trained on the basis of labels.

• 1 ein schematisiertes Kraftfahrzeug mit einer Steuerungseinheit, die zur Durchführung eines Verfahrens zum automatischen Erzeugen eines Labels ausgebildet ist,
• 2 ein skizziertes Blockschaltbild des Verfahrens zur Erzeugung des Labels gemäß Fall a),
• 3 ein skizziertes Blockschaltbild des Verfahrens zur Erzeugung des Labels gemäß Fall b) sowie
• 4 ein skizziertes Blockschaltbild des Verfahrens zur Erzeugung des Labels und des Trainings eines selbstlernenden Systems.

Exemplary embodiments of the invention are explained in more detail below with reference to the figures. These show in a somewhat simplified representation:

• 1 a schematic motor vehicle with a control unit which is designed to carry out a method for automatically generating a label,
• 2nd a sketched block diagram of the method for generating the label according to case a),
• 3rd a sketched block diagram of the method for generating the label according to case b) and
• 4th a sketched block diagram of the method for generating the label and training a self-learning system.

Parts with the same effect are always shown in the figures with the same reference symbols.

In 1 is a schematically illustrated motor vehicle 2nd shown. In the motor vehicle 2nd it is in particular an at least partially autonomous and preferably a fully autonomous passenger vehicle. The car 2nd has a control unit 4th on. In the exemplary embodiment, the motor vehicle has 2nd two sensor units 6a , 6b on. The sensor units 6a , 6b are in the embodiment with the control unit 4th connected, which in the exemplary embodiment continues to be part of an assistance system 8th is. With the assistance system 8th in the exemplary embodiment, it is, for example, a (camera-based) object recognition system. The sensor units are therefore concerned 6a , 6b in the exemplary embodiment in particular sensors for generating image data. That is, in the exemplary embodiment is one sensor unit, for example 6a as a camera and the other sensor unit 6b designed as a radar sensor.

The control unit 4th is for performing a method for automatically generating a label 10th for training a self-learning system 12th educated. In the following, different methods (cases) are explained with the help of which the label 10th is produced.

2nd shows a block diagram showing the creation of the label 10th according to a first embodiment variant (case a)). Here, each of the sensor units 6 a sensor signal S is generated and sent to a module 14 , which according to the exemplary embodiment 2nd than the self-learning system 12th trained, transmitted. Ie the first sensor unit 6a generates a first sensor signal S _a and the second sensor unit 6b generates a second sensor signal S _b . The sensor signals S _a , S _b are included in the same, i.e. one and the same real situation. In this case, the real situation is understood to mean, for example, a journey through a street on and / or on which there are several objects (for example people and / or other vehicles).

Through the module 14 is based on the two sensor signals S _a , S _b generates an output signal Sout. The module evaluates this 14 on the one hand the first sensor signal S _a with regard to a first piece of information (for example with regard to a height and / or width of another motor vehicle). On the other hand, evaluate the module 14 the second sensor signal S _b with regard to a second piece of information (for example a depth, ie a length of the further motor vehicle). The output signal Sout thus contains processed information I. to the real situation. Under the prepared situations I. As already mentioned, data relating to the sensor signals, which are preferably embodied as image data, are in this case S _a , S _b understood objects. The prepared information I. in the exemplary embodiment thus contain, for example, the first information and the second information.

The output signal Sout is then from the module 14 to a label generator 16 transmitted. The label generator 16 is designed in such a way that the label is based on the output signal Sout 10th to create. For this purpose, the evaluated and prepared information I. merged. The label 10th preferably has an input signal and an output signal in the manner of a data pair. The self-learning system is based on the input signal and the output signal 12th , which in the exemplary embodiment is the module 14 is trained. Training the self-learning system 12th For example, the label 10th the self-learning system 12th is fed such that the self-learning system 12th receives the input signal and the output signal and adjusts and / or adjusts signal processing in such a way that feeding the input signal (without additional specification of the output signal) would result in an output of the output signal (specified in the training).

Such a training method is also referred to as "supervised learning". Alternatively or in addition, a training method in the manner of a reinforcement learning is also conceivable, in which the self-learning system 12th after generating an output signal on the basis of a predetermined input signal, receives an evaluation on the basis of which the self-learning system 12th then adjust and / or adjust.

In 3rd is a creation of the label 10th according to a second embodiment variant (case b)). This is in 3rd a block diagram of the method is shown. According to the second embodiment variant (case b)), the two sensor units also operate 6a , 6b one sensor signal each S _a , S _b generated. Then one sensor signal each S _a , S _b one module at a time 14 transmitted. That is, for example the sensor signal S _a to a first module 14a and the sensor signal S _b to a second module 14b transmitted. Both modules 14a , 14b are self-learning systems to be trained in the exemplary embodiment 12th , in particular designed as neural networks. Every module 14a , 14b then generates an output signal S _{out, a} , S _{out, b} , which each processed information I. included to the real situation. These output signals S _{out, a} , S _{out, b} are then from the modules 14a , 14b to the label generator 16 transmitted. This then generates on the basis of the two output signals S _{out, a} , S _{out, b} , the label 10th , which thus different information from two sensor units 6a , 6b contains. The label created in this way 10th is then used, for example, to train the self-learning systems according to the type of methods already mentioned 12th used and provided.

In 4th is a sketched block diagram of the method for generating the label in a simplified manner. In addition, the training of a self-learning system is schematized 12th shown.

The from the sensor units 6a , 6b generated sensor signals S _a , S _b are in accordance with the embodiment 4th each as a self-learning system 12a , 12b trained module 14a , 14b transmitted. Then, based on the modules 14a , 14b generated output signals S _{out, a} , S _{out, b} in one label generator each 16a , 16b one label each 10a , 10b generated. Alternatively, the two labels are created 10a , 10b just a label generator 16 used.

The labels created 10a , 10b then become a mutual learning of the modules 14a , 14b used. That is, a label 10a , which preferably revised information on the real situation of the first sensor unit 6a is used to train the second module 14b used and vice versa. This method of training is also called "tandem learning", according to which, for example, within the control unit 4th train operating systems mutually.

The generation of the sensor signals S _a , S _b and thus the creation of the labels 10a , 10b in the exemplary embodiment is preferably carried out while the motor vehicle is traveling 2nd . The training, especially the mutual training of the modules 14a , 14b through and by means of the generated labels 10a , 10b in the exemplary embodiment is preferably carried out after driving the motor vehicle 2nd . This configuration has the advantage that modules that are better trained and thus “updated” before or when you start driving again 14a , 14b are available and therefore as part of the assistance system 8th optimize this advantageously.

The invention is not restricted to the exemplary embodiments described above. Rather, other variants of the invention can also be derived from this by the person skilled in the art without departing from the subject matter of the invention. In particular, all of the individual features described in connection with the exemplary embodiments can also be combined with one another in other ways without departing from the subject matter of the invention.

Reference list

2nd

Motor vehicle

4th

Control unit

6a, b

Sensor unit

8th

Assistance system

10a, b

Label

12a, b

self-learning system

14a, b

module

16a, b

Label generator

I.

prepared information

S _a , S _b

Sensor signal

S _{out, a} , S _{out, b}

Output signal

Claims (10)

Method for automatically generating a label (10) for training a self-learning system (12), comprising the following steps: - Generating a sensor signal (S _a , S _b ) with the help of at least two sensor units (6a, 6b) for the same real one Situation, - Provision of several modules (14) for processing the sensor signals (S _a , S _b ), at least one of the modules (14) being designed as a self-learning system (12) - Transmission of at least one of the sensor signals (S _a , S _b ) to a module (14), - generation of output signals (S _out ) by the at least one module (14), the output signals (S _out ) containing processed information (I) on the real situation, - generation of the label (10 ) by either - a) transmitting the sensor signals (S _a , S _b ) of at least two sensor units (6a, 6b) to a module (14) and generating the label (10) on the basis of the output signal (S _out ) of this module becomes, or - b) the processed information tions (I) of at least two modules (14) that have received sensor signals (S _a , S _b ) from two different sensor units (6a, 6b) are merged and on the basis of the merged information (I) the label (10) is generated so that different information from two sensor units (6a, 6b) is used for the generated label (10) - providing the generated label (10) for training the self-learning system (12). Method according to the preceding claim, wherein in case a) the module (14) for generating the processed information (I) is designed as a conventional data processing program. Procedure according to Claim 1 , wherein the self-learning system (12) is a neural network and the label (10) is used as a training file for the modules (14). Method according to one of the preceding claims, wherein the at least two sensor units (6a, 6b) are different sensors for generating environmental data. Method according to the preceding claim, wherein in case a) the label (10) is generated in such a way that - first of all from the module (14) a first sensor signal (S _a ) from the one sensor unit (6a), which is used as a camera for generating a 2D Is formed, is evaluated with regard to a first piece of information, - from the module (14) a second sensor signal (S _b ) of the second sensor unit (6b), which as one A sensor is designed to generate 3D data, is evaluated with regard to a second piece of information, - the two evaluated pieces of information are combined to generate the label (10). Method in particular according to one of the preceding claims, comprising the steps: - providing two different sensor units (6a, 6b), with each sensor unit (6a, 6b) being assigned a module (14a, 14b) which is used as a self-learning system (12a, 12b ) is formed, - generation of a sensor signal (S _a , S _b ) using the two sensor units (6a, 6b) for the same real situation, - transmission of at least one of the sensor signals (6a, 6b) to a module (14a, 14b ), - Generating a respective output signal (S _{out, a} , S _{out, b} ) by the modules (14a, 14b), the output signals (S _{out, a} , S _{out, b} ) being processed information (I) on the real situation contain, - generation of at least two labels (10a, 10b), one label (10a) containing information from the first sensor unit (6a) and the other label (10b) containing information from the second sensor unit (6b), the module ( 14a, 14b) each generated label (10a, 10b) as a training file f r is used the respective other module (14a, 14b). Method according to the preceding claim, wherein the self-learning system (12) is used in a motor vehicle (2) and, in each case after a journey section or a defined journey time, the other module (14a, 14b) is trained by means of the respectively generated label (10a, 10b ) he follows. Method according to one of the preceding claims, in which the self-learning system (12) is part of an assistance system (8). Motor vehicle with two sensor units (6a, 6b) and with a control unit (4) which is designed to carry out the method according to one of the preceding claims. Computer program product comprising commands which cause the control unit (4) of the motor vehicle (2) according to Claim 9 the procedural steps Claim 1 executes.

Images