DE102019203953A1 - Method and device for determining a trained decision network - Google Patents

Abstract

A method (600) for determining a trained decision network (140) is presented, the method (600) comprising a step of reading in (610) a first sensor signal vector (ST1) and a second sensor signal vector (ST2), the first sensor signal vector ( ST1) has at least one measured value of a first sensor (115) of a first sensor technology and the second sensor signal vector (ST2) has at least one measured value of a second sensor (120) of a second sensor technology different from the first sensor technology, the first sensor signal vector (ST1) being an object ( 110) from a perspective of the first sensor (115) and the second sensor signal vector (ST2) represents the object (110) from a perspective of the second sensor (120). The method (600) further comprises a step of adapting (620) a first network (125) with the first sensor signal vector (ST1) as the input vector and the second sensor signal vector (ST2) as the reference vector and adapting (620) a second network (130) the second sensor signal vector (ST2) as the input vector and the first sensor signal vector (ST1) as the reference vector. Finally, the method (600) comprises a step of training (630) parameters of a decision network (140) using the first and second networks (125) in order to obtain the trained decision network (140).

Description

The present invention relates to a method and a device for determining a trained decision network according to the main claims.

In vehicle systems, components are often used for evaluating sensor signals that contain evaluation algorithms specially designed for the type of sensor. In this case, however, several or a large number of separate evaluation algorithms are to be kept available for the evaluation of sensor signals from sensors of different sensor technologies, which sometimes drastically increases the numerical and / or circuit complexity of the signal processing of the sensor signals.

Against this background, the present invention creates an improved method and an improved device for determining a trained decision network as well as an improved method and an improved device for identifying at least one object from data of at least a first sensor signal vector and / or a second sensor signal vector according to the main claims. Advantageous refinements result from the subclaims and the following description.

• - Einlesen eines ersten Sensorsignalvektors und eines zweiten Sensorsignalvektors, wobei der erste Sensorsignalvektor zumindest einen Messwert eines ersten Sensors einer ersten Sensortechnologie und der zweite Sensorsignalvektor zumindest einen Messwert eines zweiten Sensors einer von der ersten Sensortechnologie unterschiedlichen zweiten Sensortechnologie aufweist, wobei der erste Sensorsignalvektor ein Objekt aus einem Blickwinkel des ersten Sensors repräsentiert und der zweite Sensorsignalvektor das Objekt aus einem Blickwinkel des zweiten Sensors repräsentiert; und
• - Adaptieren eines ersten Netzwerks mit dem ersten Sensorsignalvektor als Eingangsvektor und dem zweiten Sensorsignalvektor als Referenzvektor und Adaptieren eines zweiten Netzwerks mit dem zweiten Sensorsignalvektor als Eingangsvektor und dem ersten Sensorsignalvektor als Referenzvektor; und
• - Trainieren von Parametern eines Entscheidungsnetzwerks unter Verwendung des ersten und zweiten Netzwerks, um das trainierte Entscheidungsnetzwerk zu erhalten.

The approach presented here creates a method for determining a trained decision network, the method having the following steps:

• Reading in a first sensor signal vector and a second sensor signal vector, the first sensor signal vector having at least one measured value from a first sensor of a first sensor technology and the second sensor signal vector having at least one measured value from a second sensor of a second sensor technology different from the first sensor technology, the first sensor signal vector having an object represents a viewing angle of the first sensor and the second sensor signal vector represents the object from a viewing angle of the second sensor; and
• - Adapting a first network with the first sensor signal vector as input vector and the second sensor signal vector as reference vector and adapting a second network with the second sensor signal vector as input vector and the first sensor signal vector as reference vector; and
• - training parameters of a decision network using the first and second networks to obtain the trained decision network.

A sensor signal vector can be understood to be a one-dimensional or multi-dimensional sensor signal. Sensor technology can be understood as a physical measuring principle by means of which the sensor can acquire measured values of physical quantities. For example, a sensor technology of a radar sensor can be based on the measuring principle of evaluating a radar beam reflected on an object in order to obtain a three-dimensional position and / or movement of the object in relation to the radar sensor. A sensor technology of a lidar sensor can for example be based on the measuring principle of evaluating a laser beam reflected on an object in order to detect a three-dimensional position and / or movement of the object in relation to the lidar sensor. Here, for example, the position detected three-dimensionally with the lidar sensor can be mapped in a Cartesian coordinate, whereas the position detected three-dimensionally with the radar sensor is mapped in spherical coordinates. Furthermore, both the first sensor and the second sensor should each output a sensor signal vector that images the same object, but from the respective perspective of the relevant sensor. However, it is also conceivable that the sensor is designed as an image sensor, for example a camera or an ultrasonic sensor, and supplies corresponding data as sensor signal vectors. A network such as the first network and / or the second network can be understood, for example, as a trainable descriptor that enables a reduction in the complexity and redundancy of the sensor signal vector, but still allows an extraction of features that allow conclusions to be drawn about the presence of the object. Such a network can for example be a feature-learning network. For example, such a network can be implemented as a neural network, an artificial normal network or especially an auto-encoder. Adaptation can be understood to mean the change in at least one parameter such as, for example, the weight of a node in the network. For this purpose, for example, one of the sensor signal vectors can be used as an input signal or input vector, whereas the other sensor signal vector is used as a reference vector, so that the network or the parameters of the network are adapted or adjusted so that the first sensor signal vector is mapped onto the second sensor signal vector and vice versa . A decision network can be understood to mean, for example, a connecting network such as a generating generic network (GAN = Generative Adversarial Network). Such a network as a decision network can also be implemented as a neural network or as an alternative structure from the field of artificial intelligence. It is also conceivable that the decision network in Structure or parameters is at least partially identical or similar to a structure or parameters of the aforementioned networks, or contains a network or a combination of the aforementioned first and second networks.

The approach proposed here is based on the knowledge that by training the parameters of the decision network, a significantly more compact and simpler form for evaluating data from sensors of different sensor technologies becomes possible. By linking the adaptation of the first network with the two sensor signal vectors as the input vector or reference vector and the second network with the two sensor signal vectors in the opposite constellation and then training the decision network, a very general model can be determined as a decision network to be used in a subsequent application this decision network to be able to recognize one or more objects from the sensor signal vectors. In this way, a large number of different detection algorithms for evaluating the data from sensors with different sensor technologies can advantageously be dispensed with. This not only makes it possible to improve or reduce the numerical or circuit engineering complexity for object detection, but also to make more efficient use of the available memory.

According to one embodiment of the approach proposed here, in the training step, the parameters of the decision network can be adapted using disturbance variables, and the status values of the first network and / or status values of the second network can be added. Such an embodiment offers the advantage of enabling particularly fast and reliable determination or corresponding training of the parameters of the decision network, so that when the decision network is used later, objects that are mapped by features in the sensor signal vectors can be recognized quickly and reliably.

An embodiment of the approach presented here is particularly favorable in which, in the training step, the parameters of the decision network are trained by a GAN structure using the first and second networks. Such a GAN structure can be understood as the aforementioned structure of a generating generic network (GAN = Generative Adversarial Network) as a connecting network. Such a generating generic network can be understood as an algorithm for unsupervised learning. According to the online encyclopedia Wikipedia, such a network includes, for example, two artificial neural networks that perform a zero-sum game, with a first network as a generator creating network candidates and the second network as a discriminator evaluating the candidates. The aim of the generator network is to learn to generate results according to a certain distribution. The discriminator, on the other hand, is trained to distinguish the results of the generator from the data from the real, given distribution. The goal of the generator network is to produce results that the discriminator cannot distinguish. As a result, the generated distribution should gradually adjust to the real distribution. Such an embodiment of the approach proposed here offers the advantage of creating a decision network in order to be able to extract information about the presence and / or the position and / or the speed of the object in the sensor data as quickly and clearly as possible from the features of the sensor signal vectors.

According to a further embodiment, the parameters of the decision network can be adapted in the training step such that the first network is used to determine input values of the decision network and the second network is used to provide reference variables of the decision network. Alternatively or additionally, in the step of adapting, the parameters of the decision network can be adapted such that the second network is used to determine input values of the decision network and the first network is used to provide reference variables of the decision network. Such an embodiment of the approach presented here offers the advantage of training the decision network quickly and precisely in order to be able to reliably extract objects or features from the sensor signal vectors in a subsequent use.

An embodiment of the approach proposed here is particularly favorable in which, in the training step, the parameters of the decision network are initially used using the first network to determine input values of the decision network and the second network is used to provide reference variables of the decision network and then using the second network for determining input values of the decision network and the first network for providing reference variables of the decision network. By means of such a cascade of training steps, the training of the decision network can be further improved with regard to the precision in the extraction of features or objects for later use.

In order to determine the trained decision network as quickly as possible, according to a further embodiment, in the step of adapting, the first and second networks can be trained simultaneously and / or in parallel.

An embodiment of the approach presented here is also favorable in which, in the adapting step, the first and / or second network to be adapted is designed as a neural network, in particular as an artificial neural network and / or as an auto-encoder. Such an embodiment offers the advantage of being able to use sophisticated algorithms or structures for the first and / or second network which can deliver input values or reference values with high precision, with which the decision network is subsequently trained.

The approach presented here can be used particularly advantageously in the field of evaluating signals from vehicle sensors in order to also achieve a reduction in data rates that are required, for example, for mobile applications such as the localization of a vehicle using a reference map stored in a central computer. According to a further embodiment of the approach presented here, measured values from the first sensor of a vehicle can be read in as the first sensor signal vector and measured values from the second sensor of a vehicle can be read in as the second sensor signal vector, in particular the first sensor being designed as a radar sensor and the second sensor is designed as a lidar sensor or wherein the first sensor is designed as a lidar sensor and the second sensor is designed as a radar sensor.

An embodiment of the approach presented here is particularly cheap and easy to implement, in which, in the reading-in step, the measured value of a sensor designed as a radar sensor is a distance detected in relation to the object, an azimuth angle, an elevation angle, a Doppler frequency shift value between a transmitted and a radar beam received from the object or a value representing a radar cross-section is read in. Alternatively or additionally, in the reading-in step, an intensity value of a laser beam reflected from the object, an SNR value, an x value, a y value, a z value of a laser beam reflected from the object can be used as a measured value from a sensor designed as a lidar sensor a value representing a sensor coordinate system can be read.

According to a further embodiment, the advantages of the approach presented here can be implemented as a method for recognizing at least one object from data of at least one first sensor signal vector and / or one second sensor signal vector. The method comprises the following steps: Reading in the first sensor signal vector and the second sensor signal vector, the first sensor signal vector having at least one measured value from a first sensor of a first sensor technology and the second sensor signal vector having at least one measured value from a second sensor of a second sensor technology different from the first sensor technology, wherein the first sensor signal vector represents an object from a point of view of the first sensor and the second signal vector represents the object from a point of view of the second sensor, wherein in the step of reading in a decision network trained according to a variant presented here is also read in; and detecting the object from the first and / or the second sensor signal vector as an input vector of the trained decision network.

With such an embodiment, the previously trained decision network can be used very efficiently, so that, for example, the aforementioned reduction in the data rate in the transmission of features or recognized objects can be implemented very easily.

An embodiment of the approach presented here is particularly efficient, in which a step of marking the detected object is provided in a map representing the surroundings of the first and / or second sensor. In this way, the detected objects can be integrated into the map, for example for later evaluation or transmission, so that, for example, other vehicles can adjust their operation or function to the detected objects in the vicinity of the vehicle.

For this purpose, according to a further embodiment, the first and / or second sensor signal vector, which was provided by a corresponding first and second sensor of a vehicle, can be read in in the reading-in step. In this way, it can be ensured that, for example, different features of the same object, which can be clearly mapped using different sensor technologies in different environmental scenarios, lead to precise detection of the object, regardless of the actual current environmental scenario around the vehicle.

The approach presented here can also be implemented as a device that is set up to carry out the steps of a variant of the method for determining a trained decision network and / or to execute and / or control the steps of a variant of the method for recognizing at least one object from data of at least one first sensor signal vector and / or one second sensor signal vector in corresponding units.

A device can be an electrical device that processes electrical signals, for example sensor signals, and outputs control signals as a function thereof. The device can have one or more suitable interfaces which can be designed in terms of hardware and / or software. In the case of a hardware design, the interfaces can, for example, be part of an integrated circuit in which functions of the device are implemented. The interfaces can also be separate, integrated circuits or at least partially consist of discrete components. In the case of a software design, the interfaces can be software modules that are present, for example, on a microcontroller alongside other software modules.

Also of advantage is a computer program product with program code which can be stored on a machine-readable carrier such as a semiconductor memory, a hard disk memory or an optical memory and is used to carry out the method according to one of the embodiments described above when the program is on a computer or a device is performed.

• 1 eine schematische Darstellung eines Fahrzeugs mit einer Vorrichtung gemäß einem Ausführungsbeispiel zur Erkennung von zumindest einem Objekt;
• 2 ein Blockschaltbild eines beispielhaften AutoEncoders als Ausführungsbeispiel eines Teils einer für das erste und/oder zweite Netzwerk verwendbaren Netzwerk-Architektur;
• 3 eine schematische Darstellung der Vorgehensweise gemäß einem Ausführungsbeispiel für einen ersten Schritt des hier vorgeschlagenen Verfahrens bzw. Ansatzes;
• 4 eine schematische Darstellung einer Vorgehensweise gemäß einem Ausführungsbeispiel für einen zweiten Schritt des hier vorgeschlagenen Verfahrens bzw. Ansatzes;
• 5 eine schematische Darstellung einer Struktur oder Arbeitsweise des Verknüpfungsnetzwerkes;
• 6 ein Ablaufdiagramm eines Ausführungsbeispiels eines Verfahrens zur Ermittlung eines trainierten Entscheidungsnetzwerks; und
• 7 ein Ablaufdiagramm eines Ausführungsbeispiels eines Verfahrens zur Erkennung von zumindest einem Objekt.

The invention is explained in more detail by way of example with reference to the accompanying drawings. Show it:

• 1 a schematic representation of a vehicle with a device according to an embodiment for the detection of at least one object;
• 2 a block diagram of an exemplary AutoEnoder as an exemplary embodiment of a part of a network architecture that can be used for the first and / or second network;
• 3 a schematic representation of the procedure according to an exemplary embodiment for a first step of the method or approach proposed here;
• 4th a schematic representation of a procedure according to an exemplary embodiment for a second step of the method or approach proposed here;
• 5 a schematic representation of a structure or mode of operation of the linking network;
• 6th a flowchart of an embodiment of a method for determining a trained decision network; and
• 7th a flowchart of an embodiment of a method for recognizing at least one object.

In the following description of preferred exemplary embodiments of the present invention, identical or similar reference symbols are used for the elements shown in the various figures and having a similar effect, a repeated description of these elements being dispensed with.

1 shows a schematic representation of a vehicle 100 with a device 105 for the detection of at least one object 110 from data of at least one first sensor signal vector ST1 and / or a second sensor signal vector ST2 . At the object 110 it can be, for example, another vehicle that is driven by the first sensor 115 and / or the second sensor 120 to recognize and for example in an autonomous driving mode of the vehicle 100 is to be considered. The first sensor signal vector ST1 is from a first sensor 115 provided, which in the present case is designed as a radar sensor, for example. The second sensor signal vector ST2 is from a second sensor 120 provided, which in the present case is designed as a LIDAR sensor, for example. The first sensor 115 and the second sensor 120 are thus designed in each case with different sensor technologies, so that, for example, a position and speed of the object 110 in relation to the first sensor 115 and / or the second sensor 120 is recorded by different physical measurement methods, so that, for example, the measurement of parameters for the object is also disrupted 110 When measuring by a sensor such a disturbance may be due to the measurement of the parameters to the object 110 can be reduced or compensated by the second sensor. For example, the occurrence of fog could result in a measurement using the second sensor designed as a LIDAR sensor 120 be subject to interference, whereas a measurement with the first sensor designed as a radar sensor 115 due to the different measuring method is not subject to interference.

The first sensor signal vector ST1 is now a first network 125 supplied, in which a reduction in the complexity of the first sensor signal vector ST1 is executed to get from a state value of a first descriptor 126 corresponding first output signal 127 of the first network 125 a conclusion about features or parameters of the object 110 in relation to the first sensor 115 (for example a distance or Positions of the object 100 in relation to the first sensor 115 ) to be able to close The second sensor signal vector ST2 is now a second network 130 supplied, in which a reduction in the complexity of the second sensor signal vector ST2 is executed to get from a state value of a second descriptor 131 corresponding second output signal 132 of the second network 130 a conclusion about features or parameters of the object 110 in relation to the second sensor 120 (for example a distance or positions of the object 100 in relation to the second sensor 120 ) close. The first network 125 and the second network 130 be trained so that from the first output signal 127 no more conclusions about the sensor technology of the first sensor 115 is possible and / or second output signal 132 no conclusions about the sensor technology of the second sensor 120 is possible. This allows a general representation of the object 110 or the parameter of the object 110 in relation to the first sensor 115 or second sensor 120 achieve that in a subsequent unit 135 can be used for detection, so that the object 110 from the first sensor signal vector ST1 and / or the second sensor signal vector ST2 can be detected. The first network 125 and the second network 130 can be part of the trained decision-making network 140 be. In unity 135 for detection can be in a first subunit 145 a feature extractor can be implemented, for example from the first output signal 127 and / or the second output signal 132 Features of the object 110 in relation to the first sensor 115 and / or the second sensor 120 can extract, such as a distance, a position, a size or a movement of this object 110 in relation to the corresponding sensor. In a card management unit 150 as a further sub-unit of the unit 135 can then be used for detection by the feature extractor 150 extracted features and read in, for example, in a corresponding digital map 155 can be stored so that, for example, for further operation of the vehicle 100 there is information about the position of the object 110 is located. It is also conceivable that from the unit 135 a corresponding feature signal for detection 160 is output which, for example, one of the feature extractor 150 extracted feature directly to other or further components of the vehicle 100 forwards.

In order to enable a particularly simple feature recognition, a trained decision network is advantageously used 140 is used in which a sensor technology-independent descriptor is used as the first descriptor 126 and second descriptor 131 is used, for example, to carry out a card matching using at least one feature recognizer and the corresponding information in the card 155 to deposit. In this way, for example, it is also possible to use only one of the sensor signal vectors ST1 or. ST2 of the respective sensors 115 or. 120 those on the object 110 indicative features are extracted, so that on the one hand it is possible to avoid having to keep a sensor-specific network or a corresponding descriptor available for each sensor and also to receive additional information in the event of a fault in one of the sensors 115 or. 120 would not receive. In order to obtain such a generalized descriptor, a feature extraction network (ie the first half of the feature-learning network) is used for each sensor technology 125 or. 130 ) is used to extract the features required for feature recognition from the radar / LIDAR / et cetera input signal as a sensor signal vector ST1 or. ST2 were used. The advantage of the approach / method proposed here compared to an individual feature extraction / feature representation is a significantly more compact map or algorithm, since only the one feature representation or the corresponding descriptor needs to be saved that matches the two sensor signal descriptors. Another application is the association of features between sensor signal vectors for fusion applications. In previous applications, on the other hand, an individual localization layer with its features is stored in a map for each sensor technology. The approach proposed here or the method presented here reduces the amount of data by using a generic feature descriptor that is sensor technology-independent and can be extracted from sensor data such as LIDAR / radar / etc. sensor data.

Now to the decision-making network 140 or the first network 125 and / or the second network 130 to be interpreted in such a way that the corresponding first output signal 127 and / or the second output signal 132 can be determined from a sensor technology-independent descriptor, a training of the corresponding or relevant components must be carried out in a previous stage. This training can take place, for example, in a laboratory environment in a separate device outside the vehicle 100 out 1 are carried out, but for example using exemplary signal values from two sensors whose sensor technology is different and which were recorded when a vehicle was driven, for example. In order to improve the understanding of the approach proposed here, the functions of a feature-learning network are therefore initially discussed in general, which is the first network 125 or as a second network 130 from the 1 can be used.

The approach presented here uses an architecture of a so-called "feature-learning network". An example of such an architecture could be an AutoEncoder, as described in more detail below.

2 Figure 3 shows a block diagram of an exemplary AutoEnoder 200 as an exemplary embodiment of part of a network architecture that can be used for the first and / or second network with an encoder 210 , a bottleneck 215 and a decoder 220 . The encoder 210 reduces one dimension of the input space, i.e. the signal space of the input variable x, through the use of convolution layers, such as those used in neural networks. The next part of this architecture is the "bottleneck" 215 which represents the greatest reduction in the dimension of the input x. This representation of the input variable x in a lower dimension is also often called a “feature vector” or “descriptor”. The network, here the AutoEncoder, learns in a training step of the network 200 how the current input signal x is represented or reconstructed using only a feature vector / descriptor with low dimension. The next part of the network architecture is the "decoder" 220 , which enlarges the dimension space step-by-step in order to obtain a reconstructed input signal x 'which advantageously reaches the same dimension as the input signal x. In the ideal case, the resulting result of the decoder x 'is identical to the input x, as in the 2 is shown. For the approach presented here, the first network 125 and the second network 130 used the components of the encoder 210 and the bottle neck 215 correspond, in which the complexity-reduced variant of the input signal x is included. For example, the contents of the bottle neck 215 then the first output signal 127 or the second output signal 132 accordingly if the first network 125 and / or the second network 130 according to a configuration of one in the 2 represented network 230 correspond.

3 shows a schematic representation of a procedure according to an exemplary embodiment for a first step of the method or approach proposed here, in which at least two feature-learning networks such as the first network 125 and the second network 130 from the 1 to be trained or adapted. The first trait-learning network 125 the sensor signal vector is used as the input variable ST1 fed, then the first network 125 against the data / measured values of the second sensor signal vector ST2 is trained; it thus becomes the second sensor signal vector ST2 as a reference vector and the first sensor signal vector ST1 as the input vector for the first network 125 used. The second trait-learning network 130 the sensor signal vector is used as the input variable ST2 added, then the second network 130 against the data / measured values from the first sensor signal vector ST1 be trained. The aim of such network training is that the descriptors / feature vectors that have been learned 300a or. 300b are able to completely represent / reconstruct the corresponding information of the other sensor signal vector in the original dimension and resolution. It should therefore go through the first network 125 the first sensor signal vector ST1 to a first descriptor 300a be reduced so from this first descriptor 300a the second sensor signal vector ST2 can be generated. Alternatively or additionally, through the second network 130 the second sensor signal vector ST2 to a second descriptor 300b be reduced so from this second descriptor 300b the first sensor signal vector ST1 can be generated.

The typical input signals or measured values of a first sensor signal vector ST1 as a radar sensor, the parameters r, Az, El, r ', RCS (r = distance, distance, Az = azimuth, El = elevation angle, r' = Doppler shift value, RCS = radar cross-sectional value) per detection / target or object . Current high-performance radar sensors also recognize around 6000 targets per detection in each measurement cycle.

The typical input signals or measured values of a second sensor signal vector ST2 as LIDAR sensors are X, Y, Z, I, SNR (X = X position, Y = Y position, Z = Z position based on a Cartesian coordinate system of the sensor in question, I = intensity of the reflected laser beam, SNR = Signal-to-noise ratio) per detection / target or object. Current high-performance lidar sensors detect around 100,000 targets per detection in each measurement cycle.

4th shows a schematic representation of a procedure according to an exemplary embodiment for a second step of the method or approach proposed here, in which several descriptors such as the descriptors 300a and 300b to a common representation as a descriptor 400 be generalized, which then input values of an object 100 from any sensor such as the sensors 115 and 120 and in the same way for the relevant sensor signal vectors ST1 and ST2 describes. This generalized descriptor 400 can then be used as the first descriptor 126 or second descriptor 131 in the trained decision network 140 from the 1 be used. To such a generalized representation of the descriptors 300a or. 300b to achieve an Association Network (= linking network) 410 used to get from baseline of the two feature-learning networks 125 and 120 of input values, as the sensor signal vectors ST1 and ST2 To be able to distinguish characteristics for the same object. These characteristics are then used in the generalized descriptor 400 stored or in the networks accordingly 115 and 120 linked to a corresponding reduction in complexity from the sensor signal vectors ST1 and ST2 on the generalized descriptor 400 which is then used for the evaluation of both the sensor signal vector ST1 as well as the sensor signal vector ST2 can be used.

5 shows a schematic representation of a structure or mode of operation of the linking network 410 which, for example, can correspond to a GAN structure (GAN = Generative Adversarial Networks = generating generic network). In repeated training of descriptors 300a or. 300b through the GAN structure 410 can then for example be the first descriptor 300a as a training set and the second descriptor 300b can be used as a reference set towards which the training set is to be optimized. However, the reference set has a loss function that simulates disturbances 500 added. This way it becomes a discriminator 510 learned in such a way that the in the second descriptor 300b mapped objects of the corresponding feature-learning network, which contain too large an error, by the discriminator 510 recognized and discarded. In this way, the quality of the feature recognition can be increased by the discriminator network used here, by the similarity of the descriptors 300a and 300b to maximize and from this the generalized descriptor 400 to obtain. The trait-learning networks 125 and 130 can be in the linking network 410 can be trained serially or in parallel, ie the first descriptor can be used first 300a as a training set and the second descriptor 300b can be used as a reference sentence and in parallel or afterwards also the second descriptor 300b as a training set and the first descriptor 300a can be used as a reference rate. In other words, it becomes the first feature-learning network, for example 125 retrained while the second feature-learning network 130 is used as a reference and vice versa.

Furthermore, data from other sensors such as cameras or ultrasound can also be used. Other methods / procedures for feature learning networks and interconnection networks can also be used.

6th shows a flow chart of an exemplary embodiment of a method 600 to determine a trained decision network. The process consists of one step 610 of reading in a first sensor signal vector and a second sensor signal vector, the first sensor signal vector having at least one measured value from a first sensor of a first sensor technology and the second sensor signal vector having at least one measured value from a second sensor of a second sensor technology different from the first sensor technology, the first sensor signal vector having an object represents a viewing angle of the first sensor and the second sensor signal vector represents the object from a viewing angle of the second sensor. The method also includes 600 one step 620 of adapting a first network with the first sensor signal vector as the input vector and the second sensor signal vector as the reference vector and adapting a second network with the second sensor signal vector as the input vector and the first sensor signal vector as the reference vector. Finally, the procedure includes 600 one step 630 training parameters of a decision network using the first and second networks to obtain the trained decision network.

7th shows a flow chart of an exemplary embodiment of a method 700 for the detection of at least one object from data of at least one first sensor signal vector and / or one second sensor signal vector. The procedure 700 includes one step 710 of reading in the first sensor signal vector and the second sensor signal vector, the first sensor signal vector having at least one measured value from a first sensor of a first sensor technology and the second sensor signal vector having at least one measured value from a second sensor of a second sensor technology different from the first sensor technology, the first sensor signal vector having an object represents a viewing angle of the first sensor and the second signal vector represents the object from a viewing angle of the second sensor, wherein in the step of reading in a decision network trained according to a variant described here is also read in. The method also includes 700 one step 720 the detection of the object from the first and / or the second sensor signal vector as input vectors of the trained decision network. Furthermore, a step 730 the marking of the detected object in a map representing the surroundings of the first and / or second sensor can be provided.

The exemplary embodiments described and shown in the figures are selected only as examples. Different exemplary embodiments can be combined with one another completely or with regard to individual features. An exemplary embodiment can also be supplemented by features of a further exemplary embodiment.

Furthermore, method steps according to the invention can be repeated and carried out in a sequence other than that described.

If an exemplary embodiment comprises a “and / or” link between a first feature and a second feature, this can be read in such a way that the exemplary embodiment according to one embodiment has both the first feature and the second feature and according to a further embodiment either only the first Has feature or only the second feature.

List of reference symbols

100

vehicle

105

Device for recognizing at least one object

110

object

115

first sensor

ST1

first sensor signal vector

120

second sensor

125

first network

126

first descriptor

127

first output signal

ST2

second sensor signal vector

130

second network

131

second descriptor

132

second output signal

135

Detection unit

140

Decision-making network

145

Feature extractor

150

Card management unit

155

digital map

160

Feature signal

200

AutoEncoder

210

Encoder

215

Bottleneck

220

decoder

230

network

300a

first descriptor

300b

second descriptor

400

generalized descriptor

410

Link network

500

Loss function

510

Discriminator

600

Method for determining a trained decision network

610

Reading step

620

Step of adaptation

630

Step of exercising

700

Method for recognizing at least one object

710

Reading step

720

Detecting step

Claims (15)

Method (600) for determining a trained decision network (140), the method (600) comprising the following steps: - Reading in (610) a first sensor signal vector (ST1) and a second sensor signal vector (ST2), the first sensor signal vector (ST1) at least one measured value of a first sensor (115) of a first sensor technology and the second sensor signal vector (ST2) at least one measured value of a second Sensor (120) has a second sensor technology different from the first sensor technology, the first sensor signal vector (ST1) representing an object (110) from a perspective of the first sensor (115) and the second sensor signal vector (ST2) representing the object (110) from a Represents the angle of view of the second sensor (120); - Adapting (620) a first network (125) with the first sensor signal vector (ST1) as the input vector and the second sensor signal vector (ST2) as the reference vector and adapting (620) a second network (130) with the second sensor signal vector (ST2) as the input vector and the first sensor signal vector (ST1) as a reference vector; and - training (630) parameters of a decision network (140) using the first and second networks (125) to obtain the trained decision network (140). Method (600) according to Claim 1 , characterized in that in the training step (630) the parameters of the decision network (140) are adapted using disturbance variables (500), the state values of the first network (125) and / or state values of the second network (130) are added. The method (600) according to one of the preceding claims, characterized in that in the training step (630) the parameters of the decision network (140) are trained by a GAN structure using the first (125) and second network (130). Method (600) according to one of the preceding claims, characterized in that In the training step (630) the parameters of the decision network (140) are adapted in such a way that the first network (125) for determining input values of the decision network (140) and the second network (130) for providing reference variables of the decision network (140) is used and / or wherein in the step of adapting (620) the parameters of the decision network (140) are adapted such that the second network (1230) for determining input values of the decision network (140) and the first network (125) for providing Reference variables of the decision network (140) is used. Method (600) according to Claim 4 , characterized in that in the training step (630) the parameters of the decision network (140) initially using the first network (125) to determine input values of the decision network (140) and the second network (125) to provide reference variables of the decision network (140) are used and then used using the second network (125) to determine input values of the decision network (140) and the first network (125) to provide reference variables of the decision network (140). Method (600) according to one of the preceding claims, characterized in that in the step of adapting (620) the first (125) and second (130) networks are adapted simultaneously and / or in parallel. The method (600) according to any one of the preceding claims, characterized in that in the step of adapting (620) the first (125) and / or second (130) network to be adapted as a neural network, in particular as an artificial neural network and / or as an auto-encoder is designed. Method (600) according to one of the preceding claims, characterized in that in the step (610) of reading in as the first sensor signal vector (ST1) measured values of the first sensor (115) of a vehicle (100) and measured values of the second sensor signal vector (ST2) second sensor (120) of the vehicle (100), in particular wherein the first sensor (115) is designed as a radar sensor and the second sensor (115) is designed as a lidar sensor or wherein the first sensor (115) is designed as a lidar sensor and the second sensor (115) is designed as a radar sensor. Method (600) according to Claim 8 , characterized in that in the step (610) of reading in as a measured value of a sensor (115) designed as a radar sensor, a distance detected in relation to the object (110), an azimuth angle, an elevation angle, a Doppler frequency shift value between a transmitted and a The radar beam received by the object (110) or a value representing a radar cross-section is read in and / or that, in the reading-in step (610) as a measured value from a sensor (120) designed as a lidar sensor, an intensity value of a laser beam reflected by the object (110), an SNR value, an x value, a y value, a z value of a laser beam reflected from the object (110) is read in with reference to a value representing a sensor coordinate system. A method (700) for recognizing at least one object (110) from data of at least one first sensor signal vector (ST1) and / or a second sensor signal vector (ST2), the method (700) having the following steps: reading (710) the first Sensor signal vector (ST1) and the second sensor signal vector (ST2), the first sensor signal vector (ST1) at least one measured value from a first sensor (115) of a first sensor technology and the second sensor signal vector (ST2) at least one measured value from a second sensor (120) one of the first sensor technology has different second sensor technology, the first sensor signal vector (ST1) representing an object (110) from a perspective of the first sensor (115) and the second sensor signal vector (ST2) representing the object (110) from a perspective of the second sensor (120) represents, wherein in step (710) of reading in further a according to one of the Claims 1 to 9 trained decision network (140) is read; and detecting (720) the object (110) from the first (ST1) and / or the second sensor signal vector (ST2) as input vectors of the trained decision network (140). Method (700) according to Claim 10 , characterized by a step (730) of marking the detected object (110) in a map (155) representing the surroundings of the first (115) and / or second sensor (120). Method (700) according to one of the Claims 10 or 11 , wherein in step (710) of reading in the first (ST1) and / or second sensor signal vector (ST2) are read, which were provided by a corresponding first (115) and second sensor (120) of a vehicle (100). Device (105) which is set up to carry out the steps of the method (600) according to one of the preceding claims 1 to 9 and / or the steps of the method (700) according to one of the preceding claims Claims 10 to 12th to be executed and / or controlled in appropriate units. Computer program which is set up to carry out the steps of the method (600, 700) according to one of the preceding Claims 1 to 9 or 10 to 12th execute and / or control. Machine-readable storage medium on which the computer program is based Claim 14 is stored.

Images