DE102019130484A1 - Method and device for training an ensemble of neural networks - Google Patents

Abstract

A device for training an ensemble of N neural function networks and a neural distribution network is described. The N function networks are each designed to provide output data for an input data record. The distribution network is designed to select a function network from the N function networks for the input data record, through which the input data record is to be processed in order to generate output data for the output data of the ensemble of N function networks for the input Provide data record. The device is set up, in an epoch of an iterative learning algorithm, using the distribution network to distribute one or more training data sets to N subgroups for the corresponding N function networks, and the N function networks using the corresponding N subgroups of training Adjust records. Furthermore, the device is set up to determine error values for the one or more training data sets on the basis of the N adapted function networks, and to adapt the distribution network on the basis of the error values.

Description

The invention relates to a device and a corresponding method for teaching an ensemble of (deep) neural networks for a specific overall function, in particular for object recognition and / or for object classification.

An at least partially automated vehicle can be set up to evaluate sensor data from one or more surroundings sensors of the vehicle (e.g. an image camera, a lidar sensor, a radar sensor, etc.), e.g. to detect one or more objects in the surroundings of the vehicle. The vehicle can then be automatically guided longitudinally and / or transversely as a function of the one or more detected objects.

The detection of objects on the basis of the sensor data from one or more environment sensors can take place on the basis of specifically learned neural networks, in particular so-called deep neural networks. The quality of the object recognition can possibly be increased by using a so-called ensemble of several neural networks. The individual neural networks of an ensemble can be specialized in specific subtasks of object recognition. The output values of the individual neural networks can be combined in an aggregator in order to provide an overall evaluation of the sensor data.

A challenge when using an ensemble of several neural networks is the assignment of specializations to the individual neural networks and the training of the individual neural networks for the respective specialization. In this context, heuristics are often used, which, however, typically do not lead to an optimal distribution of tasks for the individual neural networks.

The present document deals with the technical task of training an ensemble of several neural networks in an automatic and optimized manner for an overall task or for an overall function. In other words, the present document deals with the technical task of distributing an overall function in an automatic and optimized manner to the individual neural networks of an ensemble.

The object is achieved in each case by the independent claims. Advantageous embodiments are described, inter alia, in the dependent claims. It is pointed out that additional features of a patent claim dependent on an independent patent claim without the features of the independent patent claim or only in combination with a subset of the features of the independent patent claim can form a separate invention independent of the combination of all features of the independent patent claim, which can be made the subject of an independent claim, a divisional application or a subsequent application. This applies equally to technical teachings described in the description, which can form an invention that is independent of the features of the independent patent claims.

According to one aspect, a device (e.g. a computer or a server) for training an ensemble of N (artificial) neural function networks (or function networks) and an (artificial) neural distribution network (or distribution network) described. N is an integer greater than 1, in particular N> 5 or N> 10. The function networks and the distribution network can be learned on the basis of training data with a large number of training data sets that describe or represent a specific overall function. The individual training data sets can each include a training input data set and target or training output data, the target or training output data indicating which result of the trained ensemble of N neural function networks for the training -Input data record should be provided in order to realize the overall function.

The distribution network can be trained in order to distribute individual partial functions of the overall function to individual functional networks. The individual function networks can be trained for the different sub-functions. The individual sub-functions can be described or represented by subsets of the training data. The distribution network can thus be trained in order to divide the large number of training data sets into different subsets for the different functional networks, so that different sub-functions of the overall function are represented by the different subsets. The individual training data records of a subset for a functional network can each include a training input data record and target output data, the target output data indicating which result is provided by the trained functional network for the training input data record should be in order to realize the partial function of the functional network.

The N function networks can each be designed and / or structured to provide (actual) output data for an input data set. The output data can comprise one or more output values or data elements. For example, the output data can be an or comprise several data bits (possibly also just a single data bit). A functional network can in particular be or comprise a deep neural network or a deep neural network (for example with more than 3 or 5 or 10 hidden layers). The input data record can include, for example, an image from a camera (for example a matrix with image pixels). The output data can be designed to indicate whether and, if applicable, where an object of a certain object type and / or a certain object size is located in the image. The individual functional networks can optionally be specialized in different object types (e.g. passenger cars, trucks, buses, two-wheelers, pedestrians, etc.) and / or in different object sizes (e.g. relatively small, medium-sized, large). As an alternative or in addition, the individual functional networks can be specialized in different lighting conditions, e.g. daylight, night, tunnel situation, sunrise, sunset, relatively weak lighting conditions, etc.

The device can be set up to initialize the N function networks in each case in such a way that quasi-random output data are provided for an input data record. In other words, neuron parameters of the individual function networks can optionally be initialized quasi-randomly. The iterative learning algorithm described in this document can cause the individual function networks to provide (actual) output data which, at least on average, correspond to and / or approximate the target output data indicated by the respective subset of the training data.

Furthermore, the device can be set up to initialize the N function networks in each case differently (quasi-randomly). In particular, the neuron parameters of the individual function networks can be initialized in different ways (quasi-randomly). As an alternative or in addition, the N function networks can be initialized in such a way that N different output data are provided for the same input data record from the N function networks. In this way, the quality of the convergence of the N function networks to different specializations in each case can be increased.

The distribution network can be designed to display or select for the input data record at least or precisely one function network from the N function networks through which the input data record is to be processed in order to provide output data. The output data provided by the selected function network can then be used as output data of the ensemble of N function networks for the input data set.
The distribution network can thus be designed to select one (possibly exactly one) function network from the N function networks for the input data record, through which the input data record is to be processed (possibly in an exclusive manner) in order to To provide output data for the output data of the ensemble of N function networks for the input data set.

In other words, the distribution network can be designed as an assignment function which assigns an input data record to a (possibly exactly one) function network. The assignment of an input data record to a functional network can be viewed as an action of the distribution network. The assigned function network can then process the input data record in order to provide output data at the output of the assigned function network. This output data can then (possibly unchanged or one-to-one) be used as output data of the ensemble of the N function networks. The ensemble of N function networks can, if necessary, be trained in such a way that an input data record is processed by precisely one function network in order to provide output data for the entire ensemble of N function networks.

The device can be set up to initialize the distribution network in such a way that a functional network through which the input data record is to be processed is displayed quasi-randomly for an input data record. The learning algorithm described in this document can then cause the distribution network to assign input data records to the different functional networks in accordance with a learned division of the overall function into sub-functions.

Using the iterative learning algorithm, the N function networks and the distribution network can thus be trained on the basis of the training data in such a way that individual sub-functions of the overall function indicated by the training data are gradually assigned to the individual function networks by the distribution network . The distribution network can be gradually trained to assign the training data records relevant for the respective sub-function to the respective function network in order to further improve the specialization of the function network for this sub-function.

In particular, the device can be set up, in an epoch or in an iteration of the iterative learning algorithm, based on the distribution network, one or more (in particular T) training data sets from the training data to N subgroups or subsets for the N function networks to distribute. T is an integer greater than or equal to 1, in particular T> = 8, T> = 128, T> = 512, or T> = 1024. In particular, T is typical much larger than N (e.g. by a factor of 10 or more). The distribution of the one or more training data records can take place in such a way that a training data record is assigned to only one part of the N subgroups (possibly only exactly one subgroup). If T should be less than N, no training data set can be assigned to a subgroup. If necessary, a subgroup can therefore be empty. Typically, however, T is significantly greater than N, so that each of the N subgroups typically comprises at least one of the T training data sets.

An epoch of the learning algorithm can be a phase of the learning algorithm in which all available training data sets are used at least once in order to train the ensemble of N function networks. In a subsequent epoch, the available training data sets can then be used again in order to further adapt the ensemble of N function networks until a convergence criterion is met. In each epoch, the training data sets can optionally be used for at least or precisely one adaptation of the neuron parameters of the functional networks and / or the distribution network.

The device can also be set up, in the epoch of the iterative learning algorithm, to adapt and / or to learn the N function networks on the basis of the corresponding N subgroups or subsets of training data sets. In other words, each individual functional network can (if necessary exclusively) be adapted or learned using the training data records from the corresponding subgroup of training data records. In still other words, the individual function networks can each be selectively adapted and / or learned with the respectively assigned subgroup of training data sets.

When adapting or learning a function network, the individual neuron parameters of the neurons of the function network can be adapted. In particular, the device can be set up to adapt a function network on the basis of the subgroup of training data sets assigned to the function network in order to average the deviation of the actual output data determined for the training data sets from the subgroup from the training data. Data records of the subgroup displayed target output data.

By adapting a function network, the function network is adapted to the sub-function that is described by the subgroup of training data records that have been assigned to the function network by the distribution network. A specialization of the individual functional networks can thus take place. A monitored learning of the individual function networks can thus take place on the basis of the respectively assigned subgroup of training data sets.

The device can also be set up to determine error values for the one or more training data sets (in particular T error values for the T training data sets) on the basis of the N adapted function networks. As already explained above, a training data record typically comprises a training input data record and target output data for the training input data record (as so-called “ground truth”). The target output data can display the output data that should be provided by the trained ensemble of N function networks for the training input data set (after the ensemble of N function networks has been successfully trained).

The device can be set up to determine actual output data on the basis of the adapted function network to which the training data set was assigned. The error value for the training data set can then be determined on the basis of the actual output data and on the basis of the target output data, in particular on the basis of the difference between the actual output data and the target output data.

Furthermore, the device can be set up to adapt the distribution network using the error values for the one or more training data sets (in particular using the T error values for the T training data sets). The error values can be viewed as an indication of how well the overall function indicated by the training data sets is solved or provided by the ensemble of N function networks. The distribution network can be adapted in order to reduce (at least on average) the error values, in particular the T error values. The distribution network can be adapted by means of reinforcement learning, in particular by means of deep Q learning or by means of a deep Q network (DQN). In this case, rewards for reinforcement learning can be determined on the basis of the error values, in particular on the basis of the T error values.

The device thus enables a distribution network and an ensemble of N function networks to be trained automatically (without using a heuristic) in such a way that an overall function is divided in an optimized manner into N sub-functions that are provided by the individual function networks become. In this way, the quality of an overall function provided by several neural networks can be increased. Furthermore, the computational complexity for providing the overall function can be reduced in this way (since only one functional network has to be active for an input data record).

In the learning algorithm described in this document, the distribution network learns from each other about the error values provided by the function networks, and the function networks learn by adapting the training data sets for the individual function networks can be assigned by the adapted distribution network. This mutual learning of the distribution network and the functional networks can gradually increase the quality of the overall function.

As already explained above, the one or more (in particular the T) training data records typically each comprise a training input data record. The training input data record of a training data record and the function network to which the training data record was assigned (or the assignment action corresponding to the function network) can be viewed as a state-action pair.

The device can be set up to adapt a neural Q network as an approximation for a Q function (in the sense of reinforcement learning) on the basis of the T state-action pairs and the corresponding T error values. The Q function or the Q network can display a corresponding number of expected rewards for a large number of status-action pairs. A Q network can thus be iteratively adapted and / or learned. The adapted distribution network can then be determined on the basis of the adapted Q network. In particular, the distribution network can result from the Q network in that, on the basis of the Q network, the (assignment) action is determined for an input data record, through which the reward displayed by the Q network is increased, in particular maximized, becomes. The distribution network can thus be part of the Q network.

By iterative learning of a Q network or a corresponding Q function, the quality of the learned ensemble of functional networks can be increased further.

The device can be set up to repeat a plurality of epochs of the learning algorithm until a termination criterion is reached. The assignment of the individual training data sets to the different functional networks can be different in different epochs (due to the adaptation of the distribution network). Typically, however, the allocation behavior of the distribution network converges with an increasing number of epochs. The termination criterion for the iterative learning algorithm can include a maximum number of epochs and / or a convergence of the ensemble of N function networks and / or the distribution network. Iterative learning of the ensemble of N function networks and the distribution network can thus take place in order to provide a high quality overall function.

The device can be set up to randomly assign at least some of the T training data records to one of the N function networks in an epoch of the learning algorithm. The proportion of T training data records that are randomly assigned can be reduced as the number of epochs of the learning algorithm increases. In this way, it can be ensured in a reliable manner that in an initial phase of the learning algorithm for the N function networks N different sub-functions can be identified, and that the N function networks can gradually be learned for the different sub-functions. The quality of the overall function provided can thus be further increased.

The device can be set up to adapt the N function networks and the distribution network sequentially on the basis of respectively different groups of training data sets within an epoch of the learning algorithm. In particular, different groups, each with T training data records, can be formed and / or provided sequentially from the training data. In this way, the learning of the ensemble of functional networks can be further improved.

According to a further aspect, a device for determining output data of an ensemble of N neural function networks for an input data record is described, where N is an integer greater than 1. The device can be set up to determine the input data record on the basis of sensor data from one or more sensors (in particular environment sensors of a vehicle).

The device can be set up to assign the input data record by means of a neural distribution network to one, in particular precisely one, of the N function networks. The ensemble of N functional networks and / or the distribution network can have been trained with the device described in this document.

The device can also be set up to use the assigned function network to determine output data of the assigned function network for the input data set as output data of the ensemble of N function networks. In particular, output data that indicate an object within the input data record can be provided as output data.

By using an ensemble of N neural function networks in context With a neural distribution network, a high quality overall function can be provided in an efficient manner through partial functions of the individual function networks.

The device can be set up to provide and / or operate at least one vehicle function of a vehicle (e.g. for automated driving) on the basis of the output data of the ensemble of N function networks. In this way, a high-quality vehicle function can be provided.

According to a further aspect, a (road) motor vehicle (in particular a passenger car or a truck or a bus or a motorcycle) is described which comprises at least one of the devices described in this document.

According to a further aspect, a (computer-implemented) method for training an ensemble of N neural function networks and a neural distribution network is described. The N function networks are each designed and / or structured to provide output data for an input data record. The distribution network is designed and / or structured to display at least (in particular precisely) one function network from the N function networks for the input data record, through which the input data record is to be processed in order to provide output data as a basis for the output data of the ensemble of N function networks for the input data set.

The method comprises (in an epoch of an iterative learning algorithm) the distribution of one or more training data sets to N subgroups for the corresponding N function networks using the distribution network, as well as adapting or learning the N function networks using the corresponding N subgroups of training data sets (with a function network being selectively adapted or learned using the one or more training data sets of the corresponding subgroup). Furthermore, the method includes the determination of error values for the one or more training data sets on the basis of the N adapted function networks, as well as the adaptation of the distribution network on the basis of and / or on the basis of the error values determined.

According to a further aspect, a (computer-implemented) method for determining output data of an ensemble of N neural function networks for an input data set is described. The method comprises the assignment of the input data record by means of a neural distribution network to one, in particular to exactly one, of the N function networks. Furthermore, the method comprises determining, on the basis of the assigned function network, output data of the assigned function network for the input data set as output data of the ensemble of N function networks.

According to a further aspect, a software (SW) program is described. The software program can be set up to be executed on a processor (e.g. on a control unit of a vehicle) and thereby to execute one of the methods described in this document.

According to a further aspect, a storage medium is described. The storage medium can comprise a software program which is set up to be executed on a processor and thereby to execute one of the methods described in this document.

It should be noted that the methods, devices and systems described in this document can be used both alone and in combination with other methods, devices and systems described in this document. Furthermore, any aspects of the methods, devices and systems described in this document can be combined with one another in diverse ways. In particular, the features of the claims can be combined with one another in diverse ways.

• 1 beispielhafte Komponenten eines Fahrzeugs;
• 2a ein beispielhaftes neuronales Netz;
• 2b ein beispielhaftes Neuron;
• 3a eine beispielhafte Vorrichtung zum Anlernen eines Ensembles von neuronalen Netzen;
• 3b eine beispielhafte Vorrichtung zur Nutzung eines angelernten Ensembles von neuronalen Netzen;
• 4a eine beispielhafte Vorrichtung zum Anlernen eines Verteil-Netzes und der Funktions-Netze eines Ensembles von neuronalen Netzen;
• 4b eine beispielhafte Vorrichtung zur Nutzung eines angelernten Verteil-Netzes und eines Ensembles mit angelernten Funktions-Netzen;
• 5a ein beispielhaftes Verfahren zum Anlernen eines Ensembles von neuronalen Funktions-Netzen; und
• 5b ein beispielhaftes Verfahren zur Nutzung eines Ensembles von angelernten neuronalen Funktions-Netzen.

The invention is described in more detail below on the basis of exemplary embodiments. Show it

• 1 exemplary components of a vehicle;
• 2a an exemplary neural network;
• 2 B an exemplary neuron;
• 3a an exemplary device for training an ensemble of neural networks;
• 3b an exemplary device for using a trained ensemble of neural networks;
• 4a an exemplary device for training a distribution network and the functional networks of an ensemble of neural networks;
• 4b an exemplary device for using a trained distribution network and an ensemble with trained functional networks;
• 5a an exemplary method for training an ensemble of neural function networks; and
• 5b an exemplary method for using an ensemble of trained neural function networks.

As stated at the beginning, the present document deals with the training and the use of an ensemble of neural networks, for example for the detection of objects in the vicinity of a vehicle. In this context shows 1 exemplary components of a vehicle 100 . The vehicle 100 includes one or more environment sensors 102 that are set up, environment data (ie sensor data) relating to the environment of the vehicle 100 capture. Exemplary environmental sensors 102 are an image camera, a radar sensor, a lidar sensor and / or an ultrasonic sensor.

The vehicle also includes 100 typically one or more vehicle sensors 103 which are set up, state data (ie sensor data) in relation to a state variable of the vehicle 100 capture. Exemplary state variables are the driving speed and / or the (longitudinal) acceleration of the vehicle 100 .

The vehicle also includes 100 one or more actuators 104 for automatic longitudinal and / or lateral guidance of the vehicle 100 . Exemplary actuators 104 are a drive motor, a braking device and / or a steering device.

A control unit 101 of the vehicle 100 can be set up one or more actuators 104 to operate automatically on the basis of the environment data and / or on the basis of the status data. In particular, the control unit 101 be set up on the basis of the sensor data of the one or more environment sensors 102 an object in the vicinity of the vehicle 100 to detect. The one or more actuators 104 can then be operated as a function of the detected object, in particular around the vehicle 100 at least partially automated. The object detection can take place using the methods described in this document.

To detect an object on the basis of the sensor data from one or more environmental sensors 102 At least one neural network can be used that has been trained on the basis of training data for the task of object recognition. 2a and 2 B show exemplary components of a neural network 200 , especially a feedforward network. The network 200 comprises two input neurons or input nodes in the example shown 202 , which at a specific point in time t each have a current value of an input variable as an input value 201 take up. The one or more input nodes 202 are part of an entry layer 211 . In general, the network 200 be designed, an input data record with one or more input values 201 to record.

The neural network 200 also includes neurons 220 in one or more hidden layers 212 of the neural network 200 . Each of the neurons 220 can use the individual output values of the neurons of the previous layer as input values 212 , 211 have (or at least part of them). In each of the neurons 220 processing takes place in order to obtain an output value of the neuron as a function of the input values 220 to determine. The output values of the neurons 220 the last hidden layer 212 can be in an output neuron or output node 220 an output layer 213 processed to the one or more output values 203 of the neural network 200 to determine. In general, the network 200 be designed, output data with one or more output values 203 provide.

2 B illustrates the exemplary signal processing within a neuron 220 , especially within the neurons 202 the one or more hidden layers 212 and / or the output layer 213 . The input values 221 of the neuron 220 are with individual weights 222 weighted to in a total unit 223 a weighted sum 224 of the input values 221 to be determined (if necessary, taking into account a bias or offset 227 ). Through an activation function 225 can be the weighted sum 224 to an initial value 226 of the neuron 220 can be mapped. You can use the activation function 225 For example, the range of values can be limited. For a neuron 220 can eg a sigmoid function or a hyperbolic tangent (tanh) function or a rectified linear unit (ReLU), eg f (x) = max (0, x) as activation function 225 be used. If necessary, the value of the weighted sum 224 with an offset 227 be moved.

A neuron 220 thus has weights 222 and / or an offset, if applicable 227 as a neuron parameter. The neuron parameters of the neurons 220 of a neural network 200 can be learned in a training phase to cause the neural network 200 approximates a certain function and / or models a certain behavior.

Learning a neural network 200 can be done using the backpropagation algorithm, for example. For this purpose, a learning algorithm for the input values can be used in a first phase of a q ^{th epoch} 201 at the one or more input nodes 202 of the neural network 200 corresponding output values 203 at the exit of the one or more output neurons 220 be determined. On the basis of the initial values 203 the error value of an optimization or error function can be determined. In the present case, a temporal difference (TD) error can serve as an optimization or error function for teaching a distribution network, as explained below.

In a second phase of the q ^th epoch of the learning algorithm, the error or the error value is propagated back from the output to the input of the neural network in order to transfer the neuron parameters of the neurons in layers 220 to change. The error function determined at the output can be partially based on each individual neuron parameter of the neural network 200 can be derived in order to determine an extent and / or a direction for adapting the individual neuron parameters. This learning algorithm can be repeated iteratively for a large number of epochs until a predefined convergence and / or termination criterion is reached.

As stated at the beginning, it can be advantageous to use different neural networks for an overall task, e.g. for the recognition of objects from different groups of object types, each of which specializes in solving a sub-task. For example, a specifically learned neural network can be provided for each group of object types, for example for the group of pedestrians, for the group of passenger cars, for the group of trucks, for the group of two-wheelers, etc. The overall task can then be solved jointly by the different neural networks, i.e. by an ensemble of neural networks. The individual neural networks trained for a specific subtask are also referred to in this document as functional networks.

3a shows an exemplary device 300 for teaching an ensemble 304 of functional networks 303 . The device 300 includes a distribution unit 302 that is set up a training record 301 (possibly exactly) one of the functional networks 303 assign. The available training data (with a large number of training data sets 301 ) can thus through a distribution unit 302 on the different functional networks 303 be divided. The distribution process takes place depending on the sub-task of the respective functional network 303 . For example, the individual training records 301 depending on the object type on which the respective training data set relates 301 refers to the functional network specialized in the respective object type 303 be assigned to.

A training data set 301 typically comprises a training input data set (with one or more training input values 201 ) and the associated training output data (with one or more training output values 203 ). Through a variety of training data sets 301 a specific overall function is typically described, with the training output data from a training data set 301 show how the ensemble is 304 of functional networks 303 for the training input data set from the training data set 301 should behave.

The distribution unit 302 thus divides the training data into N subsets for the N function networks 303 on, where the subset for a function network 303 the subtask represented by this functional network 303 is to be fulfilled. The individual functional networks 303 can then be taught in with the respectively assigned subset (e.g. as in connection with the 2a and 2 B described). So can N function networks 303 which are specialized in N different subtasks of an overall task.

3b shows an exemplary device 310 for the use of a trained ensemble 304 with N function networks 303 . An input data record 311 can from all functional networks 303 are processed and each have a starting value 313 (or provide the respective output data). The initial values 313 (or the output data) of the individual function networks 303 can in an agglomeration unit 314 processed, e.g. superimposed, are made to produce an overall output value 315 (or overall output data). The input data record 311 can, for example, have different objects with different object types. An object of a certain object type can then be used in a reliable manner on the basis of the functional network specialized for this object type 303 detected and as an output value 313 this functional network 303 to be provided. By superimposing the output values 313 of the different functional networks 303 can then be used as an overall baseline 315 Information regarding the different objects with different object types can be provided.

By using an ensemble 304 of functional networks 303 that are specialized in partial tasks or partial functions, the quality of an overall function, such as object recognition, can be increased. Furthermore, the complexity of the individual functional networks 303 be reduced.

A challenge when using an ensemble 304 of functional networks 303 is the division of an overall task into sub-tasks for the individual functional networks 303 . With others Words, when using an ensemble 304 of functional networks 303 The question arises as to how an overall task represented by training data can be optimally adapted to different subtasks for the individual functional networks 303 can be split to form an ensemble 304 of functional networks 303 to learn that solves the overall task as optimally as possible. A distribution unit that works on the basis of a heuristic 302 , as in connection with 3a described, typically cannot result in an optimized distribution of tasks.

4a shows an exemplary device 400 for teaching an ensemble 304 of functional networks 303 who have favourited a neural network 402 includes, which is iteratively learned in a training phase to the training data sets 301 the training data on the different functional networks 303 to distribute, and thereby the individual functional networks 303 assign different subtasks. The neural network 402 the distribution unit 302 is also referred to in this document as the distribution network.

Learning the functional networks 303 and the distribution network 402 can take place in a joint training process. The training data, ie the large number of training data sets 301 , can be divided into K groups, each with T training data sets 301 be divided. The T training records 301 a group can be sent sequentially to the distribution network 402 be handed over. The distribution network 402 can then each individual training data set 301 one or more of the N function networks 303 to assign. The distribution network 402 can for example be set up as input values 201 the training input data set of a training data set 301 to receive. Furthermore, the distribution network 402 possibly N output values 203 have those for the corresponding N function networks 303 indicates whether the training record 301 the respective functional network 303 should be assigned or not.

After distributing the T training records 301 on the N function networks 303 there are then N subsets with training data sets 301 for teaching in the corresponding N functional networks 303 available. A training data set can be used 301 (at least in a relatively early phase of the iterative learning algorithm) may be contained in several subsets. A functional network 303 can with the training records 301 from the subset for the function network 303 be learned at least partially. For example, one or more iterations of the related to the 2a and 2 B described learning procedure are carried out to the neuron parameters 222 , 227 of the functional networks 303 adapt.

As a result of the (partial) learning process of a functional network 303 results for each training data set 301 (at least) an error value of the error function. It thus results for each of the T training data sets 301 at least one error value of the error function, the error value of the error function being determined by the functional network 303 is provided to which the respective training data set 301 was assigned. The (at least) T error values are an indication of how well the T training datasets performed 301 overall task described by the ensemble 304 of functional networks 303 could be solved and / or how well that could be solved by the T training records 301 overall task described by the distribution network 402 on the N function networks 303 could be divided. The error values can therefore be used for the distribution network 402 to learn.

For teaching the distribution network 402 Reinforcement Learning (RL) can be used. A training data set can be used 301 , especially the training input data set 311 for a functional network 303 from a training data set 301 , be considered as state s. The functional network 303 that the training record 301 has been assigned (e.g. the one or more output values 302 of the distribution network 402 for the training data set 301 ) can be viewed as action a. Furthermore, the error value of the error function determined by the assigned function network 303 for the training data set 301 is determined to be regarded as a reward (especially a negative reward) r. In other words, the reward r can depend on the error value and / or can be a function of the error value. In a preferred example, the reward r is the negative (possibly scaled) error value.

Die Q-Funktion kann ggf. durch ein neuronales Netz approximiert werden, wobei das neuronale Netz als Eingangswerte 201 ein Zustands-Aktions-Paar aufnimmt, und als Ausgangswert 203 eine Belohnung bereitstellt. Die Zustands-Aktions-Belohnungs-Tupel (s, a, r) für die T Trainings-Datensätze 301 können dabei als Trainingsdaten zum Anlernen des neuronalen Netzes für die Q-Funktion verwendet werden. Das die Q-Funktion approximierende neuronale Netz kann als Q-Netz bezeichnet werden. Zum Anlernen des Q-Netzes kann der o.g. Backpropagation Algorithmus verwendet werden. Zu diesem Zweck kann der sogenannte Temporal Difference (TD)-Fehler δ als Fehlerfunktion betrachtet werden, und diese Fehlerfunktion kann dazu verwendet werden, die einzelnen Neuron-Parameter 222, 227 des Q-Netzes anzupassen. Der TD-Fehler für einen Zustands-Aktions-Belohnungs-Tupel (s, a, r) kann sich dabei als ergeben: $$\delta =Q\left(s,a\right)-\left(r+\gamma \underset{a}{\text{max}}\left(Q\left(s,a\right)\right)\right)$$

wobei Q(s, a) durch die jeweils aktuelle Version des Q-Netzes gegeben ist, und wobei γ ein Diskontierungsfaktor ist.The individual state-action-reward tuples (s, a, r) for the T training data sets 301 can be used to learn a Q-function that displays the resulting (expected, cumulative, future) reward for a state-action pair, ie $$Q\left(s,a\right)\to \mathrm{R.}$$

The Q function can optionally be approximated by a neural network, with the neural network as input values 201 receives a state-action pair, and as an output value 203 provides a reward. The state-action-reward tuples (s, a, r) for the T training records 301 can be used as training data for teaching the neural network for the Q-function. The neural network approximating the Q function can be referred to as a Q network. The above-mentioned backpropagation Algorithm can be used. For this purpose, the so-called temporal difference (TD) error δ can be viewed as an error function, and this error function can be used to calculate the individual neuron parameters 222 , 227 of the Q network. The TD error for a state-action-reward tuple (s, a, r) can result as: $$\delta =Q\left(s,a\right)-\left(r+\gamma \underset{a}{\text{Max}}\left(Q\left(s,a\right)\right)\right)$$

where Q (s, a) is given by the current version of the Q network, and where γ is a discounting factor.

Aus dem aktualisierten Q-Netz ergibt sich über den o.g. Zusammenhang das Verteil-Netz 402 bzw. die Verteil-Funktion. Das Verteil-Netz 402 kann somit als Teil des Q-Netzes betrachtet werden.A state-action pair results from a distribution network 402 corresponding distribution function π, ie a = π (s). The distribution function π should be learned in such a way that for a training input data set 311 (ie for a state s) action a indicated by the distribution function π (ie by the action a indicated by the distribution network 402 for the training input data set 311 displayed function network 303 ) the (future, expected and / or accumulated) reward R is maximized, ie $$\pi \left(s\right)=\underset{a}{\text{argmax}}\left(Q\left(s,a\right)\right)$$

The distribution network results from the updated Q network via the above-mentioned relationship 402 or the distribution function. The distribution network 402 can thus be regarded as part of the Q network.

Alternatively or in addition, a neural network, in particular a DQN (Deep-Q Network), can be learned directly that is designed to output the action a for a specific state s, through which the (future, expected and / or cumulative) reward R is maximized. In particular, a neural network can be learned that approximates π (s).

After updating the Q network and / or the distribution network 402 (especially the DQN) can use the updated distribution network 402 to be used, T further training data sets 301 from the training data to the N function networks 303 to distribute to the N function networks 303 in one or more iterations based on the assigned training data sets 301 to learn and in order for the individual training data sets 301 Identify error values as rewards. There are then again new status-action-reward tuples (s, a, r) with which again the Q network and / or the distribution network 402 can be updated.

This process can be repeated until all training records 301 from the training data were used. An epoch of the learning algorithm can then be regarded as completed. The training data can then be used again in a subsequent epoch (for example sequentially in groups of T training data records each time 301 ) to the N function networks 303 be divided to iteratively the N function nets 303 and the Q network or the distribution network 402 adapt. For training the ensemble 304 of functional networks 303 the training data can thus be used repeatedly in several epochs until a termination criterion is reached. Exemplary termination criteria are the achievement of a maximum number of epochs and / or the convergence of the functional networks and / or the Q network or the distribution network 402 .

The distribution of the training data sets 301 on the individual functional networks 303 can occur at least partially randomly in the individual epochs. The proportion of the randomly assigned training data sets 301 be reduced as the number of epochs increases. For example, 90% or more of the training records can be used in the first epoch 301 randomly assigned. The other training records 301 can be based on the Q network or the distribution network 402 can be assigned (e.g. to the functional network 303 for which the maximum or the minimum output value 203 results). The percentage of randomly assigned training records 301 can then be gradually reduced in the following epochs, for example down to 10% or less, or 5% or less or 1% or less. In this way, the quality of the learning algorithm can be increased.

4b shows an exemplary device 410 for the use of a trained ensemble 304 of functional networks 303 and a trained distribution network 402 . An input data record 311 can be used as an input value 201 to the distribution network 402 and the distribution network 402 can N initial value 203 for the N function networks 303 provide. An initial value can be used 203 for a functional network 303 show how well the input record is 311 of the respective functional network 303 can be processed. It can then be based on the initial values 203 of the distribution network 402 the functional network 303 can be selected for which the output value 203 shows the highest match or relevance. The input data record 311 can then go through the selected function network 303 processed to the (total) baseline value 315 (or the total output data) for the input data record 311 provide.

5a shows a flow diagram of an exemplary (computer-implemented) method 500 for teaching an ensemble 304 of N neural function networks 303 and a distribution network 402 . N is an integer greater than 1, for example N> 5 or N> 9. The N functional networks 303 can each be designed (in particular structurally created) for an input data record 311 (e.g. for a picture from a camera 102 ) Output data 203 (e.g. a reference to an object in the image of the camera 102 ) to provide.

The distribution network 402 can be designed (in particular structurally created) for the input data record 311 the functional network 303 from the N function networks 303 display by which the input data set 311 should be processed to the output data 203 of the ensemble 304 of N functional networks 303 for the input data record 311 provide. In other words, the distribution network 402 can be designed for an assignment function in which an input data record 311 one, in particular exactly one, of the N function networks 303 is assigned. For this purpose, the distribution network 402 eg N output values 203 for the N function networks 303 exhibit. The N output values 203 can determine the relevance of the corresponding N function networks 303 for the input data record 311 Show. The output values 203 be defined in such a way that the relevance increases with increasing value or, alternatively, that relevance decreases with increasing value. The highest or lowest baseline value 203 can then the functional network 303 which the input data record 311 is assigned.

If necessary, a training input data record can be used during the training phase 301 several (e.g. all functional networks 303 for which the initial value 203 is larger or smaller than a certain threshold value). With increasing convergence, an assignment to exactly one functional network then typically takes place 303 ).

The procedure 500 includes, in an epoch of an iterative learning algorithm, the distribution 501 of T training records 301 to N subgroups for the N function networks 303 based on the distribution network 402 . T is an integer greater than or equal to 1. Typically, T> 100, or T> 500, or T> 1000.

The method also includes 500 customizing 502 of the N function networks 303 based on the corresponding N subgroups of training data sets 301 . In other words, each of the N function networks 303 can be based on the respective functional network 303 assigned training records 301 be adjusted. For this purpose, for example, one or more iterations or epochs of a backpropagation algorithm can be used for the respective functional network 303 be performed. Adapting a functional network 303 includes adapting the neuron parameters 222 , 227 of the respective functional network 303 .

The procedure 500 further comprises determining 503 of T error values for the T training data sets 301 based on the N adapted functional networks 303 . The error value for a training record 301 can use the adapted function network 303 which the training data set 301 was assigned. For this purpose, using the adapted function network 303 the output data 203 for the training input data set 311 of the training data set 301 determined and with the target output data 203 from the training data set 301 be compared.

The procedure also includes 500 customizing 504 of the distribution network 402 based on the T error values. In particular, the T tuples (each from a training input data record 311 , an assignment to a function network 303 , and an error value or the corresponding reward) can be used to control the distribution network 402 adapt. First, a Q network can be adapted to approximate a Q function, and then the adapted distribution network on the basis of the adapted Q network 402 provided and / or calculated.

It should be noted that if a training input record 311 several functional networks 303 is assigned, a tuple can be provided for each assignment. More than T tuples can thus possibly be provided. In particular, training input data records 311 a tuple can be provided in each case.

The training records 301 typically describe a certain overall function in sum. By the procedure described 500 this overall function can be optimized for the individual function network 303 be divided in order to increase the quality of the overall function.

5b shows a flow diagram of an exemplary (computer-implemented) method 510 to determine output data 203 of an ensemble 304 of N neural function networks 303 for an input data record 311 (e.g. for a picture from a camera). N is an integer greater than 1, in particular N> 5 or N> 10. The procedure 500 can for example through the control unit 101 of a vehicle 100 are executed.

The procedure 510 includes assigning 511 of the input data record 311 by means of a neural distribution network 402 to one, in particular to exactly one, of the N function networks 303 . The distribution network 402 together with the N functional networks 303 have been trained to optimize coordination between the distribution function of the distribution network 402 and the sub-functions of the individual function networks 303 to enable.

The method also includes 510 determining 512 , based on the assigned functional network 303 , of output data 203 of the assigned function network 303 for the input data record 311 as output data 203 of the ensemble 304 of N functional networks 303 . In particular, those from the assigned function network 303 provided output data 203 directly and / or unchanged as output data 203 of the ensemble 304 of functional networks 303 be used.

With the measures described in this document, an ensemble 304 of functional networks 303 be provided for an overall function, which is an optimized distribution of the overall function on the individual function networks 303 having. In this way, the quality of the overall function provided can be increased.

The present invention is not restricted to the exemplary embodiments shown. In particular, it should be noted that the description and the figures are only intended to illustrate the principle of the proposed methods, devices and systems by way of example.

Claims (16)

Device (400) for training an ensemble (304) of N neural function networks (303) and a neural distribution network (402); where N is an integer greater than 1; wherein the N function networks (303) are each designed to provide output data (203) for an input data record (311); wherein the distribution network (402) is designed to select a function network (303) from the N function networks (303) for the input data record (311) through which the input data record (311) is to be processed, to provide output data (203) which represent output data (203) of the ensemble (304) of N function networks (303) for the input data set (311); wherein the device (400) is set up, in an epoch of an iterative learning algorithm, - using the distribution network (402) to distribute one or more training data sets (301) to N subgroups for the corresponding N function networks (303); - to adapt each of the N function networks (303) in each case on the basis of the corresponding subgroup of training data sets (301); - to determine error values for the training data sets (301) on the basis of the N adapted function networks (303); and - adapt the distribution network (402) based on the error values. Device (400) according to Claim 1 wherein the device (400) is set up to adapt the distribution network (402) by means of reinforcement learning, in particular by means of deep Q-learning; and - determine rewards for reinforcement learning based on the error values. Device (400) according to one of the preceding claims, wherein - the one or more training data sets (301) each comprise a training input data set (311); - the training input data record (311) of a training data record (301) and the function network (303) to which the training data record (301) was assigned form a state-action pair; and - the device (400) is set up, - on the basis of the state-action pairs and the corresponding error values for the one or more training data sets (301) to adapt a neural Q network as an approximation for a Q function, which has a corresponding one for a large number of state-action pairs Displays variety of expected rewards; and - Determine the adapted distribution network (402) on the basis of the adapted Q network. Device (400) according to one of the preceding claims, wherein - A training data record (301) comprises a training input data record (311) and target output data (203) for the training input data record (311); and - the device (400) is set up, - using the adapted function network (303) to which the training data set (311) was assigned, to determine actual output data (203); and - the error value for the training data set (311) on the basis of the actual output data (203) and on the basis of the target output data (203), in particular on the basis of a difference between the actual output data (203) and the target output data (203 ), to determine. Device (400) according to one of the preceding claims, wherein the device (400) is set up to adapt a function network (303) on the basis of the subgroup of training data sets (301) assigned to the function network (303) in order to, on average to reduce a deviation between the actual output data (203) determined for the training data records (301) from the subgroup and the target output data (203) displayed in the training data records (301) of the subgroup. Device (400) according to one of the preceding claims, wherein - The device (400) is set up to repeat a plurality of epochs of the learning algorithm until a termination criterion is reached; and - The termination criterion comprises in particular a maximum number of epochs and / or a convergence of the ensemble (304) of N function networks (303). Device (400) according to one of the preceding claims, wherein the device (400) is set up - to randomly assign at least some of the one or more training data sets (301) to one of the N function networks (303) in an epoch of the learning algorithm; and - to reduce a proportion of the one or more training data sets (301) that are randomly assigned as the number of epochs of the learning algorithm increases. Device (400) according to one of the preceding claims, wherein the device (400) is set up, within an epoch of the learning algorithm, the N function networks (303) and the distribution network (402) sequentially on the basis of different groups of training sessions -Data sets (301) to adapt. Device (400) according to one of the preceding claims, wherein - a functional network (303) is a deep neural network; and or - The ensemble (304) of N function networks (303) is designed to record an image of a camera (102) as an input data record (301) and to output at least one object detected in the image as output data (203); and or - A training data record (301) comprises a training input data record (311) and target output data (203) for the training input data record (311), which are provided by the ensemble (304) of N neural function networks ( 303) should be provided for the training input data set (311). Device (400) according to one of the preceding claims, wherein the device (400) is set up - to initialize the distribution network (402) in such a way that a function network (303) through which the input data record (311) is to be processed is displayed quasi-randomly for an input data record (311); and or - to initialize the N function networks (303) in each case in such a way that quasi-random output data (203) are provided for an input data record (311). Device (410) for determining output data (203) of an ensemble (304) of N neural function networks (303) for an input data set (311); where N is an integer greater than 1; wherein the device (410) is set up, - to assign the input data record (311) by means of a neural distribution network (402) to one, in particular exactly one, of the N function networks (303); and - Using the assigned function network (303) to determine output data (203) of the assigned function network (303) for the input data record (311) as output data (203) of the ensemble (304) of N function networks (303) . Device (410) according to Claim 11 , the ensemble (304) of N function networks (303) and the distribution network (402) using a device (400) according to one of the Claims 1 to 10 was determined. Device (410) according to one of the Claims 11 to 12th wherein the device (410) is set up to determine the input data record (311) on the basis of sensor data from one or more sensors (102); and / or - to display a detected object as output data (203) on the basis of the input data record (311). Device (410) according to one of the Claims 11 to 13th wherein the device (410) is set up to provide and / or operate at least one vehicle function of a vehicle (100) on the basis of the output data (203) of the ensemble (304) of N function networks (303). Method (500) for training an ensemble (304) of N neural function networks (303) and a neural distribution network (402); where N is an integer greater than 1; wherein the N function networks (303) are each designed to provide output data (203) for an input data record (311); wherein the distribution network (402) is designed to select a function network (303) from the N function networks (303) for the input data record (311) through which the input data record (311) is to be processed, to provide output data (203) which represent output data (203) of the ensemble (304) of N function networks (303) for the input data set (311); wherein the method (500) comprises, in an epoch of an iterative learning algorithm, - distribution (501) of one or more training data sets (301) to N subgroups for the corresponding N function networks (303) on the basis of the distribution network (402) ; - Adapting (502) each individual one of the N function networks (303) on the basis of the respectively corresponding subgroup of training data sets (301); - Determining (503) error values for the one or more training data sets (301) on the basis of the N adapted function networks (303); and - adapting (504) the distribution network (402) on the basis of the error values. Method (510) for determining output data (203) of an ensemble (304) of N neural function networks (303) for an input data record (311); where N is an integer greater than 1; wherein the method comprises (510), - Assigning (511) the input data record (311) by means of a neural distribution network (402) to one, in particular to exactly one, of the N function networks (303); and - Determination (512), based on the assigned function network (303), of output data (203) of the assigned function network (303) for the input data set (311) as output data (203) of the ensemble (304) of N functions - Networks (303).

Images