DE202021102084U1 - Device for determining network configurations of a neural network while fulfilling a number of secondary conditions - Google Patents

Abstract

Device for determining a Pareto set of network configurations, the network configurations for a neural network being determined for a given application and the given application being determined in the form of provided training data, the device comprising a machine-readable storage medium on which commands are stored that were used in Running from a computer caused the computer to perform a procedure that included:
a) providing (S1) a plurality of randomly selected network configurations (α);
b) determining hyperparameters (A) for each of the provided network configurations (α) by means of an optimizer;
c) training of neural networks configured with the provided network configurations (α) and depending on the respectively associated determined hyperparameters (A) on the provided training data;
d) determining a performance (accurancy) on at least part of the training data provided for each of the trained neural networks;
e) evaluating the trained neural networks with regard to predetermined optimization goals (n_pars);
f) adding a tuple comprising the network configurations (a), the determined hyperparameters (A) and the determined performance (accurancy) and the evaluated optimization goals (n_pars) to a further training data set;
g) adapting the optimizer for the further training data set, in particular as a function of network configurations (a) and the determined performance (accurancy) for predicting suitable hyperparameters (A);
h) Repeated execution of the following steps:
i. Selecting (S3) network configurations (α) from the further training data set;
ii. randomly mutating (S4) the selected network configurations; and
iii. Carrying out steps b) to g) with the mutated network configurations;
i) Repeatedly performing the following steps:
i. Selecting (S5) network configurations from the further training data set;
ii. Compressing (Pruning) (S6) the selected network configurations;
iii. Carrying out steps b) to g) with the compressed network configurations;
j) Selecting (S7) the network configurations (α) from the further training data set which correspond to a Pareto set in terms of performance and at least one of the further optimization goals.

Description

The invention relates to a device for determining a plurality of network configurations of a neural network for a given training data set, so that the configurations meet a plurality of secondary conditions with regard to optimization goals.

State of the art

The properties of neural networks are largely determined by their architecture. The architecture of a neural network is defined, for example, by its network configuration, which is given, among other things, by the number of neuron layers, the type of neuron layers (linear transformations, non-linear transformations, normalization, linking with other neuron layers, etc.), the number of weights and the like. In particular with increasing complexity of the applications or the tasks to be solved, the random finding of suitable network configurations is time-consuming, since each candidate of a network configuration must first be trained in order to be able to evaluate its performance.

Various approaches are known from the prior art for the architecture search of neural networks in order to find a configuration of a neural network for a specific application that is optimized with regard to a prediction error and with regard to one or more optimization goals.

For example disclosed EP 19719232 a method for determining a suitable network configuration for a neural network for a given application from a set of optimal network configurations.

Advantages of the invention

One object of the invention is to provide a method which enables a set of optimal network configurations to be found with little computational effort, each of which is optimal with regard to a plurality of secondary conditions (that is, on a Pareto front). The set of optimal network configurations is also called the Pareto set in the following.

One advantage of the invention is that with the claimed method the Pareto front is shifted further in a positive direction, that is to say a superior Pareto front can be found compared to known methods.

Another advantage is that the network configuration that is optimal for a given hardware with limited resources can be selected on the basis of the Pareto front. For example, the Pareto front can be used to select those network configurations that optimally utilize a hardware limitation (e.g. storage space, computing power, energy budget, bandwidth, etc.) and achieve the highest performance under this limitation. This is a significant advantage, since in many practical applications differently designed hardware components are provided for the solution of the same task for customers and the suitable network configurations for the differently designed hardware components can be applied efficiently using the Pareto front.

Disclosure of the invention

In a first aspect, the invention relates to a computer-implemented method for determining a Pareto set of network configurations. A Pareto set can be understood to mean a plurality of network configurations in which it is not possible to improve one (target) property, in particular an optimization target, without having to worsen another at the same time. A network configuration can be understood to mean an architecture of the neural network, ie a structure as characterized, for example, by the following features: number of layers, number of filters, size of filters, dropout rate, number of neurons per layer, skip connection, batch -Normalization layers, pooling layers, etc.

The network configurations for a neural network are preferably determined for a given application and the given application is determined in the form of provided training data.

The method begins with providing a plurality of randomly selected network configurations (a).

This is followed by determining hyperparameters (A) for each of the network configurations (a) provided by means of a hyperparameter optimizer. The hyperparameters (A) can be, for example, a learning rate, regularizations, a learning algorithm for training the neural network (e.g. Ada, SGD, etc.), etc. The hyperparameter optimizer preferably uses a Bayesian optimization.

This is followed by training of neural networks configured depending on the network configurations provided (a) and depending on the respective associated determined hyperparameters (A) on the training data provided and determining a performance (accurancy) for each of the trained neural networks, as well as adapting the Hyperparameter optimizer depending on the used network configurations (a), hyperparameters (A) and thus achieved performance (accurancy) and an evaluation of the trained neural networks with regard to at least one further predetermined optimization target (n_pars) in addition to the optimization target of the performance (accurancy) of the neural network. There are preferably a plurality of further optimization goals and at least one optimization goal characterizes a limited hardware resource. An optimization goal is particularly preferably a number of parameters of the neural network (n_pars).

This is followed by adding a tuple comprising the network configurations (a), hyperparameters (A) used for the training step and the determined performance and the evaluated optimization target (n_pars) to a further training data set. The performance can characterize the performance of the appropriately configured neural network, such as, for example, a classification accuracy or an error rate.

• Auswählen von Netzkonfigurationen aus dem weiteren Trainingsdatensatz, zufälliges Mutieren der ausgewählten Netzkonfigurationen und ein Evaluieren der mutierten Netzkonfigurationen gemäß den oben erläuterten Schritten über ein Trainieren von neuronalen Netzen konfiguriert abhängig von den mutierten Netzkonfigurationen bis Auswerten der neuronalen Netze und Hinzufügen des Tupels zu dem weiteren Trainingsdatensatz.

The following steps are then repeated for the first time, in particular until a first termination criterion is met:

• Selecting network configurations from the further training data set, randomly mutating the selected network configurations and evaluating the mutated network configurations in accordance with the steps explained above via training of neural networks configured depending on the mutated network configurations to evaluating the neural networks and adding the tuple to the further training data set.

A second repeated execution of the following steps then follows, in particular until a second termination criterion is met. The first and second termination criterion can define the occurrence of at least one of the following events: a predetermined number of repetitions has been achieved, or a predetermined value for the performance has been achieved by at least one of the network configuration, or a predetermined budget of time, computing power or the like has been used up . If the termination criterion is a time budget, then this is preferably distributed as follows: The entire time budget has been divided into fractions of 0.35, 0.40 and 0.25, with the first 35% of the total time budget for the first steps of random generation of network configurations and evaluating these and creating the further training data set and 40% or 25% of the remaining time of the total time budget spent on the first repetition and second repetition. Accordingly, the budget of the first termination criterion is 40% of the total allocated time for the procedure and the budget of the second termination criterion is 25% of the total allocated time for the procedure.

During this second repeated execution, the following steps are then carried out: Compressing (pruning) the selected network configurations and carrying out the steps explained above for evaluating the compressed network configurations. The performance is preferably determined here on the training data. Furthermore, the performance for the other steps can be determined on validation data.

This is followed by selection of the network configurations (a) from the further training data set, which correspond to a Pareto set in terms of performance and at least one further optimization goal.

The Pareto set is preferably determined with regard to the performance of a prediction error and at least one further optimization goal, such as a number of the parameters of the neural network.

Optionally, after the method for determining the Pareto front has been completed, the suitable network configuration can be selected from the current network configuration set. From the set of the optimal network configuration, depending on a predetermined optimization target, that network configuration can be taken from the Pareto front that fulfills this optimization target. For example, the appropriate network configuration can be selected from the current network configuration set based on an overall cost function that depends on the prediction error and resource costs with regard to the at least one optimization goal.

It is proposed that the network configurations be mutated by randomly adding layers or by combining at least two randomly drawn network configurations.

For this purpose, additional folding layers or fully meshed layers are preferably added at a random position. This means that the network configuration is changed in such a way that it characterizes a structure of the neural network which differs from the original neural network with the original network configuration in that a layer was added at a random position within the neural network.

dabei sind w die Netzparameter (Gewichtungen) des neuronalen Netzes N und w̃ die Netzparameter des variierten neuronalen Netzes TN. X entspricht dem Raum, auf dem das neuronale Netz angewendet wird.It is also conceivable that the mutation is carried out by network morphisms in accordance with predetermined rules that can be determined with the aid of an operator. General is a Network morphism is an operator T that maps a neural network N onto a network TN, where: $$N^w\left(x\right)={\left(TN\right)}^{\tilde{w}}\left(x\right)\text{For}x\in X$$

where w are the network parameters (weightings) of the neural network N and w̃ the network parameters of the varied neural network TN. X corresponds to the space in which the neural network is applied.

It is also proposed that, in the compressing step, neurons of the appropriately configured neural network are sorted according to a sum of their initial weight values, and that the smallest sums are removed having a predetermined number of neurons.

A so-called "knowledge distillation" can also be carried out for the compression, see e.g. Georey Hinton, Oriol Vinyals, and Je Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv: 1503.02531, 2015.

It is conceivable that after the compression a retraining takes place, in which case the behavior of the unpruned neural network is imitated. In other words, input variables from the training data set are used here as training data and not the assigned output variables from the training data set as labels, but the determined output variables of the unpruned neural network.

Furthermore, it is proposed that after the mutation, the neural network configured with the mutated network configurations, depending on the achieved output variables of the neural networks configured with the network configurations that were used for the mutation, be pre-trained.

It is advantageous if weight values of the neural networks configured with the network configurations that were used for the mutation are reused for the neural networks configured with the mutated network configurations.

Furthermore, it is proposed that the probability of the selection is higher for network configurations that are closer to the Pareto front and network configurations that are not on the Pareto front are selected with the same probability.

Preferably, if the further training data set only contains network configurations that lie on a Pareto front, a network configuration is randomly drawn from the further training data set. If, on the other hand, the further training data record also contains network configurations that are not on a Pareto front, a network configuration is randomly drawn from the further training data record.

The drawn network configuration is preferably removed from the further training data set.

It is also proposed that the hyperparameters (A) be optimized by means of a constrained Bayesian optimization.

The hyperparameters are preferably non-architectural hyperparameters (e.g. learning rate, weight decay, dropout rate, batch size, etc.). The restricted Bayesian optimization can be understood as a Bayesian optimization which contains all hyperparameters (architectural or not), but its acquisition function is optimized limited to the given architecture. In other words, Bounded Bayesian Optimization knows the architectural hyperparameters but cannot select them. In this way, the optimizer can select the best hyperparametric configuration for a given architecture.

It is further proposed that at least one of the optimization goals characterize a physical property of a hardware component, a neural network configured with one of the network configurations being executed on the hardware component.

Furthermore, a suitable network configuration can be selected from the Pareto set based on an overall cost function that depends on the performance, in particular the prediction error, and resource costs with regard to the at least one optimization goal.

It is also proposed that when the network configurations are optimized, they are set up in such a way that they can be used for computer-based vision, in particular for image classifications.

It is also proposed that the network configurations for a neural network be optimized for use as an image classifier. The image classifier assigns an input image to one or more classes of a predetermined classification. For example, images of serially manufactured, nominally identical products can be used as input images. The image classifier can, for example, be trained to assign the input images to one or more of at least two possible classes that represent a quality assessment of the respective product.

The concept of the image basically encompasses any distribution of information arranged in a two- or multi-dimensional grid. This information can be, for example, intensity values of image pixels that were recorded with any imaging modality, such as with an optical camera, with a thermal imaging camera or with ultrasound. However, any other data, such as audio data, radar data or LIDAR data, can also be translated into images and then classified in the same way.

According to a second aspect, a use of a neural network with a network configuration that has been created with the above method for the specified application is for the specified application, the neural network in particular for realizing functions of a technical system, in particular a robot Vehicle, a tool or a machine tool is designed. Here, for example, the neural network configured with one of the network configurations can determine an output variable depending on a detected sensor variable of a sensor, depending on which a control variable can then be determined by means of, for example, a control unit, in particular the technical system.

In a third aspect, the invention relates to a method for providing a neural network with a network configuration that has been created using a method according to the first aspect, the neural network in particular for realizing functions of a technical system, in particular a robot, a vehicle, of a tool or a machine tool.

According to a further aspect, a control device, in particular for controlling functions of a technical system, in particular a robot, a vehicle, a tool or a machine tool, is provided with a neural network that is configured using the above method.

In further aspects, the invention relates to a device and a computer program, each of which is set up to carry out the above methods, and a machine-readable storage medium on which this computer program is stored.

• 1 eine schematische Darstellung eines beispielhaften neuronalen Netzes;
• 2 eine mögliche Netzkonfiguration eines neuronalen Netzes;
• 3 ein Flussdiagramm zur Darstellung eines Verfahrens zum Ermitteln einer pareto-optimalen Menge von Netzkonfigurationen für das Ermitteln eines geeigneten Netzkonfigurationskandidaten für eine vorgegebene Anwendung;
• 4 eine schematische Darstellung einer Pareto-Front von Netz-konfigurationen abhängig von dem Vorhersagefehler und einem weiteren Optimierungsparameter, insbesondere einem Ressourcen-Nutzungsparameter; und
• 5 einen möglichen Aufbau einer Vorrichtung zum Ausführen des Verfahrens;

Embodiments of the invention are explained in more detail below with reference to the accompanying drawings. In the drawings show:

• 1 a schematic representation of an exemplary neural network;
• 2 a possible network configuration of a neural network;
• 3 a flowchart to illustrate a method for determining a Pareto-optimal set of network configurations for determining a suitable network configuration candidate for a given application;
• 4th a schematic representation of a Pareto front of network configurations depending on the prediction error and a further optimization parameter, in particular a resource usage parameter; and
• 5 a possible structure of a device for carrying out the method;

1 shows an example of the structure of a neural network 1 , which usually consists of several cascaded layers of neurons 2 of several neurons each 3 having. The layers of neurons 2 have an entrance layer 2E for creating input data, several intermediate layers 2Z and an output layer 2A to output calculation results.

entsprechen, wobei O_j dem Neuronenausgang des Neurons, φ der Aktivierungsfunktion, x_i dem jeweiligen Eingangswert des Neurons, w_i,j einem Gewichtungsparameter für den i-ten Neuroneneingang in der j-ten Neuronenschicht und θ_j einer Aktivierungsschwelle entsprechen. Die Gewichtungsparameter, die Aktivierungsschwelle und die Wahl der Aktivierungsfunktion können als Neuronenparameter in Registern des Neurons gespeichert sein.The neurons 3 the layers of neurons 2 can do a conventional neuron function $$O_j=\phi \left({\displaystyle \sum _{i=1}^{\mathrm{M.}}\left(x_i{w}_{i,j}\right)-{\theta }_j}\right)$$

_{O j} correspond to the neuron output of the neuron, φ the activation function, x _i the respective input value of the neuron, w _{i, j} a weighting parameter for the i-th neuron _{input in the j-th neuron layer and θ j} an activation threshold. The weighting parameters, the activation threshold and the selection of the activation function can be stored as neuron parameters in registers of the neuron.

The neuron outputs of a neuron 3 can each be used as neuron inputs to neurons 3 of the remaining neuron layers, ie one of the subsequent or one of the preceding neuron layers 2 , passed or if it is a neuron 3 the output layer 2A is output as a calculation result.

Neural networks formed in this way 1 can be implemented as software or with the help of calculation hardware that maps part or the entire neural network as an electronic (integrated) circuit. Such computation hardware is usually chosen to build a neural network when the computation is to be carried out very quickly, which could not be achieved with implementation in software.

The structure of the corresponding software or hardware is specified by the network configuration, which is determined by a large number of configuration parameters. The network configuration determines the calculation rules of the neural network. The configuration parameters include in a conventional network configuration, as for example in 1 is shown schematically, the number of neuron layers, the respective number of neurons in each neuron layer, the network parameters given by the weightings, the activation threshold and an activation function, information on the coupling of a neuron with input and output neurons and the like.

Apart from the network configuration described above, further configurations of neural networks are possible in which neurons are provided which are coupled on the input side to neurons from different neuron layers and on the output side to neurons from different neuron layers. Furthermore, neuron layers can also be provided in this regard which provide feedback, ie the input side with neuron layers which are provided on the output side of the relevant neuron layer with regard to the data flow. In this regard shows 2 schematically a possible configuration of a neural network with several layers L1 to L6 which are initially carried out in a manner known per se, as in 1 shown schematically, are coupled to one another, ie neuron inputs are connected to neuron outputs of the preceding neuron layer. Furthermore, the neuron layer L3 an area on the input side with neuron outputs of the neuron layer L5 is coupled. The neuron layer can also L4 provide, on the input side with outputs of the neuron layer L2 to be connected.

A method for determining an optimized network configuration for a neural network based on a predetermined application is to be carried out below. The application is essentially determined by the size of input variable vectors and the output variable vectors assigned to them, which represent the training data that define a desired network behavior or a specific task.

In 3 describes a method for determining a set of suitable network configurations for a neural network based on a desired application. The amount obtained in this way is intended to facilitate the selection of a network configuration for the desired application. The network configuration should therefore specify a neural network that can be used and is suitable for a specific application and that is optimized with regard to a prediction error and additionally with regard to at least one further optimization goal. In particular, the set of network configurations should correspond to a Pareto front or a Pareto set of network configurations that are optimized with regard to a plurality of optimization goals. The optimization goals can be, for example: performance, required resources of the neural network 1 in the form of the costs of the respective resource (such as the number of parameters), etc. The performance can be a classification accuracy or prediction errors, for example on validation data. Further optimization goals can relate to properties of the resource for the calculation hardware, among other things, such as, for example: a memory size, an evaluation speed, compatibility with regard to special hardware, evaluation energy consumption and the like.

The flowchart included in 3 is shown describes an embodiment of the method according to the invention to determine network configurations on the Pareto front.

In step S1 a providing ( S1 ) a plurality of randomly selected network configurations (a).

This is followed by step S2 . This step comprised a number of sub-steps:

First will be in step S2 a determination of hyperparameters (A) for each of the network configurations (a) provided is carried out by means of an optimizer.

This is followed by training of neural networks configured as a function of the network configurations (α) and as a function of the respectively associated determined hyperparameters (A) on the provided training data.

A performance (accurancy) is then determined for each of the trained neural networks.

The optimizer is then adapted depending on network configurations (a), hyperparameters (λ) and performance (accurancy). A parameterization of the optimizer is therefore optimized in such a way that, given a network configuration, it outputs those hyperparameters that lead to the highest performance.

This is followed by an evaluation of the neural networks configured as a function of the provided network configurations (a) with regard to predetermined optimization goals. One of the optimization goals can be a number of the parameters (n_pars) of the neural network.

Then finally follows in step S2 adding the tuple comprising network configurations (a), hyperparameters (λ) and performance (accurancy) and the optimization goals (n_pars) to a further training data set.

After the substeps of S2 have been carried out, the following steps are repeated.

First is step S3 executed, in which a selection of network configurations from the further training data set is carried out. After step S3 becomes a random mutation S4 of the selected network configurations. Thereupon will step S2 executed with the mutated network configurations. Ie all the substeps of the step described above S2 are now executed with the mutated network configurations. This is in 3 through the connection 31 shown. This sequence of steps just described S3 , S4 and S2 are executed several times until a termination criterion is met, for example a specified time budget has been used up.

The following steps are then carried out repeatedly. This repetition begins with step S5 . A selection of network configurations from the further training data set is carried out here. Then step follows S6 , which a compression (English. Pruning) of the selected network configurations from step S5 performs. With these compressed network configurations, all substeps are made from step S2 executed. This is due to the dashed connection 32 shown. After this S2 has been executed, this repetition begins again with S5. This sequence just described is carried out several times until a further termination criterion is met, for example a further, specified time budget has been used up.

After repeating the steps S5 , S6 and S2 have ended, a selection follows ( S7 ) the network configurations (a) from the further training data set, which corresponds to a Pareto set in terms of performance and at least one further optimization goal.

In 4th is, for example, the course of a Pareto front ( 40 ) a set of network configurations ( 41 ) with regard to a classification accuracy (negative accurancy) and the resource costs as a further optimization goal. In 4th the further optimization goal is the number of parameters n_pars of the network. Represented by 4th becomes a final state in which the procedure is after 3 has used up the time budget and has almost converged, ie the Pareto front no longer shifts.

The sequence of steps executed several times after the last iteration cycle ( S5 , S6 and S2) determined network configurations of the current network configuration set now represent a basis for selecting a suitable network configuration for the application determined by the training data. This can be done, for example, by specifying a total cost function that takes into account the prediction error and the resource costs. In practice, one would decide on the basis of the application in question which network configuration of the current network configuration set (current Pareto front) is best suited for the selected application. This can be done using a restriction specification. As an example scenario, a network configuration can be selected from the Pareto front that does not exceed a network size of, for example, 1GB memory.

The above method enables the architecture search for network configurations to be accelerated in an improved manner, since the evaluation of the performance wedge / prediction errors of the variants of network configurations can be carried out considerably more quickly.

The network configurations determined in this way can be used to select a suitable configuration of a neural network for a given task. The optimization of the network configuration is closely related to the task at hand. The task results from the specification of training data, so that the training data must first be defined before the actual training, from which the optimized / suitable network configuration for the task at hand is determined. For example, image recognition or classification methods can be defined by training data that contain input images, object assignments, and object classifications. In this way, network configurations can basically be determined for all tasks defined by training data.

Thus, a neural network configured in this way can be used in a control device of a technical system, in particular in a robot, a vehicle, a tool or a machine tool, in order to determine output variables as a function of input variables. In particular, sensor data or variables determined as a function of sensor data come into question as input variables of the neural network. The sensor data can originate from sensors of the technical system or can be received from the technical system externally. Depending on the output variables of the neural network, at least one actuator of the technical system is controlled with a control signal by a computing unit of the control device of the technical system. For example, a movement of a robot or vehicle can be controlled or a drive unit or a driver assistance system of a vehicle can be controlled.

5 shows schematically a training device 500 comprising a provider 51 that provides input images from a training data set. Input images are the neural networks to be trained 52 supplied, which determine output variables from this. Output variables and input images become an assessor 53 supplied, which determines updated hyper- / parameters from this, which are transmitted to the parameter memory P and replace the current parameters there.

The one from the training device 500 The methods carried out can be implemented as a computer program on a machine-readable storage medium 54 be stored and by a processor 55 are executed.

The term “computer” encompasses any device for processing specifiable arithmetic rules. These calculation rules can be in the form of software, or in the form of hardware, or also in a mixed form of software and hardware.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• EP 19719232 [0004]

Claims (8)

Device for determining a Pareto set of network configurations, the network configurations for a neural network being determined for a given application and the given application being determined in the form of provided training data, the device comprising a machine-readable storage medium on which commands are stored that were used in Running from a computer caused the computer to perform a procedure that included: a) providing (S1) a plurality of randomly selected network configurations (α); b) determining hyperparameters (A) for each of the provided network configurations (α) by means of an optimizer; c) training of neural networks configured with the provided network configurations (α) and depending on the respectively associated determined hyperparameters (A) on the provided training data; d) determining a performance (accurancy) on at least part of the training data provided for each of the trained neural networks; e) evaluating the trained neural networks with regard to predetermined optimization goals (n_pars); f) adding a tuple comprising the network configurations (a), the determined hyperparameters (A) and the determined performance (accurancy) and the evaluated optimization goals (n_pars) to a further training data set; g) adapting the optimizer for the further training data set, in particular as a function of network configurations (a) and the determined performance (accurancy) for predicting suitable hyperparameters (A); h) Repeated execution of the following steps: i. Selecting (S3) network configurations (α) from the further training data set; ii. randomly mutating (S4) the selected network configurations; and iii. Carrying out steps b) to g) with the mutated network configurations; i) Repeatedly performing the following steps: i. Selecting (S5) network configurations from the further training data set; ii. Compressing (Pruning) (S6) the selected network configurations; iii. Carrying out steps b) to g) with the compressed network configurations; j) Selecting (S7) the network configurations (α) from the further training data set which correspond to a Pareto set in terms of performance and at least one of the further optimization goals. Device according to Claim 1 The stored commands are designed such that the method that the computer executes when these commands are executed on the computer runs in such a way that the network configurations are mutated by randomly adding layers and / or combining at least two randomly drawn network configurations he follows. Device according to Claim 1 or 2 The stored commands are designed such that the method that the computer executes when these commands are executed on the computer runs in such a way that when compressing (S6) neurons are sorted according to a sum of their output weights, and a predetermined set of neurons having the smallest sums removed. Device according to one of the Claims 1 to 3 The stored commands are designed in such a way that the method that the computer executes when these commands are executed on the computer runs in such a way that after mutating (S4) the neural network is configured with the mutated network configurations as a function of the output variables achieved of the neural networks configured with the network configurations that were used for the mutation can be pre-trained. Device according to one of the Claims 1 to 4th , wherein the stored commands are designed such that the method that the computer executes when these commands are executed on the computer runs in such a way that the selection (S3) of the network configurations to be mutated and / or the selection (S5) of the compressing network configurations depending on a non-dominant sorting (English non-dominated sorting) and an epsilon-greedy method (English epsilon greedy) takes place. Device according to Claim 5 , wherein the stored commands are designed in such a way that the method which the computer executes when these commands are executed on the computer runs in such a way that a probability of selection is higher for network configurations which are closer to the Pareto front and where Network configurations that are not on the Pareto front can be selected with the same probability. Device according to one of the Claims 1 to 6th , the stored commands being designed in such a way that the method that the computer executes when these commands are executed on the computer runs in such a way that the hyperparameters (A) by means of a constrained Bayesian optimization be determined. Device according to one of the Claims 1 to 7th , wherein the stored commands are designed in such a way that the method that the computer executes when these commands are executed on the computer takes place in such a way that at least one of the optimization goals characterizes a physical property of a hardware component, in particular a neural network configured with one of the network configurations being executed on the hardware component.

Images