EP1093639A2 - Reseau neuronal, et procede et dispositif pour l'entrainement d'un reseau neuronal - Google Patents

Reseau neuronal, et procede et dispositif pour l'entrainement d'un reseau neuronal

Info

Publication number
EP1093639A2
EP1093639A2 EP99942765A EP99942765A EP1093639A2 EP 1093639 A2 EP1093639 A2 EP 1093639A2 EP 99942765 A EP99942765 A EP 99942765A EP 99942765 A EP99942765 A EP 99942765A EP 1093639 A2 EP1093639 A2 EP 1093639A2
Authority
EP
European Patent Office
Prior art keywords
neural network
training
data set
training data
phase
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP99942765A
Other languages
German (de)
English (en)
Inventor
Volkmar Sterzing
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Siemens AG
Original Assignee
Siemens AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens AG filed Critical Siemens AG
Publication of EP1093639A2 publication Critical patent/EP1093639A2/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Definitions

  • the invention relates to training a neural network.
  • a technical system is a technical system, for example a chemical reactor or a wastewater treatment plant, a process to be modeled, generally any technical system, which is carried out using measured physical values or also recorded, i.e. recorded values, for example in image processing, can be modeled.
  • Values describing the technical system are measured as the basis for the training process.
  • the physical measured values are digitized and subjected to preprocessing so that they can be processed with a computer.
  • a neural network is trained on the basis of the measured training data.
  • a training date has at least one input variable and at least one output variable assigned to the input variable, the target value.
  • a gradient descent method for example the back propagation method, can be used as the training method.
  • Global optimization methods such as the BFGS method or genetic algorithms are used.
  • a training data record is further understood to mean a quantity with any number of training data.
  • the neural network is trained with the values determined in the second measurement phase, which form a second training data set, in a second training phase.
  • the values of the first training phase which are no longer measured in the second measuring phase, are no longer taken into account in the second training phase.
  • [4] describes a neural network with a large number of neural subnets connected in parallel, which are trained with a common training data set, the individual subnets being successively trained in succession.
  • [5] discloses two neural networks which are connected to one another in such a way that output values of a first neural network are input values of a second neural network. The same input data are used for both neural networks.
  • [6] describes a learning method for a neural network in which the neural network is trained in such a way that it forms an image of a current process. Furthermore, an background network is provided which is trained during operation with representative process data in such a way that it forms an image of the process over a longer period of time.
  • the invention is therefore based on the problem of specifying a neural network, as well as a method and an arrangement for training a neural network, in which my In a second training phase, training data from a first training phase that is no longer required can be taken into account as part of the training.
  • the first neural network is trained in a first training phase with a first training data record.
  • the first neural network and the second neural network are trained in a second training phase with a second training data set, network parameters of the first neural network remaining unchanged in the second training phase.
  • the arrangement for training a neural network which comprises a first neural network and a second neural network, has a processor which is set up in such a way that the following steps can be carried out:
  • the first neural network is trained in a first training phase with a first training data record
  • the first neural network and the second neural network are trained in a second training phase with a second training data record
  • the neural network has a first neural network and a second neural network
  • the first neural network can be trained in a first training phase with a first training data record
  • the second neural network can be trained in a second training phase with a second training data record -
  • the network parameters of the first neural network m of the second training phase are unchangeable.
  • the generalization performance of the trained neural network is considerably increased by the invention.
  • the stability of the model formed by the neural network for the technical system is increased in that even incomplete training data, which do not contain the variables to be modeled, can be used for the modeling.
  • training data the input variables of which are sometimes not measured and / or used in a second training phase, for modeling the technical system, i.e. for training the neural network, which is used as a model for the technical system.
  • At least one input variable of the first neural network is used as the input variable of the second neural network. This further training further improves the quality of the trained neural network with regard to the technical system to be modeled.
  • the invention can advantageously be used in any neural network for modeling a technical system, in particular in a system in which it is not yet known at the beginning of the training which measured values actually have a significant influence on the technical system.
  • FIG. 1 shows a sketch of a neural network, which represents an exemplary embodiment of the invention
  • Figures 2a to 2c a sketch of a clearing system with a measuring arrangement ( Figure 2a), a neural network, which with the training data of a first
  • Measurement phase is trained (Figure 2b) and the trained first neural network after the first training phase (Figure 2c); 3 shows the clearing system with a measuring arrangement, m a second measuring phase a second
  • Training data record is determined.
  • FIG. 2a shows a clearing system 201.
  • values 203 of the clearing system 201 are measured by at least one measuring device 204 in a first training phase.
  • the following system parameters 202 are provided in a first training phase: time of measurement,
  • the first measurement phase is characterized in that a significantly larger number of different values 203 is measured compared to a second measurement phase described below, but often only over a relatively short period of time.
  • the system parameters and the measured values 203 for the respective system parameters 202 are each stored as tuples and all determined tuples m in the first measurement phase form a first training data set 205.
  • a first neural network 210 (see FIG. 2b) is trained with the first training data set.
  • the first neural network 210 has an input layer 211 with input neurons 214, a hidden layer 212 with hidden neurons 215 and an output layer with output neurons 216.
  • the first neural network 210 is trained with the first training data set 205.
  • network parameters weights of couplings between neurons and couplings between neurons
  • the system parameters 202 are applied to the input layer 211 and output variables of the first neural network 210 are determined.
  • Output variables are compared with the respective values 203 assigned to the system parameters 202 in such a way that a training error is determined.
  • the first neural network 210 is trained with the traming error.
  • a trained first neural network 220 is shown in FIG. 2c.
  • the trained first neural network 220 in turn has an input layer 221 with output neurons 224, a hidden layer 222 with hidden neurons 225 and an output layer 223 with output neurons 226.
  • the network parameters of the trained first neural network 220 are different from the network parameters of the untrained first neural network 210.
  • the first training phase takes place off-lme, i.e. During the first training phase, no new training data for the first training data set are determined within the clearing system 201.
  • the clarification system is designated 301 in FIG.
  • a second training phase only some of the values 203 of the first training phase are measured by at least one measuring device 304 with different system parameters 302.
  • the second training phase is characterized by the fact that a smaller number of values 203 are measured, in this case only the chemical total chemical demand COD and the concentration of the ammonium CINH «I.
  • the second measurement phase is carried out over a longer period than the first measurement phase.
  • the second training phase can be characterized by being online, i.e. During the second training phase, values 303 m of the second measuring phase can still be measured on the clearing system 301 and fed to a second training data set 305 as training data.
  • F g.1 shows a neural network 101 which comprises the first neural network 102 which was trained in the first training phase and a second neural network 103.
  • the outputs of the first neural network 102 are connected to inputs of the second neural network 103. Further inputs of the second neural network 103 are provided for recording the system parameters which were used in the second measurement phase to measure the values 303 m in the second measurement phase.
  • the system parameters 302 of the second measurement phase are applied as input variables both to the trained first neural network 102 and to the second neural network 103.
  • output variables are formed for the input variables that are applied, which are applied to further inputs of the second neural network 103.
  • these input variables are processed further and fed to an output layer 105 of the second neural network 105.
  • the output layer 105 of the neural network likewise forms the output layer of the neural network 101.
  • At least one output variable is formed in the output layer 105.
  • the output variable is compared with the measured values 303 of the second measurement phase, the deviation of the output variable being determined with the respective measured value 303 and the neural network 101 being trained with the error determined therefrom such that the following network parameters of the first neural network 102 are not to be changed.
  • the couplings and the weights of the couplings between the input layer 221 and the hidden layer 222 and between the hidden layer 222 and the output layer 223 are not changed.
  • the neural network 101 is thus clearly trained with the second trunk data set 305 using a learning method, with the network parameters of the first neural network 102 not being changed.
  • an output variable y zo can be modeled better if both the on-lme measured variables and the offline measured variables are input variables of a neural network are available, i.e. the initial size is formed according to the following regulation:
  • f (.) denotes a non-linear mapping
  • NNl (.) denotes a non-linear mapping, which is realized by the first neural network.
  • NN2 (.) denotes a non-linear mapping, which is realized by the neural network.
  • trunk data whose measured values 203 m are used in a first training phase but are no longer measured and no longer used in the second measuring phase, for modeling the technical system as a whole.
  • This additional information gain leads to an improved modeling of the technical system by a neural network.
  • the invention enables effective use of the trunk data of the first training data set 205 both as the target variable and as the input variable of the neural network 210, without this resulting in restrictions on the use of the neural network 101.
  • the invention is not restricted to any specific structure of a neural network.
  • Both the number of neurons and the number of layers in the neural network can be specified as desired.
  • the invention can be used in a wide variety of fields, for example in the context of:
  • Financial data modeling for example, to take into account major changes that occur in a financial market, e.g. with the introduction of the euro.
  • the trained first neural network 220 should not be overtrained, ie no overfitting g should occur. In this case, it is better to accept a somewhat lower approximation quality of the trained first neural network 220 in order to achieve a stable, trained first neural network 220.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Feedback Control In General (AREA)
  • Testing Or Calibration Of Command Recording Devices (AREA)
  • Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)

Abstract

Selon l'invention, dans une première phase d'entraînement, un premier réseau neuronal (102) est entraîné à l'aide d'un premier ensemble de données d'entraînement. Le premier réseau neuronal et un second réseau neuronal (103) sont, dans une seconde phase d'entraînement, entraînés avec un second ensemble de données d'entraînement, des paramètres du premier réseau neuronal restant inchangés dans la seconde phase d'entraînement.
EP99942765A 1998-07-08 1999-07-01 Reseau neuronal, et procede et dispositif pour l'entrainement d'un reseau neuronal Withdrawn EP1093639A2 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE19830539 1998-07-08
DE19830539 1998-07-08
PCT/DE1999/001952 WO2000003355A2 (fr) 1998-07-08 1999-07-01 Reseau neuronal, et procede et dispositif pour l'entrainement d'un reseau neuronal

Publications (1)

Publication Number Publication Date
EP1093639A2 true EP1093639A2 (fr) 2001-04-25

Family

ID=7873363

Family Applications (1)

Application Number Title Priority Date Filing Date
EP99942765A Withdrawn EP1093639A2 (fr) 1998-07-08 1999-07-01 Reseau neuronal, et procede et dispositif pour l'entrainement d'un reseau neuronal

Country Status (3)

Country Link
EP (1) EP1093639A2 (fr)
JP (1) JP2002520719A (fr)
WO (1) WO2000003355A2 (fr)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB9920950D0 (en) 1999-09-06 1999-11-10 Ici Ltd Apparatus and method for reducing residual solvent levels
CN102663495B (zh) * 2012-02-22 2014-12-10 天津大学 一种用于非线性器件建模的神经网络数据生成方法
JP6164639B2 (ja) 2013-05-23 2017-07-19 国立研究開発法人情報通信研究機構 ディープ・ニューラルネットワークの学習方法、及びコンピュータプログラム
DE102015226656B4 (de) * 2015-12-23 2019-10-10 Siemens Aktiengesellschaft Verfahren und Softsensor zum Ermitteln einer Leistung eines Energieerzeugers
DE102018206108A1 (de) * 2018-04-20 2019-10-24 Zf Friedrichshafen Ag Generieren von Validierungsdaten mit generativen kontradiktorischen Netzwerken
DE102019204136A1 (de) * 2019-03-26 2020-10-01 Robert Bosch Gmbh Verfahren und Vorrichtung für Training und Herstellung eines künstlichen neuronalen Netzes
EP3716238B1 (fr) * 2019-03-27 2023-03-15 Siemens Aktiengesellschaft Procédé de détermination d'une stratégie d'évacuation pour l'évacuation d'un bâtiment

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0727748A1 (fr) * 1995-02-17 1996-08-21 BODAMER, Edgar Méthode et dispositif pour apprentissage non-supervisé multicouche en utilisant une hiérarchie de réseaux neuronaux

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO0003355A2 *

Also Published As

Publication number Publication date
WO2000003355A3 (fr) 2000-04-20
WO2000003355A2 (fr) 2000-01-20
JP2002520719A (ja) 2002-07-09

Similar Documents

Publication Publication Date Title
DE102012009502A1 (de) Verfahren zum Trainieren eines künstlichen neuronalen Netzes
DE19531967A1 (de) Verfahren zum Training eines neuronalen Netzes mit dem nicht deterministischen Verhalten eines technischen Systems
EP1145192A2 (fr) Configuration d'elements informatiques interconnectes, procede de determination assistee par ordinateur d'une dynamique a la base d'un processus dynamique et procede d'apprentissage assiste par ordinateur d'une configuration d'elements informatiques interconnectes
WO2000008599A2 (fr) Ensemble d'elements de calcul relies les uns aux autres, procede de determination assistee par ordinateur d'une dynamique a la base d'un processus dynamique, et procede d'entrainement assiste par ordinateur pour un ensemble d'elements de calcul relies les uns aux autres
EP0925541B1 (fr) Dispositif et procede de production assistee par ordinateur d'au moins un vecteur de donnees d'entrainement artificiel pour un reseau neuronal
WO2000003355A2 (fr) Reseau neuronal, et procede et dispositif pour l'entrainement d'un reseau neuronal
WO1997044743A1 (fr) Procede d'optimisation d'un ensemble de regles floues au moyen d'un ordinateur
EP1327959B1 (fr) Réseau neuronal pour modéliser un système physique et procédé de construction de ce réseau neuronal
EP0890153B1 (fr) Procede de determination de poids aptes a etre elimines, d'un reseau neuronal, au moyen d'un ordinateur
DE112020005613T5 (de) Neuromorphe Einheit mit Kreuzschienen-Array-Struktur
DE10139682A1 (de) Verfahren zum Generieren von neuronalen Netzen
EP0978052B1 (fr) Selection assistee par ordinateur de donnees d'entrainement pour reseau neuronal
DE19703964C1 (de) Verfahren zur Transformation einer zur Nachbildung eines technischen Prozesses dienenden Fuzzy-Logik in ein neuronales Netz
WO1998001812A1 (fr) Procede de determination assistee par ordinateur d'une fonction de convergence d'un systeme
DE102020210795A1 (de) Künstliches neuronales Netz
EP1194890B1 (fr) Dispositif, procede, produit comportant un programme d'ordinateur et support de stockage lisible par ordinateur pour la compensation assistee par ordinateur d'un etat de desequilibre d'un systeme technique
EP1114398B1 (fr) Procede pour entrainer un reseau neuronal, procede de classification d'une sequence de grandeurs d'entree au moyen d'un reseau neuronal, reseau neuronal et dispositif pour l'entrainement d'un reseau neuronal
EP1190383B1 (fr) Procede de determination assistee par ordinateur de l'appartenance d'une grandeur d'entree donnee a un groupe
EP3710992A1 (fr) Réseau neuronal artificiel et procédé associé
DE102004059684B3 (de) Verfahren und Anordnung sowie Computerprogramm mit Programmmcode-Mitteln und Computerprogramm-Produkt zur Ermittlung eines zukünftigen Systemzustandes eines dynamischen Systems
WO2022152683A1 (fr) Détermination d'une valeur de confiance d'un réseau de neurones artificiels
WO2020193481A1 (fr) Procédé et dispositif d'apprentissage et de réalisation d'un réseau neuronal artificiel
WO2020169416A1 (fr) Module d'apprentissage quantifié pouvant être entraîné
WO2003079285A2 (fr) Procede et systeme ainsi que programme informatique dote de moyens de code de programme et produit de programme informatique servant a ponderer des grandeurs d'entree pour une structure neuronale, ainsi que structure neuronale associee
WO2005055133A2 (fr) Procede et dispositif et programme d'ordinateur comportant des moyens a code de programme, et produit programme d'ordinateur pour la determination d'un etat futur d'un systeme dynamique

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20001206

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20030113