WO2003046886A1 - Procede de classement d'une serie temporelle d'evenements au moyen d'un reseau contenant des neurones pulses - Google Patents

Procede de classement d'une serie temporelle d'evenements au moyen d'un reseau contenant des neurones pulses Download PDF

Info

Publication number
WO2003046886A1
WO2003046886A1 PCT/DE2002/004225 DE0204225W WO03046886A1 WO 2003046886 A1 WO2003046886 A1 WO 2003046886A1 DE 0204225 W DE0204225 W DE 0204225W WO 03046886 A1 WO03046886 A1 WO 03046886A1
Authority
WO
WIPO (PCT)
Prior art keywords
neurons
neural network
neuron
input variables
value
Prior art date
Application number
PCT/DE2002/004225
Other languages
German (de)
English (en)
Inventor
Gustavo Deco
Jan Storck
Original Assignee
Siemens Aktiengesellschaft
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Aktiengesellschaft filed Critical Siemens Aktiengesellschaft
Publication of WO2003046886A1 publication Critical patent/WO2003046886A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
    • G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks

Definitions

  • the invention relates to a method for classifying a temporal sequence of input variables using a neural network containing pulsed neurons, a neural network which contains pulsed neurons in the form of oscillating neurons, and an arrangement for classifying a temporal sequence of input variables using a pulsed neuron containing neural network.
  • a neural network has neurons that are at least partially linked to one another. Input neurons of the neural network are supplied with input signals as input variables.
  • the neural network usually has several layers. Depending on a neuron supplied to the neural network and an activation function provided for the neuron, a neuron generates a signal, which in turn is fed to neurons of a further layer as an input variable according to a predeterminable weighting. In an output layer, an output variable is generated in an output neuron as a function of quantities that are supplied to the output neuron by neurons of the previous layer.
  • the neural network encodes information by action potentials or "spikes” that characterize neural firing events (Rieke, F., Warland, D., de Ruyter van Steveninck, R. and Bialek,. (1997) , Spikes: Exploring the neural code. Cambridge: The MIT Press).
  • spatiotemporal firing patterns therefore encode information regarding sensory stimuli.
  • different classes of Stimuli can be distinguished by different types of spatio-temporal firing patterns.
  • the maximization of the transinformation as a means of describing the differentiability to achieve this goal has recently been proposed (Deco, G. and Schürmann, B.
  • the invention is also based on the object of specifying a corresponding neural network.
  • the invention enables the classification of temporal sequences in question with less computational effort by using oscillating neurons that allow the phase to be the only relevant parameter for the temporal pulse profile.
  • the calculation of the mutual information based on a cost function is compared to that dealt with above State of the art in terms of accuracy and especially in terms of time significantly improved because the density determination required for entropy calculation takes place in a room of much smaller dimensions. It is crucial that the need for classification in a minimum of time is already implicitly ensured by the fact that the cost function depends solely on the phases, which in turn can be calculated immediately after a single oscillation.
  • the cost function is preferably determined on the basis of a differentiation value I (T) which satisfies the following rule:
  • T is the observation time of an output pulse
  • s is the random variable that corresponds to the stimulus class
  • tj denotes the time corresponding to the i-th firing of a neuron a, tci (cl> , • • •, t kc (cK) the maximum values are less than T, and ci, ..., c ⁇ are certain code neurons.
  • I (T) / (r, ⁇ ⁇ >, ⁇ W, ..., ⁇ ⁇ *> ⁇ )
  • ⁇ i (a> denotes the time corresponding to the ith firing of a neuron a with respect to a specific reference value, ie, the phases.
  • the input variables are preferably measured physical signals.
  • the method according to the invention and the arrangement according to the invention can thus be used in the context of the description of a technical system, in particular for the description, for example for examining a multichannel signal, has been recorded by an electroencephalograph. and describes an electroencephalogram.
  • the method and the system according to the invention can also be used for analyzing multivariate financial data in the field of the financial market and for analyzing economic relationships.
  • the method and the arrangement according to the invention are also suitable for the implementation of software for a processor as well as for hardware.
  • a preferred field of application of the method and the system according to the invention is in the field of speech analysis.
  • Fig. 2 a second class of stimuli
  • Fig. 4 phase distributions of code neurons after learning, in response to a stimulus 1
  • Fig. 5 phase distributions of code neurons before learning, in response to a stimulus 2
  • the constant ⁇ describes the decay of the membrane potential when there are no input signals.
  • the synaptic strength is denoted by w.
  • the constant ⁇ is the internal drive (drive) that leads to a periodic pulse train when there are no pulse signals: charge is collected until the membrane potential V (t) reaches a predetermined threshold ⁇ , which leads to a pulse generation (discharge). After the pulse has been generated, the model is reset to a predetermined initial potential V (0) (in the present case this potential is set to zero), and the charging process starts again.
  • Each neuron i is described by a membrane potential V ⁇ which follows an equation of the type of equation (1).
  • the neural network containing N neurons is described by the following system of differential equations:
  • w ⁇ j denotes the synaptic strength between a neuron i and a neuron, the direction running from j to i, Ii (t) denotes the external stimulus, which acts as an additional input variable with constant weight, and ⁇ ⁇ denotes the internal drive of the neuron.
  • each neuron has an absolute refractory period after the emission of a pulse during which it cannot fire again.
  • I ⁇ (t) is fed into the input neurons with a strength of 0.2.
  • the architecture used here is a fully linked network similar to that described in Storck, J. and Deco, G. (1998), Spike-Based Hebbian Learning for Stimulus Discrimination, In Artificial Neural Networks - ICANN '98, Skövde, Sweden, Springer-Verlag, Heidelberg, suggests: Each neuron sends its action potentials and receives input from all other neurons via synaptic efficiencies with adaptive strength. The axonal transmission delay was chosen randomly in the range between 0 and 2 ms.
  • a cost function is introduced for global optimization.
  • the pure parameters that is, the synaptic efficiencies zen are designed so that the stimulus presented at the entrance can be classified as reliably as possible. For this reason, the mutual information between the stimulus class and the pulse response of the network is introduced as a measure of the differentiability.
  • the aim here is not, as has been the case up to now, to reconstruct the input or the input variables from the output or the output variables, but to derive the name of the class to which the presented stimulus belongs, namely from a set of given classes.
  • the random variable that corresponds to the class of the stimulus is denoted by s, ie the results of s are s (j) with the probability pj.
  • a measure of the distinguishability for an observation time T of the output pulses can be defined by the mutual information between the random variables s and the pyramidal pulse times of certain code neurons Ci, ...., c ⁇ , ie by the following differentiation value:
  • ti ⁇ a denotes the time it takes for the i th firing of a t ( ⁇ t (ct)
  • Neurons corresponds to a, and where ⁇ * denote the maximum values that are smaller than T.
  • ⁇ ⁇ (a> denotes the time corresponding to the ith firing of a neuron a with respect to a specific reference value, that is, the phases (since the firing time of one of the code neurons can be used as a self-reference, so that the number of relative phases is actually reduced to K - 1).
  • the external stimulus (the input or the input variable) consists of three non-homogeneous Poisson processes which are fed into the network simultaneously and represent a three-dimensional input current.
  • 1 shows such a three-dimensional input current for a first stimulus class
  • FIG. 2 for a second stimulus class.
  • Both classes consist of three dimensions, each with a non-homogeneous Poisson process for each dimension.
  • the sine wave rates follow (in FIGS. 1 and 2 the upper curve profiles in each case), while the third component (in FIGS.
  • the functions that describe the change in the different Poisson rates as a function of time in the three inputs define two different input classes.
  • One class is sampled on the basis of the three rate curves (for input variable 1, 2 or 3), as shown in FIG. 1, while the rates for class 2 are shown in FIG. 2.
  • These two stimuli differ only in the coherence (the temporal relationship) of their various components, which makes the classification task a kind of temporal clustering.
  • the neural network for this experiment consists of fully linked neurons. Due to the three-dimensional nature of the input signal, there are consequently three input neurons that receive the time-dependent stimulus. Such an input neuron receives no input or input zero if the Poisson process does not generate a pulse to which it belongs and a stimulus 0.2 in the case of an input pulse event. The remaining or hidden neurons receive no external input at all. In order to keep the computing effort low, the cost function is determined from a subset of the total number of neurons. These code neurons, which are taken into account when calculating the mutual information, are chosen arbitrarily, but to the exclusion of the input neurons.
  • the statistics required to calculate the mutual information probabilities are obtained by the pulse patterns (ie, the phase patterns) in response to 500 input samples for each of the two different stimuli.
  • a sample (name) in turn consists of a string of input pulses with which the network is then driven until the phases are measured and the procedure can proceed to the next sampling.
  • I (T) is calculated and the weight update of the current iteration can be started. Before the whole process is restarted for.
  • an optimization method can be used, such as the ALOPEX algorithm described in Storck, J. and Deco, G. (1998), Spike-Based Hebbian Learning for Stimulus Discrimination. Artificial Neural Networks - ICANN '98, Skövde, Sweden, Springer-Verlag, Heidelberg.
  • the complete information which determines the value of the cost function, and thus the degree of distinctness, is contained in the common distribution of the phases, the size of which is equal to the number of code neurons.
  • a visualization of the common distribution of the phases of the code neurons is therefore only possible for up to two of them.
  • the present experiment uses three code neurons, which leads to two relative phases, for which the result is shown in FIGS. 3 to 6. However, even for a larger number of code neurons in even more complex applications, the probability structure be derived from the two-dimensional distributions of subsets.
  • phase distributions of the code neurons before and after the learning show in detail the phase distributions of the code neurons before and after the learning.
  • the phase distributions in response to stimulus 1 (FIG. 3) and stimulus 2 (FIG. 5) show an unstructured form due to the random initialization of the synaptic efficiencies. Their similarity leads to a low value of the cost function, i.e. the mutual information.

Abstract

L'invention concerne un procédé de classement d'une série temporelle de grandeurs d'entrée à l'aide d'un réseau neuronal contenant des neurones pulsés. Selon ledit procédé, on utilise comme neurones pulsés des neurones oscillants, et, sur la base d'une valeur de décision, l'information réciproque entre la classe de stimuli des grandeurs d'entrée et le comportement en pulsation (= fonction de coût) des neurones est déterminée.
PCT/DE2002/004225 2001-11-22 2002-11-14 Procede de classement d'une serie temporelle d'evenements au moyen d'un reseau contenant des neurones pulses WO2003046886A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE10157220.4 2001-11-22
DE10157220A DE10157220A1 (de) 2001-11-22 2001-11-22 Verfahren zum Klassifizieren einer zeitlichen Folge von Eingangsgrößen unter Verwendung eines gepulste Neuronen enthaltenden neuronalen Netzes, neuronales Netz und Anordnung zum Durchführen des Verfahrens

Publications (1)

Publication Number Publication Date
WO2003046886A1 true WO2003046886A1 (fr) 2003-06-05

Family

ID=7706524

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/DE2002/004225 WO2003046886A1 (fr) 2001-11-22 2002-11-14 Procede de classement d'une serie temporelle d'evenements au moyen d'un reseau contenant des neurones pulses

Country Status (2)

Country Link
DE (1) DE10157220A1 (fr)
WO (1) WO2003046886A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5771306A (en) * 1992-05-26 1998-06-23 Ricoh Corporation Method and apparatus for extracting speech related facial features for use in speech recognition systems
US6021387A (en) * 1994-10-21 2000-02-01 Sensory Circuits, Inc. Speech recognition apparatus for consumer electronic applications
DE19949637A1 (de) * 1999-10-14 2001-04-19 Dietrich Kuehner Verfahren und Vorrichtungen zur Geräuscherkennung und -trennung sowie Lärmüberwachung und -prognose

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5771306A (en) * 1992-05-26 1998-06-23 Ricoh Corporation Method and apparatus for extracting speech related facial features for use in speech recognition systems
US6021387A (en) * 1994-10-21 2000-02-01 Sensory Circuits, Inc. Speech recognition apparatus for consumer electronic applications
DE19949637A1 (de) * 1999-10-14 2001-04-19 Dietrich Kuehner Verfahren und Vorrichtungen zur Geräuscherkennung und -trennung sowie Lärmüberwachung und -prognose

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ANDREAS V.M. HERZ: "How is time represented in the brain", OXFORD UNIVERITY PRESS, 2001, Oxford, pages 1 - 17, XP002233540, Retrieved from the Internet <URL:http://www.itb.biologie.hu-berlin.de/~herz/dresden.pdf> [retrieved on 20030305] *
B. W. VAN SCHOOTEN: "Solving computational problems using coherent oscillation", M.SC. THESIS, September 1997 (1997-09-01), pages 1 - 50, XP002233539, Retrieved from the Internet <URL:http://www.few.eur.nl/few/people/jvandenberg/master theses/boris.pdf> [retrieved on 20030305] *

Also Published As

Publication number Publication date
DE10157220A1 (de) 2003-06-12

Similar Documents

Publication Publication Date Title
EP2165296B1 (fr) Procédé et dispositif de prédiction assistée par calcul de déplacements intentionnels
DE102005046747B3 (de) Verfahren zum rechnergestützten Lernen eines neuronalen Netzes und neuronales Netz
EP1183619A2 (fr) Reseau semantique d&#39;ordre n, operant en fonction d&#39;une situation
DE60002290T2 (de) Verfahren zum Nachweis von Anomalien in einem Signal
DE102020212515A1 (de) Verfahren und Vorrichtung zum Trainieren eines maschinellen Lernsystems
WO1997044743A1 (fr) Procede d&#39;optimisation d&#39;un ensemble de regles floues au moyen d&#39;un ordinateur
DE102019205085A1 (de) Selbstüberwachung einer auf künstlicher Intelligenz basierenden Funktion
EP1519301A2 (fr) Dispositif de classification d&#39;événements physiologiques
DE19537010C2 (de) Lernverfahren und -anordnung zur Nachbildung eines dynamischen Prozesses
EP1227425B1 (fr) Système pour découvrir la source des attaques de maladies
DE102016124205A1 (de) Computer-implementiertes Verfahren zur Optimierung eines Herstellungsprozesses
WO2003046886A1 (fr) Procede de classement d&#39;une serie temporelle d&#39;evenements au moyen d&#39;un reseau contenant des neurones pulses
DE102020212514A1 (de) Verfahren zur Ermittlung eines Ausgabesignals mittels eines maschinellen Lernsystems
DE102021201833A1 (de) Vorrichtung zur Verarbeitung von mindestens einem Eingangsdatensatz unter Verwendung eines neuronalen Netzes sowie Verfahren
DE102021204040A1 (de) Verfahren, Vorrichtung und Computerprogramm zur Erstellung von Trainingsdaten im Fahrzeug
WO2021089591A1 (fr) Procédé d&#39;entraînement d&#39;un réseau de neurones artificiels, programme informatique, support d&#39;enregistrement, dispositif, réseau de neurones artificiels et utilisation du réseau de neurones artificiels
DE4225885C2 (de) Korrelationsermittlungsverfahren und Verknüpfungsstruktur-Schätzverfahren für Neuronen
EP2284576B1 (fr) Procédé et système de prévision d&#39;événements
DE102018216078A1 (de) Verfahren und Vorrichtung zum Betreiben eines Steuerungssystems
DE19549300C1 (de) Verfahren zur rechnergestützten Ermittlung einer Bewertungsvariablen eines Bayesianischen Netzwerkgraphen
EP1114398B1 (fr) Procede pour entrainer un reseau neuronal, procede de classification d&#39;une sequence de grandeurs d&#39;entree au moyen d&#39;un reseau neuronal, reseau neuronal et dispositif pour l&#39;entrainement d&#39;un reseau neuronal
AT411143B (de) Vorrichtung zum auswerten psychologischer und biomedizinischer rohdaten
DE102006056106A9 (de) System und Verfahren zur Herstellung und Anwendung von Signal-Klassifikatoren
WO2020094415A1 (fr) Procédé de mesure et dispositif de mesure servant à déterminer une propriété technique d&#39;un code de programme servant à piloter un appareil électronique
EP1519303A2 (fr) Dispositif pour la classification des événements physiologiques

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): CN JP US

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LU MC NL PT SE SK TR

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP