WO2021192137A1 - Unité de calcul de réservoir, dispositif de réservoir, procédé de conception pour unité de calcul de réservoir et procédé de commande pour dispositif de réservoir - Google Patents

Unité de calcul de réservoir, dispositif de réservoir, procédé de conception pour unité de calcul de réservoir et procédé de commande pour dispositif de réservoir Download PDF

Info

Publication number
WO2021192137A1
WO2021192137A1 PCT/JP2020/013621 JP2020013621W WO2021192137A1 WO 2021192137 A1 WO2021192137 A1 WO 2021192137A1 JP 2020013621 W JP2020013621 W JP 2020013621W WO 2021192137 A1 WO2021192137 A1 WO 2021192137A1
Authority
WO
WIPO (PCT)
Prior art keywords
reservoir
equation
calculation unit
unit
algebraic
Prior art date
Application number
PCT/JP2020/013621
Other languages
English (en)
Japanese (ja)
Inventor
一紀 中田
Original Assignee
Tdk株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tdk株式会社 filed Critical Tdk株式会社
Priority to PCT/JP2020/013621 priority Critical patent/WO2021192137A1/fr
Publication of WO2021192137A1 publication Critical patent/WO2021192137A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Definitions

  • the present invention relates to a reservoir calculation unit, a reservoir device, a method for designing a reservoir calculation unit, and a method for controlling a reservoir device.
  • a neuromorphic device is an element that imitates the human brain by means of a neural network. Neuromorphic devices artificially mimic the relationship between neurons and synapses in the human brain.
  • the neuromorphic device has, for example, hierarchically arranged chips (neurons in the brain) and transmission means (synapses in the brain) connecting them.
  • Neuromorphic devices increase the percentage of correct answers to questions by learning by means of communication (synapses). Learning is to find knowledge that can be used in the future from information, and neuromorphic devices weight the input data.
  • a recurrent neural network is known as one of the neural networks.
  • Recurrent neural networks can handle non-linear time series data.
  • Non-linear time-series data is data whose values change over time, such as stock prices and the number of influenza pandemics.
  • the recurrent neural network can process time-series data by returning the processing results of the neurons in the latter layer to the neurons of the previous layer.
  • Reservoir computing is one means of realizing a recurrent neural network. Reservoir computing performs recursive processing by interacting signals. Reservoir computing, for example, mimics the operating mechanism of the cerebellum, and performs recursive data processing, data conversion (for example, coordinate conversion), and the like.
  • Non-Patent Document 1 describes a reservoir with a one-dimensional ring topology.
  • the reservoir device consists of multiple units corresponding to the nodes in reservoir computing. Each unit performs an operation. If the calculation accuracy of each unit is increased to the precision of floating point or fixed point, the number of arithmetic units constituting the unit becomes enormous.
  • the present invention has been made in view of the above circumstances, and is a reservoir calculation unit, a reservoir device, and a reservoir calculation capable of improving calculation efficiency while maintaining the expressive power (complexity) of data output from a reservoir device.
  • the reservoir calculation unit comprises a circuit designed based on an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values.
  • the reservoir device has a plurality of reservoir calculation units according to the above aspect, and each of the reservoir calculation units is connected to at least one or more other reservoir calculation units.
  • the method for designing the reservoir calculation unit according to the third aspect is the method for designing the reservoir calculation unit used for the reservoir device, which is an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values. It has a step of converting into a circuit diagram and a step of converting the calculation formula of the algebraic equation into a circuit diagram.
  • the method for controlling the reservoir device according to the fourth aspect is as an external input to a reservoir calculation unit designed based on an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values.
  • the parameters of the algebraic equation are changed with time to supply.
  • the reservoir calculation unit, the reservoir device, the design method of the reservoir calculation unit, and the control method of the reservoir device according to the above aspects are to improve the calculation efficiency while maintaining the expressiveness (complexity) of the data output from the reservoir device. Can be done.
  • the reservoir device is a device of processing in reservoir computing.
  • Reservoir computing is an example of a recurrent neural network.
  • FIG. 1 is a conceptual diagram of a neural network simulated by the reservoir device according to the first embodiment.
  • the neural network NN shown in FIG. 1 is a conceptual schematic diagram of reservoir computing.
  • Neural networks NN shown in Figure 1 it has an input layer L in the reservoir R and the output layer L out. Input layer L in and the output layer L out is connected to the reservoir R.
  • Input layer L in conveys a signal inputted from the outside to the reservoir R.
  • Input layer L in, for example, comprises a plurality of neurons n 1.
  • Input signal input to each neuron n 1 of the input layer L in from the outside is transmitted to the reservoir R.
  • the reservoir R is storing the input signal input from the input layer L in, is converted into another signal.
  • the signals only interact and do not learn.
  • the input signals change non-linearly. That is, the input signal replaces another signal while retaining the original information.
  • the input signals change over time by interacting with each other in the reservoir R.
  • a plurality of neurons n 2 are randomly connected to the reservoir R. For example, the signal output from the neuron n 2 at a certain time t may return to the original neuron n 2 at a certain time t + 1.
  • neurons n 2 it can process in consideration of the time t and time t + 1 of the signal, can be recursively process the information.
  • the output layer L out outputs a signal from the reservoir R.
  • the output signal output from the output layer L out has the information of the input signal and is replaced with another signal.
  • An example of such conversion is the replacement of the Cartesian coordinate system (x, y, z) with the spherical coordinate system (r, ⁇ , ⁇ ).
  • Output layer L out for example, comprises a plurality of neurons n 3. Learning is performed from the reservoir R to the output layer L out. Learning is performed in each neuron n 2 and the output layer L out neuron n 3 and the connecting transmission path of the reservoir R (synapses in the brain). The output layer L out outputs the learning result to the outside. Neurons n 1 , n 2 , and n 3 are sometimes referred to as nodes.
  • FIG. 2 is an example of a circuit diagram of the reservoir device according to the present embodiment.
  • the reservoir device 100 shown in FIG. 2 has a plurality of units 10.
  • the unit 10 is an example of a reservoir calculation unit. Each unit 10 is connected to at least one or more other units 10.
  • the i-th unit 10 is referred to as a unit 10 i.
  • the output of unit 10 i shown in FIG. 2 is connected to, for example, unit 10 i-1 and unit 10 1 + 1.
  • the output signal of unit 10 i becomes an input signal of unit 10 i-1 and the unit 10 1 + 1.
  • a signal at time n is output to the unit 10 1 + 1.
  • a signal at time n + 1 is output to the unit 10 1-1.
  • the time n + 1 signal is produced, for example, by a delay circuit.
  • the reservoir device 100 shown in FIG. 2 has a flip-flop circuit 20 connected to the unit 10.
  • the flip-flop circuit 20 latches the data and holds and stores it until the next clock.
  • the flip-flop circuit 20 delays the signal.
  • the unit 10 i signals of different time is inputted, the information is processed recursively.
  • FIG. 3 is an example of a circuit of each unit 10 constituting the reservoir device according to the present embodiment.
  • the unit 10 consists of a circuit designed based on an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values.
  • the design method of the circuit constituting the unit 10 will be described.
  • the method of designing the circuit constituting the unit 10 includes a step of converting a continuous differential equation into an algebraic equation superdiscretized into two or multiple values and a step of converting an arithmetic equation of the algebraic equation into a circuit diagram. Have.
  • ⁇ , a, b are constants and ⁇ is time.
  • -ax is a component that produces vibration
  • bF (x- ⁇ ) is a non-linear function that includes time.
  • the reservoir device 100 processes the input signal non-linearly and converts it into another signal. The processing of the above differential equation is appropriate as the processing required for each unit 10 of the reservoir device 100.
  • U j n and V j n correspond to state variables.
  • the parameters ⁇ m, A, E, ⁇ , and ⁇ are all constants.
  • the continuous numerical change is approximated to the discretized discrete numerical change.
  • continuous operations are required.
  • the numerical change is approximated to the discretized discrete value, the number of arithmetic processes is reduced and the calculation load is lightened. Further, since the number of arithmetic processes is reduced, the number of arithmetic elements required for the unit 10 is reduced.
  • the circuit shown in FIG. 3 is designed after converting the above algebraic equation into a data flow graph and expressing it.
  • the optimum configuration when implementing an algebraic equation with a recursive structure as described above is not always trivial, and it is necessary to optimize the data flow graph showing the data flow and the arrangement of arithmetic units.
  • the calculation of the circuit shown in FIG. 3 is expressed by the following equation.
  • U i is a state variable
  • a and B are parameters that take discrete values.
  • I (n) may be a parameter (constant) as in Non-Patent Document 2, or may be an input signal that changes with time in order to contribute to the reservoir calculation.
  • the unit 10 shown in FIG. 3 has a plurality of multiplexers 1A, 1B, 1C, 1D, a plurality of subtractors 2, a multiplier 3, and an adder 4.
  • the two signals U 1-1 (n) and U 1 + 1 (n) input to the unit 10 are input to the multiplexer 1A.
  • the multiplexer 1A outputs the larger signal M (U i ) of the two signals U 1-1 (n) and U 1 + 1 (n).
  • the signal M (U i ) is branched into two.
  • One of the branched signals M (U i ) is obtained by the subtractor 2 for the difference ⁇ M (U i ) from 0, and is input to the next multiplexer 1B.
  • the multiplexer 1B outputs the larger signal of the signal I and the difference ⁇ M (U i).
  • the output signal is input to the multiplexer 1C.
  • the signal input to the multiplexer 1C is represented by max [I (n), ⁇ M (U i )].
  • the other branched signal M ( Ui ) is integrated by the multiplier 3 and input to the subtractor 2.
  • the subtractor 2 outputs the difference between the constant A and the signal 2M (U i).
  • the output signal is branched, one is input to the adder 4 and the other is input to the multiplexer 1D.
  • the adder 4 inputs the difference between the constant B and the signal A-2M ( Ui ) to the multiplexer 1C.
  • the multiplexer 1C outputs the larger signal of the input signals.
  • the signal output from the multiplexer 1C is represented by max [I (n), ⁇ M (U i ), A + B-2M (U i )].
  • the multiplexer 1D outputs the larger signal of the input signals.
  • the signal output from the multiplexer 1D is represented by max [0, A-2M ( Ui )].
  • the difference between the output signals from the multiplexer 1C and the multiplexer 1D is obtained by the subtractor 2 and becomes the output of the unit 10. That is, the circuit of the unit 10 is designed based on the above algebraic equation.
  • the circuits that make up the unit 10 change depending on the algebraic equation. That is, the corresponding data flow graph and circuit configuration differ depending on the original differential equation.
  • the circuit shown in FIG. 3 is only an example designed based on the algebraic equation shown in Non-Patent Document 2.
  • the reservoir device can reduce the calculation load by super-discretizing the differential equations that are calculated in each unit and converting them into equivalent algebraic equations. Further, by simplifying the arithmetic processing in each unit, the number of arithmetic elements required for each unit is reduced, and physical mounting is easy. In addition, by mathematically converting the differential equation into an algebraic equation, the representation (complexity) of the data output from the reservoir device is maintained while retaining the desirable properties (for example, integrability and boundability) of the original differential equation. The) is also maintained.
  • the reservoir device converts the differential equations calculated in each unit into equivalent algebraic equations by super-discretization, and further changes the parameters of the algebraic equations with time to input the reservoir device. Reservoir calculation is performed assuming that.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Neurology (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

Une unité de calcul de réservoir selon un mode de réalisation de la présente invention est pourvue d'un graphe de flux de données conçu sur la base d'une équation algébrique dans laquelle une équation différentielle continue a été ultra-discrétisée en deux valeurs ou de multiples valeurs.
PCT/JP2020/013621 2020-03-26 2020-03-26 Unité de calcul de réservoir, dispositif de réservoir, procédé de conception pour unité de calcul de réservoir et procédé de commande pour dispositif de réservoir WO2021192137A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/013621 WO2021192137A1 (fr) 2020-03-26 2020-03-26 Unité de calcul de réservoir, dispositif de réservoir, procédé de conception pour unité de calcul de réservoir et procédé de commande pour dispositif de réservoir

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/013621 WO2021192137A1 (fr) 2020-03-26 2020-03-26 Unité de calcul de réservoir, dispositif de réservoir, procédé de conception pour unité de calcul de réservoir et procédé de commande pour dispositif de réservoir

Publications (1)

Publication Number Publication Date
WO2021192137A1 true WO2021192137A1 (fr) 2021-09-30

Family

ID=77891256

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/013621 WO2021192137A1 (fr) 2020-03-26 2020-03-26 Unité de calcul de réservoir, dispositif de réservoir, procédé de conception pour unité de calcul de réservoir et procédé de commande pour dispositif de réservoir

Country Status (1)

Country Link
WO (1) WO2021192137A1 (fr)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10211856B1 (en) * 2017-10-12 2019-02-19 The Boeing Company Hardware scalable channelizer utilizing a neuromorphic approach

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10211856B1 (en) * 2017-10-12 2019-02-19 The Boeing Company Hardware scalable channelizer utilizing a neuromorphic approach

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SATSUMA, JUNKICHI ET AL.: "What is ultradiscrete?", MATHEMATICAL SCIENCES, vol. 37, no. 9, 1 September 1999 (1999-09-01), pages 5 - 11, ISSN: 0386-2240 *

Similar Documents

Publication Publication Date Title
Sharma et al. Activation functions in neural networks
Beidokhti et al. Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques
Meier A mixed-signal universal neuromorphic computing system
Wang et al. Double almost periodicity for high-order Hopfield neural networks with slight vibration in time variables
Darbon et al. Neural network architectures using min-plus algebra for solving certain high-dimensional optimal control problems and Hamilton–Jacobi PDEs
Dong et al. Application of fractional theory in quantum back propagation neural network
Adetiba et al. Implementation of efficient multilayer perceptron ANN neurons on field programmable gate array chip
Ranganath et al. Design of MAC unit in artificial neural network architecture using Verilog HDL
WO2021192137A1 (fr) Unité de calcul de réservoir, dispositif de réservoir, procédé de conception pour unité de calcul de réservoir et procédé de commande pour dispositif de réservoir
MacLennan A review of analog computing
Kumaresan et al. Optimal control for stochastic linear quadratic singular system using neural networks
Heidari et al. Using artificial neural networks in solving heat conduction problems
Zhu et al. NN-Poly: Approximating common neural networks with Taylor polynomials to imbue dynamical system constraints
Atencia et al. FPGA implementation of a systems identification module based upon Hopfield networks
Abrol et al. Artificial neural network implementation on FPGA chip
Wang et al. A Hammerstein-Wiener recurrent neural network with universal approximation capability
Behera et al. System identification using recurrent neural network
Islam et al. FPGA Implementation of Nerve Cell Using Izhikevich Neuronal Model As Spike Generator (SG)
Yamada et al. Dynamical linking of positive and negative sentences to goal-oriented robot behavior by hierarchical rnn
Mohamed et al. Brain-inspired machine learning algorithm: Neural network optimization
WO2023175722A1 (fr) Programme d'apprentissage et apprenant
US20230023123A1 (en) Information processing system, information processing device, information processing method, and information processing program
Ellery Bootstrapping Neural Electronics from Lunar Resources for In-Situ Artificial Intelligence Applications
Cheng et al. A Bidirectional Deep Learning Approach for Designing MEMS Sensors
JP2024034865A (ja) 機械学習システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20927504

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20927504

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP