WO2021192137A1 - Reservoir calculation unit, reservoir device, design method for reservoir calculation unit, and control method for reservoir device - Google Patents

Reservoir calculation unit, reservoir device, design method for reservoir calculation unit, and control method for reservoir device Download PDF

Info

Publication number
WO2021192137A1
WO2021192137A1 PCT/JP2020/013621 JP2020013621W WO2021192137A1 WO 2021192137 A1 WO2021192137 A1 WO 2021192137A1 JP 2020013621 W JP2020013621 W JP 2020013621W WO 2021192137 A1 WO2021192137 A1 WO 2021192137A1
Authority
WO
WIPO (PCT)
Prior art keywords
reservoir
equation
calculation unit
unit
algebraic
Prior art date
Application number
PCT/JP2020/013621
Other languages
French (fr)
Japanese (ja)
Inventor
一紀 中田
Original Assignee
Tdk株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tdk株式会社 filed Critical Tdk株式会社
Priority to PCT/JP2020/013621 priority Critical patent/WO2021192137A1/en
Publication of WO2021192137A1 publication Critical patent/WO2021192137A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Definitions

  • the present invention relates to a reservoir calculation unit, a reservoir device, a method for designing a reservoir calculation unit, and a method for controlling a reservoir device.
  • a neuromorphic device is an element that imitates the human brain by means of a neural network. Neuromorphic devices artificially mimic the relationship between neurons and synapses in the human brain.
  • the neuromorphic device has, for example, hierarchically arranged chips (neurons in the brain) and transmission means (synapses in the brain) connecting them.
  • Neuromorphic devices increase the percentage of correct answers to questions by learning by means of communication (synapses). Learning is to find knowledge that can be used in the future from information, and neuromorphic devices weight the input data.
  • a recurrent neural network is known as one of the neural networks.
  • Recurrent neural networks can handle non-linear time series data.
  • Non-linear time-series data is data whose values change over time, such as stock prices and the number of influenza pandemics.
  • the recurrent neural network can process time-series data by returning the processing results of the neurons in the latter layer to the neurons of the previous layer.
  • Reservoir computing is one means of realizing a recurrent neural network. Reservoir computing performs recursive processing by interacting signals. Reservoir computing, for example, mimics the operating mechanism of the cerebellum, and performs recursive data processing, data conversion (for example, coordinate conversion), and the like.
  • Non-Patent Document 1 describes a reservoir with a one-dimensional ring topology.
  • the reservoir device consists of multiple units corresponding to the nodes in reservoir computing. Each unit performs an operation. If the calculation accuracy of each unit is increased to the precision of floating point or fixed point, the number of arithmetic units constituting the unit becomes enormous.
  • the present invention has been made in view of the above circumstances, and is a reservoir calculation unit, a reservoir device, and a reservoir calculation capable of improving calculation efficiency while maintaining the expressive power (complexity) of data output from a reservoir device.
  • the reservoir calculation unit comprises a circuit designed based on an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values.
  • the reservoir device has a plurality of reservoir calculation units according to the above aspect, and each of the reservoir calculation units is connected to at least one or more other reservoir calculation units.
  • the method for designing the reservoir calculation unit according to the third aspect is the method for designing the reservoir calculation unit used for the reservoir device, which is an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values. It has a step of converting into a circuit diagram and a step of converting the calculation formula of the algebraic equation into a circuit diagram.
  • the method for controlling the reservoir device according to the fourth aspect is as an external input to a reservoir calculation unit designed based on an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values.
  • the parameters of the algebraic equation are changed with time to supply.
  • the reservoir calculation unit, the reservoir device, the design method of the reservoir calculation unit, and the control method of the reservoir device according to the above aspects are to improve the calculation efficiency while maintaining the expressiveness (complexity) of the data output from the reservoir device. Can be done.
  • the reservoir device is a device of processing in reservoir computing.
  • Reservoir computing is an example of a recurrent neural network.
  • FIG. 1 is a conceptual diagram of a neural network simulated by the reservoir device according to the first embodiment.
  • the neural network NN shown in FIG. 1 is a conceptual schematic diagram of reservoir computing.
  • Neural networks NN shown in Figure 1 it has an input layer L in the reservoir R and the output layer L out. Input layer L in and the output layer L out is connected to the reservoir R.
  • Input layer L in conveys a signal inputted from the outside to the reservoir R.
  • Input layer L in, for example, comprises a plurality of neurons n 1.
  • Input signal input to each neuron n 1 of the input layer L in from the outside is transmitted to the reservoir R.
  • the reservoir R is storing the input signal input from the input layer L in, is converted into another signal.
  • the signals only interact and do not learn.
  • the input signals change non-linearly. That is, the input signal replaces another signal while retaining the original information.
  • the input signals change over time by interacting with each other in the reservoir R.
  • a plurality of neurons n 2 are randomly connected to the reservoir R. For example, the signal output from the neuron n 2 at a certain time t may return to the original neuron n 2 at a certain time t + 1.
  • neurons n 2 it can process in consideration of the time t and time t + 1 of the signal, can be recursively process the information.
  • the output layer L out outputs a signal from the reservoir R.
  • the output signal output from the output layer L out has the information of the input signal and is replaced with another signal.
  • An example of such conversion is the replacement of the Cartesian coordinate system (x, y, z) with the spherical coordinate system (r, ⁇ , ⁇ ).
  • Output layer L out for example, comprises a plurality of neurons n 3. Learning is performed from the reservoir R to the output layer L out. Learning is performed in each neuron n 2 and the output layer L out neuron n 3 and the connecting transmission path of the reservoir R (synapses in the brain). The output layer L out outputs the learning result to the outside. Neurons n 1 , n 2 , and n 3 are sometimes referred to as nodes.
  • FIG. 2 is an example of a circuit diagram of the reservoir device according to the present embodiment.
  • the reservoir device 100 shown in FIG. 2 has a plurality of units 10.
  • the unit 10 is an example of a reservoir calculation unit. Each unit 10 is connected to at least one or more other units 10.
  • the i-th unit 10 is referred to as a unit 10 i.
  • the output of unit 10 i shown in FIG. 2 is connected to, for example, unit 10 i-1 and unit 10 1 + 1.
  • the output signal of unit 10 i becomes an input signal of unit 10 i-1 and the unit 10 1 + 1.
  • a signal at time n is output to the unit 10 1 + 1.
  • a signal at time n + 1 is output to the unit 10 1-1.
  • the time n + 1 signal is produced, for example, by a delay circuit.
  • the reservoir device 100 shown in FIG. 2 has a flip-flop circuit 20 connected to the unit 10.
  • the flip-flop circuit 20 latches the data and holds and stores it until the next clock.
  • the flip-flop circuit 20 delays the signal.
  • the unit 10 i signals of different time is inputted, the information is processed recursively.
  • FIG. 3 is an example of a circuit of each unit 10 constituting the reservoir device according to the present embodiment.
  • the unit 10 consists of a circuit designed based on an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values.
  • the design method of the circuit constituting the unit 10 will be described.
  • the method of designing the circuit constituting the unit 10 includes a step of converting a continuous differential equation into an algebraic equation superdiscretized into two or multiple values and a step of converting an arithmetic equation of the algebraic equation into a circuit diagram. Have.
  • ⁇ , a, b are constants and ⁇ is time.
  • -ax is a component that produces vibration
  • bF (x- ⁇ ) is a non-linear function that includes time.
  • the reservoir device 100 processes the input signal non-linearly and converts it into another signal. The processing of the above differential equation is appropriate as the processing required for each unit 10 of the reservoir device 100.
  • U j n and V j n correspond to state variables.
  • the parameters ⁇ m, A, E, ⁇ , and ⁇ are all constants.
  • the continuous numerical change is approximated to the discretized discrete numerical change.
  • continuous operations are required.
  • the numerical change is approximated to the discretized discrete value, the number of arithmetic processes is reduced and the calculation load is lightened. Further, since the number of arithmetic processes is reduced, the number of arithmetic elements required for the unit 10 is reduced.
  • the circuit shown in FIG. 3 is designed after converting the above algebraic equation into a data flow graph and expressing it.
  • the optimum configuration when implementing an algebraic equation with a recursive structure as described above is not always trivial, and it is necessary to optimize the data flow graph showing the data flow and the arrangement of arithmetic units.
  • the calculation of the circuit shown in FIG. 3 is expressed by the following equation.
  • U i is a state variable
  • a and B are parameters that take discrete values.
  • I (n) may be a parameter (constant) as in Non-Patent Document 2, or may be an input signal that changes with time in order to contribute to the reservoir calculation.
  • the unit 10 shown in FIG. 3 has a plurality of multiplexers 1A, 1B, 1C, 1D, a plurality of subtractors 2, a multiplier 3, and an adder 4.
  • the two signals U 1-1 (n) and U 1 + 1 (n) input to the unit 10 are input to the multiplexer 1A.
  • the multiplexer 1A outputs the larger signal M (U i ) of the two signals U 1-1 (n) and U 1 + 1 (n).
  • the signal M (U i ) is branched into two.
  • One of the branched signals M (U i ) is obtained by the subtractor 2 for the difference ⁇ M (U i ) from 0, and is input to the next multiplexer 1B.
  • the multiplexer 1B outputs the larger signal of the signal I and the difference ⁇ M (U i).
  • the output signal is input to the multiplexer 1C.
  • the signal input to the multiplexer 1C is represented by max [I (n), ⁇ M (U i )].
  • the other branched signal M ( Ui ) is integrated by the multiplier 3 and input to the subtractor 2.
  • the subtractor 2 outputs the difference between the constant A and the signal 2M (U i).
  • the output signal is branched, one is input to the adder 4 and the other is input to the multiplexer 1D.
  • the adder 4 inputs the difference between the constant B and the signal A-2M ( Ui ) to the multiplexer 1C.
  • the multiplexer 1C outputs the larger signal of the input signals.
  • the signal output from the multiplexer 1C is represented by max [I (n), ⁇ M (U i ), A + B-2M (U i )].
  • the multiplexer 1D outputs the larger signal of the input signals.
  • the signal output from the multiplexer 1D is represented by max [0, A-2M ( Ui )].
  • the difference between the output signals from the multiplexer 1C and the multiplexer 1D is obtained by the subtractor 2 and becomes the output of the unit 10. That is, the circuit of the unit 10 is designed based on the above algebraic equation.
  • the circuits that make up the unit 10 change depending on the algebraic equation. That is, the corresponding data flow graph and circuit configuration differ depending on the original differential equation.
  • the circuit shown in FIG. 3 is only an example designed based on the algebraic equation shown in Non-Patent Document 2.
  • the reservoir device can reduce the calculation load by super-discretizing the differential equations that are calculated in each unit and converting them into equivalent algebraic equations. Further, by simplifying the arithmetic processing in each unit, the number of arithmetic elements required for each unit is reduced, and physical mounting is easy. In addition, by mathematically converting the differential equation into an algebraic equation, the representation (complexity) of the data output from the reservoir device is maintained while retaining the desirable properties (for example, integrability and boundability) of the original differential equation. The) is also maintained.
  • the reservoir device converts the differential equations calculated in each unit into equivalent algebraic equations by super-discretization, and further changes the parameters of the algebraic equations with time to input the reservoir device. Reservoir calculation is performed assuming that.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Neurology (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

A reservoir calculation unit according to an embodiment of the present invention is provided with a data flow graph designed on the basis of an algebraic equation in which a continuous differential equation has been ultra-discretized as two values or multiple values.

Description

リザーバ計算ユニット、リザーバデバイス、リザーバ計算ユニットの設計方法およびリザーバデバイスの制御方法Reservoir calculation unit, reservoir device, reservoir calculation unit design method and reservoir device control method
　本発明は、リザーバ計算ユニット、リザーバデバイス、リザーバ計算ユニットの設計方法およびリザーバデバイスの制御方法に関する。 The present invention relates to a reservoir calculation unit, a reservoir device, a method for designing a reservoir calculation unit, and a method for controlling a reservoir device.
　ニューロモルフィックデバイスは、ニューラルネットワークにより人間の脳を模倣した素子である。ニューロモルフィックデバイスは、人間の脳におけるニューロンとシナプスとの関係を人工的に模倣している。 A neuromorphic device is an element that imitates the human brain by means of a neural network. Neuromorphic devices artificially mimic the relationship between neurons and synapses in the human brain.
　ニューロモルフィックデバイスは、例えば、階層状に配置されたチップ（脳におけるニューロン）と、これらの間を繋ぐ伝達手段（脳におけるシナプス）と、を有する。ニューロモルフィックデバイスは、伝達手段（シナプス）が学習することで、問題の正答率を高める。学習は将来使えそうな知識を情報から見つけることであり、ニューロモルフィックデバイスでは入力されたデータに重み付けをする。 The neuromorphic device has, for example, hierarchically arranged chips (neurons in the brain) and transmission means (synapses in the brain) connecting them. Neuromorphic devices increase the percentage of correct answers to questions by learning by means of communication (synapses). Learning is to find knowledge that can be used in the future from information, and neuromorphic devices weight the input data.
　ニューラルネットワークの一つとして、リカレントニューラルネットワークが知られている。リカレントニューラルネットワークは、非線形な時系列のデータを扱うことができる。非線形な時系列のデータは、時間の経過とともに値が変化するデータであり、株価やインフルエンザの流行者数はその一例である。リカレントニューラルネットワークは、後段の階層のニューロンでの処理結果を前段の階層のニューロンに戻すことで、時系列のデータを処理することができる。 A recurrent neural network is known as one of the neural networks. Recurrent neural networks can handle non-linear time series data. Non-linear time-series data is data whose values change over time, such as stock prices and the number of influenza pandemics. The recurrent neural network can process time-series data by returning the processing results of the neurons in the latter layer to the neurons of the previous layer.
　リザーバコンピューティングは、リカレントニューラルネットワークを実現する一つの手段である。リザーバコンピューティングは、信号を相互作用させることで、再帰的な処理を行う。リザーバコンピューティングは、例えば、小脳の動作機構を模倣しており、再帰的なデータの処理やデータの変換（例えば、座標の変換）等を行う。 Reservoir computing is one means of realizing a recurrent neural network. Reservoir computing performs recursive processing by interacting signals. Reservoir computing, for example, mimics the operating mechanism of the cerebellum, and performs recursive data processing, data conversion (for example, coordinate conversion), and the like.
　リザーバコンピューティングの概念を実際の素子に物理的に実装する試みが行われている。リザーバコンピューティングの概念を実際の素子にマッピングしたものを、以下、リザーバデバイスと称する。例えば、非特許文献１には、１次元リングトポロジーのリザーバが記載されている。 Attempts are being made to physically implement the concept of reservoir computing on actual devices. A device that maps the concept of reservoir computing to an actual device is hereinafter referred to as a reservoir device. For example, Non-Patent Document 1 describes a reservoir with a one-dimensional ring topology.
　リザーバデバイスは、リザーバコンピューティングにおけるノードに対応する複数のユニットからなる。それぞれのユニットは演算を行う。それぞれのユニットの演算精度を浮動小数点又は固定小数点の精度まで高めると、ユニットを構成する演算器の数が膨大になる。 The reservoir device consists of multiple units corresponding to the nodes in reservoir computing. Each unit performs an operation. If the calculation accuracy of each unit is increased to the precision of floating point or fixed point, the number of arithmetic units constituting the unit becomes enormous.
　本発明は上記事情に鑑みてなされたものであり、リザーバデバイスから出力されるデータの表現力（複雑さ）を維持しつつ、演算効率を向上することができるリザーバ計算ユニット、リザーバデバイス、リザーバ計算ユニットおよびリザーバデバイスの設計方法を提供する。 The present invention has been made in view of the above circumstances, and is a reservoir calculation unit, a reservoir device, and a reservoir calculation capable of improving calculation efficiency while maintaining the expressive power (complexity) of data output from a reservoir device. Provides a method of designing a unit and a reservoir device.
（１）第１の態様にかかるリザーバ計算ユニットは、連続的な微分方程式を２値又は多値に超離散化した代数方程式に基づいて設計された回路からなる。 (1) The reservoir calculation unit according to the first aspect comprises a circuit designed based on an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values.
（２）上記態様にかかるリザーバ計算ユニットにおいて、前記微分方程式は、εｄｘ／ｄτ＝－ａｘ＋ｂＦ（ｘ－τ）で表記されてもよく、ε、ａ、ｂは定数であり、τは時間であり，εは時定数である。 (2) In the reservoir calculation unit according to the above aspect, the differential equation may be expressed by εdx / dτ = -ax + bF (x-τ), where ε, a, b are constants and τ is time. ， Ε is a time constant.
（３）第２の態様にかかるリザーバデバイスは、上記態様にかかるリザーバ計算ユニットを複数有し、前記リザーバ計算ユニットはそれぞれ少なくとも一つ以上の別のリザーバ計算ユニットに接続されている (3) The reservoir device according to the second aspect has a plurality of reservoir calculation units according to the above aspect, and each of the reservoir calculation units is connected to at least one or more other reservoir calculation units.
（４）第３の態様にかかるリザーバ計算ユニットの設計方法は、リザーバデバイスに用いられるリザーバ計算ユニットの設計方法であって、連続的な微分方程式を２値又は多値に超離散化した代数方程式に変換する工程と、前記代数方程式の演算式を回路図に変換する工程と、を有する。 (4) The method for designing the reservoir calculation unit according to the third aspect is the method for designing the reservoir calculation unit used for the reservoir device, which is an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values. It has a step of converting into a circuit diagram and a step of converting the calculation formula of the algebraic equation into a circuit diagram.
（５）第４の態様にかかるリザーバデバイスの制御方法は、連続的な微分方程式を２値又は多値に超離散化した代数方程式に基づいて設計されたリザーバ計算ユニットに対して、外部入力として前記代数方程式のパラメータを時間的に変化させたものを供給する。 (5) The method for controlling the reservoir device according to the fourth aspect is as an external input to a reservoir calculation unit designed based on an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values. The parameters of the algebraic equation are changed with time to supply.
　上記態様にかかるリザーバ計算ユニット、リザーバデバイスおよびリザーバ計算ユニットの設計方法およびリザーバデバイスの制御方法は、リザーバデバイスから出力されるデータの表現力（複雑さ）を維持しつつ、演算効率を向上することができる。 The reservoir calculation unit, the reservoir device, the design method of the reservoir calculation unit, and the control method of the reservoir device according to the above aspects are to improve the calculation efficiency while maintaining the expressiveness (complexity) of the data output from the reservoir device. Can be done.
第１実施形態にかかるリザーバデバイスが模擬するニューラルネットワークの概念図である。It is a conceptual diagram of the neural network simulated by the reservoir device which concerns on 1st Embodiment. 本実施形態にかかるリザーバデバイスの回路構成の一例である。This is an example of the circuit configuration of the reservoir device according to this embodiment. 本実施形態にかかるリザーバデバイスを構成するそれぞれのユニットの回路構成の一例である。This is an example of the circuit configuration of each unit constituting the reservoir device according to the present embodiment.
　以下、本実施形態について、図を適宜参照しながら詳細に説明する。以下の説明で用いる図面は、本発明の特徴をわかりやすくするために便宜上特徴となる部分を拡大して示している場合があり、各構成要素の寸法比率などは実際とは異なっていることがある。以下の説明において例示される材料、寸法等は一例であって、本発明はそれらに限定されるものではなく、本発明の効果を奏する範囲で適宜変更して実施することが可能である。 Hereinafter, the present embodiment will be described in detail with reference to the figures as appropriate. The drawings used in the following description may be enlarged for convenience in order to make the features of the present invention easy to understand, and the dimensional ratios of the respective components may differ from the actual ones. be. The materials, dimensions, etc. exemplified in the following description are examples, and the present invention is not limited thereto, and can be appropriately modified and carried out within the range in which the effects of the present invention are exhibited.
　本実施形態に係るリザーバデバイスは、リザーバコンピューティングにおける処理をデバイス化したものである。リザーバコンピューティングは、リカレントニューラルネットワークの一例である。 The reservoir device according to the present embodiment is a device of processing in reservoir computing. Reservoir computing is an example of a recurrent neural network.
「第１実施形態」
（リザーバコンピューティング）
　図１は、第１実施形態にかかるリザーバデバイスが模擬するニューラルネットワークの概念図である。図１に示すニューラルネットワークＮＮは、リザーバコンピューティングの概念模式図である。図１に示すニューラルネットワークＮＮは、入力層ＬｉｎとリザーバＲと出力層Ｌｏｕｔとを有する。入力層Ｌｉｎ及び出力層Ｌｏｕｔは、リザーバＲに接続されている。 "First embodiment"
(Reservoir computing)
FIG. 1 is a conceptual diagram of a neural network simulated by the reservoir device according to the first embodiment. The neural network NN shown in FIG. 1 is a conceptual schematic diagram of reservoir computing. Neural networks NN shown in Figure 1, it has an input layer L in the reservoir R and the output layer L out. Input layer L in and the output layer L out is connected to the reservoir R.
　入力層Ｌｉｎは、外部から入力された信号をリザーバＲに伝える。入力層Ｌｉｎは、例えば、複数のニューロンｎ１を含む。外部から入力層Ｌｉｎのそれぞれのニューロンｎ１に入力された入力信号は、リザーバＲに伝わる。 Input layer L in conveys a signal inputted from the outside to the reservoir R. Input layer L in, for example, comprises a plurality of neurons n 1. Input signal input to each neuron n 1 of the input layer L in from the outside is transmitted to the reservoir R.
　リザーバＲは、入力層Ｌｉｎから入力された入力信号を貯留し、別の信号に変換する。リザーバＲ内では、信号は相互作用するだけであり、学習しない。入力信号が互いに相互作用すると、入力信号が非線形に変化する。すなわち、入力信号は、元の情報を保有しつつ別の信号に置き換わる。入力信号は、リザーバＲ内で互いに相互作用することで、時間の経過とともに変化する。リザーバＲは、複数のニューロンｎ２がランダムに接続されている。例えば、ある時刻ｔにあるニューロンｎ２から出力された信号は、ある時刻ｔ＋１において元のニューロンｎ２に戻る場合がある。ニューロンｎ２では、時刻ｔ及び時刻ｔ＋１の信号を踏まえた処理ができ、情報を再帰的に処理できる。 The reservoir R is storing the input signal input from the input layer L in, is converted into another signal. Within the reservoir R, the signals only interact and do not learn. When the input signals interact with each other, the input signals change non-linearly. That is, the input signal replaces another signal while retaining the original information. The input signals change over time by interacting with each other in the reservoir R. A plurality of neurons n 2 are randomly connected to the reservoir R. For example, the signal output from the neuron n 2 at a certain time t may return to the original neuron n 2 at a certain time t + 1. In neurons n 2, it can process in consideration of the time t and time t + 1 of the signal, can be recursively process the information.
　出力層Ｌｏｕｔは、リザーバＲからの信号を出力する。出力層Ｌｏｕｔから出力される出力信号は、入力信号の情報を持ちつつ、別の信号に置き換わっている。当該変換の一例として、直交座標系（ｘ，ｙ，ｚ）から球面座標系（ｒ，θ，φ）への置き換えが挙げられる。出力層Ｌｏｕｔは、例えば、複数のニューロンｎ３を含む。リザーバＲから出力層Ｌｏｕｔに至る際に、学習が行われる。学習は、リザーバＲのそれぞれのニューロンｎ２と出力層Ｌｏｕｔのニューロンｎ３とを繋ぐ伝達経路（脳におけるシナプス）で行われる。出力層Ｌｏｕｔは、学習の結果を外部に出力する。ニューロンｎ１、ｎ２、ｎ３はノードと言われることもある。 The output layer L out outputs a signal from the reservoir R. The output signal output from the output layer L out has the information of the input signal and is replaced with another signal. An example of such conversion is the replacement of the Cartesian coordinate system (x, y, z) with the spherical coordinate system (r, θ, φ). Output layer L out, for example, comprises a plurality of neurons n 3. Learning is performed from the reservoir R to the output layer L out. Learning is performed in each neuron n 2 and the output layer L out neuron n 3 and the connecting transmission path of the reservoir R (synapses in the brain). The output layer L out outputs the learning result to the outside. Neurons n 1 , n 2 , and n 3 are sometimes referred to as nodes.
　図２は、本実施形態にかかるリザーバデバイスの回路図の一例である。図２に示すリザーバデバイス１００は、複数のユニット１０を有する。ユニット１０は、リザーバ計算用ユニットの一例である。ユニット１０はそれぞれ少なくとも一つ以上の別のユニット１０に接続されている。 FIG. 2 is an example of a circuit diagram of the reservoir device according to the present embodiment. The reservoir device 100 shown in FIG. 2 has a plurality of units 10. The unit 10 is an example of a reservoir calculation unit. Each unit 10 is connected to at least one or more other units 10.
　例えば、ｉ番目のユニット１０をユニット１０ｉと称する。図２に示すユニット１０ｉの出力は、例えば、ユニット１０ｉ－１及びユニット１０１＋１と接続されている。ユニット１０ｉの出力信号は、ユニット１０ｉ－１及びユニット１０１＋１の入力信号となる。ユニット１０１＋１には、時刻ｎの信号が出力される。ユニット１０１ー１には、時刻ｎ＋１の信号が出力される。 For example, the i-th unit 10 is referred to as a unit 10 i. The output of unit 10 i shown in FIG. 2 is connected to, for example, unit 10 i-1 and unit 10 1 + 1. The output signal of unit 10 i becomes an input signal of unit 10 i-1 and the unit 10 1 + 1. A signal at time n is output to the unit 10 1 + 1. A signal at time n + 1 is output to the unit 10 1-1.
　時刻ｎ＋１の信号は、例えば遅延回路によって生み出される。図２に示すリザーバデバイス１００は、ユニット１０と接続されたフリップフロップ回路２０を有する。フリップフロップ回路２０は、データをラッチし、次のクロックまで保持、記憶する。フリップフロップ回路２０は、信号を遅延させる。ユニット１０ｉには、異なる時刻の信号が入力され、情報が再帰的に処理される。 The time n + 1 signal is produced, for example, by a delay circuit. The reservoir device 100 shown in FIG. 2 has a flip-flop circuit 20 connected to the unit 10. The flip-flop circuit 20 latches the data and holds and stores it until the next clock. The flip-flop circuit 20 delays the signal. The unit 10 i, signals of different time is inputted, the information is processed recursively.
　図３は、本実施形態にかかるリザーバデバイスを構成するそれぞれのユニット１０の回路の一例である。ユニット１０は、連続的な微分方程式を２値又は多値に超離散化した代数方程式に基づいて設計された回路からなる。 FIG. 3 is an example of a circuit of each unit 10 constituting the reservoir device according to the present embodiment. The unit 10 consists of a circuit designed based on an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values.
　ユニット１０を構成する回路の設計方法について説明する。ユニット１０を構成する回路の設計方法は、連続的な微分方程式を２値又は多値に超離散化した代数方程式に変換する工程と、代数方程式の演算式を回路図に変換する工程と、を有する。 The design method of the circuit constituting the unit 10 will be described. The method of designing the circuit constituting the unit 10 includes a step of converting a continuous differential equation into an algebraic equation superdiscretized into two or multiple values and a step of converting an arithmetic equation of the algebraic equation into a circuit diagram. Have.
　まず連続的な微分方程式を決定する。微分方程式は、任意の方程式を用いることができる。例えば、εｄｘ／ｄτ＝－ａｘ＋ｂＦ（ｘ－τ）で表記される微分方程式を用いる。上式において、ε、ａ、ｂは定数であり、τは時間である。上記の微分方程式において、－ａｘは振動を生み出す成分であり、ｂＦ（ｘ－τ）は時間を含む非線形関数である。リザーバデバイス１００は、入力された信号を非線形に処理し、別の信号に変換する。上記の微分方程式の処理は、リザーバデバイス１００のそれぞれのユニット１０に求められる処理として適切である。 First, determine a continuous differential equation. Any equation can be used as the differential equation. For example, a differential equation expressed by εdx / dτ = -ax + bF (x-τ) is used. In the above equation, ε, a, b are constants and τ is time. In the above differential equation, -ax is a component that produces vibration, and bF (x-τ) is a non-linear function that includes time. The reservoir device 100 processes the input signal non-linearly and converts it into another signal. The processing of the above differential equation is appropriate as the processing required for each unit 10 of the reservoir device 100.
　次いで、連続的な微分方程式を代数方程式に変換する。以下、非特許文献２で示されたものを一例として、下記の連続的な微分方程式を変換する。 Next, convert continuous differential equations to algebraic equations. Hereinafter, the following continuous differential equations are converted by taking the one shown in Non-Patent Document 2 as an example.
Figure JPOXMLDOC01-appb-M000001
Figure JPOXMLDOC01-appb-M000001
　まず微分方程式を差分方程式に変換する。微分方程式を差分方程式に変換すると、微分方程式の空間変数と時間変数とが離散化される。空間変数と時間変数とは、量子化する。上記の微分方程式のｘを一定の幅を持った範囲ｊに変換し、時間ｔを時間範囲ｎに変換した差分方程式は以下になる。 First, convert the differential equation to the difference equation. When a differential equation is converted into a difference equation, the spatial and temporal variables of the differential equation are discretized. Spatial variables and time variables are quantized. The difference equation obtained by converting x of the above differential equation into a range j having a constant width and converting time t into a time range n is as follows.
Figure JPOXMLDOC01-appb-M000002
Figure JPOXMLDOC01-appb-M000002
　次いで、差分方程式を代数方程式に変換する。差分方程式を代数方程式に変換すると、微分方程式の状態変数も離散化される。状態変数の離散化は、超離散化と言われる。超離散化によって、状態変数は二値あるいは多値に量子化する。差分方程式から代数方程式への変換は、まず以下の変数変換を行う。 Next, convert the difference equation to an algebraic equation. When the difference equation is converted to an algebraic equation, the state variables of the differential equation are also discretized. Discretization of state variables is called superdiscretization. By super-discretization, state variables are quantized into binary or multi-valued. To convert the difference equation to the algebraic equation, first perform the following change of variables.
Figure JPOXMLDOC01-appb-M000003
Figure JPOXMLDOC01-appb-M000003
　上式において、Ｕｊ ｎ及びＶｊ ｎは状態変数に対応する。パラメータλｍ、Ａ、Ｅ、Γ、Φはいずれも定数である。 In the above equation, U j n and V j n correspond to state variables. The parameters λm, A, E, Γ, and Φ are all constants.
　以下の式に基づき、上記変数変換した結果の超離散極限を求める。 Based on the following formula, find the superdiscrete limit as a result of the above variable transformation.
Figure JPOXMLDOC01-appb-M000004
Figure JPOXMLDOC01-appb-M000004
　その結果、代数方程式は以下の式となる。 As a result, the algebraic equation becomes the following equation.
Figure JPOXMLDOC01-appb-M000005
Figure JPOXMLDOC01-appb-M000005
　微分方程式を代数方程式に変換すると、連続的な数値変化が、離散化した飛び飛びの数値変化に近似される。連続的な数値変化をユニット１０で処理するためには、連続的な演算が必要である。これに対し、数値変化が離散化した飛び飛びの値に近似されると、演算処理数が少なくなり、計算負荷が軽くなる。また演算処理数が少なくなるため、ユニット１０に必要な演算素子の数が低減される。 When the differential equation is converted to an algebraic equation, the continuous numerical change is approximated to the discretized discrete numerical change. In order for the unit 10 to process continuous numerical changes, continuous operations are required. On the other hand, when the numerical change is approximated to the discretized discrete value, the number of arithmetic processes is reduced and the calculation load is lightened. Further, since the number of arithmetic processes is reduced, the number of arithmetic elements required for the unit 10 is reduced.
　次いで、代数方程式の演算式を実際の回路を構成するためのデータフローグラフに変換する。図３に示す回路は、上記の代数方程式をデータフローグラフに変換して表現したうえで設計されたものである。上記のように再帰構造を持つ代数方程式を実装する際の最適な構成は必ずしも自明ではなく、データの流れと演算器の配置を示すデータフローグラフを最適化する必要がある。図３に示す回路の演算は、以下の式で表される。 Next, the arithmetic expression of the algebraic equation is converted into a data flow graph for constructing an actual circuit. The circuit shown in FIG. 3 is designed after converting the above algebraic equation into a data flow graph and expressing it. The optimum configuration when implementing an algebraic equation with a recursive structure as described above is not always trivial, and it is necessary to optimize the data flow graph showing the data flow and the arrangement of arithmetic units. The calculation of the circuit shown in FIG. 3 is expressed by the following equation.
Figure JPOXMLDOC01-appb-M000006
Figure JPOXMLDOC01-appb-M000006
　上式において、Ｕiは状態変数であり、Ａ及びＢは離散値を取るパラメータである。Ｉ(ｎ)は、非特許文献２と同様にパラメータ（定数）でもよく、リザーバ計算に資するために時間的に変化する入力信号でもよい。 In the above equation, U i is a state variable, and A and B are parameters that take discrete values. I (n) may be a parameter (constant) as in Non-Patent Document 2, or may be an input signal that changes with time in order to contribute to the reservoir calculation.
　図３に示すユニット１０は、複数のマルチプレクサ１Ａ，１Ｂ，１Ｃ，１Ｄと複数の減算器２と乗算器３と加算器４とを有する。ユニット１０に入力された二つの信号Ｕ１－１（ｎ）、Ｕ１＋１（ｎ）は、マルチプレクサ１Ａに入力される。マルチプレクサ１Ａは、二つの信号Ｕ１－１（ｎ）、Ｕ１＋１（ｎ）のうち大きい方の信号Ｍ（Ｕｉ）を出力する。 The unit 10 shown in FIG. 3 has a plurality of multiplexers 1A, 1B, 1C, 1D, a plurality of subtractors 2, a multiplier 3, and an adder 4. The two signals U 1-1 (n) and U 1 + 1 (n) input to the unit 10 are input to the multiplexer 1A. The multiplexer 1A outputs the larger signal M (U i ) of the two signals U 1-1 (n) and U 1 + 1 (n).
　信号Ｍ（Ｕｉ）は２つに分岐される。分岐した信号Ｍ（Ｕｉ）の一方は、減算器２で０との差分－Ｍ（Ｕｉ）が求められ、次のマルチプレクサ１Ｂに入力される。マルチプレクサ１Ｂは、信号Ｉと差分－Ｍ（Ｕｉ）とのうち大きい方の信号を出力する。出力された信号は、マルチプレクサ１Ｃに入力される。信号Ｉを、時間的に変位するＩ（ｎ）とすると、リザーバデバイス１００の処理の非線形性が高まる。マルチプレクサ１Ｃに入力される信号は、ｍａｘ［Ｉ（ｎ），－Ｍ（Ｕｉ）］で表記される。 The signal M (U i ) is branched into two. One of the branched signals M (U i ) is obtained by the subtractor 2 for the difference −M (U i ) from 0, and is input to the next multiplexer 1B. The multiplexer 1B outputs the larger signal of the signal I and the difference −M (U i). The output signal is input to the multiplexer 1C. When the signal I is I (n) which is displaced in time, the non-linearity of the processing of the reservoir device 100 is increased. The signal input to the multiplexer 1C is represented by max [I (n), −M (U i )].
　これに対して、分岐した他方の信号Ｍ（Ｕｉ）は、乗算器３で積算され、減算器２に入力される。減算器２は、定数Ａと信号２Ｍ（Ｕｉ）との差分を出力する。出力された信号は分岐し、一方は加算器４、他方はマルチプレクサ１Ｄに入力される。加算器４は、定数Ｂと信号Ａ－２Ｍ（Ｕｉ）との差分をマルチプレクサ１Ｃに入力する。 On the other hand, the other branched signal M ( Ui ) is integrated by the multiplier 3 and input to the subtractor 2. The subtractor 2 outputs the difference between the constant A and the signal 2M (U i). The output signal is branched, one is input to the adder 4 and the other is input to the multiplexer 1D. The adder 4 inputs the difference between the constant B and the signal A-2M ( Ui ) to the multiplexer 1C.
　マルチプレクサ１Ｃは、入力された信号のうち大きい方の信号を出力する。マルチプレクサ１Ｃから出力される信号は、ｍａｘ［Ｉ（ｎ），－Ｍ（Ｕｉ），Ａ＋Ｂ－２Ｍ（Ｕｉ）］で表記される。 The multiplexer 1C outputs the larger signal of the input signals. The signal output from the multiplexer 1C is represented by max [I (n), −M (U i ), A + B-2M (U i )].
　マルチプレクサ１Ｄは、入力された信号のうち大きい方の信号を出力する。マルチプレクサ１Ｄから出力される信号は、ｍａｘ［０，Ａ－２Ｍ（Ｕｉ）］で表記される。 The multiplexer 1D outputs the larger signal of the input signals. The signal output from the multiplexer 1D is represented by max [0, A-2M ( Ui )].
　マルチプレクサ１Ｃ及びマルチプレクサ１Ｄからの出力信号は、減算器２で差分が求められ、ユニット１０の出力となる。すなわち、ユニット１０の回路は、上記の代数方程式に基づいて設計されている。 The difference between the output signals from the multiplexer 1C and the multiplexer 1D is obtained by the subtractor 2 and becomes the output of the unit 10. That is, the circuit of the unit 10 is designed based on the above algebraic equation.
　ユニット１０を構成する回路は、代数方程式によって変わる。すなわち、元となる微分方程式によって対応するデータフローグラフ及び回路構成は異なるものとなる。図３に示す回路は、非特許文献２で示された代数方程式に基づいて設計された一例にすぎない。 The circuits that make up the unit 10 change depending on the algebraic equation. That is, the corresponding data flow graph and circuit configuration differ depending on the original differential equation. The circuit shown in FIG. 3 is only an example designed based on the algebraic equation shown in Non-Patent Document 2.
　本実施形態にかかるリザーバデバイスは、それぞれのユニットでの演算にあたる微分方程式を、超離散化させ等価な代数方程式に変換することで、演算負荷を低減することができる。またそれぞれのユニットでの演算処理が簡便化されることで、それぞれのユニットに必要な演算素子数が少なくなり、物理実装しやすい。また微分方程式を数学的に代数方程式に変換したことにより、元の微分方程式の持つ望ましい性質（例えば、可積分性や有界性）を保持しつつ、リザーバデバイスから出力されるデータの表現（複雑さ）も維持される。 The reservoir device according to the present embodiment can reduce the calculation load by super-discretizing the differential equations that are calculated in each unit and converting them into equivalent algebraic equations. Further, by simplifying the arithmetic processing in each unit, the number of arithmetic elements required for each unit is reduced, and physical mounting is easy. In addition, by mathematically converting the differential equation into an algebraic equation, the representation (complexity) of the data output from the reservoir device is maintained while retaining the desirable properties (for example, integrability and boundability) of the original differential equation. The) is also maintained.
　本実施形態にかかるリザーバデバイスは、それぞれのユニットでの演算にあたる微分方程式を、超離散化により等価な代数方程式に変換し、さらに代数方程式のパラメータを時間的に変化させることで、リザーバデバイスの入力とみなしてリザーバ計算を行う。 The reservoir device according to the present embodiment converts the differential equations calculated in each unit into equivalent algebraic equations by super-discretization, and further changes the parameters of the algebraic equations with time to input the reservoir device. Reservoir calculation is performed assuming that.
１０　ユニット
１００　リザーバデバイス 10 units 100 reservoir device

Claims (5)

  1. 　連続的な微分方程式を２値又は多値に超離散化した代数方程式に基づいて設計されたデータフローグラフを備えた、リザーバ計算ユニット。 Reservoir calculation unit equipped with a data flow graph designed based on algebraic equations in which continuous differential equations are super-discretized into binary or multi-values.
  2. 　前記微分方程式は、εｄｘ／ｄτ＝－ａｘ＋ｂＦ（ｘ－τ）で表記され、
    　ε、ａ、ｂは定数であり、τは時間である、請求項１に記載のリザーバ計算ユニット。     The differential equation is expressed by εdx / dτ = -ax + bF (x-τ).
    The reservoir calculation unit according to claim 1, wherein ε, a, and b are constants and τ is time.
  3. 　請求項１又は２に記載のリザーバ計算ユニットを複数有し、
    　前記リザーバ計算ユニットはそれぞれ少なくとも一つ以上の別のリザーバ計算ユニットに接続されている、リザーバデバイス。     Having a plurality of reservoir calculation units according to claim 1 or 2,
    A reservoir device, each of which is connected to at least one or more separate reservoir computing units.
  4. 　リザーバデバイスに用いられるリザーバ計算ユニットの設計方法であって、
    　連続的な微分方程式を２値又は多値に超離散化した代数方程式に変換する工程と、
    　前記代数方程式の演算式をデータフローグラフに変換する工程と、を有する、リザーバ計算ユニットの設計方法。     A method of designing a reservoir calculation unit used in a reservoir device.
    The process of converting a continuous differential equation into a binary or multi-valued algebraic equation,
    A method for designing a reservoir calculation unit, which comprises a step of converting an arithmetic expression of the algebraic equation into a data flow graph.
  5. 　連続的な微分方程式を２値又は多値に超離散化した代数方程式に基づいて設計されたリザーバ計算ユニットに対して、外部入力として前記代数方程式のパラメータを時間的に変化させたものを供給する、リザーバデバイスの制御方法。 A temporally changed parameter of the algebraic equation is supplied as an external input to a reservoir calculation unit designed based on an algebraic equation obtained by superdiscretizing a continuous differential equation into two or multiple values. , How to control the reservoir device.
PCT/JP2020/013621 2020-03-26 2020-03-26 Reservoir calculation unit, reservoir device, design method for reservoir calculation unit, and control method for reservoir device WO2021192137A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/013621 WO2021192137A1 (en) 2020-03-26 2020-03-26 Reservoir calculation unit, reservoir device, design method for reservoir calculation unit, and control method for reservoir device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/013621 WO2021192137A1 (en) 2020-03-26 2020-03-26 Reservoir calculation unit, reservoir device, design method for reservoir calculation unit, and control method for reservoir device

Publications (1)

Publication Number Publication Date
WO2021192137A1 true WO2021192137A1 (en) 2021-09-30

Family

ID=77891256

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/013621 WO2021192137A1 (en) 2020-03-26 2020-03-26 Reservoir calculation unit, reservoir device, design method for reservoir calculation unit, and control method for reservoir device

Country Status (1)

Country Link
WO (1) WO2021192137A1 (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10211856B1 (en) * 2017-10-12 2019-02-19 The Boeing Company Hardware scalable channelizer utilizing a neuromorphic approach

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10211856B1 (en) * 2017-10-12 2019-02-19 The Boeing Company Hardware scalable channelizer utilizing a neuromorphic approach

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SATSUMA, JUNKICHI ET AL.: "What is ultradiscrete?", MATHEMATICAL SCIENCES, vol. 37, no. 9, 1 September 1999 (1999-09-01), pages 5 - 11, ISSN: 0386-2240 *

Similar Documents

Publication Publication Date Title
Sharma et al. Activation functions in neural networks
Beidokhti et al. Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques
Meier A mixed-signal universal neuromorphic computing system
Aguilar-Leal et al. Distributed parameter system identification using finite element differential neural networks
Wu et al. Numerical solutions of wavelet neural networks for fractional differential equations
Darbon et al. Neural network architectures using min-plus algebra for solving certain high-dimensional optimal control problems and Hamilton–Jacobi PDEs
Dong et al. Application of fractional theory in quantum back propagation neural network
WO2021192137A1 (en) Reservoir calculation unit, reservoir device, design method for reservoir calculation unit, and control method for reservoir device
Becerikli et al. Modeling and prediction with a class of time delay dynamic neural networks
Zaman Farsa et al. Function approximation by hardware spiking neural network
US20230023123A1 (en) Information processing system, information processing device, information processing method, and information processing program
MacLennan A review of analog computing
Kumaresan et al. Optimal control for stochastic linear quadratic singular system using neural networks
Islam et al. FPGA Implementation of Nerve Cell Using Izhikevich Neuronal Model As Spike Generator (SG)
Kadam et al. Artificial neural network approximation of fractional-order derivative operators: analysis and DSP implementation
Zhu et al. NN-Poly: Approximating common neural networks with Taylor polynomials to imbue dynamical system constraints
Heidari et al. Using artificial neural networks in solving heat conduction problems
Abrol et al. Artificial neural network implementation on FPGA chip
Wang et al. A Hammerstein-Wiener recurrent neural network with universal approximation capability
Behera et al. System identification using recurrent neural network
Elreesh et al. FPGA fuzzy controller design for magnetic ball levitation
Atallah et al. A novel approach for data-free, physics-informed neural networks in fluid mechanics using the principle of minimum pressure gradient
Yamada et al. Dynamical linking of positive and negative sentences to goal-oriented robot behavior by hierarchical rnn
Mohamed et al. Brain-inspired machine learning algorithm: Neural network optimization
WO2023175722A1 (en) Learning program and learner

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20927504

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20927504

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP