US20200110432A1 - Reservoir computing operations using multi-mode photonic integrated circuits - Google Patents
Reservoir computing operations using multi-mode photonic integrated circuits Download PDFInfo
- Publication number
- US20200110432A1 US20200110432A1 US16/154,584 US201816154584A US2020110432A1 US 20200110432 A1 US20200110432 A1 US 20200110432A1 US 201816154584 A US201816154584 A US 201816154584A US 2020110432 A1 US2020110432 A1 US 2020110432A1
- Authority
- US
- United States
- Prior art keywords
- coefficients
- electrical signals
- neuronal
- speckle pattern
- circuit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02F—OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
- G02F7/00—Optical analogue/digital converters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06E—OPTICAL COMPUTING DEVICES; COMPUTING DEVICES USING OTHER RADIATIONS WITH SIMILAR PROPERTIES
- G06E1/00—Devices for processing exclusively digital data
- G06E1/02—Devices for processing exclusively digital data operating upon the order or content of the data handled
- G06E1/04—Devices for processing exclusively digital data operating upon the order or content of the data handled for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
- G06E1/045—Matrix or vector computation
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B6/00—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
- G02B6/10—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B6/00—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
- G02B6/10—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
- G02B6/12—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
- G02B6/12004—Combinations of two or more optical elements
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B6/00—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
- G02B6/10—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
- G02B6/12—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
- G02B6/122—Basic optical elements, e.g. light-guiding paths
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06E—OPTICAL COMPUTING DEVICES; COMPUTING DEVICES USING OTHER RADIATIONS WITH SIMILAR PROPERTIES
- G06E3/00—Devices not provided for in group G06E1/00, e.g. for processing analogue or hybrid data
- G06E3/001—Analogue devices in which mathematical operations are carried out with the aid of optical or electro-optical elements
- G06E3/005—Analogue devices in which mathematical operations are carried out with the aid of optical or electro-optical elements using electro-optical or opto-electronic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06E—OPTICAL COMPUTING DEVICES; COMPUTING DEVICES USING OTHER RADIATIONS WITH SIMILAR PROPERTIES
- G06E3/00—Devices not provided for in group G06E1/00, e.g. for processing analogue or hybrid data
- G06E3/008—Matrix or vector computation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
- G06F17/12—Simultaneous equations, e.g. systems of linear equations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
- G06N3/065—Analogue means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
- G06N3/0675—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means using electro-optical, acousto-optical or opto-electronic means
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B6/00—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
- G02B6/10—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
- G02B6/12—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
- G02B2006/12083—Constructional arrangements
- G02B2006/1209—Multimode
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B6/00—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
- G02B6/10—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
- G02B6/12—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
- G02B2006/12133—Functions
- G02B2006/12142—Modulator
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B6/00—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
- G02B6/10—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
- G02B6/12—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
- G02B2006/12133—Functions
- G02B2006/1215—Splitter
Definitions
- This application generally relates to reservoir computing.
- Reservoir computing is a recently developed class of machine learning, and can be useful for time domain applications.
- Reservoir computing techniques can include performing matrix operations, such as linear or nonlinear matrix multiplication. However, when matrix dimensions can be on the order of 1000 s by 100000 s or more, the matrix operations can take a significant amount of computational time and power.
- Embodiments of the present invention provide reservoir computing operations using multi-mode photonic integrated circuits (PICs).
- PICs photonic integrated circuits
- a method for performing an operation can include receiving, by different physical locations of a multi-mode waveguide, an input signal and a plurality of coefficients imposed on laser light.
- the method also can include generating, by the multi-mode waveguide, a speckle pattern based on the different physical locations, the input signal, and the plurality of coefficients.
- the method also can include adjusting at least one of the coefficients based on the speckle pattern.
- the input signal is imposed onto the laser light by an input optical modulator, and the plurality of coefficients respectively are imposed onto the laser light by neuronal optical modulators.
- the input optical modulator and the neuronal optical modulators are coupled to the multi-mode waveguide via respective waveguides.
- optionally adjusting at least one of the coefficients based on the speckle pattern includes generating one or more electrical signals based on a received portion of the speckle pattern.
- an array of photodetectors respectively coupled to the neuronal optical modulators generates the one or more electrical signals based on the received portion of the speckle pattern.
- the coefficient imposed on the laser light by the neuronal optical modulators optionally is adjusted based on the one or more electrical signals.
- the neuronal optical modulators respond nonlinearly to the one or more electrical signals.
- the photodetectors optionally receive the speckle pattern via respective waveguides.
- the method includes generating an output signal based collectively on the one or more electrical signals.
- adjusting the at least one of the coefficients can include adjusting a gain of at least one of the one or more electrical signals based on a comparison of the output signal to the input signal to the output signal.
- the input signal optionally can be time-varying, and the output signal can be predictive of the input signal.
- the laser light can be generated by a continuous-wave, single wavelength laser.
- a circuit for performing an operation can include a multi-mode waveguide configured to receive, at different physical locations, an input signal and a plurality of coefficients imposed on laser light.
- the multi-mode waveguide can be configured to generate a speckle pattern based on the different physical locations, the input signal, and the plurality of coefficients.
- the circuit also can include circuitry configured to adjust at least one of the coefficients based on the speckle pattern.
- the circuit includes an input optical modulator configured to impose the input signal onto the laser light; and the circuitry includes neuronal optical modulators respectively configured to impose the plurality of coefficients onto the laser light.
- the circuit further includes respective waveguides coupling the input optical modulator and the neuronal optical to the multi-mode waveguide.
- the circuitry optionally can be configured to generate one or more electrical signals based on a received portion of the speckle pattern and to adjust the at least one of the coefficients based on the speckle pattern based on the one or more electrical signals.
- the circuitry can include an array of photodetectors coupled to one of the neuronal optical modulators and configured to generate the one or more electrical signals based on the received portion of the speckle pattern, wherein the coefficient imposed on the laser light by that neuronal optical modulator is adjusted based on the one or more electrical signals.
- the neuronal optical modulators optionally can be configured to respond nonlinearly to the one or more electrical signals.
- the circuit includes respective waveguides coupling the photodetectors to the multi-mode waveguide so as to receive the speckle pattern. Additionally, or alternatively, the circuitry optionally is configured to generate an output signal based collectively on the one or more electrical signals.
- the circuitry further optionally is configured to adjust the at least one of the coefficients by adjusting a gain of at least one of the one or more electrical signals based on a comparison of the output signal to the input signal to the output signal.
- the input signal is time-varying, and the output signal is predictive of the input signal.
- the laser light is generated by a continuous-wave, single wavelength laser.
- FIG. 1 schematically illustrates a graphical representation of a prior art reservoir computing network.
- FIG. 2 schematically illustrates a mathematical formulation of a prior art reservoir computing operation.
- FIG. 3A schematically illustrates components of a reservoir computing circuit including a multi-mode photonic integrated circuit, according to one exemplary configuration.
- FIG. 3B schematically illustrates components of another reservoir computing circuit including a multi-mode photonic integrated circuit, according to one exemplary configuration.
- FIGS. 4A-4B schematically illustrate plan and perspective views of a multi-mode waveguide suitable for use in multi-mode photonic integrated circuits such as illustrated in FIGS. 3A-3B , according to one exemplary configuration.
- FIGS. 5A-5B respectively schematically illustrate simulated propagation of light through the multi-mode waveguide based on light input at different physical locations of the waveguide, according to one exemplary configuration.
- FIGS. 6A-6B are plots respectively illustrating complex and real distributions of simulated speckle output from the multi-mode waveguide of FIGS. 4A-4B , according to one exemplary configuration.
- FIGS. 7A-7B are plots respectively illustrating complex and real distributions of random numbers.
- FIGS. 8A-8B are plots illustrating prediction, by reservoir computing operations, of a Mackey-Glass time series.
- FIGS. 9A-9B respectively schematically illustrate matrices formed by a completely interconnected reservoir of neurons and partially interconnected reservoirs of neurons, according to exemplary configurations provided herein.
- FIGS. 10A-10B schematically illustrate operations for forming matrices such as respectively illustrated in FIGS. 9A-9B .
- FIG. 11 schematically illustrates components of another reservoir computing circuit including a multi-mode photonic integrated circuit, according to one exemplary configuration.
- FIG. 12 illustrates steps in an exemplary method for performing reservoir computing operations using a multi-mode photonic integrated circuit, according to one example.
- Embodiments of the present invention provide reservoir computing operations using multi-mode photonic integrated circuits (PICs).
- PICs photonic integrated circuits
- the present multi-mode PICs can execute reservoir computing operations in real-time, with relatively low power consumption, and at relatively high frequencies by performing matrix operations, such as linear or nonlinear matrix multiplications, in the optical domain using a multi-mode waveguide and adjusting the time-varying values of “neurons” in the reservoir computer based on such matrix operations.
- FIG. 1 schematically illustrates a graphical representation of a prior art reservoir computing network.
- one or more inputs u(t) are connected to each of the “neurons” x i (t) of the network with input coefficients a i .
- each neuron x i (t) is connected to all other neurons with network coefficients w ij .
- each of the neurons x i (t) in the network is connected to one or more outputs y(t) with output coefficients b i .
- time-varying values of the neurons x i (t) can be expressed as a column vector x(t) of size n ⁇ 1
- the values of the input coefficients a i can be expressed as a column vector a of size n ⁇ 1
- the values of the network coefficients w ij can be expressed as a square matrix w of size n ⁇ n
- the values of the output coefficients b can be expressed as a row vector b of size 1 ⁇ n.
- FIG. 2 schematically illustrates a mathematical formulation of a prior art reservoir computing operation for adjusting the values of output coefficients b i . This operation follows discrete time steps t by using the formula illustrated in FIG. 2 which can be expressed as:
- ⁇ ( ) is a nonlinear activation function which is sufficiently nonlinear over the range of values produced by the network.
- ⁇ ( ) is a nonlinear activation function which is sufficiently nonlinear over the range of values produced by the network.
- the present multi-mode PIC can involve only O(n) operations at an estimated energy cost of about 100 fJ/operation, resulting in a comparable power dissipation of 40 mW for the same matrix size and clock frequency as the example provided for commercial electronic ICs—a power dissipation savings of several orders of magnitude.
- the present multi-mode PIC only involves O(n) operations, this performance improvement (power dissipation savings) can scale with the number of nodes (neurons) in the network. Indeed, as one example a 4 ⁇ increase in nodes (neurons) can provide an additional order of magnitude reduction in power consumption relative to an electronic IC with the same number of nodes. Furthermore, the present multi-mode PIC can operate at speeds above 30 GHz, and therefore potentially can compute matrix operations an order of magnitude faster, enabling new applications in the RF domain which electronic ICs cannot address.
- FIG. 3A schematically illustrates components of a reservoir computing circuit 300 including a multi-mode photonic integrated circuit, according to one exemplary configuration.
- Reservoir computing circuit 300 illustrated in FIG. 3A includes photonic integrated circuit (PIC) 310 configured to receive at least one input signal Vin, one or more light sources 310 , detector array 330 configured to generate electrical signals based on light output by PIC 310 , optional amplifiers 340 configured to amplify the electrical signals generated by detector array 330 , variable gain amplifiers 350 configured to apply respective output coefficients to the electrical signals from detector array 330 or from optional amplifiers 340 , arithmetic circuit 360 configured to combine the outputs of variable gain amplifiers 350 with one another to generate an output predictive of the input signal Vin, and amplifier gain controller 370 coupled to the variable gain amplifiers 350 and configured to adjust the output coefficients respectively applied by variable gain amplifiers 350 based on a comparison of y(t) to Vin so as to cause y(t) to predict (be similar to or the same as) Vin.
- PIC
- PIC 310 can include splitter 311 , a modulator array including an input optical modulator 312 and a plurality of neuronal optical modulators 313 , and multi-mode waveguide 314 .
- a continuous-wave single-frequency laser source serves as the light source 320 for the entire PIC 310 , and is suitably coupled to splitter 311 of PIC 310 , e.g., via a waveguide (not specifically labeled).
- Splitter 311 can be configured to split the light received from light source 320 between any suitable number of optical waveguides (not specifically labeled) which respectively are coupled to optical modulators of the modulator array.
- splitter 311 can split the light received from light source 320 between n optical waveguides which feed n neuronal optical modulators 313 , as well as input optical modulator 312 .
- the optical modulators of the modulator array can include any suitable type of intensity and/or phase modulator.
- Each optical modulator of the modulator array also receives a respective electrical signal based on which that modulator modulates the intensity and/or phase of the light received from splitter 311 .
- input optical modulator 312 receives electrical input signal Vin, based upon which input optical modulator modulates the light it receives from splitter 311 .
- Vin can be received from any suitable signal source that need not necessarily be considered to be part of reservoir computing circuit 300 .
- input optical modulator 312 can receive Vin via a suitable wired or wireless signaling pathway from a separate signal source (not specifically illustrated).
- Exemplary sources of Vin can include, but are not limited to, radar systems, communication systems, data processing, brain-machine interfaces, and robotics. Further exemplary sources that suitably can be used to provide Vin, and exemplary applications of reservoir computing, can be found in Schrauwen et al., “An overview of reservoir computing: theory, applications and implementations,” ESANN'2007 proceedings—European Symposium on Artificial Neural Networks, Bruges, Belgium, 25-27 Apr. 2007, pages 471-482, ISBN 2-930307-07-2, the entire contents of which are incorporated by reference herein.
- MZM Mach-Zehnder modulator
- modulators such as absorptive modulators based on the Franz-Keldysh effect or the quantum confined Stark effect, on-off keying, or other interferometric modulators, or resonant cavity modulators such as microring modulators, can also suitably be used.
- Neuronal optical modulators 313 respectively receive electrical signals from detectors of detector array 330 or from respective amplifiers 340 , based upon which they respectively modulate the light they receive from splitter 311 .
- the use of the term “neuronal” for optical modulators 313 is intended to indicate that the respective output light intensity from these modulators can be considered to represent the states of the neurons x i (t) of a reservoir computer in a manner such as described further below.
- the optical modulators of the modulator array have a nonlinear response function. That is, in some configurations the intensity or phase of light respectively transmitted by the modulators of the modulator array can be a nonlinear function of the electrical signals respectively applied to those modulators.
- This nonlinear function can be considered to correspond to ⁇ ( ) in equation (1).
- the nonlinear response function is cos( ) 2 , which is the response function of a Mach-Zehnder intensity modulator.
- Native nonlinearity of the modulator can be used to implement ⁇ ( ).
- the modulator can be designed and configured so as to implement a desired nonlinear function ⁇ ( ).
- the outputs from the optical modulators 312 , 313 of the array are then input via respective waveguides (not specifically labeled) to a multi-mode waveguide 314 having a sufficient number of modes, e.g., having at least as many transverse nodes as there are optical inputs to waveguide 314 , e.g., n+1 transverse modes.
- a multi-mode waveguide 314 having a sufficient number of modes, e.g., having at least as many transverse nodes as there are optical inputs to waveguide 314 , e.g., n+1 transverse modes.
- an irregular multi-mode waveguide with sufficient length generates a random optical speckle pattern at the output of the waveguide due to the different propagation constants of the transverse optical modes.
- each waveguide output from a respective optical modulator 312 , 313 has a different physical position entering the multi-mode waveguide 314 . Therefore, the modulated light entering multi-mode waveguide 314 from these different respective positions can excite a different longitudinal mode, optionally with a different relative strength, and therefore produce a respective unique speckle pattern at the output of multi-mode waveguide 314 .
- multi-mode waveguide 314 can include a fiber, or a planar waveguide.
- PIC 310 optionally can include a reticle (not specifically illustrated) to couple the respective outputs of the modulator array into multi-mode waveguide 314 .
- Exemplary characteristics of multi-mode optics 130 are provided elsewhere herein and in U.S. Pat. No. 9,413,372 to Valley, the entire contents of which are incorporated by reference herein.
- For details of another exemplary multi-mode waveguide that suitably can be used in system 300 see Redding et al., “Evanescently coupled multimode spiral spectrometer.” Optica 3.9: 956-962 (2016).
- Multi-mode waveguide 314 can be configured so as to output a speckle pattern based on laser light it receives from input optical modulator 312 and neuronal optical modulators 313 .
- multi-mode waveguide it is meant a passive optical component that supports a plurality of electromagnetic propagation modes for light that is input thereto from different physical locations, in which different of such propagation modes coherently interfere with one another so as to produce a speckle pattern.
- speckle pattern it is meant an irregular, aperiodic pattern in which at least a first portion of the pattern includes an optical intensity profile that is different than an optical intensity profile of at least a second portion of the pattern that is spatially separated from the first portion of the pattern.
- optical intensity profile it is meant the respective intensities (amplitudes) of the light in different regions of space.
- a length and width of the multi-mode waveguide 314 can be selected so as to provide a sufficient number of electromagnetic propagation modes, e.g., at least n+1 electromagnetic propagation modes.
- the width can be selected to provide the n+1 modes, and the length can be selected to provide sufficient mixing of the modes.
- At the end of multi-mode waveguide 314 are a suitable number of output waveguides respectively coupled to photodetectors of detector array 330 , e.g., n output waveguides connected to n photodetectors.
- Each output waveguide receives a portion of the speckle pattern generated by multimode waveguide 314 , which portion can contain contributions from some or all of the modes excited by the inputs to multimode waveguide 314 , that is, by the outputs from the optical modulators 312 , 313 which are input to waveguide 314 at respective physical locations.
- detector array 330 is configured to generate electrical signals based on light output by PIC 310 . More specifically, in some configurations each photodetector of detector array 330 is coupled to multi-mode waveguide 314 so as to generate an electrical signal based on the portion of the speckle pattern received by that photodetector.
- amplifiers 340 are configured to amplify the electrical signals generated by detector array 330 .
- Variable gain amplifiers 350 are configured to apply respective output coefficients to the electrical signals from detector array 330 or from optional amplifiers 340 responsive to control by amplifier gain controller 370 , and arithmetic circuit 360 configured to combine the outputs of variable gain amplifiers 350 with one another to generate and provide to amplifier gain controller 370 an output y(t) predictive of the input signal Vin.
- g 1 ⁇ ( t ) w 11 ⁇ x 1 ⁇ ( t ) + w 12 ⁇ x 2 ⁇ ( t ) + ... ⁇ ⁇ w 1 ⁇ nXn ⁇ ( t ) + a 1 ⁇ u ⁇ ( t ) ( 3 )
- g(t) represents the optical field amplitude in the i th output waveguide
- the elements w ij represent the transmission coefficients from the j th input waveguide to the i th output waveguide.
- These transmission coefficients w ij are determined by the modes of the multi-mode waveguide 314 that are excited based on the locations of respective input waveguides (from the n modulators 313 ) and the locations of the respective output waveguides (to the photodetectors of detector array 330 ), and correspond to the elements of square matrix w in equation 1.
- the input coefficient values a also are determined by the modes of the multi-mode waveguide 314 that are excited based on the location of the input waveguide from input modulator 312 and the locations of the respective output waveguides (to the photodetectors of detector array 330 ), and correspond to the elements of column vector a in equation 1.
- the time-varying value u(t) corresponds to Vin, which is imposed by input modulator 312 .
- the respective electrical output signals from the photodetectors of detector array 330 can be used to control the neuronal optical modulators 313 in the analog domain and to apply the nonlinear function ⁇ ( ) in a feedback loop.
- the photodetectors of detector array 330 optionally can be connected to electronic amplifiers 340 which are coupled to and drive neuronal optical modulators 313 , thereby changing the light intensity at the output of modulator output from x(t) to the next value in the reservoir computer operation sequence, x(t+1), in accordance with equation (1).
- the nonlinear activation function ⁇ ( ) can be applied using any suitable component or combination of components in circuit 300 , for example, can be applied using any suitable combination of neuronal optical modulators 313 , detectors 330 , and/or amplifiers 340 .
- the photodetectors 330 generate voltages linearly proportional to the number of photons they respectively receive
- the electronic amplifiers 340 produce voltages linearly proportional to the currents of the photodetectors they are respectively coupled
- Other nonlinearities can also be utilized in the photodetectors (e.g., photodiodes) 330 and/or electronic amplifiers 340 , for example depending on photodiode bias and amplifier operating point.
- reservoir computing circuit 300 generates y(t) by generating a sum, by arithmetic circuit 360 , of the outputs of an array of variable gain electronic amplifiers 350 which receive the outputs of detector array 300 directly, or indirectly via optional amplifiers 340 .
- the coefficients b are trained during implementation of equations (1) and (2), and the variable gain amplifiers 350 respectively apply the values of b to x(t) and are trained by amplifier gain controller 370 which receives y(t) and adjusts the respective gains of those amplifiers to make y(t) similar to or the same as Vin.
- FIG. 3B schematically illustrates components of another reservoir computing circuit 300 ′ including a multi-mode photonic integrated circuit, according to one exemplary configuration.
- Reservoir computing circuit 300 ′ can be configured similarly as reservoir computing circuit 300 , e.g., includes PIC 310 ′ (including splitter 311 ′, modulator array 312 ′, 313 ′, and multimode waveguide 314 ′), input laser 320 ′, detector array 330 ′, and amplifiers 340 ′ respectively configured similarly as PIC 310 (including splitter 311 , modulator array 312 , 313 , and multimode waveguide 314 ), input laser 320 , detector array 330 , and amplifiers 340 ) illustrated in FIG. 3A .
- PIC 310 ′ including splitter 311 ′, modulator array 312 ′, 313 ′, and multimode waveguide 314 .
- the output of detector array 330 ′ can be provided to amplifiers 340 ′ in a similar manner as detector array 330 provides output to amplifiers 340 such as described with reference to FIG. 3A .
- the respective outputs of amplifiers 340 ′ which collectively can be considered to correspond to y(t) can be digitized using ADCs 380 ′ and provided to processor and memory 370 ′ which can sum the outputs of ADCs 380 ′ to generate output y(t) and compares such outputs to Vin.
- the output of modulator array 313 ′ can be controlled using processor and memory 370 ′ which, based upon the comparison between y(t) and Vin, provides outputs to DACs 390 ′ which provide analog control signals to modulators 313 ′.
- the values of b are set within the processor and memory 370 ′.
- FIGS. 4A-4B schematically illustrate plan and perspective views of a multi-mode waveguide suitable for use in multi-mode photonic integrated circuits such as illustrated in FIGS. 3A-3B , according to one exemplary configuration.
- Multi-mode waveguide 414 can have a generally spiral shape, allowing for a relatively long waveguide within a relatively small footprint.
- Multi-mode waveguide 414 can be coupled to, e.g., can be integrally formed with, a plurality of input waveguides 415 which respectively can be coupled to and receive modulated light from input optical modulator 312 and neuronal optical modulators 313 illustrated in FIGS. 3A-3B .
- Each input waveguide 415 can be coupled to a different physical location at the input of multi-mode waveguide 414 .
- Multi-mode waveguide 413 also can be coupled to, e.g., can be integrally formed with, a plurality of output waveguides 416 which respectively can be coupled to and output light from multimode waveguide 413 to photodetectors of detector array 330 , 330 ′ respectively illustrated in FIGS. 3A-3B .
- Each output waveguide 416 can be coupled to a different physical location at the output of multi-mode waveguide 413 .
- Light input on different ones of the input waveguides 415 can excite different transverse modes of multi-mode waveguide 413 and can generate different speckle patterns received by respective ones of the output waveguides 416 .
- FIGS. 5A-5B respectively schematically illustrate simulated propagation of light through the multi-mode waveguide based on light input at different physical locations of the waveguide, according to one exemplary configuration.
- the simulations were performed with the finite-difference time-domain (FDTD) method on the exemplary 5 input and 5 output multi-mode waveguide device illustrated in FIGS. 4A-4B .
- This particular 5 ⁇ 5 device was chosen to reduce the simulation to a reasonable time, and it should be appreciated that similar results can be obtained for devices having other configurations and greater or lesser numbers of inputs and/or outputs.
- FDTD finite-difference time-domain
- FIG. 4A-4B and 5A-5B illustrate an example spiral multi-mode waveguide 413 coupled to five input waveguides 415 and five output waveguides 416
- the present multi-mode waveguides can have any suitable shape, any suitable number of inputs and outputs, and can support any suitable number of transverse excitation modes.
- a test simulation was performed using a physical implementation of the multi-mode waveguide 413 of FIGS. 4A-4B with random input amplitudes.
- the result from the test simulation was compared to the output patterns from simulations performed such as described with reference to FIGS. 5A-5B to verify the matrix multiplication functionality of the device.
- FIGS. 6A-6B are plots respectively illustrating complex and real distributions of simulated speckle output from the multi-mode waveguide of FIGS. 4A-4B , according to one exemplary configuration.
- FIGS. 6A-6B show the distribution of transmission coefficients from 25 simulations of the multi-mode waveguide of FIGS. 4A-4B . As can be expected, the coefficients follow a complex normal distribution centered at zero.
- FIGS. 7A-7B are plots respectively illustrating complex and real distributions of random numbers, shown for comparison, from which it may be understood that the simulated speckle output follows a similar distribution as the random distribution.
- FIGS. 8A-8B are plots illustrating prediction, by reservoir computing operations, of the Mackey-Glass time series. More specifically, FIG. 8A shows the result of the original reservoir computing program, which compares the target signal with the signal predicted by the reservoir computer operation. In this implementation of the program the random coefficients are uniformly distributed and the nonlinear activation function is a hyperbolic tangent function. In comparison, FIG. 8B shows the result of the modified reservoir computing program, where the random coefficients have been replaced by the complex normal distribution and the nonlinear activation function is cosine squared function. In this particular case, the modified operation ( FIG. 8B ) may be understood to perform better than the original ( FIG. 8A ), with a mean-squared error that is two orders of magnitude lower.
- the number of nodes (neurons) in the reservoir is equal to the number of modulators in array 313 .
- the number of nodes can be scaled up simply by increasing the number of optical inputs and the corresponding width to the multimode waveguide, it potentially can become impractical to do so beyond several hundred nodes because of the area occupied by the modulators and photodetectors.
- the random speckle patterns generated by the multimode waveguide are not only position dependent, but also wavelength dependent. This allows the speckle pattern to be changed with the wavelength of the input light source, such as a tunable laser.
- FIGS. 9A-9B respectively schematically illustrate matrices formed by a completely interconnected reservoir of neurons and partially interconnected reservoirs of neurons, according to exemplary configurations provided herein
- FIGS. 10A-10B schematically illustrate operations for forming matrices such as respectively illustrated in FIGS. 9A-9B
- the laser can be tuned to a number of discrete wavelengths ( ⁇ 1 , ⁇ 2 , . . . ⁇ 16 in this example) to increase the number of nodes in the reservoir computer network.
- each sub-matrix multiplication corresponding to a single one of wavelengths ⁇ 1 , ⁇ 2 , . . .
- FIG. 10A shows in detail an exemplary procedure for scaling the reservoir size with multiple wavelengths.
- a matrix including four sub-matrices each corresponding to a wavelength is used, whereas the example illustrated in FIG. 9A includes sixteen sub-matrices each corresponding to a wavelength.
- Computing each matrix multiplication can include an operation for each different wavelength (four operations being shown in FIG. 10A ).
- each discrete wavelength can produce a different speckle pattern, therefore resulting in a different matrix of transmission coefficients w( ⁇ n ).
- Both the input vector x(t) and output vector g(t) can be split in multiple parts, e.g., two parts in FIG.
- the first step sets the input wavelength to ⁇ 1 and records g 1 (t) while setting the modulator array 313 to the values of x 1 (t) and input modulator 312 to u(t).
- the input wavelength is then changed to ⁇ 2 and the new values of g 1 (t) are added to the previous values while the modulator array 313 is set to the values of x 2 (t) and the input modulator 312 is set to zero output. This process is then repeated for input wavelengths ⁇ 3 and ⁇ 4 to compute g 2 (t).
- the outputs from the photodetector (e.g., photodiode) array can be integrated with a capacitive element, either by the photodetector internal capacitance or external capacitor, to sum the values of g n (t) while the wavelength is changed to ⁇ n .
- these values can be stored in memory, with the summation taking place in the digital electronics.
- FIG. 10B illustrates an exemplary configuration in which a quasi-block diagonal matrix multiplication can be accomplished by using fewer discrete input wavelengths ( ⁇ 1 , ⁇ 2 , . . . ⁇ 7 in this example). Rather than using a discrete wavelength corresponding to each possible matrix, some sub-matrices can be omitted from the computation, e.g., the upper right sub-matrix and lower left sub-matrix are omitted from the computation illustrated in FIG. 10B . This effectively creates a reservoir network where multiple sub-networks are only partially connected.
- a single input waveguide can be used in place of multiple-input waveguides if combined with a suitable multiplexing scheme.
- multiplexing schemes include time-domain multiplexing and wavelength division multiplexing.
- time-domain multiplexing the matrix multiplication can be performed by sequentially encoding the states of the neural network, x i (t), on the neuronal optical modulators while integrating the outputs of the photodetectors.
- wavelength division multiplexing an array of laser sources with different wavelengths can be used in place of a single laser source, because each wavelength will have a unique speckle pattern.
- the laser array can then be either directly modulated, or externally modulated to encode the states of the neural network on the laser output.
- the modulated laser outputs can be combined before entering the multi-mode waveguide, or input to the multi-mode waveguide at different positions.
- FIG. 11 schematically illustrates components of another reservoir computing circuit including a multi-mode photonic integrated circuit, according to one exemplary configuration.
- two parallel photonic reservoir computing circuits 1110 , 1111 each configured similarly as PIC 310 , 310 ′ respectively described above with reference to FIGS. 3A-3B , have the output photodetectors wired in a differential detection scheme.
- differential detection can allow for nonlinear activation functions which have both positive and negative values, similar to the more commonly used hyperbolic tangent, tanh( ).
- a reservoir computing circuit such as described with reference to FIG. 3A-3B or 11 can include a common substrate, such as a silicon substrate, on which any suitable number of laser, modulators, multi-mode waveguide, photodetectors (e.g., photodiodes), amplifiers, and arithmetic circuit are integrated.
- An example of a low-power modulator that suitably can be used with the present reservoir computing circuits is a microring or microdisk resonator modulator in a silicon photonics platform.
- An example of a low-power photodetector that suitably can be used with the present reservoir computing circuits includes a germanium photodiode in a silicon photonics platform, which can be the same platform in which the modulator is provided.
- FIG. 12 illustrates steps in an exemplary method for performing reservoir computing operations using a multi-mode photonic integrated circuit, according to one example.
- method 1200 includes receiving, by different physical locations of a multi-mode waveguide, an input signal and a plurality of coefficients imposed on laser light (operation 1210 ).
- different physical locations of multi-mode waveguide 314 , 314 ′ respectively illustrated in FIGS. 3A-3B can receive laser light it receives via respective waveguides from respective optical modulators.
- the input signal can be imposed onto the laser light by an input optical modulator 312 , 312 ′, and the plurality of coefficients respectively can be imposed onto the laser light by neuronal optical modulators 313 , 313 ′ in a manner such as described elsewhere herein.
- the laser light can be, for example, generated by a continuous-wave, single wavelength laser.
- the input optical modulator 312 , 312 ′ and the neuronal optical modulators 313 , 313 ′ are coupled to the multi-mode waveguide 314 , 314 ′ via respective waveguides.
- Method 1200 illustrated in FIG. 12 also can include generating, by the multi-mode waveguide, a speckle pattern based on the different physical locations, the input signal, and the plurality of coefficients (operation 1220 ).
- multimode waveguide 314 , 314 ′ respectively illustrated in FIGS. 3A-3B can generate such a speckle pattern in a manner such as described elsewhere herein.
- Method 1200 illustrated in FIG. 12 also includes adjusting at least one of the coefficients based on the speckle pattern (operation 1230 ).
- adjusting at least one of the coefficients based on the speckle pattern can include generating one or more electrical signals based on a received portion of the speckle pattern, for example in a manner such as described with reference to FIGS. 3A-3B .
- an array of photodetectors respectively coupled to the neuronal optical modulators can generate the one or more electrical signals based on the received portion of the speckle pattern, and the coefficient imposed on the laser light by the neuronal optical modulators can be adjusted based on the one or more electrical signals. For example, in the configuration illustrated in FIG.
- the coefficients respectively imposed on the laser light by neuronal optical modulators 313 can be adjusted based on control by amplifier gain controller 370 of variable gain amplifiers 350 based upon the outputs of detector array 330 .
- the coefficients respectively imposed on the laser light by neuronal optical modulators 313 ′ can be adjusted based on control by processor and memory 370 ′ of signals that are applied to neuronal optical modulators 313 ′ via DACs 390 ′.
- the neuronal optical modulators e.g., 313 , 313 ′
- the neuronal optical modulators can respond nonlinearly to the one or more electrical signals in a manner such as described elsewhere herein.
- the photodetectors e.g., 330 , 330 ′
- the photodetectors can receive the speckle pattern via respective waveguides.
- an output signal can be generated that is based collectively on the one or more electrical signals.
- arithmetic circuit 360 can sum the outputs of variable gain amplifiers 350 to generate output y(t).
- processor and memory 370 ′ can sum the outputs of ADCs 380 ′ to generate output y(t)
- adjusting the at least one of the coefficients can include adjusting a gain of at least one of the one or more electrical signals based on a comparison of the output signal to the input signal to the output signal. For example, in the configuration of FIG.
- amplifier gain controller 370 can control the electronic signals applied to neuronal optical modulators 313 .
- processor and memory 370 ′ can control the values respectively applied by DACs 390 ′ to neuronal optical modulators 313 ′.
- the input signal can be time-varying, and the output signal can be predictive of the input signal as a result of training the reservoir computing network.
- the energy use for a single operation for the present PIC primarily is that used by an optical modulator (e.g., one of neuronal optical modulators 313 , 313 ′ respectively illustrated in FIGS. 3A-3B ) and by a photodetector (e.g., one of detector array 330 , 330 ′ respectively illustrated in FIGS. 3A-3B ) which generates an electrical output used to control the optical modulator at the next time step; the multi-mode waveguide (e.g., waveguide 314 ) operates passively and therefore uses no energy.
- an optical modulator e.g., one of neuronal optical modulators 313 , 313 ′ respectively illustrated in FIGS. 3A-3B
- a photodetector e.g., one of detector array 330 , 330 ′ respectively illustrated in FIGS. 3A-3B
- the multi-mode waveguide e.g., waveguide 314
- An example of a low-power modulator that suitably can be used with the present reservoir computing circuits is a microring or microdisk resonator modulator in a silicon photonics platform that requires a 1V drive voltage to modulate the optical signal On/Off with 30 GHz bandwidth.
- An example of a low-power photodetector that suitably can be used with the present reservoir computing circuits includes a germanium photodiode in a silicon photonics platform with 50 GHz bandwidth and a quantum efficiency of 0.9.
- the energy used to perform a reservoir computing operation can be considered to be or include that of converting the light input to the photodetector at the current time step to the light output from the modulator at the next time step.
- the capacitance of the modulator can be expressed as:
- the total charge required to produce the 1V drive can be calculated from:
- the total number of electrons needed to produce this charge then can be:
- the energy cost per operation of the optical devices using values from typical silicon photonics foundries is estimated to be on the order of 100 fJ. It should appreciated that other such value is only an estimate and can depend on the particular configuration used.
- the present PICs, and reservoir computing circuits incorporating such PICs solve the problem of power demand for large scale computing operations, such as matrix multiplications in artificial intelligence applications.
- the higher operating frequency of the present PIC as compared to electronic circuitry for performing matrix multiplications, can enable new applications in radio frequency (RF) signal processing which may not be achieved due to the low clock frequencies of conventional digital ICs.
- RF radio frequency
- industrial and commercial applications of the present PICs and reservoir computing circuits can include, but are not limited to, the applications of reservoir computing in general.
- these applications include speech recognition, time series prediction, signal classification, and control systems (e.g. robotics).
- control systems e.g. robotics
- signal classification can be performed on RF signals up to the Nyquist limit of 1 ⁇ 2 the clock frequency of the present PICs. With current foundry specifications of 30 GHz bandwidth for modulators and photodetectors, this translates to applying classification tasks to RF signals up to 15 GHz.
- Another example is in control systems, where systems that have dynamics at sub-nanosecond time scales can be addressed by the present PICs.
Abstract
Description
- This application generally relates to reservoir computing.
- Reservoir computing is a recently developed class of machine learning, and can be useful for time domain applications. Reservoir computing techniques can include performing matrix operations, such as linear or nonlinear matrix multiplication. However, when matrix dimensions can be on the order of 1000 s by 100000 s or more, the matrix operations can take a significant amount of computational time and power.
- Embodiments of the present invention provide reservoir computing operations using multi-mode photonic integrated circuits (PICs).
- Under one aspect, a method for performing an operation is provided. The method can include receiving, by different physical locations of a multi-mode waveguide, an input signal and a plurality of coefficients imposed on laser light. The method also can include generating, by the multi-mode waveguide, a speckle pattern based on the different physical locations, the input signal, and the plurality of coefficients. The method also can include adjusting at least one of the coefficients based on the speckle pattern.
- In some configurations, optionally the input signal is imposed onto the laser light by an input optical modulator, and the plurality of coefficients respectively are imposed onto the laser light by neuronal optical modulators. Optionally, the input optical modulator and the neuronal optical modulators are coupled to the multi-mode waveguide via respective waveguides. Additionally, or alternatively, optionally adjusting at least one of the coefficients based on the speckle pattern includes generating one or more electrical signals based on a received portion of the speckle pattern. Optionally, an array of photodetectors respectively coupled to the neuronal optical modulators generates the one or more electrical signals based on the received portion of the speckle pattern. The coefficient imposed on the laser light by the neuronal optical modulators optionally is adjusted based on the one or more electrical signals. Optionally, the neuronal optical modulators respond nonlinearly to the one or more electrical signals. Additionally, or alternatively, the photodetectors optionally receive the speckle pattern via respective waveguides. In some configurations, optionally the method includes generating an output signal based collectively on the one or more electrical signals. Optionally, adjusting the at least one of the coefficients can include adjusting a gain of at least one of the one or more electrical signals based on a comparison of the output signal to the input signal to the output signal. The input signal optionally can be time-varying, and the output signal can be predictive of the input signal. As a further or alternative option, the laser light can be generated by a continuous-wave, single wavelength laser.
- Under another aspect, a circuit for performing an operation is provided. The circuit can include a multi-mode waveguide configured to receive, at different physical locations, an input signal and a plurality of coefficients imposed on laser light. The multi-mode waveguide can be configured to generate a speckle pattern based on the different physical locations, the input signal, and the plurality of coefficients. The circuit also can include circuitry configured to adjust at least one of the coefficients based on the speckle pattern.
- In some configurations, the circuit includes an input optical modulator configured to impose the input signal onto the laser light; and the circuitry includes neuronal optical modulators respectively configured to impose the plurality of coefficients onto the laser light. Optionally, the circuit further includes respective waveguides coupling the input optical modulator and the neuronal optical to the multi-mode waveguide. Additionally, or alternatively, the circuitry optionally can be configured to generate one or more electrical signals based on a received portion of the speckle pattern and to adjust the at least one of the coefficients based on the speckle pattern based on the one or more electrical signals. Optionally, the circuitry can include an array of photodetectors coupled to one of the neuronal optical modulators and configured to generate the one or more electrical signals based on the received portion of the speckle pattern, wherein the coefficient imposed on the laser light by that neuronal optical modulator is adjusted based on the one or more electrical signals. Additionally, or alternatively, the neuronal optical modulators optionally can be configured to respond nonlinearly to the one or more electrical signals. Additionally, or alternatively, the circuit includes respective waveguides coupling the photodetectors to the multi-mode waveguide so as to receive the speckle pattern. Additionally, or alternatively, the circuitry optionally is configured to generate an output signal based collectively on the one or more electrical signals. Additionally, or alternatively, the circuitry further optionally is configured to adjust the at least one of the coefficients by adjusting a gain of at least one of the one or more electrical signals based on a comparison of the output signal to the input signal to the output signal. Optionally, the input signal is time-varying, and the output signal is predictive of the input signal. Additionally, or alternatively, optionally the laser light is generated by a continuous-wave, single wavelength laser.
- The patent or application file includes at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
-
FIG. 1 schematically illustrates a graphical representation of a prior art reservoir computing network. -
FIG. 2 schematically illustrates a mathematical formulation of a prior art reservoir computing operation. -
FIG. 3A schematically illustrates components of a reservoir computing circuit including a multi-mode photonic integrated circuit, according to one exemplary configuration. -
FIG. 3B schematically illustrates components of another reservoir computing circuit including a multi-mode photonic integrated circuit, according to one exemplary configuration. -
FIGS. 4A-4B schematically illustrate plan and perspective views of a multi-mode waveguide suitable for use in multi-mode photonic integrated circuits such as illustrated inFIGS. 3A-3B , according to one exemplary configuration. -
FIGS. 5A-5B respectively schematically illustrate simulated propagation of light through the multi-mode waveguide based on light input at different physical locations of the waveguide, according to one exemplary configuration. -
FIGS. 6A-6B are plots respectively illustrating complex and real distributions of simulated speckle output from the multi-mode waveguide ofFIGS. 4A-4B , according to one exemplary configuration. -
FIGS. 7A-7B are plots respectively illustrating complex and real distributions of random numbers. -
FIGS. 8A-8B are plots illustrating prediction, by reservoir computing operations, of a Mackey-Glass time series. -
FIGS. 9A-9B respectively schematically illustrate matrices formed by a completely interconnected reservoir of neurons and partially interconnected reservoirs of neurons, according to exemplary configurations provided herein. -
FIGS. 10A-10B schematically illustrate operations for forming matrices such as respectively illustrated inFIGS. 9A-9B . -
FIG. 11 schematically illustrates components of another reservoir computing circuit including a multi-mode photonic integrated circuit, according to one exemplary configuration. -
FIG. 12 illustrates steps in an exemplary method for performing reservoir computing operations using a multi-mode photonic integrated circuit, according to one example. - Embodiments of the present invention provide reservoir computing operations using multi-mode photonic integrated circuits (PICs). The present multi-mode PICs can execute reservoir computing operations in real-time, with relatively low power consumption, and at relatively high frequencies by performing matrix operations, such as linear or nonlinear matrix multiplications, in the optical domain using a multi-mode waveguide and adjusting the time-varying values of “neurons” in the reservoir computer based on such matrix operations.
-
FIG. 1 schematically illustrates a graphical representation of a prior art reservoir computing network. In it, one or more inputs u(t) are connected to each of the “neurons” xi(t) of the network with input coefficients ai. Within the network of neurons, each neuron xi(t) is connected to all other neurons with network coefficients wij. Additionally, each of the neurons xi(t) in the network is connected to one or more outputs y(t) with output coefficients bi. Note that the time-varying values of the neurons xi(t) can be expressed as a column vector x(t) of size n×1, the values of the input coefficients ai can be expressed as a column vector a of size n×1, the values of the network coefficients wij can be expressed as a square matrix w of size n×n, and the values of the output coefficients b can be expressed as a row vector b ofsize 1×n. - One aspect of such a reservoir computing network is that the input coefficients ai and network coefficients wij are random and fixed. The only training required for such a reservoir computing network takes place at the output coefficients bi, which are adjusted to produce the desired system response.
FIG. 2 schematically illustrates a mathematical formulation of a prior art reservoir computing operation for adjusting the values of output coefficients bi. This operation follows discrete time steps t by using the formula illustrated inFIG. 2 which can be expressed as: -
x(t+1)=ƒ(w·x(t)+au(t)) (1) -
y(t)=b·x(t) (2) - In equation (1), ƒ( ) is a nonlinear activation function which is sufficiently nonlinear over the range of values produced by the network. For further details of reservoir computing and nonlinear activation functions, see Schrauwen et al., “An overview of reservoir computing: theory, applications and implementations,” ESANN'2007 proceedings—European Symposium on Artificial Neural Networks, Bruges, Belgium, 25-27 Apr. 2007, pages 471-482, ISBN 2-930307-07-2, the entire contents of which are incorporated by reference herein. A commonly used nonlinear activation function is the hyperbolic tangent, tanh( ). However, many other nonlinear functions can achieve the desired result. For further details of exemplary nonlinear functions that can be used in reservoir computing, see Dong et al., “Scaling up echo-state networks with multiple light scattering,” arXiv: 1609.05204v3, 5 pages (submitted on Sep. 15, 2016 and last updated Feb. 13, 2018), the entire contents of which are incorporated by reference herein.
- Similar to other machine learning operations, most of the computational cost in a reservoir computing network such as illustrated in
FIG. 1 implementing an operation such as illustrated inFIG. 2 and expressed in equations (1) and (2) occurs in the matrix multiplications at each step, which computational cost can be relatively large. In particular, the operation w·x(t) between the n×n matrix w and the n×1 column vector x(t) represent most of the computational cost. In exemplary electronics, evaluating this matrix product w·x(t) can involve at least O(n2) operations, which can carry an estimated energy cost of about 1 pJ/operation. As used herein, the term “about” means within an order of magnitude of the stated value. For commercial electronic integrated circuits (ICs) which process 128×128 matrices, this can result in power dissipation on the order of 50 W at 3 GHz clock frequency, not including data transfer and supporting subsystems which can increase the total system power by a factor of ten. In comparison, and as described in greater detail herein, the present multi-mode PIC can involve only O(n) operations at an estimated energy cost of about 100 fJ/operation, resulting in a comparable power dissipation of 40 mW for the same matrix size and clock frequency as the example provided for commercial electronic ICs—a power dissipation savings of several orders of magnitude. Because the present multi-mode PIC only involves O(n) operations, this performance improvement (power dissipation savings) can scale with the number of nodes (neurons) in the network. Indeed, as one example a 4× increase in nodes (neurons) can provide an additional order of magnitude reduction in power consumption relative to an electronic IC with the same number of nodes. Furthermore, the present multi-mode PIC can operate at speeds above 30 GHz, and therefore potentially can compute matrix operations an order of magnitude faster, enabling new applications in the RF domain which electronic ICs cannot address. -
FIG. 3A schematically illustrates components of areservoir computing circuit 300 including a multi-mode photonic integrated circuit, according to one exemplary configuration.Reservoir computing circuit 300 illustrated inFIG. 3A includes photonic integrated circuit (PIC) 310 configured to receive at least one input signal Vin, one or morelight sources 310,detector array 330 configured to generate electrical signals based on light output byPIC 310,optional amplifiers 340 configured to amplify the electrical signals generated bydetector array 330,variable gain amplifiers 350 configured to apply respective output coefficients to the electrical signals fromdetector array 330 or fromoptional amplifiers 340,arithmetic circuit 360 configured to combine the outputs ofvariable gain amplifiers 350 with one another to generate an output predictive of the input signal Vin, andamplifier gain controller 370 coupled to thevariable gain amplifiers 350 and configured to adjust the output coefficients respectively applied byvariable gain amplifiers 350 based on a comparison of y(t) to Vin so as to cause y(t) to predict (be similar to or the same as) Vin. -
PIC 310 can includesplitter 311, a modulator array including an inputoptical modulator 312 and a plurality of neuronaloptical modulators 313, andmulti-mode waveguide 314. In one nonlimiting example, a continuous-wave single-frequency laser source serves as thelight source 320 for theentire PIC 310, and is suitably coupled tosplitter 311 ofPIC 310, e.g., via a waveguide (not specifically labeled).Splitter 311 can be configured to split the light received fromlight source 320 between any suitable number of optical waveguides (not specifically labeled) which respectively are coupled to optical modulators of the modulator array. For example,splitter 311 can split the light received fromlight source 320 between n optical waveguides which feed n neuronaloptical modulators 313, as well as inputoptical modulator 312. - The optical modulators of the modulator array, e.g., input
optical modulator 312 and neuronaloptical modulators 313, can include any suitable type of intensity and/or phase modulator. Each optical modulator of the modulator array also receives a respective electrical signal based on which that modulator modulates the intensity and/or phase of the light received fromsplitter 311. For example, inputoptical modulator 312 receives electrical input signal Vin, based upon which input optical modulator modulates the light it receives fromsplitter 311. Vin can be received from any suitable signal source that need not necessarily be considered to be part ofreservoir computing circuit 300. For example, inputoptical modulator 312 can receive Vin via a suitable wired or wireless signaling pathway from a separate signal source (not specifically illustrated). Exemplary sources of Vin can include, but are not limited to, radar systems, communication systems, data processing, brain-machine interfaces, and robotics. Further exemplary sources that suitably can be used to provide Vin, and exemplary applications of reservoir computing, can be found in Schrauwen et al., “An overview of reservoir computing: theory, applications and implementations,” ESANN'2007 proceedings—European Symposium on Artificial Neural Networks, Bruges, Belgium, 25-27 Apr. 2007, pages 471-482, ISBN 2-930307-07-2, the entire contents of which are incorporated by reference herein. For further details of an example Mach-Zehnder modulator (MZM) that can be used in the modulator array to impose signals on laser light, see U.S. Patent Publication No. 2018/0165248 to Valley et al., the entire contents of which are incorporated by reference herein. Other modulators, such as absorptive modulators based on the Franz-Keldysh effect or the quantum confined Stark effect, on-off keying, or other interferometric modulators, or resonant cavity modulators such as microring modulators, can also suitably be used. - Neuronal
optical modulators 313 respectively receive electrical signals from detectors ofdetector array 330 or fromrespective amplifiers 340, based upon which they respectively modulate the light they receive fromsplitter 311. In this regard, note that the use of the term “neuronal” foroptical modulators 313 is intended to indicate that the respective output light intensity from these modulators can be considered to represent the states of the neurons xi(t) of a reservoir computer in a manner such as described further below. In some configurations, the optical modulators of the modulator array have a nonlinear response function. That is, in some configurations the intensity or phase of light respectively transmitted by the modulators of the modulator array can be a nonlinear function of the electrical signals respectively applied to those modulators. This nonlinear function can be considered to correspond to ƒ( ) in equation (1). In one nonlimiting example, the nonlinear response function is cos( )2, which is the response function of a Mach-Zehnder intensity modulator. Native nonlinearity of the modulator can be used to implement ƒ( ). Alternatively, the modulator can be designed and configured so as to implement a desired nonlinear function ƒ( ). - The outputs from the
optical modulators multi-mode waveguide 314 having a sufficient number of modes, e.g., having at least as many transverse nodes as there are optical inputs towaveguide 314, e.g., n+1 transverse modes. For example, an irregular multi-mode waveguide with sufficient length generates a random optical speckle pattern at the output of the waveguide due to the different propagation constants of the transverse optical modes. For further details, see Valley et al., “Multimode waveguide speckle patterns for compressive sensing,” Optics Letters 41, 2529-2532 (2016), the entire contents of which are incorporated by reference herein. In the configuration illustrated inFIG. 3 , each waveguide output from a respectiveoptical modulator multi-mode waveguide 314. Therefore, the modulated light enteringmulti-mode waveguide 314 from these different respective positions can excite a different longitudinal mode, optionally with a different relative strength, and therefore produce a respective unique speckle pattern at the output ofmulti-mode waveguide 314. - In various configurations,
multi-mode waveguide 314 can include a fiber, or a planar waveguide.PIC 310 optionally can include a reticle (not specifically illustrated) to couple the respective outputs of the modulator array intomulti-mode waveguide 314. Exemplary characteristics of multi-mode optics 130 are provided elsewhere herein and in U.S. Pat. No. 9,413,372 to Valley, the entire contents of which are incorporated by reference herein. For details of another exemplary multi-mode waveguide that suitably can be used insystem 300, see Redding et al., “Evanescently coupled multimode spiral spectrometer.” Optica 3.9: 956-962 (2016). For another example of a waveguide that suitably can be used asmulti-mode waveguide 314, see Piels et al., “Compact silicon multimode waveguide spectrometer with enhanced bandwidth,”Scientific Reports 7, 1-7 (2017), the entire contents of which are incorporated by reference herein. -
Multi-mode waveguide 314 can be configured so as to output a speckle pattern based on laser light it receives from inputoptical modulator 312 and neuronaloptical modulators 313. By “multi-mode waveguide” it is meant a passive optical component that supports a plurality of electromagnetic propagation modes for light that is input thereto from different physical locations, in which different of such propagation modes coherently interfere with one another so as to produce a speckle pattern. By “speckle pattern” it is meant an irregular, aperiodic pattern in which at least a first portion of the pattern includes an optical intensity profile that is different than an optical intensity profile of at least a second portion of the pattern that is spatially separated from the first portion of the pattern. By “optical intensity profile” it is meant the respective intensities (amplitudes) of the light in different regions of space. - A length and width of the
multi-mode waveguide 314 can be selected so as to provide a sufficient number of electromagnetic propagation modes, e.g., at least n+1 electromagnetic propagation modes. For example, the width can be selected to provide the n+1 modes, and the length can be selected to provide sufficient mixing of the modes. At the end ofmulti-mode waveguide 314 are a suitable number of output waveguides respectively coupled to photodetectors ofdetector array 330, e.g., n output waveguides connected to n photodetectors. Each output waveguide receives a portion of the speckle pattern generated bymultimode waveguide 314, which portion can contain contributions from some or all of the modes excited by the inputs tomultimode waveguide 314, that is, by the outputs from theoptical modulators detector array 330 is configured to generate electrical signals based on light output byPIC 310. More specifically, in some configurations each photodetector ofdetector array 330 is coupled tomulti-mode waveguide 314 so as to generate an electrical signal based on the portion of the speckle pattern received by that photodetector. Optionally,amplifiers 340 are configured to amplify the electrical signals generated bydetector array 330.Variable gain amplifiers 350 are configured to apply respective output coefficients to the electrical signals fromdetector array 330 or fromoptional amplifiers 340 responsive to control byamplifier gain controller 370, andarithmetic circuit 360 configured to combine the outputs ofvariable gain amplifiers 350 with one another to generate and provide toamplifier gain controller 370 an output y(t) predictive of the input signal Vin. - Operation for the
PIC 310 withinreservoir computer circuit 300 can be described as follows. For each waveguide input to the multi-mode waveguide 314 (from the modulator array), the speckle pattern generated bymulti-mode waveguide 314 distributes light randomly across the output waveguides (to the detector array). Therefore, the optical fields in each output waveguide can be expressed as: -
- In equations (3)-(5), g(t) represents the optical field amplitude in the ith output waveguide, and the elements wij represent the transmission coefficients from the jth input waveguide to the ith output waveguide. These transmission coefficients wij are determined by the modes of the
multi-mode waveguide 314 that are excited based on the locations of respective input waveguides (from the n modulators 313) and the locations of the respective output waveguides (to the photodetectors of detector array 330), and correspond to the elements of square matrix w inequation 1. The input coefficient values a also are determined by the modes of themulti-mode waveguide 314 that are excited based on the location of the input waveguide frominput modulator 312 and the locations of the respective output waveguides (to the photodetectors of detector array 330), and correspond to the elements of column vector a inequation 1. The time-varying value u(t) corresponds to Vin, which is imposed byinput modulator 312. From equations (3)-(5), it may be understood that, responsive to inputs from inputoptical modulator 312 and neuronaloptical modulators 313,multi-mode waveguide 314 generates the function g(t)=w·x(t)+au(t), which corresponds to the argument of the nonlinear function ƒ( ) in equation (1), passively and without any power dissipation during this computation step. Applying the nonlinear function ƒ( ) to the argument g(t)=w·x(t)+au(t) yields the next time step values x(t+1) for the set of reservoir computer neurons (nodes), in accordance with equation (1). - In the exemplary configuration illustrated in
FIG. 3A , the respective electrical output signals from the photodetectors ofdetector array 330, corresponding to respective elements gi(t), can be used to control the neuronaloptical modulators 313 in the analog domain and to apply the nonlinear function ƒ( ) in a feedback loop. For example, the photodetectors ofdetector array 330 optionally can be connected toelectronic amplifiers 340 which are coupled to and drive neuronaloptical modulators 313, thereby changing the light intensity at the output of modulator output from x(t) to the next value in the reservoir computer operation sequence, x(t+1), in accordance with equation (1). The nonlinear activation function ƒ( ) can be applied using any suitable component or combination of components incircuit 300, for example, can be applied using any suitable combination of neuronaloptical modulators 313,detectors 330, and/oramplifiers 340. In one example in which the nonlinear activation function is applied substantially entirely using neuronaloptical modulators 313, thephotodetectors 330 generate voltages linearly proportional to the number of photons they respectively receive, theelectronic amplifiers 340 produce voltages linearly proportional to the currents of the photodetectors they are respectively coupled, and each neuronaloptical modulator 313 applies a nonlinear activation function ƒ( ) to the value g(t) that it receives viadetector array 330 andoptional amplifiers 340, e.g., ƒ( )=cos( )2 for a Mach-Zehnder modulator. Other nonlinearities can also be utilized in the photodetectors (e.g., photodiodes) 330 and/orelectronic amplifiers 340, for example depending on photodiode bias and amplifier operating point. - As noted further above, the reservoir computing operations expressed in equations (1) and (2) further include generation of the reservoir computing circuit output, y(t)=b·x(t) in accordance with equation (2), where y(t) is predictive of u(t), which in
FIG. 3A corresponds to Vin applied to inputoptical modulator 312. In the configuration ofFIG. 3A ,reservoir computing circuit 300 generates y(t) by generating a sum, byarithmetic circuit 360, of the outputs of an array of variable gainelectronic amplifiers 350 which receive the outputs ofdetector array 300 directly, or indirectly viaoptional amplifiers 340. For example, the coefficients b are trained during implementation of equations (1) and (2), and thevariable gain amplifiers 350 respectively apply the values of b to x(t) and are trained byamplifier gain controller 370 which receives y(t) and adjusts the respective gains of those amplifiers to make y(t) similar to or the same as Vin. - As another option, the electrical outputs from the photodetectors of
detector array 330, which receive the respective elements g(t) frommulti-mode waveguide 314, can be digitized with traditional electronic analog to digital converters (ADCs) and remainder of the reservoir computing operation computed in the digital domain. For example,FIG. 3B schematically illustrates components of anotherreservoir computing circuit 300′ including a multi-mode photonic integrated circuit, according to one exemplary configuration.Reservoir computing circuit 300′ can be configured similarly asreservoir computing circuit 300, e.g., includesPIC 310′ (includingsplitter 311′,modulator array 312′, 313′, andmultimode waveguide 314′),input laser 320′,detector array 330′, andamplifiers 340′ respectively configured similarly as PIC 310 (includingsplitter 311,modulator array input laser 320,detector array 330, and amplifiers 340) illustrated inFIG. 3A . Inreservoir computing circuit 300′ the output ofdetector array 330′ can be provided toamplifiers 340′ in a similar manner asdetector array 330 provides output toamplifiers 340 such as described with reference toFIG. 3A . However, in the configuration illustrated inFIG. 3B , the respective outputs ofamplifiers 340′, which collectively can be considered to correspond to y(t), can be digitized usingADCs 380′ and provided to processor andmemory 370′ which can sum the outputs ofADCs 380′ to generate output y(t) and compares such outputs to Vin. The output ofmodulator array 313′ can be controlled using processor andmemory 370′ which, based upon the comparison between y(t) and Vin, provides outputs to DACs 390′ which provide analog control signals tomodulators 313′. In such a configuration, the values of b are set within the processor andmemory 370′. -
FIGS. 4A-4B schematically illustrate plan and perspective views of a multi-mode waveguide suitable for use in multi-mode photonic integrated circuits such as illustrated inFIGS. 3A-3B , according to one exemplary configuration.Multi-mode waveguide 414 can have a generally spiral shape, allowing for a relatively long waveguide within a relatively small footprint.Multi-mode waveguide 414 can be coupled to, e.g., can be integrally formed with, a plurality ofinput waveguides 415 which respectively can be coupled to and receive modulated light from inputoptical modulator 312 and neuronaloptical modulators 313 illustrated inFIGS. 3A-3B . Eachinput waveguide 415 can be coupled to a different physical location at the input ofmulti-mode waveguide 414. Multi-mode waveguide 413 also can be coupled to, e.g., can be integrally formed with, a plurality of output waveguides 416 which respectively can be coupled to and output light from multimode waveguide 413 to photodetectors ofdetector array FIGS. 3A-3B . Each output waveguide 416 can be coupled to a different physical location at the output of multi-mode waveguide 413. Light input on different ones of theinput waveguides 415 can excite different transverse modes of multi-mode waveguide 413 and can generate different speckle patterns received by respective ones of the output waveguides 416. - For example,
FIGS. 5A-5B respectively schematically illustrate simulated propagation of light through the multi-mode waveguide based on light input at different physical locations of the waveguide, according to one exemplary configuration. The simulations were performed with the finite-difference time-domain (FDTD) method on the exemplary 5 input and 5 output multi-mode waveguide device illustrated inFIGS. 4A-4B . This particular 5×5 device was chosen to reduce the simulation to a reasonable time, and it should be appreciated that similar results can be obtained for devices having other configurations and greater or lesser numbers of inputs and/or outputs. In the simulations respectively illustrated inFIGS. 5A-5B , either the first or the fourth one of theinput waveguides 415 was excited and the simulation was run until the light input on thatwaveguide 415 had propagated through the multi-mode waveguide 413 and output waveguides 416, where the output amplitudes were recorded. It may be understood fromFIGS. 5A and 5B that for light respectively input on the first input waveguide andfourth input waveguide 415, the multi-mode waveguide imparted a distinct (unique) output transfer function. AlthoughFIGS. 4A-4B and 5A-5B illustrate an example spiral multi-mode waveguide 413 coupled to fiveinput waveguides 415 and five output waveguides 416, it should be appreciated that the present multi-mode waveguides can have any suitable shape, any suitable number of inputs and outputs, and can support any suitable number of transverse excitation modes. - A test simulation was performed using a physical implementation of the multi-mode waveguide 413 of
FIGS. 4A-4B with random input amplitudes. The result from the test simulation was compared to the output patterns from simulations performed such as described with reference toFIGS. 5A-5B to verify the matrix multiplication functionality of the device. - To assess whether or not the random speckle generated by multimode waveguides is viable for reservoir computing, the distribution of transmission coefficients was measured and a representative random distribution was then tested in a simple reservoir computing program.
FIGS. 6A-6B are plots respectively illustrating complex and real distributions of simulated speckle output from the multi-mode waveguide ofFIGS. 4A-4B , according to one exemplary configuration.FIGS. 6A-6B show the distribution of transmission coefficients from 25 simulations of the multi-mode waveguide ofFIGS. 4A-4B . As can be expected, the coefficients follow a complex normal distribution centered at zero.FIGS. 7A-7B are plots respectively illustrating complex and real distributions of random numbers, shown for comparison, from which it may be understood that the simulated speckle output follows a similar distribution as the random distribution. - The randomly generated coefficients following the complex normal distribution were tested in a simple reservoir computing program with the task of predicting a Mackey-Glass time series.
FIGS. 8A-8B are plots illustrating prediction, by reservoir computing operations, of the Mackey-Glass time series. More specifically,FIG. 8A shows the result of the original reservoir computing program, which compares the target signal with the signal predicted by the reservoir computer operation. In this implementation of the program the random coefficients are uniformly distributed and the nonlinear activation function is a hyperbolic tangent function. In comparison,FIG. 8B shows the result of the modified reservoir computing program, where the random coefficients have been replaced by the complex normal distribution and the nonlinear activation function is cosine squared function. In this particular case, the modified operation (FIG. 8B ) may be understood to perform better than the original (FIG. 8A ), with a mean-squared error that is two orders of magnitude lower. - In PIC configurations such as illustrated in
FIGS. 3A-3B , the number of nodes (neurons) in the reservoir is equal to the number of modulators inarray 313. Although the number of nodes can be scaled up simply by increasing the number of optical inputs and the corresponding width to the multimode waveguide, it potentially can become impractical to do so beyond several hundred nodes because of the area occupied by the modulators and photodetectors. However, the random speckle patterns generated by the multimode waveguide are not only position dependent, but also wavelength dependent. This allows the speckle pattern to be changed with the wavelength of the input light source, such as a tunable laser. - For example,
FIGS. 9A-9B respectively schematically illustrate matrices formed by a completely interconnected reservoir of neurons and partially interconnected reservoirs of neurons, according to exemplary configurations provided herein, andFIGS. 10A-10B schematically illustrate operations for forming matrices such as respectively illustrated inFIGS. 9A-9B . As shown inFIG. 9A , the laser can be tuned to a number of discrete wavelengths (λ1, λ2, . . . λ16 in this example) to increase the number of nodes in the reservoir computer network. Instead of operating in a real time mode, each sub-matrix multiplication corresponding to a single one of wavelengths λ1, λ2, . . . λ16 can be processed sequentially; for example,FIG. 10A shows in detail an exemplary procedure for scaling the reservoir size with multiple wavelengths. In the example illustrated inFIG. 10A , a matrix including four sub-matrices each corresponding to a wavelength is used, whereas the example illustrated inFIG. 9A includes sixteen sub-matrices each corresponding to a wavelength. Computing each matrix multiplication can include an operation for each different wavelength (four operations being shown inFIG. 10A ). For example, each discrete wavelength can produce a different speckle pattern, therefore resulting in a different matrix of transmission coefficients w(λn). Both the input vector x(t) and output vector g(t) can be split in multiple parts, e.g., two parts inFIG. 10A : x1(t), x2(t), g1 (t), and g2(t). The first step sets the input wavelength to λ1 and records g1(t) while setting themodulator array 313 to the values of x1(t) andinput modulator 312 to u(t). The input wavelength is then changed to λ2 and the new values of g1 (t) are added to the previous values while themodulator array 313 is set to the values of x2(t) and theinput modulator 312 is set to zero output. This process is then repeated for input wavelengths λ3 and λ4 to compute g2(t). In some configurations, the outputs from the photodetector (e.g., photodiode) array can be integrated with a capacitive element, either by the photodetector internal capacitance or external capacitor, to sum the values of gn(t) while the wavelength is changed to λn. In other configurations, these values can be stored in memory, with the summation taking place in the digital electronics. - Alternative methods of scaling reservoir network size are also considered. In some cases, a fully connected network of neurons may not be necessary.
FIG. 10B illustrates an exemplary configuration in which a quasi-block diagonal matrix multiplication can be accomplished by using fewer discrete input wavelengths (λ1, λ2, . . . λ7 in this example). Rather than using a discrete wavelength corresponding to each possible matrix, some sub-matrices can be omitted from the computation, e.g., the upper right sub-matrix and lower left sub-matrix are omitted from the computation illustrated inFIG. 10B . This effectively creates a reservoir network where multiple sub-networks are only partially connected. - While this description has primarily focused on a particular configuration which uses multiple input and multiple output waveguides to a sufficiently long multi-mode waveguide, other configurations can be used. For example, a single input waveguide can be used in place of multiple-input waveguides if combined with a suitable multiplexing scheme. Examples of possible multiplexing schemes include time-domain multiplexing and wavelength division multiplexing. For time-domain multiplexing, the matrix multiplication can be performed by sequentially encoding the states of the neural network, xi(t), on the neuronal optical modulators while integrating the outputs of the photodetectors. For wavelength division multiplexing, an array of laser sources with different wavelengths can be used in place of a single laser source, because each wavelength will have a unique speckle pattern. The laser array can then be either directly modulated, or externally modulated to encode the states of the neural network on the laser output. The modulated laser outputs can be combined before entering the multi-mode waveguide, or input to the multi-mode waveguide at different positions.
- Other variations of the detection scheme may also be used to achieve unique nonlinear activation functions. For example,
FIG. 11 schematically illustrates components of another reservoir computing circuit including a multi-mode photonic integrated circuit, according to one exemplary configuration. In the configuration shown inFIG. 11 , two parallel photonicreservoir computing circuits 1110, 1111, each configured similarly asPIC FIGS. 3A-3B , have the output photodetectors wired in a differential detection scheme. Using differential detection can allow for nonlinear activation functions which have both positive and negative values, similar to the more commonly used hyperbolic tangent, tanh( ). - Note that any suitable arrangement and types of laser, optical modulators, multi-mode waveguides, photodetectors, amplifiers, arithmetic circuits and substrate(s) carrying such elements can be used. For example, any suitable combination of elements of the present circuits can be integrated in one or more suitable substrates. In one configuration, a reservoir computing circuit such as described with reference to
FIG. 3A-3B or 11 can include a common substrate, such as a silicon substrate, on which any suitable number of laser, modulators, multi-mode waveguide, photodetectors (e.g., photodiodes), amplifiers, and arithmetic circuit are integrated. An example of a low-power modulator that suitably can be used with the present reservoir computing circuits is a microring or microdisk resonator modulator in a silicon photonics platform. An example of a low-power photodetector that suitably can be used with the present reservoir computing circuits includes a germanium photodiode in a silicon photonics platform, which can be the same platform in which the modulator is provided. -
FIG. 12 illustrates steps in an exemplary method for performing reservoir computing operations using a multi-mode photonic integrated circuit, according to one example. In the nonlimiting configuration illustrated inFIG. 12 ,method 1200 includes receiving, by different physical locations of a multi-mode waveguide, an input signal and a plurality of coefficients imposed on laser light (operation 1210). For example, different physical locations ofmulti-mode waveguide FIGS. 3A-3B can receive laser light it receives via respective waveguides from respective optical modulators. In one exemplary configuration, the input signal can be imposed onto the laser light by an inputoptical modulator optical modulators optical modulator optical modulators multi-mode waveguide -
Method 1200 illustrated inFIG. 12 also can include generating, by the multi-mode waveguide, a speckle pattern based on the different physical locations, the input signal, and the plurality of coefficients (operation 1220). For example,multimode waveguide FIGS. 3A-3B can generate such a speckle pattern in a manner such as described elsewhere herein. -
Method 1200 illustrated inFIG. 12 also includes adjusting at least one of the coefficients based on the speckle pattern (operation 1230). For example, adjusting at least one of the coefficients based on the speckle pattern can include generating one or more electrical signals based on a received portion of the speckle pattern, for example in a manner such as described with reference toFIGS. 3A-3B . Illustratively, an array of photodetectors respectively coupled to the neuronal optical modulators can generate the one or more electrical signals based on the received portion of the speckle pattern, and the coefficient imposed on the laser light by the neuronal optical modulators can be adjusted based on the one or more electrical signals. For example, in the configuration illustrated inFIG. 3A , the coefficients respectively imposed on the laser light by neuronaloptical modulators 313 can be adjusted based on control byamplifier gain controller 370 ofvariable gain amplifiers 350 based upon the outputs ofdetector array 330. Or, for example, in the configuration illustrated inFIG. 3B , the coefficients respectively imposed on the laser light by neuronaloptical modulators 313′ can be adjusted based on control by processor andmemory 370′ of signals that are applied to neuronaloptical modulators 313′ viaDACs 390′. Regardless of the particular configuration, optionally the neuronal optical modulators (e.g., 313, 313′) can respond nonlinearly to the one or more electrical signals in a manner such as described elsewhere herein. As a further or alternative option, the photodetectors (e.g., 330, 330′) can receive the speckle pattern via respective waveguides. - In a manner such as described elsewhere herein, an output signal can be generated that is based collectively on the one or more electrical signals. For example, in the configuration illustrated in
FIG. 3A ,arithmetic circuit 360 can sum the outputs ofvariable gain amplifiers 350 to generate output y(t). As another example, in the configuration illustrated inFIG. 3B , processor andmemory 370′ can sum the outputs ofADCs 380′ to generate output y(t) Optionally, adjusting the at least one of the coefficients can include adjusting a gain of at least one of the one or more electrical signals based on a comparison of the output signal to the input signal to the output signal. For example, in the configuration ofFIG. 3A , based on a comparison of y(t) to Vin,amplifier gain controller 370 can control the electronic signals applied to neuronaloptical modulators 313. As another example, in the configuration ofFIG. 3B , based on a comparison of y(t) to Vin, processor andmemory 370′ can control the values respectively applied byDACs 390′ to neuronaloptical modulators 313′. As noted elsewhere herein, the input signal can be time-varying, and the output signal can be predictive of the input signal as a result of training the reservoir computing network. - Further information regarding an estimation of energy cost per operation of the present PICs, for example when integrated into reservoir computing circuits, can illustrate why such PICs provide a significant advance relative to all-electronic based devices for use in reservoir computing circuits.
- For example, as can be understood from the exemplary configurations provided above with reference to
FIGS. 3A-3B andFIG. 11 , the energy use for a single operation for the present PIC primarily is that used by an optical modulator (e.g., one of neuronaloptical modulators FIGS. 3A-3B ) and by a photodetector (e.g., one ofdetector array FIGS. 3A-3B ) which generates an electrical output used to control the optical modulator at the next time step; the multi-mode waveguide (e.g., waveguide 314) operates passively and therefore uses no energy. An example of a low-power modulator that suitably can be used with the present reservoir computing circuits is a microring or microdisk resonator modulator in a silicon photonics platform that requires a 1V drive voltage to modulate the optical signal On/Off with 30 GHz bandwidth. An example of a low-power photodetector that suitably can be used with the present reservoir computing circuits includes a germanium photodiode in a silicon photonics platform with 50 GHz bandwidth and a quantum efficiency of 0.9. The energy used to perform a reservoir computing operation can be considered to be or include that of converting the light input to the photodetector at the current time step to the light output from the modulator at the next time step. This energy cost can be calculated by thinking backward through the signal path starting with the 1V drive used by the modulator. For transmission of an RF signal to drive the modulator, the impedance is matched to a 50 Ohm transmission line, which combined with knowledge of the bandwidth of the modulator allows us to calculate the junction capacitance with a simple RC model. In this case, the capacitance of the modulator can be expressed as: -
- Given the capacitance of the modulator, the total charge required to produce the 1V drive can be calculated from:
-
Q=CV=1.06·10−13 F·1V=1.06·10−13 C (7). - The total number of electrons needed to produce this charge then can be:
-
- With an exemplary detector quantum efficiency of 0.9 and photon energy of about 0.8 eV for photons at about a 1550 nm wavelength, this can be converted to the number of photons and total energy of the photons required to produce the 1V drive as follows:
-
-
- As may be understood from these estimation, the energy cost per operation of the optical devices using values from typical silicon photonics foundries is estimated to be on the order of 100 fJ. It should appreciated that other such value is only an estimate and can depend on the particular configuration used.
- In view of the foregoing, it should be appreciated that the present PICs, and reservoir computing circuits incorporating such PICs, solve the problem of power demand for large scale computing operations, such as matrix multiplications in artificial intelligence applications. In addition to the power reduction, the higher operating frequency of the present PIC, as compared to electronic circuitry for performing matrix multiplications, can enable new applications in radio frequency (RF) signal processing which may not be achieved due to the low clock frequencies of conventional digital ICs.
- It further should be appreciated that industrial and commercial applications of the present PICs and reservoir computing circuits can include, but are not limited to, the applications of reservoir computing in general. At present, these applications include speech recognition, time series prediction, signal classification, and control systems (e.g. robotics). Because of the high clock speeds available with the present PICs, these applications suitably can be extended to systems with faster dynamics. For example, signal classification can be performed on RF signals up to the Nyquist limit of ½ the clock frequency of the present PICs. With current foundry specifications of 30 GHz bandwidth for modulators and photodetectors, this translates to applying classification tasks to RF signals up to 15 GHz. Another example is in control systems, where systems that have dynamics at sub-nanosecond time scales can be addressed by the present PICs.
- While preferred embodiments of the invention are described herein, it will be apparent to one skilled in the art that various changes and modifications may be made. For example, it should be apparent that the photonic integrated circuits and multi-mode waveguides provided herein suitably may be used to perform any suitable type of computing operation, and are not limited to use in reservoir computing. The appended claims are intended to cover all such changes and modifications that fall within the true spirit and scope of the invention.
Claims (22)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/154,584 US10627849B1 (en) | 2018-10-08 | 2018-10-08 | Reservoir computing operations using multi-mode photonic integrated circuits |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/154,584 US10627849B1 (en) | 2018-10-08 | 2018-10-08 | Reservoir computing operations using multi-mode photonic integrated circuits |
Publications (2)
Publication Number | Publication Date |
---|---|
US20200110432A1 true US20200110432A1 (en) | 2020-04-09 |
US10627849B1 US10627849B1 (en) | 2020-04-21 |
Family
ID=70052025
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/154,584 Active 2038-11-02 US10627849B1 (en) | 2018-10-08 | 2018-10-08 | Reservoir computing operations using multi-mode photonic integrated circuits |
Country Status (1)
Country | Link |
---|---|
US (1) | US10627849B1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111561953A (en) * | 2020-05-14 | 2020-08-21 | 上海科技大学 | On-chip integrated optical matrix vector multiplier based on wavelength division multiplexing and balanced detection |
CN113705804A (en) * | 2020-05-21 | 2021-11-26 | 格芯(美国)集成电路科技有限公司 | Optical nerve simulation device |
CN114265150A (en) * | 2021-12-16 | 2022-04-01 | 浙江大学 | Compact coil optical waveguide supporting long-distance low-loss transmission on chip |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11521055B2 (en) * | 2018-04-14 | 2022-12-06 | International Business Machines Corporation | Optical synapse |
TW202111467A (en) * | 2019-02-25 | 2021-03-16 | 美商萊特美特股份有限公司 | Path-number-balanced universal photonic network |
KR20220127820A (en) | 2019-12-11 | 2022-09-20 | 록클리 포토닉스 리미티드 | optical sensing module |
US11378746B2 (en) | 2020-09-29 | 2022-07-05 | The Aerospace Corporation | Reservoir computing operations using multiple propagations through a multi-mode waveguide |
WO2022136146A1 (en) * | 2020-12-22 | 2022-06-30 | Ecole Polytechnique Federale De Lausanne (Epfl) | Optical computing and reconfiguring with spatiotemporal nonlinearities in waveguides |
US20220244464A1 (en) * | 2021-02-02 | 2022-08-04 | Macom Technology Solutions Holdings, Inc. | Multimode splitter for narrowing input width photodetector connections |
US11736203B1 (en) | 2022-06-27 | 2023-08-22 | Rockwell Collins, Inc. | Radio frequency (RF) signal processor with photonic local oscillator (LO) phase control |
Family Cites Families (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US2747038A (en) | 1953-07-27 | 1956-05-22 | Leo J Perkovich | Driver's alarm |
US4851840A (en) | 1988-01-06 | 1989-07-25 | Wright State University | Optical analog to digital converter |
US5488354A (en) | 1994-12-02 | 1996-01-30 | Bobby; Colvin | Snooze alert device |
US5568127A (en) | 1995-10-27 | 1996-10-22 | Richard M. Bang | Drowsiness warning device and neck support |
US6236862B1 (en) | 1996-12-16 | 2001-05-22 | Intersignal Llc | Continuously adaptive dynamic signal separation and recovery system |
US6346124B1 (en) | 1998-08-25 | 2002-02-12 | University Of Florida | Autonomous boundary detection system for echocardiographic images |
SE521903C2 (en) | 1999-08-12 | 2003-12-16 | Ericsson Telefon Ab L M | Method and apparatus for optoelectronic A / D conversion |
US6326910B1 (en) | 2000-11-06 | 2001-12-04 | The United States Of America As Represented By The Secretary Of The Air Force | Photonic analog-to-digital conversion using light absorbers |
US6404366B1 (en) | 2000-12-01 | 2002-06-11 | The United States Of America As Represented By The Secretary Of The Navy | Photonic analog-to-digital converter utilizing wavelength division multiplexing and distributed optical phase modulation |
US6625381B2 (en) | 2001-02-20 | 2003-09-23 | Eastman Kodak Company | Speckle suppressed laser projection system with partial beam reflection |
US6600590B2 (en) | 2001-02-20 | 2003-07-29 | Eastman Kodak Company | Speckle suppressed laser projection system using RF injection |
US6445487B1 (en) | 2001-02-20 | 2002-09-03 | Eastman Kodak Company | Speckle suppressed laser projection system using a multi-wavelength doppler shifted beam |
IL145245A0 (en) | 2001-09-03 | 2002-06-30 | Jtc 2000 Dev Delaware Inc | System and method including vector-matrix multiplication |
US7321731B2 (en) | 2004-04-07 | 2008-01-22 | The Boeing Company | Optical pulse position modulation discriminator |
EP2050074A2 (en) | 2006-07-13 | 2009-04-22 | The Regents Of The University Of Colorado | Echo particle image velocity (epiv) and echo particle tracking velocimetry (eptv) system and method |
TWI371925B (en) | 2006-09-08 | 2012-09-01 | Via Tech Inc | Apparatus for processing multiple signals with a single analog-to-digital converter and method thereof |
US7868770B2 (en) | 2007-11-14 | 2011-01-11 | Al-Mutairi Sami H | Warning device for drivers and the like |
GB2459512B (en) | 2008-04-25 | 2012-02-15 | Tannoy Ltd | Control system for a transducer array |
US8514115B2 (en) | 2008-10-31 | 2013-08-20 | Telefonaktiebolaget L M Ericsson (Publ) | Optical analogue to digital converter |
US8432153B2 (en) | 2008-11-21 | 2013-04-30 | The Regents Of The University Of California | Time stretch enhanced recording scope |
US8566053B2 (en) | 2009-03-19 | 2013-10-22 | William Marsh Rice University | Method and apparatus for compressive parameter estimation and tracking |
US7834795B1 (en) | 2009-05-28 | 2010-11-16 | Bae Systems Information And Electronic Systems Integration Inc. | Compressive sensor array system and method |
US8026837B1 (en) | 2010-04-22 | 2011-09-27 | The Aerospace Corporation | Systems and methods for converting wideband signals in the optical domain |
US20150036021A1 (en) | 2011-11-10 | 2015-02-05 | Centre National De La Recherche Scientifique - Cnrs | Multiple Scattering Medium For Compressive Imaging |
US8902069B2 (en) | 2012-07-06 | 2014-12-02 | Edmund M. Martinez | Snooze alert |
US9389305B2 (en) | 2013-02-27 | 2016-07-12 | Mitsubishi Electric Research Laboratories, Inc. | Method and system for compressive array processing |
US8902096B2 (en) | 2013-03-14 | 2014-12-02 | The Aerospace Corporation | Systems and methods for converting wideband signals into the digital domain using electronics or guided-wave optics |
US9413372B1 (en) | 2015-07-30 | 2016-08-09 | The Aerospace Corporation | Systems and methods for converting radio frequency signals into the digital domain using multi-mode optics |
US10095262B2 (en) | 2016-12-12 | 2018-10-09 | The Aerospace Corporation | Systems and methods for performing linear algebra operations using multi-mode optics |
-
2018
- 2018-10-08 US US16/154,584 patent/US10627849B1/en active Active
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111561953A (en) * | 2020-05-14 | 2020-08-21 | 上海科技大学 | On-chip integrated optical matrix vector multiplier based on wavelength division multiplexing and balanced detection |
CN111561953B (en) * | 2020-05-14 | 2021-11-05 | 上海科技大学 | On-chip integrated optical matrix vector multiplier based on wavelength division multiplexing and balanced detection |
CN113705804A (en) * | 2020-05-21 | 2021-11-26 | 格芯(美国)集成电路科技有限公司 | Optical nerve simulation device |
CN114265150A (en) * | 2021-12-16 | 2022-04-01 | 浙江大学 | Compact coil optical waveguide supporting long-distance low-loss transmission on chip |
Also Published As
Publication number | Publication date |
---|---|
US10627849B1 (en) | 2020-04-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10627849B1 (en) | Reservoir computing operations using multi-mode photonic integrated circuits | |
US11853871B2 (en) | Optoelectronic computing systems | |
US11687767B2 (en) | Optoelectronic computing systems | |
KR102542496B1 (en) | Apparatus and method for optical neural networks | |
US10803259B2 (en) | Hybrid analog-digital matrix processors | |
US20210036783A1 (en) | Systems and methods for analog computing using a linear photonic processor | |
US20210173238A1 (en) | Optical modulaton for optoelectronic processing | |
US11719963B2 (en) | Optical modulation for optoelectronic processing | |
CN112912900A (en) | Photoelectric computing system | |
US20210116781A1 (en) | Systems and Methods for Activation Functions for Photonic Neural Networks | |
CN113496281A (en) | Photoelectric computing system | |
CN112823359A (en) | Photoelectric computing system | |
CN113570051A (en) | Photoelectric processing system | |
Filipovich et al. | Monolithic silicon photonic architecture for training deep neural networks with direct feedback alignment | |
Guo et al. | Multi-level encoding and decoding in a scalable photonic tensor processor with a photonic general matrix multiply (GeMM) compiler | |
Davis III et al. | Frequency-encoded deep learning with speed-of-light dominated latency | |
Sunny et al. | High-performance deep learning acceleration with silicon photonics | |
Guo | Photonic Tensor Machine and Multi-level Encoding and Decoding in Wavelength-Multiplexed Photonic Processors | |
US20230281437A1 (en) | Radio-Frequency Photonic Architecture for Deep Neural Networks, Signal Processing, and Computing | |
Zhu et al. | Coherent General-Purpose Photonic Matrix Processor |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
AS | Assignment |
Owner name: THE AEROSPACE CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SCOFIELD, ADAM C.;SHAW, THOMAS JUSTIN;VALLEY, GEORGE C.;REEL/FRAME:047109/0107 Effective date: 20181003 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
FEPP | Fee payment procedure |
Free format text: PETITION RELATED TO MAINTENANCE FEES GRANTED (ORIGINAL EVENT CODE: PTGR); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 4 |