TWI819368B - Optoelectronic computing system - Google Patents

Abstract

An optoelectronic computing system comprises: a first semiconductor die comprising a photonic integrated circuit (PIC), the PIC comprising: a plurality of optical waveguides, wherein a set of multiple input values are encoded on respective optical signals carried by the optical waveguides, an optical copying distribution network comprising a plurality of optical splitters, in which each optical splitter sends half of the power of an input optical wave at an input port to each of two output ports, and an array of optoelectronic circuitry sections, each optoelectronic circuitry section receiving an optical wave from one of the output ports of the optical copying distribution network, and each optoelectronic circuitry section including: at least one photodetector detecting at least one optical wave from the optoelectronic operation; and at least one wire integrated in the PIC electrically coupled to the photodetector and electrically coupled to an electrical output port; and a second semiconductor die comprising an electronic integrated circuit (EIC), the EIC comprising: a plurality of electrical input ports receiving respective electrical values; wherein the first semiconductor die and the second semiconductor die are electrically coupled in a controlled collapsed chip connection, with the electrical output ports of the PIC connected to the electrical input ports of the EIC.

Description

Optoelectronic computing system

The present disclosure relates to an optoelectronic computing system.

Neuromorphic computing is a method in the electronic field that approximates the operation of the brain. One prominent approach to neuromorphic computing is artificial neural networks (ANN), which are collections of artificial neurons that are interconnected in specific ways to process information in a manner similar to how the brain functions. Artificial neural networks have been widely used in artificial intelligence, speech recognition, text recognition, natural language processing and various forms of pattern recognition.

An ANN has an input layer, one or more hidden layers, and an output layer. Each layer has nodes, or artificial neurons, and the nodes are interconnected between layers. Each node of a hidden layer performs a weighted sum of signals received from nodes of the previous layer, and performs a nonlinear transformation ("activation") of the weighted sum to produce an output. The weighted sum can be calculated by performing a matrix multiplication step. Therefore, computing an ANN typically involves multiple matrix multiplication steps, which are often performed using electronic integrated circuits.

In the form of electrical signals (such as voltage or current) in analog or digital form Computations performed on the encoded electronic data are typically performed using electronic computing hardware, such as on an integrated circuit (e.g., a processor, an application-specific integrated circuit (ASIC), or a system on a chip). An analog or digital electronic device implemented on a chip; SoC), electronic circuit board, or other electronic circuit. Optical signals have been used to transmit data over long distances and shorter distances (for example, within data centers). Operations performed on such optical signals are typically performed in the context of optical data transmission, such as within devices used to switch or filter optical signals in a network. The use of optical signals in computing platforms has become more restricted. Various components and systems have been proposed for all-optical computing. Such systems may include conversion from electrical signals to electrical signals individually at inputs and outputs, but neither type of signal (electrical and optical) may be used for important operations performed in computing.

In general, in a first aspect, an optoelectronic computing system includes a first semiconductor die including a photonic integrated circuit (PIC) including a plurality of optical waveguides, wherein the optical waveguides carry A set of multiple input values is encoded on the corresponding complex optical signal; the optical replication distribution network includes a plurality of optical splitters, wherein each optical splitter transmits half of the power of the input light wave located at the input port to two output ports respectively ; and an array of optoelectronic circuit segments, each optoelectronic circuit segment receiving light waves from one of the output ports of the optical replication distribution circuit, each optoelectronic circuit segment including: at least one photodetector that detects during the optoelectronic operation At least one light wave; and at least one wire in the photonic integrated circuit electrically coupled to the photodetector and the electrical output; and a second semiconductor die including an electronic integrated circuit (EIC), the electronic integrated circuit including: a plurality of electrical input terminals receiving respective electrical values, wherein the first semiconductor die and the second semiconductor die are used to control the collapse height of the die Connection (Controlled Collapsed Chip Connection) is electrically coupled, and the electrical output end of the photonic integrated circuit is connected to the input end of the electronic integrated circuit.

Computing system embodiments may include one or more of the following features.

Each optoelectronic circuit segment contains: an optoelectronic operating module that performs one of: (1) an optical replication distribution network scaling an optical value based on one of the input values, and (2) an electrical input providing Electrical value; at least one photodetector detects at least one light wave in the optoelectronic operation, and at least one wire in the photonic integrated circuit is electrically coupled to the photodetector and the electrical output terminal.

The electronic integrated circuit further includes complex digital to analog converters (DACs), which provide electrical values to corresponding electrical output terminals, and the electrical input terminal of the photonic integrated circuit is connected to the electrical output terminal of the electronic integrated circuit .

Optical splitters are configured as nodes in a binary tree setup connected by optical waveguides that are links of the binary tree setup.

The light propagation length differs between the root of the binary tree assignment and between the different optoelectronic circuit sections.

The optical waveguides in the optical replication distribution network are disposed on the first semiconductor die to avoid passing through any optical waveguides in the optical replication distribution network.

The photonic circuit section is disposed on the first semiconductor die, and overall Arranged in plural straight lines.

The plurality of straight lines are each optically coupled to each other straight line through one or more optical waveguides in the optical replication distribution network.

Wires in some photonic integrated circuits connect photodetectors to junctions between wires from different optoelectronic circuit segments.

In another aspect, a computing system includes: a first unit configured to generate a complex modulator control signal; and a processing unit. The processing unit includes: a light source or port configured to provide a plurality of light outputs; and a first set of light modulators coupled to the light source or port and the first unit. The complex optical modulators of the first group of optical modulators are configured to be based on a digital input value corresponding to a first group of modulator control signals of the complex modulator control signals, the modulation being provided by the light source or port The complex light output is used to generate a light input vector, and the light input vector includes a complex light signal. The processing unit also includes a matrix multiplication unit including a second set of light modulators. The matrix multiplication unit is coupled to the first unit and configured to be based on a complex digital weight value corresponding to a second set of the complex modulator control signals applied to the second set of optical modulators. , convert the light input vector into an analog output vector. At least one light modulator of at least one of the first set of light modulators or the second set of light modulators is configured to modulate light based on a first of the plurality of modulator control signals. signal, and the first unit is configured to shape the first modulator control signal to include bandwidth enhancement associated with amplitude changes associated with corresponding changes in the complex consecutive digital values corresponding to the first modulator control signal.

Computing system embodiments may include one or more of the following features. plan The computing system may include a second unit coupled to the matrix multiplication unit, and the second unit is configured to convert the analog output vector into a digital output vector; and a controller. The controller may include an integrated circuit configured to: receive an artificial neural network calculation request, the artificial neural network calculation request including an input data set including a first digital input vector; receive a first plurality of neural network calculation requests. network weights; and generating, through the first unit, a first plurality of modulator control signals based on the first digital input vector and a first plurality of weight control signals based on the first plurality of neural network weights.

The first unit may include a digital to analog converter (DAC).

The computing system may include a memory unit configured to store the data set and the complex neural network weights.

The integrated circuit of the controller may be further configured to perform operations including storing the input data set and the first plurality of neural network weights in the memory unit.

The controller may include an application specific integrated circuit (ASIC), and the step of receiving the artificial neural network calculation request may include receiving the artificial neural network calculation request from a general-purpose data processor.

The first unit, the processing unit, the second unit, and the controller may be disposed on at least one of a multi-chip module or an integrated circuit. The step of receiving the artificial neural network calculation request may include receiving the artificial neural network calculation request from a second data processor, wherein the second data processor is external to the multi-chip module or integrated circuit, and the second data processor communicates through communication channel coupled to multi-chip A module or integrated circuit, and the processing unit can process data at a data rate that is at least one order of magnitude greater than the data rate of the communication channel.

The first unit, the processing unit, the second unit and the controller may be used in a photoelectric processing cycle that is repeated in a plurality of iterations. The photoelectric processing cycle includes: (1) at least a first light modulation operation based on at least one of the modulator control signals, and at least a second light modulation operation based on at least one of the weight control signals, and (2) ) at least one of (a) an electrical summing operation or (b) an electrical storage operation.

The photoelectric processing cycle may include electrical storage operations, and the electrical storage operations may be performed using a memory unit coupled to the controller. The operations performed by the controller may further include storing the input data set and the first plurality of neural network weights in the memory unit.

The photoelectric processing loop may include electrical summing operations, and the electrical summing operations may be performed using electrical summing modules within the matrix multiplication unit. The electrical summation module may be configured to generate currents corresponding to elements of the analog output vector, the currents representing the sum of corresponding elements of the optical input vector multiplied by corresponding neural network weights.

The first modulator control signal may include an analog signal associated with a plurality of predetermined amplitude levels, with each of the amplitude levels associated with a different corresponding digital value.

The first modulator control signal may include an analog signal associated with two predetermined amplitude levels, with each of the amplitude levels associated with a different corresponding binary value.

Continuous numeric values may include complex consecutive binary values within a series of binary values.

The controller may be configured to increase the time associated with the first time interval by The magnitude of the amplitude change between the first predetermined amplitude level and the second predetermined amplitude level associated with the second time interval is used to shape the first modulator control signal to include the amplitude for the initial portion of the second time interval. Bandwidth enhancement.

A series of binary values may be used to determine the amplitude level of the first modulator control signal used to modulate the optical signal according to a non-return-to-zero (NRZ) modulation mode.

The first unit may be configured by the diode structure of the first modulator in the second set of optical modulators and being connected in series between the diode structure and the circuit providing the first modulator control signal. pumping current between the capacitors to shape the first modulator control signal to include bandwidth enhancement, and the amount of charge transferred by the pumping current is based at least in part on a constant over a time period that provides successive digital values voltage to determine.

In another aspect, a computing device includes a plurality of optical waveguides coupled to a first set of optical amplitude modulators, wherein the first set of optical amplitude modulators are used to encode corresponding complex optical signals carried by the optical waveguides. A set of multiple input values. The computing device includes a plurality of replication modules, and for each of at least two subsets of the one or more optical signals, a corresponding set of the one or more replication modules is configured to replicate a portion of the one or more optical signals. A subset is divided into two or more copies of an optical signal. The computing device includes complex multiplication modules, each of the multiplication modules including an optical amplitude modulator of a second set of optical amplitude modulators, and for at least two copies of a first subset of one or more optical signals Each of the corresponding one of the multiplication modules is configured to multiply the one or more optical signals of the first subset by one or more matrix elements using the optical amplitude modulators of the second set of optical amplitude modulators value. Computing device package One or more summation modules are included, and for a result of two or more of the multiplication modules, a corresponding one of the summation modules is configured to generate an electrical signal, the electrical signal represents a result of the two or more multiplication modules. The sum of the results. At least one optical amplitude modulator of at least one of the first group of optical amplitude modulators or the second group of optical amplitude modulators is configured to use power that monotonically increases relative to the absolute value of the modulation value, Modulate the light signal by modulating the value.

Computing device embodiments may include one or more of the following features. The at least one optical amplitude modulator of at least one of the first set of optical amplitude modulators or the second set of optical amplitude modulators may comprise a coherent sensitive optical amplitude modulator configured to Based on the interference between complex light waves, the light signal is modulated by the modulation value. The light waves have a coherence length that is at least as long as the propagation distance through the coherently sensitive light amplitude modulator.

The coherence-sensitive optical amplitude modulator may include a Mach-Zehnder Interferometer (MZI). The Mach-Zehnder interferometer divides the light wave guided by the input optical waveguide into the first optical waveguide arm of the Mach-Zehnder interferometer and the Mach-Zehnder interferometer. The second optical waveguide arm of the Zehnder interferometer. The first optical waveguide arm may include an active phase shifter, the active phase shifter generates a relative phase shift with respect to a phase delay of the second optical waveguide arm, and the Mach-Zehnder interferometer may combine signals from the first optical waveguide arm and the second optical waveguide. The plurality of light waves of the arms are combined into at least one output light waveguide.

The power used to modulate the optical signal by modulating the value may include power applied to the active phase shifter.

A complex input value encoded in a set of multiple input values on a corresponding optical signal The input value may represent a complex element of the input vector multiplied by a matrix containing one or more matrix element values.

A set of multiple output values may be encoded on a complex corresponding electrical signal generated by one or more summation modules, and the complex output values in the set of multiple output values may represent a complex element of an output vector, the output vector Produced by multiplying the input vector by a matrix.

Each of the optical signals carried by the optical waveguide may include light waves having a common wavelength, which is approximately the same for all optical signals.

The replication module may include at least one replication module having an optical splitter that sends a predetermined proportion of the power of the light wave at an input port of the replication module to a first output port of the replication module, and at an input port of the replication module The input port sends the remaining proportion of the power of the light wave to the second output port of the replica module.

The optical splitter may include a waveguide optical splitter that sends a determined proportion of the power of the optical wave guided by the input optical waveguide of the replica module to the first output optical waveguide of the replica module and that is directed by the input optical waveguide of the replica module. The remaining proportion of the power of the light wave guided by the optical waveguide is sent to the second output optical waveguide of the replica module.

The guided mode of the input optical waveguide may be adiabatically coupled to the plurality of guided modes of each of the first and second output optical waveguides.

The optical splitter may include a beam splitter including at least one surface that transmits a determined proportion of the power of the light wave at the input port and reflects a remaining proportion of the power of the light wave at the input port.

At least one of the optical waveguides may include an optical fiber coupled to an optical coupler that couples a guided mode of the optical fiber to a free-space propagation mode. propagation mode).

The multiplication module may include at least one coherently sensitive optical amplitude modulator configured to multiply one or more optical signals of the first subset by one or more optical signals based on interference between complex light waves. matrix element values, the light wave has a coherence length that is at least as long as the propagation distance through the coherently sensitive optical amplitude modulator.

The coherently sensitive optical amplitude modulator may include a Mach-Zehnder interferometer (MZI) that divides light waves guided by an input optical waveguide into a first optical waveguide arm of the MZI and a first optical waveguide arm of the MZI. The second optical waveguide arm. The first optical waveguide arm may include a phase shifter, the phase shifter generates a relative phase shift with respect to a phase delay of the second optical waveguide arm, and the Mach-Zehnder interferometer may combine the signals from the first optical waveguide arm and the second optical waveguide arm. The plurality of light waves are combined into at least one output light waveguide.

The Mach-Zehnder interferometer can combine the complex light waves from the first and second optical waveguide arms into each of the first and second output optical waveguides. The first photodetector can receive the light wave from the first output optical waveguide to generate the first photocurrent, the second photodetector can receive the light wave from the second output optical waveguide to generate the second photocurrent, and the coherently sensitive light amplitude modulation The result of the inverter may include a difference between the first photocurrent and the second photocurrent.

The coherently sensitive optical amplitude modulator may include one or more ring resonators including at least one ring resonator coupled to the first optical waveguide and at least one ring resonator coupled to the second optical waveguide.

The first optical detector can receive light waves from the first optical waveguide to generate Generating a first photocurrent, the second photodetector can receive the light wave from the second optical waveguide to generate a second photocurrent, and the result of the coherently sensitive optical amplitude modulator can include the first photocurrent and the second photocurrent. the difference between.

The multiplication module may include at least one coherent insensitive optical amplitude modulator configured to multiply one or more optical signals of the first subset by one or more based on energy absorption within the light wave. Multiple matrix element values.

Coherent non-sensitive optical amplitude modulators may include electroabsorption modulators.

The one or more summing modules may include at least one summing module having the following components: (1) Two or more input conductors, each of the input conductors carrying an electrical signal in the form of an input current, the magnitude of the input current represents a corresponding result of a corresponding one of the multiplication modules, and (2) at least one output conductor carrying an electrical signal representing the sum of the corresponding results in the form of an output current, the output current being proportional to the sum of the input currents.

The two or more input conductors and the output conductors may include a plurality of conductors that meet at one or more junctions between the conductors, and the output current may be approximately equal to the sum of the input currents.

At least a first input current of the input current can be provided in the form of at least one photocurrent, the photocurrent is generated by at least one photodetector, and the photodetector receives the optical signal generated by the first multiplication module of the multiplication module .

The first input current may be provided in the form of a difference between two photocurrents generated by different corresponding photodetectors, and the photodetectors receive different corresponding optical signals generated by the first multiplication module .

One of the copies of the first subset of the one or more optical signals may consist of a single optical signal on which one of the input values is encoded.

The multiplication module corresponding to the copy of the first subset may multiply the encoded input value by a single matrix element value.

One of the copies of the first subset of the one or more optical signals may include more than one optical signal and less than the number of all optical signals in which multiple input values of the optical signal are encoded.

A multiplication module corresponding to a copy of the first subset may multiply the encoded input value by a different corresponding matrix element value.

Different multiplication modules corresponding to different corresponding copies of the first subset of the one or more optical signals may be included in different devices, and the different devices optically communicate to transmit the first subset of the one or more optical signals between the different devices. One of the copies of the set.

At least one of two or more optical waveguides, two or more replication modules, two or more multiplication modules, and one or more summing modules may be disposed on a substrate of a common device.

The common device can perform vector matrix multiplication, where input vectors can be provided as a set of optical signals and output vectors can be provided as a set of electrical signals.

The computing device may further include an accumulator that integrates an input electrical signal corresponding to an output of one of the multiplication modules or one of the summation modules, wherein the input electrical signal is encoded using time domain coding, the time domain coding being performed on a plurality of Switching amplitude modulation is used within each time slot, and the accumulator generates an output electrical signal. The output electrical signal is encoded with more than two amplitude levels, the amplitude levels corresponding to the different time domain encodings on the multiple time slots. duty cycle.

Each of two or more multiplication modules may correspond to a different subset of one or more optical signals.

The computing device may further include for each replica of the second subset of one or more optical signals, distinct from the optical signals in the first subset of one or more optical signals, the multiplication module configured to use optical amplitude modulation. Multiplying one or more optical signals of the second subset by one or more matrix element values.

In another general view, a method of operating a computing system includes: using a first set of optical amplitude modulators to encode a set of a plurality of input values on a corresponding optical signal; for at least two subsets of the one or more optical signals Each, using a corresponding set of one or more replication modules to divide a plurality of subsets of one or more optical signals into two or more copies of the optical signal; for a first subset of the one or more optical signals Each of the at least two copies uses a corresponding multiplication module to multiply the one or more optical signals of the first subset by one or more optical amplitude modulators using the optical amplitude modulators of the second set matrix element values; and for the results of two or more multiplication modules, a summation module is used to generate an electrical signal that represents the sum of the results of the two or more multiplication modules. At least one optical amplitude modulator of at least one of the first group of optical amplitude modulators or the second group of optical amplitude modulators is configured to use a monotonically increasing power relative to an absolute value of the modulation value, by modulating value to modulate the light signal.

In another general view, the computing system includes: a memory unit configured to store a data set and complex neural network weights; a digital-to-analog converter (DAC) unit configured to generate complex tones The inverter control signal and generates the complex weight control signal; the optical processor, including the laser unit, is configured to generate the complex optical output; the complex optical modulator is coupled to the laser unit Yuan and DAC unit, the optical modulator is configured to modulate the optical output generated by the laser unit based on the modulator control signal to generate an optical input vector; the optical matrix multiplication unit is coupled to the optical modulator and a DAC unit, a light matrix multiplication unit configured to convert the light input vector into a light output vector based on the weight control signal; and a light detection unit coupled to the light matrix multiplication unit and configured to generate a complex output corresponding to the light output vector voltage; an analog-to-digital converter (ADC) unit, coupled to the light detection unit, and configured to convert the output voltage into a complex digital light output; a controller, including an integrated circuit, configured to perform the following operations: receive an artificial neural network calculation request from the computer including an input data set and a first plurality of neural network weights, wherein the input data set includes a first digital input vector; store the input data set and the first plurality of neural network weights in a memory unit a first plurality of neural network weights; and generating a first plurality of modulator control signals based on the first digital input vector through the DAC unit, and generating a first plurality of weight control signals based on the first plurality of neural network weights.

Computing system embodiments may include one or more of the following features. For example, the operation may further include: obtaining a first plurality of digital light outputs corresponding to the light output vector of the light matrix multiplication unit from the ADC unit, and the first plurality of digital light outputs forming a first digital output vector; performing a nonlinear transformation on the vector to generate a first transformed digital output vector; and storing the first transformed digital output vector in the memory unit.

The computing system may have a first cycle defined as the steps of storing the input data set and the first plurality of neural network weights in the memory unit and in the memory unit. The time elapsed between steps of storing the first converted digital output vector in the volume unit. The first cycle period may be less than or equal to 1 ns.

In some embodiments, the operations may further include: outputting an artificial neural network output generated based on the first converted digital output vector.

In some embodiments, the operations may further include generating, through the DAC unit, a second plurality of modulator control signals based on the first converted digital output vector.

In some embodiments, the artificial neural network calculation request may further include a second plurality of neural network weights, and the operation may further include: based on obtaining the first plurality of digital light outputs, through the digital-to-analog converter unit based on the second The plurality of neural network weights generate a second plurality of weight control signals. The first plurality of neural network weights and the second plurality of neural network weights may correspond to different layers of the artificial neural network.

In some embodiments, the input data set may further include a second digital input vector, and the operations may further include: generating a second plurality of modulator control signals based on the second digital input vector through the DAC unit; obtaining the corresponding from the ADC unit a second plurality of digital light outputs of the light output vector of the light matrix multiplication unit, the second plurality of digital light outputs forming a second digital output vector; performing a nonlinear conversion on the second digital output vector to generate a second converted digital output vector; store the second converted digital output vector in the memory unit; and output the artificial neural network output generated based on the first converted digital output vector and the second converted digital output vector. The light output vector of the light matrix multiplication unit is generated by a second light input vector generated based on the second plurality of modulator control signals. The second light input vector is converted by the light matrix multiplication unit based on the first mentioned above-mentioned weight control signal. .

In some embodiments, the computing system may further include: an analog nonlinear unit disposed between the light detection unit and the ADC unit, the analog nonlinear unit configured to receive an output voltage from the light detection unit, apply nonlinear transfer function, and outputting the complex conversion output voltage to the ADC unit, and the operation further includes: obtaining a first plurality of conversion digital output voltages corresponding to the conversion output voltage from the ADC unit, and the first plurality of conversion digital output voltages forming a first conversion digital output vector ; and storing the first converted digital output vector in the memory unit.

In some embodiments, the integrated circuit of the controller may be configured to generate the first plurality of modulator control signals at a frequency greater than or equal to 8 GHz.

In some embodiments, the computing system may further include: an analog memory unit disposed between the DAC unit and the light modulator, the analog memory unit configured to store a complex analog voltage and output the stored analog voltage; and The analog nonlinear unit is disposed between the light detection unit and the ADC unit, and is configured to receive an output voltage from the light detection unit, apply a nonlinear transfer function, and output a complex conversion output voltage. Analog memory cells include complex capacitors.

In some embodiments, the analog memory unit may be configured to receive and store the converted output voltage of the analog nonlinear unit, and output the stored converted output voltage to the light modulator, and the operations may further include: based on generating the first The plurality of modulator control signals and the first plurality of weight control signals store the conversion output voltage of the analog nonlinear unit in the analog memory unit; output the stored conversion output voltage through the analog memory unit; and obtain a second conversion output voltage from the ADC unit. Multiple converted digital output voltages, second most The converted digital output voltages form a second converted digital output vector; and the second converted digital output vector is stored in the memory unit.

In some embodiments, the input data set requested by the artificial neural network calculation may include a complex numeric input vector. The laser unit can be configured to generate a complex number of wavelengths. The light modulator may include: a plurality of light modulator banks configured to generate a plurality of light input vectors, each light modulator bank corresponding to a wavelength and generating a corresponding light input vector having a corresponding wavelength; and light A multiplexer configured to combine the optical input vectors into a combined optical input vector including wavelengths. The light detection unit may be further configured to demultiplex wavelengths and generate a plurality of demultiplexed output voltages. The operation may include: obtaining a complex digital demultiplexed light output from the ADC unit, and the digital demultiplexed light output forming a complex first digital output vector, where each first digital output vector corresponds to a wavelength; for each first digital output vector performing a nonlinear transformation to generate a complex transformation first digital output vector; and storing the transformation first digital output vector in a memory unit. Each digital input vector corresponds to a light input vector.

In some embodiments, the artificial neural network calculation request may include a complex numeric input vector. The laser unit can be configured to generate a complex number of wavelengths. The optical modulator may include: a plurality of optical modulator groups configured to generate a plurality of optical input vectors, each optical modulator group corresponding to a wavelength and generating a corresponding optical input vector having a corresponding wavelength; and an optical multiplexer A user configured to combine the light input vectors into a combined light input vector including wavelengths. The operations may include: obtaining a first plurality of digital light outputs corresponding to a light output vector from the ADC unit, the light output vector including a wavelength, the first plurality of digital light outputs forming a first digital output vector; and performing nonlinearity on the first digital output vector. conversion, to generate a first converted digital output vector; and store the first converted digital output vector in the memory unit.

In some embodiments, the DAC unit may include a 1-bit DAC sub-unit configured to generate a complex 1-bit modulator control signal. The resolution of the ADC unit can be 1 bit. The resolution of the first digital input vector may be N bits. The operation may include: decomposing the first digital input vector into N 1-bit input vectors, each N 1-bit input vector corresponding to one of the N bits of the first digital input vector; through the 1-bit DAC subunit Generate a sequence of N 1-bit modulator control signals corresponding to N 1-bit input vectors; obtain N digital 1-bit light outputs corresponding to the sequence of N 1-bit modulator control signals from the ADC unit a sequence; constructing an N-bit digital output vector from a sequence of N digital 1-bit light outputs; performing a nonlinear transformation on the constructed N-bit digital output vector to produce a converted N-bit digital output vector; and in a memory unit Convert N-bit digital output vector stored in.

In some embodiments, the memory unit may include: a digital input vector memory configured to store the first digital input vector and including at least one static random access memory; and a neural network weight memory configured to Stores neural network weights and includes at least one dynamic random access memory.

In some embodiments, the DAC unit may include: a first DAC subunit configured to generate a modulator control signal; and a second DAC subunit configured to generate a weight control signal, wherein the first DAC subunit and the The two DAC subunits are different.

In some embodiments, the laser unit may include: a laser source configured to generate light; and an optical power splitter configured to split the light generated by the laser source into optical outputs, wherein each optical output has approximately Same power.

In some embodiments, the above-mentioned optical modulator includes a Mach-Zehnder interferometer (MZI) modulator, a ring resonator modulator or an electro-absorption modulator. of one.

In some embodiments, the light detection unit may include: a plurality of light detectors; and a plurality of amplifiers configured to convert the photocurrent generated by the light detector into an output voltage.

In some embodiments, the integrated circuit may be an application special integrated circuit.

In some embodiments, the optical matrix multiplication unit may include: an input waveguide array for receiving the optical input vector; an optical interference unit for optical communication with the input waveguide array for performing conversion of the optical input vector into a second optical signal array. linear conversion; and an output waveguide array in optical communication with the optical interference unit for guiding a second optical signal array, wherein at least one input waveguide in the input waveguide array passes through the optical interference unit and each output waveguide in the output waveguide array Optical communications.

In some embodiments, the optical interference unit may include: a plurality of interconnected MZIs, each of the interconnected MZIs including: a first phase shifter configured to change a separation ratio of the MZI; and a second phase shifter, Configure to displace one of the MZI A phase of an output, wherein the first phase shifter and the second phase shifter are coupled to the weight control signal.

In another view, the computing system includes: a memory unit configured to store a data set and a complex neural network weight; a driver unit configured to generate a complex modulator control signal and generate a complex weight control signal; an optical processor , including: a laser unit configured to generate a complex light output; a complex optical modulator coupled to the laser unit and the driver unit, the optical modulator is configured to control a signal based on the modulator, the modulation is performed by the laser The light output generated by the unit is used to generate a light input vector; the light matrix multiplication unit is coupled to the light modulator and the driver unit, and the light matrix multiplication unit is configured to convert the light input vector into a light output vector based on the weight control signal; and a light detection unit coupled to the light matrix multiplication unit and configured to generate a complex output voltage corresponding to the light output vector; a comparator unit coupled to the light detection unit and configured to convert the output voltage into a complex number a digital 1-bit light output; and a controller, including an integrated circuit, configured to: receive an artificial neural network calculation request from a computer including an input data set and a first plurality of neural network weights, wherein the input data The set includes a first digit input vector with N-bit resolution; storing the input data set and the first plurality of neural network weights in a memory unit; decomposing the first digit input vector into N 1-bit input vectors, Each N 1-bit input vector corresponds to one of the N bits of the first digital input vector; the driver unit generates a sequence of N 1-bit modulator control signals corresponding to the N 1-bit input vectors; from The comparator unit obtains a sequence of N digital 1-bit light outputs corresponding to a sequence of N 1-bit modulator control signals; constructs an N-bit digital output vector from the sequence of N digital 1-bit light outputs; constructs N bits of The digital output vector performs a nonlinear conversion to generate a converted N-bit digital output vector; and the converted N-bit digital output vector is stored in a memory unit.

In another aspect, a computing method is used to perform artificial neural network calculations in a computing system having an optical matrix multiplication unit configured to convert a light input vector into a light output vector based on a complex weight control signal , the calculation method includes: receiving an artificial neural network calculation request including an input data set and a first plurality of neural network weights from a computer, where the input data set includes a first digital input vector; storing the input data set and the first plurality of neural network weights in a memory unit a first plurality of neural network weights; generating a first plurality of modulator control signals based on the first digital input vector through a digital-to-analog converter (DAC) unit, and generating a first plurality of modulator control signals based on the first plurality of neural network weights; A weight control signal; obtaining a first plurality of digital light outputs corresponding to the light output vector of the light matrix multiplication unit from the analog-to-digital converter (ADC) unit, and the first plurality of digital light outputs form a first digital output vector; by controlling The controller performs a non-linear conversion on the first digital output vector to generate a first converted digital output vector; stores the first converted digital output vector in the memory unit; and outputs a signal generated based on the first converted digital output vector by the controller. Artificial neural network output.

In another aspect, the computing method includes: providing input information in an electronic format; converting at least a portion of the electronic input information into a light input vector; optically converting the light input vector into a light output vector based on light matrix multiplication; converting the light output vector into an electronic format; and electronically applying a nonlinear transformation to the electronically converted light output vector to provide output information in an electronic format.

Embodiments of computing methods may include one or more of the following features. For example, the calculation method may further include: repeating electronic-to-optical converting, optical transforming, and optical-to for new electronic input information corresponding to the output information provided in the electronic format. -electronic converting) and nonlinear conversion for electrical applications.

In some embodiments, the light matrix multiplication for the initial light conversion and the light matrix multiplication for the repeated light conversion may be the same and may correspond to the same layer of the artificial neural network.

In some embodiments, the light matrix multiplication for the initial light conversion and the light matrix multiplication for the repeated light conversion may be different and may correspond to different layers of the artificial neural network.

In some embodiments, the calculation method may further include: repeating electro-optical conversion, optical conversion, photoelectric conversion, and electrically applied nonlinear conversion for different parts of the electronic input information, wherein the optical matrix multiplication for the initial optical conversion and the repeated optical The transformed light matrix multiplication is the same and corresponds to the first layer of the artificial neural network.

In some embodiments, the computing method may further include: providing electronic intermediate information in an electronic format based on electronic output information for the plurality of portions of electronic input information generated by the first layer of the artificial neural network; and for electronic Each different part of the intermediate information, repeated electro-optical conversion, optical conversion, photoelectric conversion and non-linear conversion of electrical applications, where the optical matrix multiplication for the initial optical conversion and the repeated optical conversion associated with the different parts of the electronic intermediate information Matrix multiplication is the same and corresponds to the second layer of the artificial neural network.

In another aspect, a computing system includes: a light processor including a passive diffractive optical element, wherein the passive diffractive optical element is configured to convert a light input vector or matrix into a light output vector or matrix, It represents the result of matrix processing applied to a light input vector or matrix and a given vector defined by the arrangement of diffractive optical elements.

Computing system embodiments may include one or more of the following features. For example, matrix processing may include matrix multiplication between a light input vector or matrix and a given vector defined by the arrangement of diffractive optical elements.

In some embodiments, the optical processor may include an optical matrix processing unit including: an input waveguide array for receiving the optical input vector, and an optical interference unit including a passive diffractive optical element, wherein the optical interference unit is optically connected to the input waveguide array. communicating and configured to perform linear conversion of the optical input vector into a second optical signal array; and an output waveguide array in optical communication with the optical interference unit for guiding the second optical signal array, wherein at least one of the input waveguide arrays The input waveguide optically communicates with each output waveguide in the output waveguide array through the optical interference unit.

In some embodiments, the optical interference unit may include a substrate having at least one of holes or stripes, the holes having a size in the range of 100 nm to 10 μm, and the stripes having a width in the range of 100 nm to 10 μm.

In some embodiments, the optical interference unit may include a substrate having passive diffractive optical elements arranged in a two-dimensional configuration, and the substrate includes at least one of a planar substrate or a curved substrate.

In some embodiments, the substrate may comprise a planar substrate parallel to the direction of light propagation from the input waveguide array to the output waveguide array.

In some embodiments, the optical processor may include an optical matrix processing unit including: an input waveguide array for receiving the optical input matrix, and an optical interference unit including a passive diffraction optical element, wherein the optical interference unit is optically connected to the input waveguide array. communicating and configured to perform linear conversion of the optical input matrix into a second optical signal array; and an output waveguide array in optical communication with the optical interference unit for guiding the second optical signal array, wherein at least one of the input waveguide array The input waveguide optically communicates with each output waveguide in the output waveguide array through the optical interference unit.

In some embodiments, the optical interference unit may include a substrate having at least one of holes or stripes, the holes having a size in the range of 100 nm to 10 μm, and the stripes having a width in the range of 100 nm to 10 μm.

In some embodiments, the optical interference unit may include a substrate having passive diffraction optical elements arranged in a three-dimensional configuration.

In some embodiments, the substrate may have a shape of at least one of a cube, a column, a prism, or an irregular volume.

In some embodiments, the light processor may include an optical interference unit including a hologram with passive diffraction optical elements, the light processor is configured to receive modulated light representative of the light input matrix, and in the light Light is continuously converted as it passes through the hologram until the light exits the hologram as a light output matrix.

In some embodiments, the optical interference unit may include a substrate having passive diffractive optical elements, and the substrate includes at least one of silicon, silicon oxide, silicon nitride, quartz, lithium niobate, phase change materials, or polymers.

In some embodiments, the optical interference unit may include a substrate having passive diffractive optical elements, and the substrate includes at least one of a glass substrate or an acrylic substrate.

In some embodiments, passive diffractive optical elements may be formed in part from dopants.

In some embodiments, matrix processing may represent a neural network's processing of input data represented by optical input vectors.

In some embodiments, the light processor may include: a laser unit configured to generate a complex light output; a complex light modulator coupled to the laser unit and configured to modulate the light output based on the complex modulator control signal. changing the light output generated by the laser unit to generate the light input vector; the light matrix processing unit is coupled to the light modulator, the light matrix processing unit includes a passive diffraction optical element configured to generate a light input vector based on the passive diffraction The complex weights defined by the optical element convert the light input vector into the light output vector; and the light detection unit is coupled to the light matrix processing unit and configured to generate a complex output electrical signal corresponding to the light output vector.

In some embodiments, the passive diffraction optical element may be arranged in a three-dimensional configuration, the light modulator includes a two-dimensional light modulator array, and the light detection unit includes a two-dimensional light detector array.

In some embodiments, the optical matrix processing unit may include a housing module to support and protect the input waveguide array, the optical interference unit, and the output waveguide array, and the optical processor may include a receiving module configured to receive The optical matrix processing unit, the receiving module includes a first interface that enables the optical matrix processing unit to receive the optical input vector from the optical modulator, and a second interface that enables the optical matrix processing unit to transmit the optical output vector to the optical matrix processing unit. detection unit.

In some embodiments, the output electrical signal may include at least one of a complex voltage signal or a complex current signal.

In some embodiments, the computing system may include: a memory unit; a digital-to-analog converter (DAC) unit configured to generate a modulator control signal; an analog-to-digital converter (ADC) unit coupled to the light detection unit , and configured to convert the output electrical signal into a complex digital output; and the controller, including the integrated circuit, is configured to perform the following operations: receiving an artificial neural network calculation request from the computer including an input data set, wherein the input data set The method includes a first digital input vector; storing an input data set in a memory unit; and generating a first plurality of modulator control signals based on the first digital input vector through the DAC unit.

In another view, the computing method includes: 3D printing a light matrix processing unit including a passive diffractive optical element, wherein the passive diffractive optical element is configured to convert a light input vector or matrix into a light output vector or matrix, which represents The result of matrix processing applied to a light input vector or matrix and a given vector defined by the arrangement of diffractive optical elements.

In another aspect, a computational method includes using one or more laser beams to generate a hologram including passive diffractive optical elements, wherein the passive diffractive optical elements are configured to convert light input vectors or matrices into light output vectors or matrix, which represents the result of matrix processing applied to a light input vector or matrix and a given vector defined by the arrangement of diffractive optical elements.

In another aspect, a computing system includes: a light processor including a passive diffractive optical element disposed in a one-dimensional manner, wherein the passive diffractive optical element is configured to convert light input into light output, a representation of which is applied to the light input and the result of matrix processing of a given vector defined by the arrangement of diffractive optical elements.

Computing system embodiments may include one or more of the following features. For example, matrix processing may include matrix multiplication between the light input and a given vector defined by the arrangement of diffractive optical elements.

In some embodiments, the optical processor may include an optical matrix processing unit including an input waveguide for receiving optical input, an optical interference unit including a passive diffractive optical element, wherein the optical interference unit is in optical communication with the input waveguide, and configured to perform linear conversion of the light input; and an output waveguide in optical communication with the optical interference unit for guiding the light output.

In some embodiments, the optical interference unit may include a substrate having at least one of holes or grating elements, and the holes or grating elements may have dimensions in the range of 100 nm to 10 μm.

In another aspect, a computing system includes: a memory unit; a digital-to-analog converter (DAC) unit configured to generate a complex modulator control signal; and and a light processor, including: a laser unit configured to generate a complex light output; a complex light modulator coupled to the laser unit and the DAC unit, the light modulator being configured to modulate based on the modulator control signal changing the light output generated by the laser unit to generate the light input vector; the light matrix processing unit is coupled to the light modulator, the light matrix processing unit includes a passive diffraction optical element configured to generate a light input vector based on the passive diffraction The complex weights defined by the optical element convert the light input vector into the light output vector; and the light detection unit is coupled to the light matrix processing unit and configured to generate a complex output electrical signal corresponding to the light output vector. The computing system further includes: an analog-to-digital converter (ADC) unit coupled to the light detection unit and configured to convert the output electrical signal into a complex digital light output; and a controller, including an integrated circuit, configured to execute The following operations: receive an artificial neural network calculation request from the computer including an input data set, wherein the input data set includes a first digital input vector; store the input data set in a memory unit; and through the DAC unit, based on the first digital input vector Generating a first plurality of modulator control signals.

Computing system embodiments may include one or more of the following features. For example, the matrix processing unit may include a passive diffractive optical element configured to convert a light input vector into a light output vector that represents a relationship between the light input vector and a given vector defined by the passive diffractive optical element. The product of matrix multiplication.

In some embodiments, the operation further includes: obtaining a first plurality of digital light outputs corresponding to the light output vector of the light matrix processing unit from the ADC unit, the first plurality of digital light outputs forming a first digital output vector; performing a non-linear transformation on the output vector to generate a first transformed digital output vector; and storing the first transformed digital output vector in the memory unit.

In some embodiments, the computing system may have a first cycle period defined as the elapsed time between the step of storing the input data set in the memory unit and the step of storing the first converted digital output vector in the memory unit. time, and wherein the first cycle period may be less than or equal to 1 ns.

In some embodiments, the operations may further include: outputting an artificial neural network output generated based on the first converted digital output vector.

In some embodiments, the operations may further include generating, through the DAC unit, a second plurality of modulator control signals based on the first converted digital output vector.

In some embodiments, the input data set may further include a second digital input vector, and the operations may further include: generating a second plurality of modulator control signals based on the second digital input vector through the DAC unit; obtaining from the ADC unit Corresponding to a second plurality of digital light outputs of the light output vector of the light matrix processing unit, the second plurality of digital light outputs form a second digital output vector; performing a nonlinear conversion on the second digital output vector to generate a second converted digital output vector ; Store the second converted digital output vector in the memory unit; and output the artificial neural network output generated based on the first converted digital output vector and the second converted digital output vector, wherein the light output vector of the light matrix processing unit is based on A second optical input vector generated by the second plurality of modulator control signals is generated, and the second optical input vector is converted by the optical matrix multiplication unit based on weights defined by the passive diffractive optical element.

In some embodiments, the computing system may further include: an analog nonlinear unit disposed between the light detection unit and the ADC unit, the analog nonlinear unit configured to receive an output electrical signal from the light detection unit, apply nonlinear transfer function, and Outputting the complex conversion output electrical signal to the ADC unit, wherein the operation may further include: obtaining a first plurality of conversion digital output electrical signals corresponding to the conversion output electrical signal from the ADC unit, and the first plurality of conversion digital output electrical signals form a first conversion digit output vector; and storing the first converted digital output vector in the memory unit.

In some embodiments, the integrated circuit of the controller may be configured to generate a first plurality of modulator control signals at a frequency greater than or equal to 8 GHz.

In some embodiments, the computing system may further include: an analog memory unit disposed between the DAC unit and the light modulator, the analog memory unit configured to store a complex analog voltage and output the stored analog voltage; and The analog nonlinear unit is disposed between the light detection unit and the ADC unit. The analog nonlinear unit is configured to receive an output electrical signal from the light detection unit, apply a nonlinear transfer function, and output a complex conversion output electrical signal.

In some embodiments, analog memory cells may include complex capacitors.

In some embodiments, the analog memory unit may be configured to receive and store the conversion output electrical signal of the analog nonlinear unit, and output the stored conversion output electrical signal to the light modulator, and the operations may further include: based on Generate a first plurality of modulator control signals, store the conversion output electrical signal of the analog nonlinear unit in the analog memory unit; output the stored conversion output electrical signal through the analog memory unit; obtain a second plurality of conversions from the ADC unit Digital output electrical signals, the second plurality of converted digital output electrical signals form a second converted digital output vector; and the second converted digital output vector is stored in the memory unit.

In some embodiments, the artificial neural network calculation request may include a complex digital input vector, wherein the laser unit may be configured to generate a complex wavelength, and wherein the optical modulator may include a complex optical modulator group configured to generating a complex number of optical input vectors, each optical modulator group corresponding to a wavelength, and generating a corresponding optical input vector having a corresponding wavelength; and an optical multiplexer configured to combine the optical input vectors into a combined light including the wavelength Input vector. The light detection unit may be further configured to demultiplex wavelengths and generate a complex demultiplexed output electrical signal, and the operations may include: obtaining a complex digital demultiplexed light output from the ADC unit, and the digital demultiplexed light output forming a complex digital demultiplexed light output. a digital output vector, wherein each first digital output vector corresponds to a wavelength; perform a nonlinear transformation on each first digital output vector to generate a complex converted first digital output vector; and store the converted first digital output vector in the memory unit Digital output vectors, where each digital input vector corresponds to a light input vector.

In some embodiments, the artificial neural network calculation request may include a complex digital input vector, wherein the laser unit may be configured to generate a complex wavelength, and wherein the optical modulator may include a complex optical modulator group configured to generating a complex number of optical input vectors, each optical modulator group corresponding to a wavelength, and generating a corresponding optical input vector having a corresponding wavelength; and an optical multiplexer configured to combine the optical input vectors into a combined light including the wavelength Input vector. The operations may include: obtaining a first plurality of digital light outputs corresponding to a light output vector from the ADC unit, the light output vector including a wavelength, the first plurality of digital light outputs forming a first digital output vector; and performing nonlinearity on the first digital output vector. Converting to generate a first converted digital output vector; and storing the first converted digital output vector in the memory unit.

In some embodiments, the DAC unit may include: a 1-bit DAC unit configured to generate a complex 1-bit modulator control signal, wherein the resolution of the ADC unit may be 1 bit, and wherein the first digital input vector The resolution can be N bits. The operation may include: decomposing the first digital input vector into N 1-bit input vectors, each N 1-bit input vector corresponding to one of the N bits of the first digital input vector; generating through a 1-bit DAC unit A sequence of N 1-bit modulator control signals corresponding to N 1-bit input vectors; a sequence of N digital 1-bit light outputs corresponding to the sequence of N 1-bit modulator control signals is obtained from the ADC unit ; Constructing an N-bit digital output vector from a sequence of N digital 1-bit light outputs; performing a nonlinear transformation on the constructed N-bit digital output vector to produce a transformed N-bit digital output vector; and in a memory unit Store the converted N-bit digital output vector.

In some embodiments, the memory unit may include a digital input vector memory configured to store the first digital input vector and including at least one static random access memory.

In some embodiments, the laser unit may include: a laser source configured to generate light; and an optical power splitter configured to split the light generated by the laser source into optical outputs, wherein each optical output has approximately Same power.

In some embodiments, the optical modulator includes one of a Mach-Zehnder interference modulator, a ring resonance modulator, or an electroabsorption modulator.

In some embodiments, the light detection unit may include: a plurality of light detectors; and a plurality of amplifiers configured to convert the photocurrent generated by the light detector into an output electrical signal.

In some embodiments, the integrated circuits may include application special integrated circuits.

In some embodiments, the optical matrix processing unit may include: an input waveguide array for receiving optical input vectors; an optical interference unit in optical communication with the input waveguide array for performing conversion of the optical input vectors into a second optical signal array. Linear conversion, wherein the optical interference unit includes a passive diffractive optical element; and an output waveguide array in optical communication with the optical interference unit for guiding a second optical signal array, wherein at least one input waveguide in the input waveguide array passes through the optical interference unit In optical communication with each output waveguide in the array of output waveguides.

In another view, the computing system includes: a memory unit; a driver unit configured to generate a complex modulator control signal; a light processor including: a laser unit configured to generate a complex light output; complex light modulation The optical modulator is coupled to the laser unit and the driver unit, and the optical modulator is configured to modulate the optical output generated by the laser unit based on the modulator control signal to generate an optical input vector; the optical matrix processing unit is coupled to Connected to the light modulator and driver unit, the light matrix processing unit includes a passive diffractive optical element configured to convert the light input vector into a light output vector based on a complex weighted control signal defined by the passive diffractive optical element; and The detection unit is coupled to the light matrix processing unit and configured to generate a complex output electrical signal corresponding to the light output vector. The computing system also includes a comparator unit coupled to the light detection unit and configured to convert the output electrical signal into a complex digital 1-bit optical output; and a controller, including an integrated circuit, configured to perform the following operations: Receive an artificial neural network calculation request from a computer including an input data set including an N-bit resolution The first digit input vector; stores the input data set in the memory unit; decomposes the first digit input vector into N 1-bit input vectors, each N 1-bit input vector corresponds to N bits of the first digit input vector One of the elements; generate a sequence of N 1-bit modulator control signals corresponding to N 1-bit input vectors through the driver unit; obtain a sequence of N 1-bit modulator control signals corresponding to the comparator unit sequence of N digital 1-bit light outputs; construct an N-bit digital output vector from the sequence of N digital 1-bit light outputs; perform a nonlinear transformation on the constructed N-bit digital output vector to produce converted N-bit a digital output vector; and storing the converted N-bit digital output vector in a memory unit.

Computing system embodiments may include one or more of the following features. For example, the light matrix processing unit may include a light matrix multiplication unit configured to convert a light input vector into a light output vector that represents an input vector represented by the light input vector and a light matrix defined by the diffractive optical element. The product of matrix multiplications between given vectors.

In another aspect, a computing method is for performing an artificial neural network calculation in a computing system having a light matrix processing unit, the computing method comprising: receiving an artificial neural network calculation request from a computer including an input data set, the input data set including a first digital input vector; storing the input data set in the memory unit; generating a first plurality of modulator control signals based on the first digital input vector through a digital-to-analog converter (DAC) unit; by using a method including passive diffraction a light matrix processing unit of an arrangement of optical elements that converts a light input vector into a light output vector, wherein the light output vector represents the result of matrix processing applied to the light input vector and a given vector defined by the arrangement of the diffractive optical elements; from The analog-to-digital converter (ADC) unit obtains the corresponding optical matrix processing unit The first plurality of digital light outputs of the light output vector, the first plurality of digital light outputs form a first digital output vector; the controller performs nonlinear conversion on the first digital output vector to generate a first converted digital output vector ; Store the first converted digital output vector in the memory unit; and output the artificial neural network output generated based on the first converted digital output vector by the controller.

Embodiments of computing methods may include one or more of the following features. For example, converting the light input vector into a light output vector may include converting the light input vector into a light output vector representing a product of a matrix multiplication between the digital input vector and a given vector defined by the arrangement of the diffractive optical elements.

In another aspect, the computing method includes: providing the input information in an electronic format; converting at least a portion of the electronic input information into a light input vector; and converting the light input vector to the light input vector based on light matrix processing by a light processor including passive diffractive optical elements. optically converting into a light output vector; converting the light output vector into an electronic format; and electronically applying a nonlinear conversion to the electronically converted light output vector to provide output information in an electronic format.

Embodiments of computing methods may include one or more of the following features. For example, optically converting a light input vector into a light output vector may include converting the light input vector into a light output vector based on a light matrix multiplication between an input vector represented by the light input vector and a given vector defined by the passive diffractive optical element. The vector is optically converted into a light output vector.

In some embodiments, the calculation method may further include: repeating electro-optical conversion, optical conversion, photoelectric conversion, and electrically applied nonlinear conversion for new electronic input information corresponding to the output information provided in an electronic format.

In some embodiments, the light matrix processing for the initial light conversion and the light matrix processing for the repeated light conversion may be the same and may correspond to the same layer of the artificial neural network.

In some embodiments, the calculation method may further include: repeating electro-optical conversion, optical conversion, photoelectric conversion, and electrically applied nonlinear conversion for different parts of the electronic input information, wherein the optical matrix processing for the initial optical conversion and the repeated optical conversion are The transformed light matrix processing can be the same and correspond to a layer of the artificial neural network.

In another aspect, the computing system includes: an optical matrix processing unit configured to process an input vector of length N, wherein the optical matrix processing unit includes N+2 layers of directional couplers and N layers of phase shifters. , and N is a positive integer.

Computing system embodiments may include one or more of the following features. For example, the optical matrix processing unit may include no more than N+2 layers of directional couplers.

In some embodiments, the light matrix processing unit may include a light matrix method unit.

In some embodiments, the optical matrix processing unit may include a substrate and interconnected interferometers disposed on the substrate, wherein each interferometer includes an optical waveguide disposed on the substrate, and the directional coupler and phase shifter are interconnected interferometers. part of the ceremony.

In some embodiments, the optical matrix processing unit may include a layer of attenuators after the last layer of directional couplers.

In some embodiments, a layer of attenuators may include N attenuators.

In some embodiments, the computing system may include one or more homodyne detectors for detecting the output from the attenuator.

In some embodiments, N=3, and the optical matrix processing unit may include: an input terminal (terminal) configured to receive an input vector; a first layer directional coupler coupled to the input terminal; a first layer phase shifter , coupled to the first layer directional coupler; the second layer directional coupler, coupled to the first layer phase shifter; the second layer phase shifter, coupled to the second layer directional coupler; the third layer directional coupling The third layer phase shifter is coupled to the second layer phase shifter; the third layer phase shifter is coupled to the third layer directional coupler; the fourth layer directional coupler is coupled to the third layer phase shifter; and the fifth layer Directional coupler, coupled to the fourth layer directional coupler.

In some embodiments, N=4, and the optical matrix processing unit may include: an input terminal configured to receive an input vector; a first layer, a second layer, a third layer and a fourth layer of directional couplers, each layer directional couplers. The coupler is followed by a layer of phase shifters, with the first layer of directional couplers coupled to the input; a second-to-last layer of directional couplers coupled to the fourth layer of phase shifters; and finally The layer directional coupler is coupled to the penultimate layer directional coupler.

In some embodiments, N=8, and the optical matrix processing unit may include: an input configured to receive an input vector; eight layers of directional couplers, each layer of directional couplers followed by a layer of phase shifters, wherein the first layer The directional coupler is coupled to the input terminal; the penultimate layer directional coupler is coupled to the eighth layer phase shifter; and the final layer directional coupler is coupled to the penultimate layer directional coupler.

In some embodiments, the optical matrix multiplication unit may include: an input terminal configured to receive an input vector; N layers of directional couplers, each layer of directional couplers is followed by a layer of phase shifters, wherein the first layer of directional couplers is coupled to the input end; the penultimate layer of directional couplers, coupled to the Nth layer of directional couplers; and the final layer of directional couplers, coupled to the penultimate layer of directional couplers.

In some embodiments, N is an even number.

In some embodiments, each i-th layer directional coupler includes N/2 directional couplers, where i is an odd number, and each j-th layer directional coupler includes N/2-1 directional couplers, where j is an even number.

In some embodiments, for each i-th layer directional coupler for which i is an odd number, the k-th directional coupler may be coupled to the (2k-1)th and 2k-th outputs of the previous layer, k being from 1 An integer up to N/2.

In some embodiments, for each jth layer directional coupler where j is an even number, the mth directional coupler may be coupled to the (2m)th and (2m+1)th outputs of the previous layer, m is an integer from 1 to N/2-1.

In some embodiments, each i-th layer phase shifter may include N phase shifters, where i is an odd number, and each j-th layer phase shifter may include N-2 phase shifters, where j is an even number. .

In some embodiments, N may be an odd number.

In some embodiments, each layer of directional couplers may include (N-1)/2 directional couplers.

In some embodiments, each layer of phase shifters may include N-1 phase shifters.

In another aspect, a computing system includes: a generator configured to generate a first data set, wherein the generator includes a light matrix processing unit; and a discriminator configured to receive data including from the first data set. A second data set of data in the data set and data from a third data set, the data in the first data set has similar characteristics to the data in the third data set, and the data in the second data set is classified as from Data from the first data set or data from the third data set.

Embodiments of computing methods may include one or more of the following features. For example, the light matrix processing unit may include at least one of the following: (i) the above-mentioned light matrix multiplication unit, (ii) the above-mentioned passive diffractive optical element, or (iii) the above-mentioned light matrix processing unit.

In some embodiments, the third data set may include real data, the generator is configured to generate synthetic data similar to the real data, and the discriminator is configured to classify the data as real data or synthetic data.

In some embodiments, the generator may be configured to generate data sets for training autonomous vehicles, medical diagnostic systems, fraud detection systems, weather forecasting systems, financial forecasting systems, facial recognition systems, speech recognition At least one of a system or product defect detection system.

In some embodiments, the generator may be configured to generate an image that resembles an image of at least one of a real object or a real scene, and the discriminator is configured to Set up to classify received images as (i) images of real objects or real scenes, or (ii) synthetic images produced by the generator.

In some embodiments, real objects may include at least one of people, animals, cells, tissues, or products, and real scenes include scenes encountered by the vehicle.

In some embodiments, the discriminator may be configured to classify a received image as (i) a real person, a real animal, a real cell, a real tissue, a real product, or a real scene encountered by a vehicle, or (ii) The composite image produced by the generator.

In some embodiments, the vehicle may include at least one of a motorcycle, a car, a truck, a train, a helicopter, an airplane, a submarine, a ship, or a drone.

In some embodiments, the generator can be configured to generate images of tissues or cells associated with at least one of human disease, animal disease, or plant disease.

In some embodiments, the generator may be configured to generate images of tissues or cells associated with human disease, and the disease includes cancer, Parkinson's disease, sickle cell anemia, heart disease, cardiovascular disease, diabetes, chest disease or at least one of the skin diseases.

In some embodiments, the generator may be configured to generate images of tissue or cells associated with cancer, and the cancer may include at least one of skin cancer, breast cancer, lung cancer, liver cancer, prostate cancer, or brain cancer.

In some embodiments, the computing system may further include a random noise generator configured to generate random noise input to the generator, and the generator is configured to generate the first data set based on the random noise.

In another aspect, a computing system includes a random noise generator configured to generate random noise; and a generator configured to generate data based on the random noise, wherein the generator includes an optical matrix processing unit.

Computing system embodiments may include one or more of the following features. For example, the light matrix processing unit may include at least one of (i) the above-mentioned light matrix multiplication unit, (ii) the above-mentioned passive diffractive optical element, or (iii) the above-mentioned light matrix processing unit.

In another aspect, a computing system includes an optical circuit configured to perform a logic function on two input signals, the optical circuit including a first directional coupler having two inputs and two outputs , the two input terminals are configured to receive two input signals; the first pair of phase shifters is configured to modify the phase of the signals at the two output terminals of the first directional coupler; the second directional coupler, Having two input terminals and two output terminals, the two input terminals are configured to receive signals from the first pair of phase shifters; and the second pair of phase shifters are configured to modify two of the second directional couplers. The phase of the signal at the output.

Embodiments of computing methods may include one or more of the following features. For example, a phase shifter can be configured to cause the optical circuit to perform rotation:

In some embodiments, when input signals x ₁ and x ₂ are provided to the two input terminals of the first directional coupler, the phase shifter may be configured to cause the optical circuit to perform:

In some embodiments, the optical circuit may include a first photodetector configured to generate an absolute value of the signal from the second pair of phase shifters to cause the optical circuit to perform:

In some embodiments, the optical circuit may include a comparator configured to compare the output signal of the first light detector to a threshold to generate a binary value to cause the optical circuit to generate an output:

其產生輸出AND(x₁,x₂)和OR(x₁,x₂)。 In some embodiments, the optical circuit may include a feedback mechanism configured such that the output signal of the optical detector is fed back to the input end of the first directional coupler, and through the first directional coupling the first pair of phase shifters, the second pair of directional couplers and the second pair of phase shifters, and are detected by the light detector to cause the optical circuit to perform operations:

It produces the outputs AND(x ₁ ,x ₂ ) and OR(x ₁ ,x ₂ ).

其產生輸出AND(x₁,x₂)和OR(x₁,x₂)。 In some embodiments, the optical circuit may include: a third directional coupler having two input terminals and two output terminals, the two input terminals being configured to receive signals from the second pair of phase shifters; a third pair of phase shifters; The shifter is configured to modify the phase of the signal at the two output terminals of the third directional coupler; the fourth directional coupler has two input terminals and two output terminals, and the two input terminals are configured to receive signals from the third directional coupler. signals from the fourth pair of phase shifters; a fourth pair of phase shifters configured to modify the phases of the signals at the two output terminals of the fourth directional coupler; and a second photodetector configured to generate signals from the fourth pair of phase shifters. The absolute value of the shifter signal causes the optical circuit to perform operations:

It produces the outputs AND(x ₁ ,x ₂ ) and OR(x ₁ ,x ₂ ).

In some embodiments, a computing system may include a bitonic sorter configured to perform a sorting function of the bitonic sorter using optical circuits.

In some embodiments, a computing system may include a device configured to perform a hashing function using optical circuits.

In some embodiments, the hash function may include secure hash algorithm 2 (SHA-2).

Typically, computing systems used to perform computations use different types of operations to produce computational results, each of which is a signal (e.g., that is best suited to the underlying physics of the operation (e.g., in terms of energy consumption and/or speed)). : Electrical signal or light signal) is executed. For example, three such operations are: copying, summation, and multiplication. Replication can be performed using optical power splitting, summation can be performed using electrical current-based summation, and multiplication can be performed using optical amplitude modulation, as follows More detailed description below. An example of a calculation that can be performed using these three types of operations is multiplying a vector by a matrix (eg, as used in artificial neural network calculations). Various other calculations can be performed using these operations, which represent a general set of linear operations that can perform a variety of calculations, including: vector-vector dot product, vector-vector element aspect multiplication -wise multiplication), vector-scalar element wise multiplication, or matrix-matrix element-wise multiplication, but is not limited thereto. Some embodiments described herein show techniques and configurations for vector-matrix multiplication, but corresponding techniques and configurations may be used for any of these types of calculations.

Each perspective may have one or more of the following advantages.

Optoelectronic computing systems using electrical and optical signals as described herein may facilitate increased flexibility and/or efficiency. In the past, it was possible to combine optical (or photonic) integrated devices with electrical (or electronic) integrated devices on a common platform (e.g., a common semiconductor die, or on a controlled collapsed chip). connection) or “flip-chip” arrangement potential challenges associated with multiple semiconductor dies). Such potential challenges may include input/output (I/O) packaging or temperature control, for example. For those computing systems described herein, when used with a relatively large number of optical input/output ports (ports) and a relatively large number of electrical input/output ports (e.g., 4 or more optical input/output ports, 200 or more electrical input/output ports), may increase potential challenges. For example, in a controlled sag height chip connection, a semiconductor die with a photonic integrated circuit (for example: referring to Figure 1A to implement the optical processor described below) may include electrical input ports and electrical output ports, and corresponding The electrical output port and the electrical input port of the electronic integrated circuit are connected (for example: refer to Figure 1A to implement the following controller 110, memory unit 120, digital to analog converter (DAC) unit 130, and/or analog digital converter (ADC) unit 160). For example, to control the sag height, chip connections can use solder balls (or solder bumps) composed of alloys. The solder balls are in direct contact with metal pads integrated into the die. The bonding of wires and pads eliminates the need for more complex and less compact processes. Packaging requirements. These potential challenges can be mitigated using appropriate system design. For example, computing systems may use high-density packaging arrangements that use temperature control (e.g., thermoelectric cooling) to control different material types (e.g., semiconductor materials (e.g., silicon), glass materials (silica), or "silica"). )", ceramic materials, etc.), and/or use an enclosing housing as a heat sink and provide a certain degree of sealing. With this temperature stabilization technology, different thermal expansion coefficients can be limited ( coefficients of thermal expansion; CTE) and misalignment between system ports and ports of packaged high-density fiber arrays.

For copy operations, since the optical power separation is passive, no power is required to perform the operation. Additionally, the bandwidth of an electrical separator has limitations related to the RC time constant. In contrast, the bandwidth of an optical splitter is essentially unlimited. Different types of optical power splitters may be used, including waveguide optical splitters or free-space beam splitters, as described in more detail below.

For multiplication operations, one value can be encoded as an optical signal and another value can be encoded as an amplitude scaling coefficient (eg, multiplying a value in the range 0 to 1). After setting the scaling factor, the multiplication operation in the optical domain requires less (or no) adjustment of the electrical signal, thereby reducing constraints due to electrical noise, power consumption, and bandwidth limitations. By appropriately choosing the detection scheme, signed results (e.g., multiplied by a value between -1 and +1) can be obtained, as described in more detail below.

For the summation operation, different techniques can be used to achieve a result in which the magnitude of the current in the conductor is determined based on the sum of the different contributions. In the case of input current signals, a single conductor carrying an output current signal represents the sum of those input current signals when two or more conductors carrying those input current signals are combined at a junction. In the case of an input optical signal, when two or more light waves of different wavelengths are illuminated on the detector, the current signal carried on the photocurrent generated by the detector represents the sum of the power in the input optical signal. Both produce an electrical signal (e.g. current) as an output representing the sum, but one uses current as input (current input-based summation (current-input-based summation, also known as "electrical summation" performed in the "electrical domain") while the other uses light waves as input (light-input-based summation (optical-input-based summation), also known as "optoelectronic summation" performed in the "optoelectronic domain"). However, in some embodiments, summation based on current inputs is used rather than summing based on optical inputs, which enables a single optical wavelength to be used in the system, avoiding potentially complex components that may be required to provide the system and maintaining multiple wavelengths .

Combinations of these basic operations performed by these modules may be configured to provide means for performing linear operations such as vector-matrix multiplication with arbitrary matrix element magnitude. Other implementations of matrix multiplication using optical signals and interferometers for combining signals using optical interference (the description here does not use copy modules or summation modules) have been limited to providing a single matrix with certain restrictions (e.g. or diagonal matrix) vector matrix multiplication. Additionally, some other embodiments may rely on large-scale phase alignment of multiple optical signals as they propagate through a relatively large number of optical elements (eg, light modulators). Alternatively, embodiments described herein may relax this phase alignment constraint by converting the optical signal into an electrical signal after propagating through fewer optical elements (e.g., after propagating through no more than a single optical amplitude modulation). This allows the use of optical signals with reduced coherence, or even incoherent optical signals using optical modulators that do not rely on constructive/destructive interference.

For time domain encoding of optical and electrical signals, described in more detail below, analog electronic circuits can be optimized to operate at specific power levels. This may be helpful if the circuit operates at high speeds. . Such temporal encoding is useful in reducing any challenges that might be associated with precisely controlling the relatively large number of clearly distinguishable intensity levels for each symbol. In contrast, when applying precise control of the duty cycle in the time domain over multiple time slots within a single symbol duration, a relatively constant amplitude can be used (for " "On" level, with zero or near-zero amplitude at "Off" level).

By integrating photonic and electronic devices on a common substrate (e.g., silicon wafer), or by connecting the fabricated dies using a flip-chip configuration as described above, coupling in compact systems can be easily fabricated at scale. Mods. Routing signals on the substrate as optical signals rather than electrical signals can help avoid Long electronic traces and their associated challenges (eg, parasitic capacitance, inductance, and crosstalk).

For embodiments of a computing system using submatrix multiplication, different devices (e.g., different cores, different processors, different computers, different servers) can be used to simultaneously calculate each element of the output vector, which helps to alleviate some potential limitations (such as memory walls) and help the entire system scale to very large matrices. In some embodiments, different means may be used to multiply each sub-matrix by the corresponding sub-vector. This can then be done by collecting or accumulating items from different installations Set the summand (summand) to calculate the sum. Intermediate results in the form of optical signals can be easily transmitted between devices, even if they are separated by relatively large distances.

Other concepts include other combinations of the features described above and other features represented as methods, devices, systems, program products, and other means.

Specific embodiments of the subject matter described herein can be implemented to achieve one or more of the following advantages. ANN can be improved to calculate throughput, latency, or both. The power efficiency of ANN calculations can be improved.

In another aspect, a computing device includes: a plurality of optical waveguides, wherein a set of a plurality of input values is encoded on a corresponding optical signal carried by the optical waveguide; a plurality of replica modules, and for at least two of the one or more optical signals each of the subsets, a corresponding set of one or more replication modules configured to divide the one or more subsets of the optical signal into two or more copies of the optical signal; the complex multiplication module, And for each of the at least two copies of the first subset of the one or more optical signals, a corresponding multiplication module is configured to multiply the one or more optical signals of the first subset using optical amplitude modulation. With one or more matrix element values, at least one multiplication module includes an optical amplitude modulator, the optical amplitude modulator includes an input port and two output ports, and provides a pair of correlated optical signals from the two output ports, such that the difference between the amplitudes of the relevant optical signals corresponds to the result of multiplying the input value by the signed matrix element value; and one or more summation modules, and for the results of two or more multiplication modules, correspondingly A summation module is configured to generate an electrical signal representing the sum of the results of two or more multiplication modules.

Computing device embodiments may include one or more of the following features. For example, input values encoded on respective optical signals in a set of multiple optical signals may represent elements of an input vector multiplied by a matrix including one or more matrix element values.

In some embodiments, a set of multiple output values may be encoded on a corresponding electrical signal generated by one or more summation modules, and an output value in the set of multiple output values may represent an element of an output vector. , the output vector is generated by multiplying the input vector by the matrix.

In some embodiments, each optical signal carried by the optical waveguide includes an optical wave having a common wavelength, which is approximately the same for all optical signals.

In some embodiments, the replica module may include at least one replica module having an optical splitter that sends a predetermined proportion of the power of the light wave at the input port to the first output port, and the power of the light wave at the input port. The remaining proportion is sent to the second output port.

In some embodiments, the optical splitter may include a waveguide optical splitter that sends a determined proportion of the power of the optical wave guided by the input optical waveguide to the first output optical waveguide, and that transmits a prescribed proportion of the power of the optical wave guided by the input optical waveguide. The remaining proportion of the power is sent to the second output optical waveguide.

In some embodiments, the guided mode of the input optical waveguide can be adiabatically coupled to the plurality of guided modes of each of the first and second output optical waveguides.

In some embodiments, the optical splitter may include a beam splitter including at least one surface that transmits a determined proportion of the power of the light wave at the input port and reflects a remaining proportion of the power of the light wave at the input port.

In some embodiments, at least one optical waveguide may include an optical fiber coupled to an optical coupler that couples the guided mode of the optical fiber to a free-space propagation mode.

In some embodiments, the multiplication module may include at least one coherence-sensitive multiplication module. The coherence-sensitive multiplication module is configured to use optical amplitude modulation to convert the first One or more optical signals of a subset are multiplied by one or more matrix element values. The optical wave has a coherence length that is at least as long as the propagation distance through the coherence sensitive multiplication module.

In some embodiments, the coherence-sensitive multiplication module may include a Mach-Zehnder interferometer (MZI) that divides light waves guided by the input optical waveguide into a first optical waveguide arm of the MZI arm) and the second optical waveguide arm of the Mach-Zehnder interferometer, the first optical waveguide arm includes a phase shifter, the phase shifter generates a relative phase shift with respect to the phase delay of the second optical waveguide arm, and the Mach-Zehnder interferometer will The plurality of light waves from the first optical waveguide arm and the second optical waveguide arm are combined into at least one output optical waveguide.

In some embodiments, the MZI can combine the plurality of light waves from the first and second optical waveguide arms into each of the first and second output optical waveguides, and the first optical detector can receiving light waves from the first output optical waveguide to generate a first The second photodetector may receive a light wave from the second output optical waveguide to generate a second photocurrent, and the result of the coherent sensitive multiplication module may include a difference between the first photocurrent and the second photocurrent.

In some embodiments, the coherence-sensitive multiplying module may include one or more ring resonators, including at least one ring resonator coupled to the first optical waveguide and coupled to the second optical waveguide. at least one ring resonator.

In some embodiments, the first photodetector can receive the light wave from the first optical waveguide to generate the first photocurrent, and the second photodetector can receive the light wave from the second optical waveguide to generate the second light. current, and the result of the coherence sensitive multiplication module may include a difference between the first photocurrent and the second photocurrent.

In some embodiments, the multiplication module may include at least one coherence-insensitive multiplication module, and the coherence-insensitive multiplication module is configured to use optical amplitude modulation to convert the third A subset of one or more optical signals is multiplied by one or more matrix element values.

In some embodiments, the coherent insensitive multiplication module may include an electro-absorption modulator.

In some embodiments, one or more summing modules may include at least one summing module having the following components: (1) two or more input conductors, each input conductor carrying an electrical signal in the form of an input current, The magnitude of the input current represents the corresponding result of one of the multiplication modules, and (2) at least one output conductor carrying an electrical signal representative of the sum of the corresponding results in the form of an output current, the output current being proportional to the sum of the input currents.

In some embodiments, two or more input conductors and output conductors may include a plurality of conductors that meet at one or more junctions between the conductors, and the output current is approximately equal to the sum of the input currents.

In some embodiments, at least a first input current of the input current can be provided in the form of at least one photocurrent, the photocurrent is generated by at least one photodetector, and the photodetector receives the first multiplication module from the multiplication module. The light signal generated by the group.

In some embodiments, the first input current may be provided in the form of a difference between two photocurrents generated by different corresponding photodetectors, and the photodetectors receive the data generated by the first multiplication module. Different corresponding light signals generated.

In some embodiments, one of the copies of the first subset of the one or more optical signals may consist of a single optical signal on which one input value is encoded.

In some embodiments, a multiplication module corresponding to a copy of the first subset may multiply the encoded input value by a single matrix element value.

In some embodiments, one of the copies of the first subset of one or more optical signals may include more than one optical signal and less than the number of all optical signals in which multiple input values of the optical signal are encoded .

In some embodiments, a multiplication module corresponding to a copy of the first subset may multiply the encoded input value by a different corresponding matrix element value.

In some embodiments, different multiplication modules corresponding to different corresponding copies of the first subset of one or more optical signals may be included in different devices, and the different devices perform One that performs optical communications to transmit one or more copies of a first subset of an optical signal between different devices.

In some embodiments, at least one of two or more optical waveguides, two or more replication modules, two or more multiplication modules, and one or more summing modules may be disposed in a common on the base of the device.

In some embodiments, the device performs vector matrix multiplication, where the input vectors can be provided as a set of optical signals and the output vectors can be provided as a set of electrical signals.

In some embodiments, the computing device may further include an accumulator that integrates the input electrical signal corresponding to the output of the multiplication module or the summation module, wherein time domain encoding (time domain encoding) may be used to encode the input electrical signal, Time domain encoding uses on-off amplitude modulation within each of multiple time slots, and the accumulator can produce an output electrical signal that is encoded with more than two amplitude levels. Levels correspond to different duty cycles of time domain encoding over multiple time slots.

In some embodiments, each of the two or more multiplying modules corresponds to a different subset of one or more optical signals.

In some embodiments, the computing device may further include a multiplication module for each replica of the second subset of one or more optical signals, as opposed to the optical signals in the first subset of one or more optical signals. Configured to multiply the one or more optical signals of the second subset by one or more matrix element values using optical amplitude modulation.

In another aspect, a computational method includes encoding a set of a plurality of input values on a corresponding optical signal; for each of at least two subsets of the one or more optical signals, using one or more replica modules A corresponding group is used to divide a subset of one or more optical signals into two or more copies of the optical signal; for each of the at least two copies of the first subset of the one or more optical signals, use a corresponding The multiplication module uses optical amplitude modulation to multiply one or more optical signals of the first subset by one or more matrix element values, wherein at least one multiplication module includes an optical amplitude modulator, and the optical amplitude modulator includes one input port and two output ports, and providing a pair of correlated optical signals from the two output ports such that the difference between the amplitudes of the correlated optical signals corresponds to the result of multiplying the input value by the signed matrix element value; and for The results of two or more multiplication modules use a summation module to generate an electrical signal that represents the sum of the results of the two or more multiplication modules.

In another aspect, the calculation method includes: encoding a set of input values representing elements of the input vector on the corresponding optical signal; encoding a set of coefficients representing the matrix elements as a set of optical amplitude modulators coupled to the optical signal. Amplitude modulation level, wherein at least one optical amplitude modulator including one input port and two output ports provides a pair of correlated optical signals from the two output ports, such that the difference between the amplitudes of the correlated optical signals corresponds to the result of multiplying an input value by an element value of a signed matrix; and a set of output values encoding elements representing an output vector on a corresponding electrical signal, at least one of which is in the form of a current with an amplitude corresponding to the corresponding element of the input vector multiplied by the matrix The sum of the corresponding elements of a row.

Embodiments of computing methods may include one or more of the following features. For example, at least one optical signal may be provided by a first optical waveguide, and the first optical waveguide may to be coupled to an optical splitter that sends a determined proportion of the power of the light wave guided by the first optical waveguide to the second output optical waveguide and a remaining proportion of the power of the optical wave guided by the first optical waveguide. to the third optical waveguide.

In another aspect, a computing device includes: a complex optical waveguide encoding a set of input values representing elements of an input vector on a corresponding optical signal carried by the optical waveguide; a set of optical amplitude modulators coupled to the optical signal, Encoding a set of coefficients representing matrix elements as amplitude modulation levels, at least one optical amplitude modulator including one input port and two output ports provides a pair of correlated optical signals from the two output ports, such that the correlated optical signals The difference between the amplitudes corresponds to the result of multiplying the input value by the signed matrix element value; the complex summation module encodes a set of output values representing the elements of the output vector on the corresponding electrical signal, in which at least one telecommunication signal The signal is the form of a current whose amplitude corresponds to the corresponding element of the input vector multiplied by the sum of the corresponding elements of a column (row) of the matrix.

In another aspect, a computational method for multiplying an input vector by a given matrix includes: encoding a set of input values representing elements of the input vector on a corresponding set of optical signals; converting a first set of one or more a device coupled to a first set of one or more waveguides, providing a first subset of the set of optical signals, and generating a first sub-matrix of a given matrix multiplied by a value over the first subset of the set of optical signals the result of; coupling a second set of one or more devices to a second set of one or more waveguides, providing a second subset of the set of optical signals, and producing a second sub-matrix of the given matrix multiplied by the set of a result of values on a second subset of optical signals; coupling a third set of one or more devices to a third set of one or more waveguides to provide a third set of optical signals generated by the first optical splitter a copy of a subset and produces a third submatrix of the given matrix multiplied by the value on the first subset of the set of optical signals Result; coupling a fourth set of one or more devices to a fourth set of one or more waveguides, providing a copy of a second subset of the set of optical signals produced by the second optical splitter, and producing a given matrix The result of multiplying the fourth sub-matrix by the value on the second subset of the set of optical signals; where the first, second, third and fourth sub-matrices connected together form a given matrix; and where represents the output At least one output value of an element of a vector is encoded on an electrical signal, the output vector corresponding to the input vector multiplied by a given matrix, the electrical signal being generated by a device communicating with the first group of one or more devices and the second group of one or more devices. produce.

Embodiments of computing methods may include one or more of the following features. For example, each pair of a first group of one or more devices, a second group of one or more devices, a third group of one or more devices, and a fourth group of one or more devices may be mutually exclusive. (mutually exclusive).

In another aspect, a computing device includes: a first set of one or more devices configured to receive a first set of optical signals and generate a first matrix multiplied by a value encoded on the first set of optical signals; A second group of one or more devices is configured to receive a second group of optical signals and generate a result of a second matrix multiplied by a value encoded on the second group of optical signals; a third group of one or more devices is configured to Receive a third group of optical signals and generate a result of a third matrix multiplied by a value encoded on the third group of optical signals; a fourth group of one or more devices configured to receive a fourth group of optical signals and generate a fourth a result of matrix multiplication by a value encoded on a fourth set of optical signals; and a configurable connection path between a first set of one or more devices, a second set of one or more devices, a third set of one or more devices, or Between two or more of the fourth set of one or more devices, wherein a first configuration of configurable connection paths is configured to (1) provide a copy of the first set of optical signals as a second set of optical signals number, at least one of the third group of optical signals or the fourth group of optical signals, and (2) combine one or more signals from the first group of one or more devices with the second group of one or more devices. One or more signals are provided to the summing module, which is configured to generate an electrical signal representing the sum of the values encoded on the signal received by the summing module.

In another aspect, a computing device includes a first set of one or more devices configured to receive a first set of optical signals and generate a result based on optical amplitude modulation of the one or more optical signals of the first set of optical signals ; A second group of one or more devices configured to receive a second group of optical signals and generate a result based on optical amplitude modulation of the one or more optical signals of the second group of optical signals; A third group of one or more devices , is configured to receive a third group of optical signals, and generates a result based on the optical amplitude modulation of one or more optical signals of the third group of optical signals; a fourth group of one or more devices is configured to receive a fourth group of optical signals signals, and an optical amplitude modulation result of one or more optical signals based on the fourth group of optical signals; and a configurable connection path between the first group of one or more devices, the second group of one or more devices, the second group of one or more devices, and the second group of one or more devices. Between two or more of three groups of one or more devices or a fourth group of one or more devices, wherein a first configuration of configurable connection paths is configured to (1) provide a copy of the first group of optical signals as a third Three sets of optical signals, or (2) providing one or more signals from the first set of one or more devices and one or more signals from the second set of one or more devices to the summing module, summing The module is configured to generate an electrical signal representing a sum of values encoded on the signal received by the summing module.

Computing device embodiments may include one or more of the following features. For example, each pair of a first group of one or more devices, a second group of one or more devices, a third group of one or more devices, and a fourth group of one or more devices may be mutually exclusive. of.

In some embodiments, a first configuration of configurable connection paths is configured to (1) provide a copy of the first set of optical signals as a third set of optical signals, and (2) convert the first set of one or more devices from the first set of optical signals. One or more signals and one or more signals from the second set of one or more devices are provided to the summing module, the summing module being configured to generate an electrical signal, the electrical signal represented in the signal received by the summing module. The sum of the values encoded in at least two different signals.

In some embodiments, a first configuration of configurable connection paths is configured to provide a copy of the first set of optical signals as a third set of optical signals, and a second configuration of configurable connection paths can be configured to provide a copy of the first set of optical signals from the first set of optical signals. One or more signals from the one or more devices and one or more signals from the second group of one or more devices are provided to a summing module configured to generate an electrical signal represented by The sum of the values encoded in the signal received by the summation module.

In another aspect, a computing device includes: a plurality of optical waveguides, wherein a set of multiple input values is encoded on a corresponding optical signal carried by the optical waveguide; a plurality of replica modules including at least one for one or more optical signals. In each of the two subsets, a corresponding set of one or more replication modules is configured to divide the one or more subsets of the optical signal into two or more replicas of the optical signal; a complex multiplication module, including In each of the at least two copies of the first subset of the one or more optical signals, a corresponding multiplication module is configured to multiply the one or more optical signals of the first subset using optical amplitude modulation. one or more numerical values; and one or more summation modules, including results for two or more multiplication modules, the summation module being configured to generate an electrical signal, the electrical signal representing the two or more multiplication modules The sum of the results, wherein the result includes at least one result encoded on the electronic signal, and the result is Derived from a copy of an optical signal that propagates through no more than a single optical amplitude modulator before being converted into an electrical signal.

In another aspect, a computing system includes: a first unit configured to generate a complex modulator control signal; and a processor including: a light source configured to provide a complex light output; the complex light modulator coupled to The light source and the first unit, the light modulator is configured to modulate the light output provided by the light source based on the modulator control signal to generate a light input vector, the light input vector includes a complex light signal; and a matrix multiplication unit, the coupling unit Connected to the light modulator and the first unit, the matrix multiplication unit is configured to convert the light input vector into an analog output vector based on the complex weight control signal. The computing system also includes a second unit coupled to the matrix multiplication unit, and the second unit is configured to convert the analog output vector into a digital output vector; and a controller, including an integrated circuit, configured to: receive an artificial Neural network calculation request, the artificial neural network calculation request includes an input data set, the input data set includes a first digital input vector; receiving a first plurality of neural network weights; and generating, through the first unit, based on the first digital input vector A first plurality of modulator control signals and a first plurality of weight control signals are generated based on the first plurality of neural network weights.

Computing system embodiments may include one or more of the following features. For example, the first unit may include a digital-to-analog converter (DAC).

In some embodiments, the second unit may include an analog-to-digital converter (ADC).

In some embodiments, a computing system may include a memory unit configured to store data sets and complex neural network weights.

In some embodiments, the integrated circuit of the controller may be further configured to perform operations including storing the input data set and the first plurality of neural network weights in the memory unit.

In some embodiments, the first unit may be configured to generate the weight control signal.

In some embodiments, the controller may include an application specific integrated circuit (ASIC), and receiving the artificial neural network calculation request may include receiving the artificial neural network calculation request from a general purpose data processor.

In some embodiments, the first unit, the processing unit, the second unit, and the controller may be disposed on at least one of a multi-chip module or an integrated circuit. The step of receiving the artificial neural network calculation request may include receiving the artificial neural network calculation request from a second data processor, wherein the second data processor may be external to the multi-chip module or integrated circuit, and the second data processor may The processing unit is coupled to the multi-chip module or integrated circuit through a communication channel, and the processing unit can process data at a data rate that is at least one order of magnitude greater than the data rate of the communication channel.

In some embodiments, the first unit, the processing unit, the second unit, and the controller may be used for an optoelectronic processing cycle that is repeated in a plurality of iterations, and the optoelectronic processing cycle includes: (1) based on at least one of the modulator control signals; at least one first light modulation operation of one, and at least one second light modulation operation of at least one based on the weighted control signal, and (2) (a) electrical summing operation or (b) electrical storage operation At least one of them.

In some embodiments, the optoelectronic processing cycle may include electrical storage operations, and the electrical storage operations are performed using a memory unit coupled to the controller, wherein The operations performed by the controller may further include storing the input data set and the first plurality of neural network weights in the memory unit.

In some embodiments, the optoelectronic processing loop may include an electrical summation operation, and the electrical summation operation may be performed using an electrical summation module within a matrix multiplication unit, wherein the electrical summation module may be configured to generate a corresponding analog The current of an element of the output vector, where the current represents the sum of the corresponding element of the light input vector multiplied by the corresponding neural network weight.

In some embodiments, an optoelectronic processing loop may include at least one signal path on which no more than one first optical modulation operation is performed in a single loop iteration based on at least one of the modulator control signals, and based on At least one of the weighted control signals performs no more than one second light modulation operation in a single loop iteration.

In some embodiments, the first light modulation operation may be performed by one of a light modulator coupled to a light source and a matrix multiplication unit of the light output, and the second light modulation operation may be performed by a light modulator included in The matrix multiplication unit is implemented in the optical modulator.

In some embodiments, an optoelectronic processing loop may include at least one signal path on which no more than one electrical storage operation is performed in a single loop iteration.

In some embodiments, the light source may include a laser unit configured to generate a light output.

In some embodiments, the matrix multiplication unit may include: an input waveguide array for receiving an optical input vector, and the optical input vector includes a first optical signal array; an optical interference unit in optical communication with the input waveguide array for performing the optical Input direction linear conversion into a second optical signal array; and an output waveguide array in optical communication with the optical interference unit for guiding the second optical signal array, wherein at least one input waveguide in the input waveguide array passes through the optical interference unit and is in Each output waveguide in the array of output waveguides communicates optically.

In some embodiments, the optical interference unit may include: a plurality of interconnected MZIs, each of the interconnected MZIs including: a first phase shifter configured to change a separation ratio of the MZI; and a second phase shifter, Configured to shift a phase of an output of the MZI, wherein the first phase shifter and the second phase shifter are coupled to the weight control signal.

In some embodiments, the matrix multiplication unit may include: a plurality of replica modules, wherein each replica module corresponds to a subset of one or more optical signals of the optical input vector, and is configured to convert a subset of the one or more optical signals The subset is divided into two or more copies of the optical signal; complex multiplication modules, wherein each multiplication module corresponds to one or more subsets of the optical signal and is configured to use optical amplitude modulation to convert one or more of the subsets an optical signal multiplied by one or more matrix element values; and one or more summation modules, wherein each summation module is configured to generate an electrical signal representing the results of two or more of the multiplication modules the sum of.

In some embodiments, at least one multiplication module includes an optical amplitude modulator. The optical amplitude modulator includes an input port and two output ports, and can provide a pair of correlated optical signals from the two output ports, such that the correlated optical signals The difference between the amplitudes of the signals corresponds to multiplying the input values by the signed matrix element values.

In some embodiments, the matrix multiplication unit may be configured to multiply the light input vector by a matrix including one or more matrix element values.

In some embodiments, a set of multiple output values may be encoded on a corresponding electrical signal generated by one or more summation modules, and an output value in the set of multiple output values may represent an element of an output vector. , the output vector is generated by multiplying the light input vector by the matrix.

In some embodiments, the computing system may include a memory unit configured to store the input data set and the neural network weights, the second unit may include an analog-to-digital converter (ADC) unit, and the operations may further include: A first plurality of digital outputs corresponding to the analog output vector of the matrix multiplication unit are obtained from the analog-to-digital converter unit, and the first plurality of digital outputs form a first digital output vector; a nonlinear conversion is performed on the first digital output vector to generate a first converting the digital output vector; and storing the first converted digital output vector in the memory unit.

In some embodiments, the computing system has a first cycle, the first cycle being defined as the steps of storing the input data set and the first plurality of neural network weights in the memory unit and storing the first plurality of neural network weights in the memory unit. The time elapsed between steps to convert the digital output vector, and where the first cycle period is less than or equal to 1 ns.

In some embodiments, the operations may further include: outputting an artificial neural network output generated based on the first converted digital output vector.

In some embodiments, the first unit may include a digital-to-analog converter (DAC) unit, and the operations may further include: generating, through the digital-to-analog converter unit, a second plurality of modulator control signals based on the first converted digital output vector. .

In some embodiments, the first unit may include a digital-to-analog converter (DAC) unit, the artificial neural network calculation request may further include a second plurality of neural network weights, and the operations may further include: based on the first plurality of The digital output is obtained through a digital-to-analog converter unit that generates a second plurality of weight control signals based on a second plurality of neural network weights.

In some embodiments, the first plurality of neural network weights and the second plurality of neural network weights may correspond to different layers of the artificial neural network.

In some embodiments, the first unit may include a digital-to-analog converter (DAC) unit, and the input data set may further include a second digital input vector. The operations may further include: generating a second plurality of modulator control signals based on the second digital input vector through the digital-to-analog converter unit; and obtaining the second plurality of digital bits corresponding to the analog output vector of the matrix multiplication unit from the analog-to-digital converter unit. Output, a second plurality of digital outputs to form a second digital output vector; perform a nonlinear transformation on the second digital output vector to generate a second transformed digital output vector; store the second transformed digital output vector in the memory unit; and output based on An artificial neural network output generated by the first transformed digital output vector and the second transformed digital output vector. The analog output vector of the matrix multiplication unit may be generated by a second optical input vector generated based on the second plurality of modulator control signals, the second optical input vector being converted by the matrix multiplication unit based on the first mentioned weight control signal.

In some embodiments, the computing system may include a memory unit configured to store the input data set and the neural network weights, and the second unit may include an analog-to-digital converter (ADC) unit. The computing system may further include: an analog nonlinear unit, which is disposed between the matrix multiplication unit and the analog-to-digital converter unit. The nonlinear unit may be configured to receive the complex output voltage from the matrix multiplication unit, apply the nonlinear transfer function, and output the complex conversion output voltage to the analog-to-digital converter unit. The operations performed by the integrated circuit of the controller may further include: obtaining a first plurality of converted digital output voltages corresponding to the converted output voltage from the analog-to-digital converter unit, and the first plurality of converted digital output voltages forming a first converted digital output vector; and storing the first converted digital output vector in the memory unit.

In some embodiments, the integrated circuit of the controller may be configured to generate a first plurality of modulator control signals at a frequency greater than or equal to 8 GHz.

In some embodiments, the first unit may include a digital-to-analog converter (DAC) unit and the second unit may include an analog-to-digital converter (ADC) unit. The matrix multiplication unit may include: an optical matrix multiplication unit coupled to the light modulator and the digital-to-analog converter unit, the optical matrix multiplication unit being configured to convert the light input vector into the light output vector based on the weight control signal; and light detection A unit coupled to the optical matrix multiplication unit and configured to generate a complex output voltage corresponding to the optical output vector.

In some embodiments, the computing system may further include: an analog memory unit disposed between the DAC unit and the light modulator, the analog memory unit configured to store a complex analog voltage and output the stored analog voltage; and The analog nonlinear unit is disposed between the light detection unit and the ADC unit, and is configured to receive an output voltage from the light detection unit, apply a nonlinear transfer function, and output a complex conversion output voltage.

In some embodiments, analog memory cells may include complex capacitors.

In some embodiments, the analog memory unit may be configured to receive and store the converted output voltage of the analog nonlinear unit and output the stored converted output voltage to the light modulator. The operations may further include: storing the conversion output voltage of the analog nonlinear unit in the analog memory unit based on generating the first plurality of modulator control signals and the first plurality of weight control signals; and outputting the stored conversion through the analog memory unit. Output voltage; obtain a second plurality of converted digital output voltages from the analog-to-digital converter unit, the second plurality of converted digital output voltages form a second converted digital output vector; and store the second converted digital output vector in the memory unit.

In some embodiments, the computing system may include a memory unit configured to store the input data set and the neural network weights, and the input data set requested by the artificial neural network calculation may include a complex numeric input vector. The light source can be configured to produce a complex number of wavelengths. The optical modulator may include: a plurality of optical modulator groups configured to generate a plurality of optical input vectors, each optical modulator group corresponding to a wavelength and generating a corresponding optical input vector having a corresponding wavelength; and an optical multiplexer The user is configured to combine the light input vectors into a combined light input vector including wavelengths, wherein the light detection unit may be further configured to demultiplex the wavelengths and generate a plurality of demultiplexed output voltages. The operation may include: obtaining a complex digital demultiplexed light output from the analog-to-digital converter unit, and the digital demultiplexed light output forming a complex first digital output vector, wherein each first digital output vector corresponds to a wavelength; for each first performing a non-linear conversion on the digital output vector to generate a complex converted first digital output vector; and storing the converted first digital output vector in the memory unit. Each digital input vector corresponds to a light input vector.

In some embodiments, the computing system may include a memory unit configured to store the input data set and the neural network weights, the second unit may include an analog-to-digital converter (ADC) unit, and the artificial neural network calculation request may include Complex digit input vector. The light source can be configured to produce a complex number of wavelengths. The optical modulator may include: a plurality of optical modulator groups configured to generate a plurality of optical input vectors, each optical modulator group corresponding to one wavelength, and generating a corresponding optical input vector having a corresponding wavelength; and an optical multiplexer A multiplexer configured to combine the optical input vectors into a combined optical input vector including wavelengths. The operations may include: obtaining a first plurality of digital light outputs corresponding to a light output vector from the analog-to-digital converter unit, the light output vector including a wavelength, and the first plurality of digital light outputs forming a first digital output vector; performing a non-linear transformation to generate a first transformed digital output vector; and storing the first transformed digital output vector in a memory unit.

In some embodiments, the first unit may include a digital-to-analog converter (DAC) unit, the second unit may include an analog-to-digital converter (ADC) unit, and the digital-to-analog converter unit may include: a 1-bit digital-to-analog converter A subunit configured to generate a plurality of 1-bit modulator control signals. The resolution of the analog-to-digital converter unit may be 1 bit, and the resolution of the first digital input vector may be N bits. The operation may further include: decomposing the first digit input vector into N 1-bit input vectors, each of the N 1-bit input vectors corresponding to one of the N bits of the first digit input vector; through the 1-bit The digital-to-analog converter subunit generates a sequence of N 1-bit modulator control signals corresponding to N 1-bit input vectors; and obtains a sequence of N 1-bit modulator control signals corresponding to the analog-to-digital converter unit. Sequence of N digital 1-bit light outputs; starting from A sequence of N digital 1-bit light outputs constructs an N-bit digital output vector; performs a nonlinear transformation on the constructed N-bit digital output vector to produce a transformed N-bit digital output vector; and stores the transformation in a memory unit N-bit digital output vector.

In some embodiments, a computing system may include a memory unit configured to store input data sets and neural network weights. The memory unit may include: a digital input vector memory configured to store the first digital input vector and including at least one static random access memory; and a neural network weight memory configured to store the neural network weights, and includes at least one dynamic random access memory.

In some embodiments, the first unit may include a digital-to-analog converter (DAC) unit including: a first digital-to-analog converter subunit configured to generate the modulator control signal; and a second digital-to-analog converter subunit. The analog converter sub-unit is configured to generate the weight control signal, wherein the first digital-to-analog converter sub-unit and the second digital-to-analog converter sub-unit are different.

In some embodiments, the light source may include: a laser source configured to generate light; and an optical power splitter configured to split the light generated by the laser source into optical outputs, wherein each optical output has substantially the same power.

In some embodiments, the optical modulator includes one of an MZI modulator, a ring resonance modulator, or an electroabsorption modulator.

In some embodiments, the light detection unit may include: a plurality of light detectors; and a plurality of amplifiers configured to convert the photocurrent generated by the light detector into an output voltage.

In some embodiments, the integrated circuit may be an application special integrated circuit.

In some embodiments, the computing system may include a complex optical waveguide coupled between the optical modulator and the matrix multiplication unit, wherein the optical input vector may include a set of multiple input values, the set of multiple input values being The encoding is on a corresponding optical signal carried by an optical waveguide, and each optical signal carried by an optical waveguide may include light waves having a common wavelength, which is approximately the same for all optical signals.

In some embodiments, the replica module may include at least one replica module having an optical splitter that sends a predetermined proportion of the power of the light wave at the input port to the first output port, and the power of the light wave at the input port. The remaining proportion is sent to the second output port.

In some embodiments, the optical splitter may include a waveguide optical splitter that sends a determined proportion of the power of the optical wave guided by the input optical waveguide to the first output optical waveguide, and that transmits a prescribed proportion of the power of the optical wave guided by the input optical waveguide. The remaining proportion of the power is sent to the second output optical waveguide.

In some embodiments, the guided mode of the input optical waveguide can be adiabatically coupled to the plurality of guided modes of each of the first and second output optical waveguides.

In some embodiments, the optical splitter may include a beam splitter including at least one surface that transmits a determined proportion of the power of the light wave at the input port and reflects a remaining proportion of the power of the light wave at the input port.

In some embodiments, at least one optical waveguide may include an optical fiber coupled to an optical coupler that couples the guided mode of the optical fiber to the free space propagating mode.

In some embodiments, the multiplication module may include at least one coherence-sensitive multiplication module configured to use optical amplitude modulation to convert one or more of the first subset based on interference between complex light waves. The optical signal is multiplied by one or more matrix element values, and the light wave has a coherence length that is at least as long as the propagation distance through the coherence-sensitive multiplication module.

In some embodiments, the coherence-sensitive multiplication module may include a Mach-Zehnder interferometer (MZI) that divides light waves guided by an input optical waveguide into a first optical waveguide arm of the MZI and a Mach-Zehnder interferometer. The second optical waveguide arm of the Del interferometer, the first optical waveguide arm includes a phase shifter, the phase shifter generates a relative phase shift with respect to the phase delay of the second optical waveguide arm, and the Mach-Zehnder interferometer can convert light from the first optical waveguide arm. The plurality of light waves of the waveguide arm and the second optical waveguide arm are combined into at least one output optical waveguide.

In some embodiments, the MZI can combine the plurality of light waves from the first and second optical waveguide arms into each of the first and second output optical waveguides, and the first optical detector can The light wave is received from the first output optical waveguide to generate the first photocurrent, the second photodetector can receive the light wave from the second output optical waveguide to generate the second photocurrent, and the result of the coherent sensitive multiplication module can include the first light The difference between the current and the second photocurrent.

In some embodiments, the coherence-sensitive multiplying module may include one or more ring resonators, including at least one ring resonator coupled to the first optical waveguide and at least one ring resonator coupled to the second optical waveguide. Resonator.

In some embodiments, the first photodetector can receive the light wave from the first optical waveguide to generate the first photocurrent, and the second photodetector can receive the light wave from the second optical waveguide to generate the second light. current, and the result of the coherence sensitive multiplication module may include a difference between the first photocurrent and the second photocurrent.

In some embodiments, the multiplication module may include at least one coherent insensitive multiplication module configured to use optical amplitude modulation to convert one or more of the first subset based on energy absorption within the light wave. The optical signal is multiplied by one or more matrix element values.

In some embodiments, the coherent insensitive multiplying module may include an electroabsorption modulator.

In some embodiments, one or more summing modules may include at least one summing module having the following components: (1) two or more input conductors, each input conductor carrying an electrical signal in the form of an input current, The magnitude of the input current represents the corresponding result of one of the multiplication modules, and (2) at least one output conductor carrying an electrical signal representative of the sum of the corresponding results in the form of an output current, the output current being proportional to the sum of the input currents.

In some embodiments, two or more input conductors and output conductors may include a plurality of conductors that meet at one or more junctions between the conductors, and the output current is approximately equal to the sum of the input currents.

In some embodiments, at least a first input current of the input current can be provided in the form of at least one photocurrent, the photocurrent is generated by at least one photodetector, and the photodetector receives the first multiplication module from the multiplication module. The light signal generated by the group.

In some embodiments, the first input current may be provided in the form of a difference between two photocurrents generated by different corresponding photodetectors, and the photodetectors receive the data generated by the first multiplication module. Different corresponding light signals generated.

In some embodiments, one of the copies of the first subset of the one or more optical signals may consist of a single optical signal on which one input value is encoded.

In some embodiments, a multiplication module corresponding to a copy of the first subset may multiply the encoded input value by a single matrix element value.

In some embodiments, one of the copies of the first subset of one or more optical signals may include more than one optical signal and less than the number of all optical signals in which multiple input values of the optical signal are encoded .

In some embodiments, a multiplication module corresponding to a copy of the first subset may multiply the encoded input value by a different corresponding matrix element value.

In some embodiments, different multiplication modules corresponding to different respective copies of the first subset of one or more optical signals may be included in different devices, and the different devices may optically communicate to transmit one or more of the optical signals between the different devices. One of the copies of the first subset of the optical signal.

In some embodiments, at least one of two or more optical waveguides, two or more replication modules, two or more multiplication modules, and one or more summing modules may be disposed in a common on the base of the device.

In some embodiments, the device performs vector matrix multiplication, where the input vectors can be provided as a set of optical signals and the output vectors can be provided as a set of electrical signals.

In some embodiments, the computing device may further include an accumulator that integrates the input electrical signal corresponding to the output of the multiplication module or the summation module, wherein the input electrical signal may be encoded using time domain coding. The time domain coding is performed on multiple Switching amplitude modulation is used within each of the time slots, and the accumulator can generate an output electrical signal encoded with more than two amplitude levels, the amplitude levels corresponding to the time domain encoding on the multiple time slots of different duty cycles.

In some embodiments, each of the two or more multiplying modules corresponds to a different subset of one or more optical signals.

In some embodiments, the computing device may further include a multiplication module for each replica of the second subset of one or more optical signals, as opposed to the optical signals in the first subset of one or more optical signals. Configured to multiply the one or more optical signals of the second subset by one or more matrix element values using optical amplitude modulation.

In another aspect, a computing system includes a memory unit configured to store a data set and a complex neural network weight; and a driver unit configured to generate a complex modulator control signal. The computing system includes an optoelectronic processor, the optoelectronic processor including: a light source configured to provide a complex light output; a complex light modulator coupled to the light source and a driver unit, the light modulator is configured to modulate the light output generated by the light source to generate a light input vector based on the modulator control signal; the matrix multiplication unit is coupled to the light modulator and the driver unit, the matrix The multiplication unit is configured to convert the optical input vector into an analog output vector based on the complex weight control signal; and the comparator unit is coupled to the matrix multiplication unit and configured to convert the analog output vector into a complex digital 1-bit output. The computing system includes a controller including an integrated circuit configured to: receive an artificial neural network calculation request including an input data set and a first plurality of neural network weights, wherein the input data set includes an N-bit The first digit input vector of element resolution; store the input data set and the first plurality of neural network weights in the memory unit; decompose the first digit input vector into N 1-bit input vectors, each with N 1's The bit input vector corresponds to one of the N bits of the first digital input vector; the driver unit generates a sequence of N 1-bit modulator control signals corresponding to the N 1-bit input vectors; the corresponding sequence is obtained from the comparator unit A sequence of N digital 1-bit outputs from a sequence of N 1-bit modulator control signals; constructs an N-bit digital output vector from a sequence of N digital 1-bit light outputs; constructs an N-bit digital output vector Performing a nonlinear transformation on the vector to generate a transformed N-bit digital output vector; and storing the transformed N-bit digital output vector in a memory unit.

Computing system embodiments may include one or more of the following features. For example, receiving the artificial neural network calculation request may include receiving the artificial neural network calculation request from a general purpose computer.

In some embodiments, the driver unit may be configured to generate the weight control signal.

In some embodiments, the matrix multiplication unit may include: an optical matrix multiplication unit coupled to the light modulator and the driver unit, the optical matrix multiplication unit configured to convert the light input vector into the light output vector based on the weight control signal; and The light detection unit is coupled to the light matrix multiplication unit and configured to generate a complex output voltage corresponding to the light output vector.

In some embodiments, the matrix multiplication unit may include: an input waveguide array for receiving the light input vector; an optical interference unit in optical communication with the input waveguide array for performing linear conversion of the light input vector into the second optical signal array. conversion; and an output waveguide array in optical communication with the optical interference unit for guiding a second optical signal array, wherein at least one input waveguide in the input waveguide array passes through the optical interference unit and each output waveguide in the output waveguide array is optically Communication.

In some embodiments, the optical interference unit may include: a plurality of interconnected MZIs, each of the interconnected MZIs including: a first phase shifter configured to change a separation ratio of the MZI; and a second phase shifter, Configured to shift a phase of an output of the MZI, wherein the first phase shifter and the second phase shifter may be coupled to the weight control signal.

In some embodiments, the matrix multiplication unit may include: a complex replication module including each of at least two subsets of the one or more optical signals for the optical input vector, a corresponding A set configured to divide a subset of one or more optical signals into two or more copies of the optical signal; a complex multiplication module included for use in at least two copies of a first subset of the one or more optical signals In each, the corresponding multiplication module is configured to use optical amplitude modulation to multiply the one or more optical signals of the first subset by one or a plurality of matrix element values; and one or more summation modules including results for two or more multiplication modules, the summation module being configured to generate an electrical signal representing the two or more multiplication modules the sum of the results.

In some embodiments, at least one multiplication module may include an optical amplitude modulator. The optical amplitude modulator may include an input port and two output ports, and may provide a pair of correlated optical signals from the two output ports, such that the correlation The difference between the amplitudes of the optical signals corresponds to multiplying the input value by the signed matrix element value.

In some embodiments, the matrix multiplication unit may be configured to multiply the light input vector by a matrix including one or more matrix element values.

In some embodiments, a set of multiple output values may be encoded on a corresponding electrical signal generated by one or more summation modules, and an output value in the set of multiple output values may represent an element of an output vector. , the output vector is generated by multiplying the light input vector by the matrix.

In another aspect, a computational method is provided for performing artificial neural network calculations in a system having a matrix multiplication unit configured to convert an optical input vector into an analog output vector based on a complex weight control signal. The calculation method includes: receiving an artificial neural network calculation request including an input data set and a first plurality of neural network weights, wherein the input data set includes a first digital input vector; and storing the input data set and the first plurality of neural network weights in a memory unit. neural network weights; generating a first plurality of modulator control signals based on the first digital input vector, and generating a first plurality of weight control signals based on the first plurality of neural network weights; obtaining the output of the corresponding matrix multiplication unit A first plurality of digital outputs of the vector, the first plurality of digital outputs form a first digital output direction quantity; performing a nonlinear conversion on the first digital output vector by the controller to generate a first converted digital output vector; storing the first converted digital output vector in the memory unit; and using the controller to output the first converted digital output vector based on the first converted digital output vector. Output vector generated by the artificial neural network output.

Embodiments of computing methods may include one or more of the following features. For example, receiving the artificial neural network calculation request may include receiving the artificial neural network calculation request from the computer through the communication channel.

In some embodiments, generating the first plurality of modulator control signals may include generating the first plurality of modulator control signals through a digital-to-analog converter (DAC) unit.

In some embodiments, obtaining the first plurality of digital outputs may include obtaining the first plurality of digital outputs from an analog-to-digital converter (ADC) unit.

In some embodiments, the calculating method may include: applying a first plurality of modulator control signals to a plurality of optical modulators coupled to the light source and the DAC unit; and using the optical modulator to modulate the modulator based on the modulator control signal. The light output generated by the laser unit is varied to generate a light input vector.

In some embodiments, the matrix multiplication unit may be coupled to the light modulator and the DAC unit, and the calculation method may include: using the matrix multiplication unit to convert the light input vector into an analog output vector based on the weight control signal.

In some embodiments, the ADC unit may be coupled to the matrix multiplication unit, and the computing method may include converting the analog output vector into the first plurality of digital outputs using the ADC unit.

In some embodiments, the matrix multiplication unit may include an optical matrix multiplication unit coupled to the optical modulator and the DAC unit. Converting the light input vector into the analog output vector may include using a light matrix multiplication unit to convert the light input vector into a light output vector based on the weight control signal. The calculation method may include: using a light detection unit coupled to the light matrix multiplication unit to generate a complex output voltage corresponding to the light output vector.

In some embodiments, the calculation method may include: receiving the light input vector at the input waveguide array; using an optical interference unit in optical communication with the input waveguide array to perform a linear conversion of the light input vector into the second optical signal array; and using An output waveguide array in optical communication with the optical interference unit guides the second optical signal array, wherein at least one input waveguide in the input waveguide array is in optical communication with each output waveguide in the output waveguide array through the optical interference unit.

In some embodiments, the optical interference unit may include: complex interconnected Mach-Zehnder interferometers (MZIs), each of the interconnected MZIs may include a first phase shifter and a second phase shifter, and the first phase The shifter and the second phase shifter may be coupled to the weight control signal. The calculation method may include using a first phase shifter to change a separation ratio of the MZI, and using a second phase shifter to shift a phase of one output of the MZI.

In some embodiments, the calculation method may include: for each of at least two subsets of the one or more optical signals of the optical input vector, using a respective set of one or more replication modules to convert one or more A subset of the optical signal is divided into two or more copies of the optical signal; for each of the at least two copies of the first subset of the one or more optical signals, a corresponding multiplication module is used to use optical amplitude modulation Multiplying one or more optical signals of the first subset by one or more matrix element values; and for the result of two or more multiplication modules As a result, a summation module is used to generate an electrical signal that represents the sum of the results of two or more multiplication modules.

In some embodiments, at least one multiplication module may include an optical amplitude modulator. The optical amplitude modulator may include an input port and two output ports, and may provide a pair of correlated optical signals from the two output ports, such that the correlation The difference between the amplitudes of the optical signals corresponds to multiplying the input value by the signed matrix element value.

In some embodiments, the calculation method may include using a matrix multiplication unit to multiply the light input vector by a matrix including one or more matrix element values.

In some embodiments, the calculation method may include encoding a set of a plurality of output values on corresponding electrical signals generated by one or more summing modules, and using an output value in the set of the plurality of output values to represent the output vector. The output vector is generated by multiplying the light input vector by the matrix.

In another aspect, the computing method includes: providing input information in an electronic format; converting at least a portion of the electronic input information into an optical input vector; optically converting the optical input vector into an analog output vector based on matrix multiplication; and nonlinearly converting the electronic input information into an analog output vector. Ground is applied to analog output vectors to provide output information in electronic format.

Embodiments of computing methods may include one or more of the following features. For example, the calculation method may further include: repeating electro-optical conversion, photoelectric conversion, and electrically applied nonlinear conversion for new electronic input information corresponding to the output information provided in an electronic format.

In some embodiments, the matrix multiplication for the initial photoelectric conversion and the matrix multiplication for the repeated photoelectric conversion may be the same and may correspond to the same layer of the artificial neural network.

In some embodiments, the matrix multiplication for the initial photoelectric conversion and the matrix multiplication for the repeated photoelectric conversion may be different and may correspond to different layers of the artificial neural network.

In some embodiments, the calculation method may further include: repeating electro-optical conversion, photoelectric conversion, and electrically applied nonlinear conversion for different parts of the electronic input information, wherein matrix multiplication for the initial photoelectric conversion and matrix multiplication for repeated photoelectric conversion are the same and correspond to the first layer of the artificial neural network.

In some embodiments, the computing method may further include: providing electronic intermediate information in an electronic format based on electronic output information for the plurality of portions of electronic input information generated by the first layer of the artificial neural network; and for electronic Each different portion of the intermediate information, repeated electro-optical conversions, photoelectric conversions, and electrically applied nonlinear transformations, where the matrix multiplication for the initial photoelectric conversion and the matrix multiplication for the repeated photoelectric conversions associated with the different portions of the electronic intermediate information are the same , and corresponds to the second layer of the artificial neural network.

In another aspect, a computational method is provided for performing artificial neural network calculations. The calculation method includes: a first unit configured to generate a complex vector control signal and a complex weight control signal; a second unit configured to provide an optical input vector based on the vector control signal; and a matrix multiplication unit coupled to the second unit and the first unit, the matrix multiplication unit is configured to control the light output based on the weight signal Convert the input vector into an output vector. The computing system includes a controller including an integrated circuit configured to: receive an artificial neural network calculation request including an input data set and a first plurality of neural network weights, wherein the input data set includes a first digit input vector; and through the first unit, generate a first plurality of vector control signals based on the first digital input vector, and generate a first plurality of weight control signals based on the first plurality of neural network weights; wherein the first unit, the second The unit, matrix multiplication unit, and controller are used in an optoelectronic processing loop that is repeated in complex iterations, and the optoelectronic processing loop includes: (1) at least two optical modulation operations, and (2) (a) an electrical summation operation or (b) At least one of the electrical storage operations.

In another aspect, a computational method is provided for performing artificial neural network calculations. The calculation method includes: providing input information in an electronic format; converting at least a portion of the electronic input information into an optical input vector; and using a set of neural network weights to convert the optical input vector into an output vector based on matrix multiplication. Provide operations and conversion operations to be performed in a photoelectric processing loop, using different corresponding sets of neural network weights and different corresponding input information, repeating the photoelectric processing loop in a plurality of iterations, and the photoelectric processing loop includes: (1) at least two light modulations operation, and at least one of (2) (a) an electrical summing operation or (b) an electrical storage operation.

The details of one or more embodiments of the subject matter described in this disclosure are set forth in the drawings and the description below. Other features, aspects, and advantages of the present disclosure will become apparent from the specification, drawings, and claims.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which they belong. like In the event of a conflict with a patent application or patent application publication incorporated by reference, the present disclosure, including definitions, controls.

100:Artificial neural network computing system

102:Computer

110:Controller

120: Memory unit

130: Digital to analog converter unit

132: First digital-to-analog converter subunit

134: Second digital-to-analog converter subunit

140: Optical processor

142:Laser unit

144:Modulator array

146:Detection unit

150:Light matrix multiplication unit

160:Analog-to-digital converter unit

152:Input waveguide

154: Optical interference unit

156:Output waveguide

157:Configuration

158:Configuration

170: Mach-Zehnder interferometer

171: First input waveguide

172: Second input waveguide

174:First phase shifter

176: Second phase shifter

178:First output waveguide

179: Second output waveguide

200:Method

210-270: Steps

290: Schematic diagram

104:Wavelength demultiplexing artificial neural network computing system

3900: Mach-Zehnder modulator

3902a, 3902b: 1×2 port multi-mode interference coupler

3094a, 3904b: arm

3906:Phase shifter

3908:Signal line

3910: Curve graph

300: Artificial Neural Network Computing System

310: Analog nonlinear unit

302:Artificial Neural Network Computing System

320: Analog memory unit

400: Artificial Neural Network Computing System

430:Drive unit

432: First drive subunit

434: Second drive subunit

460: Comparator unit

500: Artificial Neural Network Computing System

502: Light matrix multiplication unit

504: Optical processor

506: Digital to analog converter unit

600: Optical interference unit

602:Input waveguide

604:Output waveguide

700:Artificial Neural Network Computing System

702: Optical processor

704:Laser unit

706:Modulator Array

708: Light matrix multiplication unit

710:Detection unit

712:Digital-to-analog converter unit

714:Beam Array

716: Two-dimensional beam array

718: Two-dimensional output beam array

802:Light input matrix

804: Optical interference unit

806:Light output matrix

900:Artificial Neural Network Computing System

902:Controller

904: Digital to analog converter unit

906: Optical processor

908:Laser unit

910:Laser Beam

912:Modulator

914: Modulated beam

916:Light multiplication unit

918:Output beam

920:Detection unit

930:Analog-to-digital converter unit

1002:Light input

1004: Optical interference unit

1006:Light output

2500:Method

2510-2570: Steps

2600:Artificial Neural Network Computing System

2602: Optoelectronic processor

2604: The first 2D light matrix multiplication unit

2606: Second 2D light matrix multiplication unit

142a: First laser unit

144a: First modulator array

146a: First detection unit

310a: First analogy nonlinear element

142b: Second laser unit

144b: Second modulator array

146b: Second detection unit

310b: Second analogy nonlinear unit

2700:Artificial Neural Network Computing System

2702: Optoelectronic processor

2704: The first 3D light matrix multiplication unit

2706: Second 3D light matrix multiplication unit

704a: First laser unit

706a: First modulator array

710a: First detection unit

704b: Second laser unit

706b: Second modulator array

710b: Second detection unit

2800: Neural network computing system

2802: Optical processor

2804: 2D light matrix multiplication unit

2900: Neural network computing system

2902: Optical processor

2904:3D light matrix multiplication unit

3000:Artificial Neural Network Computing System

3002: Optical processor

3004:2D light matrix multiplication unit

3100:Artificial Neural Network Computing System

3102: Optical processor

3104:3D light matrix multiplication unit

1100: Photon matrix multiplier unit

1102:Modulator

1104:Interconnect Interferometer

1106:Attenuator

1108a, 1108b, 1108c, 1108d, 1108e: directional coupler layer/first layer directional coupler, second layer directional coupler, third layer directional coupler, fourth layer directional coupler, fifth layer directional coupler

1110a, 1110b, 1110c, 1110d: Phase shifter layer/first layer phase shifter, second layer phase shifter, third layer phase shifter, fourth layer phase shifter

1200, 1204, 1208: Interconnected Mach-Zehnder interferometers

1202,1206,1210:Interconnection interferometer

1212: Compact interconnect interferometer

1302:Attenuator layer

1400: Light-generated adversarial networks

1402:Discriminator

1404:Generator

1406: Initial training image set

1408: Random noise

1410:Composite image

1500: Mach-Zehnder interferometer

1502: Phase shifter

1600: Photonic circuits

1602:Detector

1604: Comparator

1606a, 1608a: first output

1606b, 1608b: Second output

1700: Photonic circuits

1704: First output

1706: Second output

1710: Photonic circuits

1712: First Mach-Zehnder interferometer

1714:First Detector

1716: Second Mach-Zehnder interferometer

1718:Second Detector

1720: First output

1722: Second output

1800: Optoelectronic computing systems

1802A, 1802B: Optical port/light source

1803:Light path

1804A, 1804B: Copy module

1806A, 1806B, 1806C, 1806D: Multiplication module

1808:Sum module

1810A, 1810B: signal

1900:System configuration

1902:Copy module

1904: Multiplication module

1906:Light detection module

1908:Sum module

1920:System configuration

1910: Optical combiner module

1912: Optoelectronic summing module

2000: Symmetric differential configuration

2002: Current subtraction module

2010,2014:Current

2012: First photodiode detector

2016: Second photodiode detector

2018:Contact

2020:Difference current

2024: Terminal

2026:Output

2028: Resistive elements

2030: Op Amp

2032:Public terminal

2040,2044:Current

2042: The first photodiode detector

2046: Second photodiode detector

2050:Op amp

2052: Inverting terminal

2054: Non-inverting terminal

2056:Output terminal

2100: Symmetric differential configuration

2102:Current subtraction module

2110:System configuration

2112:Copy module

2114A, 2114B: summation module

2200:1×2 optical amplitude modulator

2202:Input optical splitter

2204:Phase modulator

2206:Coupler

2210: Symmetric differential configuration

2212,2214:Light detector

2216:Contact

2220: Symmetric differential configuration

2221:Input port

2222: Ring Resonator

2224,2228: path

2226: Clockwise path

2230: Symmetric differential configuration

2231:Input port

2232,2234: Ring resonator

2236,2242:path

2238: Clockwise path

2240: Counterclockwise path

2300A: Optoelectronic system configuration

2302A, 2302B: Optical amplitude modulator

2303:Waveguide splitter

2306A, 2306B: Operational amplifier

2310A: Vector matrix multiplier subsystem

2300B: Optoelectronic system configuration

2310B: Vector matrix multiplier subsystem

2400A: System configuration

2402: Optical port or light source

2403, 2405A, 2405B: Group

2404:Copy module

2410: Vector matrix multiplier subsystem

2414:Sum module

2400B: System configuration

2420:Light Emitter Array

2422: Optical receiver array

2400C: System configuration

2430: Optical switch

2440: Electric switch

2400D: System configuration

2400E: System configuration

2450:Data storage subsystem

2460: Auxiliary processing subsystem

2462: Result data

3200:Artificial Neural Network Computing System

3210: Optoelectronic processor

3220: Photoelectric matrix multiplication unit

3230:Light source

1803_1,1803_2,...,1803_ m : light path

1804_1,1804_2,...,1804_ n : Copy module

1806_11,1806_21,...,1806_ m 1,1806_12,1806_22,...,1804_ m 2,1806_1 n ,1806_2 n ,...,1806_ mn : Multiplication module

1808_1,1808_2,...,1808_ n : summation module

3300:Method

3310-3370: Steps

3290: Schematic diagram

3500 wavelength demultiplexing artificial neural network computing system/system configuration

3510: Optoelectronic processor

3520: Photoelectric matrix multiplication unit

3530_11,3530_21,...,3530_ m 1,3530_12,3530_22,...,3530_ m 2,3530_1 n ,3530_2 n ,...,3530_ mn : Multiplication module

3310:Light detection module

3320:Sum module

3600, 3700, 3800: Artificial neural network computing system

4000:homodyne detector

4002: Beam splitter

4004a, 4004b: Light detector

4006:Subtractor

4008:Output

4100:Computing Systems

4102:First light signal

4104:First modulator

4120: The first modulated light signal

4106:Second light signal

4108: Second modulator

4122: Second modulation light signal

4110:Multiplexer

4112:Optical fiber

4114: Optoelectronic chip

4116a, 4116b, 4116c, 4116d, 4118a, 4118b, 4118c, 4118d: matrix multiplication module

4118: Demultiplexer

4124,4126:Copy module

4120a: Optocoupler

4122a:Light detector

4124:Third light signal

4128:Third modulator

4132: The third modulated light signal

4126:The fourth light signal

4130: Fourth modulator

4134: The fourth modulation light signal

4136:Multiplexer

4138:Optical fiber

4140: Demultiplexer

4142,4144:Copy module

4120b: Optocoupler

4122b:Light detector

4200: Modulation value probability distribution chart

4202:Modulator power diagram

4300: Mach-Zehnder interferometer modulator

4302:Input optical splitter

4304:Active phase shifter

4306: Passive phase shifter

4308:Output light combiner

4400:Modulator circuit

4402: Pump capacitor

4404: Control voltage waveform

4405:Inverter circuit

4406: Driving voltage waveform

4408:Terminal

4410:Terminal

4412:Current source

4414:switch

4500: shares

4502:Separator

4504: Optoelectronic node

4505:MZI modulator

4507:Light detector

4510: Cable arrangement

4512A, 4512B: line

4514:Tile

4515:Connection part

4516: Bump

4520: Cable arrangement

4522:Tile

4524: Root modulator

4526:node

4528, 4530: Bump

4532:Transimpedance amplifier (TIA)

4534, 4536: Waveguide part

4540: Optical cable arrangement

4542-4548: Tile

4550: Optical cable arrangement

4552:Tile

4560:Data processing circuit

4570: Data upload circuit

4580: Optoelectronic computing systems

The present disclosure is best understood from the following detailed description when read in conjunction with the drawings. It is emphasized that, by convention, the various features of the diagrams are not to scale. On the contrary, the dimensions of the various features are arbitrarily expanded or reduced for clarity.

Figure 1A is a schematic diagram of an artificial neural network (ANN) computing system.

Figure 1B is a schematic diagram of the optical matrix multiplication unit.

Figures 1C and 1D are schematic diagrams of example configurations of interconnected Mach-Zehnder interferometers (MZIs).

Figure 1E is a schematic diagram of the MZI.

Figure 1F is a schematic diagram of a wavelength division multiplexed ANN (wavelength division multiplexed ANN) calculation system.

Figure 2A is a flowchart showing a method for performing ANN calculations.

Figure 2B is a schematic diagram showing one aspect of the method of Figure 2A.

Figures 3A and 3B are schematic diagrams of the ANN computing system.

Figure 4A is a schematic diagram of an ANN computing system with 1-bit internal resolution.

Figure 4B is a mathematical representation of the operation of the ANN computing system of Figure 4A.

Figure 5 is a schematic diagram of an artificial neural network (ANN) computing system.

Figure 6 is a schematic diagram of the optical matrix multiplication unit.

Figure 7 is a schematic diagram of an artificial neural network (ANN) computing system.

Figure 8 is a schematic diagram of the optical matrix multiplication unit.

Figure 9 is a schematic diagram of an artificial neural network (ANN) computing system.

Figure 10 is a schematic diagram of the optical matrix multiplication unit.

Figure 11 is a schematic diagram of a compact matrix multiplier unit.

Figure 12A shows a schematic comparing the photon matrix multiplier unit.

Figure 12B is a schematic of a compact interconnect interferometer.

Figure 13 is a schematic diagram of a compact matrix multiplier unit.

Figure 14 is a schematic diagram of an optical generative adversarial network.

Figure 15 is a schematic diagram of a Mach-Zehnder interferometer.

Figures 16, 17A and 17B are schematic diagrams of photonic circuits.

Figure 18 is a schematic diagram of an optoelectronic computing system.

Figures 19A and 19B are schematic diagrams of the system configuration.

Figure 20A is a schematic diagram of a symmetric differential configuration.

Figure 20B and Figure 20C are circuit diagrams of the system module.

Figure 21A is a schematic diagram of a symmetric differential configuration.

Figure 21B is a schematic diagram of the system configuration.

Figure 22A is a schematic diagram of an optical amplitude modulator.

Figures 22B to 22D are schematic diagrams of optical amplitude modulators using optical detection in a symmetric differential configuration.

Figures 23A to 23C are photoelectric circuit diagrams of the system configuration.

Figures 24A-24E are schematic diagrams of computing systems using multiple optoelectronic systems.

Figure 25 is a flowchart showing a method for performing ANN calculations.

Figures 26 and 27 are schematic diagrams of the ANN computing system.

Figure 28 is a schematic diagram of a neural network computing system using a passive 2D optical matrix multiplication unit.

Figure 29 is a schematic diagram of the neural network computing system of the passive 3D light matrix multiplication unit.

Figure 30 is a schematic diagram of an artificial neural network computing system with 1-bit internal resolution, where the system uses a passive 2D optical matrix multiplication unit.

Figure 31 is a schematic diagram of an artificial neural network computing system with 1-bit internal resolution, where the system uses a passive 3D light matrix multiplication unit.

Figure 32A is a schematic diagram of an artificial neural network (ANN) computing system.

Figure 32B is a schematic diagram of the photoelectric matrix multiplication unit.

Figure 33 is a flowchart showing a method for performing ANN calculations using an optoelectronic processor.

Figure 34 is a schematic diagram showing one aspect of the method of Figure 33.

Figure 35A is a schematic diagram of a wavelength division multiplexing ANN computing system using an optoelectronic processor.

Figure 35B and Figure 35C are schematic diagrams of the wavelength division multiplexing optoelectronic matrix multiplication unit.

Figures 36 and 37 are schematic diagrams of an ANN computing system using an optoelectronic processor.

Figure 38 is a schematic diagram of an artificial neural network computing system with 1-bit internal resolution, where the system uses an optoelectronic matrix multiplication unit.

Figure 39A is a schematic diagram of a Mach-Zehnder modulator.

Figure 39B is a graph showing the intensity-voltage curve of the Mach-Zehnder modulator of Figure 39A.

Figure 40 is a schematic diagram of a homodyne detector.

Figure 41 is a schematic diagram of a computing system including optical fibers, each of which carries signals having multiple wavelengths.

Figure 42 is a graph of modulation value probability distribution and the relationship between modulator power and modulation value.

Figure 43 Schematic diagram of a Mach-Zehnder modulator.

Figure 44 is a schematic diagram of a charge-pump bandwidth enhancement circuit.

Figures 45A to 45G are schematic layout diagrams of photonic integrated circuits and electronic integrated circuits configured to control the collapse height of chip connections on some dies.

The same drawing labels and names in the drawings identify the same components.

Figure 1A shows a schematic diagram of an artificial neural network (ANN) computing system 100. ANN computing system 100 includes a controller 110, a memory unit 120, a digital-to-analog converter (DAC) unit 130, an optical processor 140, and an analog-to-digital converter (ADC) unit 160. The controller 110 is coupled to the computer 102, the memory unit 120, the DAC unit 130 and the ADC unit 160. Controller 110 includes integrated circuitry configured to control the operation of ANN computing system 100 to perform ANN computations.

The integrated circuit of the controller 110 may be an application special integrated circuit specially configured to perform the steps of the ANN calculation procedure. For example, integrated circuits may implement microcode or firmware specific to executing ANN calculation procedures. As such, controller 110 may have a reduced instruction set relative to a general-purpose processor used in conventional computers, such as computer 102 . In some embodiments, the integrated circuitry of controller 110 may include two or more circuits configured to perform different steps of the ANN calculation procedure.

In an example operation of the ANN computing system 100, the computer 102 may send an artificial neural network calculation request to the ANN computing system 100. An ANN calculation request may include the neural network weights that define the ANN, as well as the set of input data processed by the provided ANN. The controller 110 receives the ANN calculation request and stores the input data set and neural network weights in the memory unit 120 .

The input data set can correspond to various digital information that the ANN will process. Examples of the input data set include image files, audio files, LiDAR point clouds and Global Positioning System (GPS) coordinate sequences, and will be based on receiving image files as input data set to describe the operation of the ANN computing system 100. Generally speaking, the size of the input data set can vary widely, from hundreds of data points to millions of data points or larger. For example, a digital image file with 1 megapixel resolution has approximately one million pixels, and each of the one million pixels may be a data point processed by the ANN. Due to the large number of data points in a typical input data set, the input data set is usually divided into multiple digital input vectors of smaller size to be processed individually by the optical processor 140 . As an example, for a grayscale digital image, the elements of the digital input vector can be 8-bit values representing image intensity, and the digital input vector can have elements ranging from tens of elements (for example: 32 elements, 64 elements) to hundreds of elements (for example: 256 elements, 512 elements) in length. Generally speaking, an input data set of any size can be divided into digital input vectors of a size suitable for processing by optical processor 140 . In cases where the number of elements in the input data set is not divisible by the length of the numeric input vector, zero padding can be used to pad the data set so that it is divisible by the length of the numeric input vector. The processed output of individual digit input vectors can be processed to reconstruct the complete output, which is the result of processing the input data set through an ANN. In some embodiments, a block matrix multiplication technique may be used to achieve splitting of the input data set into multiple input vectors and subsequent vector-level processing.

Neural network weights are a set of numerical values that define the connectivity of an ANN's artificial neurons, including the relative importance or weight of those connections. An ANN may include one or more hidden layers with corresponding sets of nodes. In the case of an ANN with a single hidden layer, the ANN can be defined by two sets of neural network weights, one for should be the connectivity between the input nodes and the nodes of the hidden layer, and the second group corresponds to the connectivity between the hidden layer and the output nodes. Each set of neural network weights describing connectivity corresponds to a matrix implemented by the light processor 140 . For ANNs with two or more hidden layers, additional sets of neural network weights are required to define the connectivity between the additional hidden layers. As such, under normal circumstances, the neural network weights included in the ANN calculation request may include multiple sets of neural network weights, which represent the connectivity between various layers of the ANN.

Since the input data set to be processed is usually divided into multiple smaller digital input vectors for individual processing, the input data set is usually stored in digital memory. However, memory operations between the computer 102's memory and the processor are significantly slower than the rate at which the ANN calculation system 100 can perform ANN calculations. For example, ANN computing system 100 may perform tens to hundreds of ANN calculations during a typical memory read cycle of computer 102 . As such, during the processing of the ANN calculation request, if the ANN calculation of the ANN calculation system 100 involves multiple data transfers between the ANN calculation system 100 and the computer 102, the rate of ANN calculations that can be performed by the ANN calculation system 100 can be is limited to its full processing rate. For example, if computer 102 were to access an input data set from its own memory and provide a numeric input vector to controller 110 upon request, the operation of ANN computing system 100 may vary between computer 102 and controller 110 The time required to transfer a series of data is greatly slowed down. It is worth noting that the memory access latency of the computer 102 is typically non-deterministic, which further complicates and reduces the speed at which the digital input vectors can be provided to the ANN computing system 100 . Lower its speed. Additionally, computer 102 processor cycles may be wasted managing the transfer of data between computer 102 and ANN computing system 100 .

Instead, in some embodiments, ANN computing system 100 stores the entire input data set in memory unit 120 , which is part of and dedicated to ANN computing system 100 . The dedicated memory unit 120 allows transactions between the memory unit 120 and the controller 110 and is particularly suitable for allowing smooth and uninterrupted data flow between the memory unit 120 and the controller 110 . This uninterrupted data flow can significantly improve ANN calculations by allowing optical processor 140 to perform matrix multiplications at its full processing rate without being limited by the slow memory operations of conventional computers (such as computer 102 ). Total circulation of system 100. Furthermore, because all information required in performing the ANN calculations is provided to the ANN calculation system 100 by the computer 102 in a single transaction, the ANN calculation system 100 can perform its ANN calculations in a unique manner independent of the computer 102. This unique operation of the ANN computing system 100 reduces the computing burden of the computer 102 and eliminates external dependencies in the operation of the ANN computing system 100, improving the performance of the ANN computing system 100 and the computer 102.

The internal operation of the ANN computing system 100 will now be described. The optical processor 140 includes a laser unit 142, a modulator array 144, a detection unit 146, and an optical matrix multiplication (OMM) unit 150. The optical processor 140 operates by encoding a digital input vector of length N onto an optical input vector of length N and propagating the optical input vector through the OMM unit 150 . The OMM unit 150 receives an optical input vector of length N, and Performs N×N matrix multiplication on the received light input vector. The N×N matrix multiplication performed by the OMM unit 150 is determined by the internal configuration of the OMM unit 150 . The internal configuration of the OMM unit 150 may be controlled by electrical signals, such as those generated by the DAC unit 130 .

OMM unit 150 may be implemented in various ways. Figure 1B shows a schematic diagram of the OMM unit 150. OMM unit 150 may include an array of input waveguides 152 to receive optical input vectors; an optical interference unit 154 in optical communication with the array of input waveguides 152 ; and an array of output waveguides 156 in optical communication with optical interference unit 154 . The optical interference unit 154 linearly converts the light input vector into the second optical signal array. The array of output waveguides 156 guides the second array of optical signals output by the optical interference unit 154 . At least one input waveguide in the array of input waveguides 152 is in optical communication with each output waveguide in the array of output waveguides 156 through an optical interference unit 154 . For example, for an optical input vector of length N, OMM unit 150 may include N input waveguides 152 and N output waveguides 156.

The optical interference unit may include a plurality of interconnected Mach-Zehnder interferometers (MZI). Figures 1C and 1D show schematic diagrams of configurations 157 and 158 of examples of interconnected MZIs. The MZIs may be interconnected in various ways (eg, in configurations 157 or 158 ) to achieve linear transformation of light input vectors received through the array of input waveguides 152 .

Figure 1E shows a schematic of the MZI 170. MZI 170 includes first input waveguide 171 , second input waveguide 172 , first output waveguide 178 , and second output waveguide 179 . Additionally, each MZI 170 of the plurality of interconnected MZIs includes a first phase shifter 174 configured to change a splitting ratio of the MZI 170; and a second phase shifter 176, is configured to shift the phase of one output of the MZI 170 , such as light exiting the MZI 170 through the second output waveguide 179 . The first phase shifter 174 and the second phase shifter 176 of the MZI 170 are coupled to a plurality of weight control signals generated by the DAC unit 130 . The first phase shifter 174 and the second phase shifter 176 are embodiments of reconfigurable elements of the OMM unit 150 . Examples of reconfigurable elements include thermo-optic phase shifters or electro-optic phase shifters. The thermo-optical phase shifter changes the refractive index of the waveguide and cladding material by heating the waveguide, thereby converting it into a phase change. Electro-optic phase shifters operate by applying an electric field (e.g. lithium niobate (LiNbO ₃ ), reverse biased PN junction) or current (e.g. forward biased PIN junction), which changes the refraction of the waveguide material Rate. By changing the weight control signal, the phase delay of the first phase shifter 174 and the second phase shifter 176 of each interconnected MZI 170 can be changed, which reconfigures the optical interference unit 154 of the OMM unit 150 to achieve the entire A specific matrix multiplication determined by the phase delay set on the optical interference unit 154. Additional embodiments of OMM unit 150 and optical interference unit 154 are disclosed in US Patent Publication No. US 2017/0351293 Al entitled "Apparatus and Method for Optical Neural Networks," which is fully incorporated herein by reference.

The optical input vector is generated by the laser unit 142 and the modulator array 144 . The optical input vector of length N has N independent optical signals, and the intensity of each optical signal corresponds to the value of the corresponding element of the digital input vector of length N. As an example, laser unit 142 may produce N light outputs. The N light outputs have the same wavelength and are optically coherent. The optical coherence of the light output allows the light produce optical interference with each other, which is a property exploited by the OMM unit 150 (e.g., in the operation of the MZI). Furthermore, the light outputs of the laser units 142 may be substantially the same as each other. For example, the N light outputs may be neutralized within their intensities (e.g., within 5%, within 3%, within 1%, within 0.5%, within 0.1%, or within 0.01%) and their relative phases (e.g., within Within 10 degrees, within 5 degrees, within 3 degrees, within 1 degree, within 0.1 degrees), it is roughly uniform. The uniformity of the light output can improve the faithfulness of the light input vector to the digital input vector, thereby improving the overall accuracy of the light processor 140 . In some embodiments, the light output of the laser unit 142 may have an optical power of 0.1 mW to 50 mW per output, a wavelength in the near-infrared range (eg, between 900 nm and 1600 nm), and a linewidth of less than 1 nm. The light output of the laser unit 142 may be a single transverse-mode light output.

In some embodiments, laser unit 142 includes a single laser source and an optical power splitter. A single laser source is configured to generate laser light. The optical power splitter is configured to split the light generated by the laser source into N light outputs having approximately the same intensity and phase. By splitting a single laser output into multiple outputs, optical coherence of multiple light outputs can be achieved. For example, a single laser source may be a semiconductor laser diode, a vertical-cavity surface-emitting laser (VCSEL), a distributed feedback (DFB) laser, or a distributed Bragg reflection Distributed Bragg reflector (DBR) laser. For example, the optical power splitter may be a 1:N multimode interference (MMI) splitter, a multi-stage splitter including multiple 1:2 MMI splitters or directional couplers. splitter), or star coupler (star coupler). In some other embodiments, a master-slave laser configuration may be used, where the slave laser is injection locked by the master laser to have a stable phase relationship with the master laser.

The optical output of laser unit 142 is coupled to modulator array 144 . Modulator array 144 is configured to receive optical input from laser unit 142 and to modulate the intensity of the received optical input based on a modulator control signal, which is an electrical signal. Examples of modulators include Mach-Zehnder interference (MZI) modulators, ring resonator modulators, and electro-absorption modulators. Modulator array 144 has N modulators, each modulator receiving one of the N light outputs of laser unit 142 . The modulator receives a control signal corresponding to an element of the digital input vector and modulates the intensity of the light. The control signal may be generated by DAC unit 130.

The DAC unit 130 is configured to generate a plurality of modulator control signals and generate a plurality of weight control signals under the control of the controller 110 . For example, the DAC unit 130 receives a first DAC control signal from the controller 110 , and the first DAC control signal corresponds to a digital input vector to be processed by the optical processor 140 . The DAC unit 130 generates a modulator control signal based on the first DAC control signal. The modulator control signal is an analog signal suitable for driving the modulator array 144 and the OMM unit 150 . For example, the analog signal may be a voltage or a current, depending on the technology and design of the modulators of the modulator array 144 and the OMM unit 150 . The voltage can have an amplitude ranging from ±0.1V to ±10V, and the current can have an amplitude ranging from 100μA to 100mA. In some implementations For example, the DAC unit 130 may include a modulator driver configured to buffer, amplify, or condition the analog signal so that the modulators of the modulator array 144 and the OMM unit 150 can be fully driven. For example, some types of modulators can be driven with differential control signals. In this case, the modulator driver may be a differential driver that generates a differential electrical output based on a single-ended input signal. As another example, certain types of modulators may have a 3dB bandwidth that is less than the desired processing rate of optical processor 140. In this case, the modulator driver may include a pre-emphasis circuit or other bandwidth enhancement circuit designed to extend the operating bandwidth of the modulator. For example, this bandwidth enhancement may be useful for modulators based on PIN diode structures that are forward biased to use carrier injection to modulate the guided modulated light wave. The refractive index of a portion of the waveguide. For example, if the modulator is an MZI modulator, a PIN diode structure can be used to implement a phase shifter in one or both waveguide arms of the MZI modulator. Configuring the phase shifter for forward bias operation facilitates shorter modulator length and a more compact overall design, which may be useful for OMM units 150 with a large number of modulators.

For example, in a bandwidth-enhanced pre-emphasis form, an analog electrical signal (such as a voltage or current) driving a modulator can be shaped to include a transient pulse. The transient pulse causes the analog Changes in signal level overshoot (overshoot), the analog signal level represents a given digital data value of the DAC control signal in a series of digital data values. Each digital data value can have any number of bits, including a single 1-bit data value, as assumed in the remainder of this example. Therefore, if the bit value is the same as the previous value, the analog electrical signal driving the modulator is maintained at a steady-state level (for example: the signal level X ₀ with a bit value of 0 and The higher signal level X ₁ ) with a bit value of 1. However, if a bit changes from 0 to 1, the corresponding analog electrical signal used to drive the modulator may include a momentary pulse that occurs during the bit transition before settling to a steady-state value X ₁ Initially there is a peak value of X ₁ +(X ₁ -X ₀ ). Likewise, if a bit changes from ₁ to 0, the corresponding analog electrical signal used to drive the modulator may include a momentary pulse that has a peak value at the beginning of the bit transition before settling to a steady-state value Peak value X ₀ +(X ₀ -X ₁ ). The size and length of the instantaneous pulse can be chosen to optimize bandwidth enhancement (for example, to maximize the open area of the eye diagram for non-return-to-zero (NRZ) modulation mode).

In a bandwidth-enhanced charge pump form, the analog current signal driving the modulator can be shaped into a momentary pulse that involves moving a precisely determined amount of charge. Figure 44 shows a charge pump bandwidth enhancement circuit that uses a capacitor connected in series between a voltage source and a modulator to precisely control charge flow. A portion of the circuit shown in Figure 44 may be included in the modulator driver described above. In this embodiment, the modulator is represented by modulator circuit 4400, which models the electrical characteristics of the modulator's phase shifter as a PIN diode. Modulator circuit 4400 includes a parallel connection of an ideal diode, a capacitor having capacitance _Cd , and a resistor having resistance R. Pump capacitor 4402 has capacitance C _p . Control voltage waveform 4404 is provided to an inverter circuit 4405 to produce a drive voltage waveform 4406, the amplitude of which can be precisely corrected to move a given amount of charge into or out of modulator circuit 4400 through pump capacitor 4402 . The PIN diode modeled by modulator circuit 4400 is forward biased by applying a constant voltage VDD_IO at terminal 4408. The charge pump control voltage VCP is applied to the terminal 4410 of the inverter circuit 4405 to control the amount of charge pumped when the driving voltage waveform 4406 transitions, and the corresponding optical phase shift applied by the modulator.

The value of the charge pump control voltage VCP can be adjusted before operation so that the nominal charge Q stored in the pump capacitor (charge pump capacitor) 4402 is accurately corrected based on the measured value of the capacitance C _p (for example: There may be some variability due to uncertainties during manufacturing). For example, the charge pump control voltage VCP may be equal to the nominal charge Q divided by the capacitance C _p . The resulting change in the refractive index of the portion of the waveguide that intersects the PIN diode can then provide a phase shift of the guided light wave that is related to the amount of charge moving between the PIN diode and the pump capacitor (charge pump capacitor) 4402 Q ( For example: stored through the internal capacitor C _d ) is linearly proportional. If the drive voltage changes from a low value to a high value, the current flowing from the charge pump capacitor 4402 into the PIN diode will deliver a given amount of charge in a short period of time (i.e., the integration of the positive current over time). If the drive voltage changes from a high value to a low value, the current flowing from the charge pump capacitor 4402 into the PIN diode will remove a given amount of charge over a short period of time (i.e., the integration of the negative current over time). After this relatively short switching time, steady-state current is provided by current source 4412, controlled by switch 4414, to replace the current lost due to the internal capacitor losing current through internal resistor R while maintaining the drive voltage. Charge (for example: during the hold time of a specific digital value). Using this charge pump configuration can have advantages, such as better accuracy than other techniques (including some pre-emphasis techniques), since the amount of charge moved during a short switching time depends on a constant physical parameter (C=) and steady-state control value (VCP) and is therefore precisely controllable and repeatable.

In some embodiments, reduced power consumption may be achieved by designing the modulators and/or OMM cells 150 of the modulator array 144 such that when the modulators are operated to produce modulation values that represent more commonly occurring coefficients Less power is consumed when operating the modulator, and more power is consumed when operating the modulator to produce modulation values representing more infrequently occurring coefficients. For example, power consumption can be reduced for certain data sets that are known to have certain properties. Figure 42 shows a modulation value probability distribution plot 4200 (dashed line) superimposed on a modulator power plot 4202 (solid line) for a particular design of modulator and/or OMM unit 150 of modulator array 144. . Both plots are a function of modulation values (on the horizontal axis) expressed in normalized units to represent coefficients between -1 and 1. In this embodiment, the data set includes various coefficients (for example, vector coefficients and/or matrix coefficients) used for artificial neural network calculations, such that the probability distribution function (PDF) of the coefficients is better for smaller coefficients ( i.e. coefficients with relatively small absolute values) yield higher probabilities (and thus more frequent instances). For such data sets ("low coefficient weight data sets"), reduced power consumption can be achieved by designing the modulator such that the modulator operates in a lower power state to use smaller coefficients (in the data set occur more frequently), while operating in a higher power state uses larger coefficients to perform calculations (occur less frequently in the data set).

Some optical amplitude modulators use relatively high power to modulate optical signals with small modulation values. For example, for coherent non-sensitive optical amplitude modulators, modulation values close to zero may require relatively high modulator power, such as for electroabsorption modulators, which require relatively high modulator power. current to drive a diode-based The absorber absorbs large optical power to reduce the optical amplitude of the modulated optical signal. For coherently sensitive optical amplitude modulators, modulation values close to zero may require relatively high modulator power, such as for MZI modulators, which require relatively high currents to drive diode-based phase shifter to provide a relative phase shift between the two MZI arms for destructive optical interference, thereby reducing the optical amplitude of the modulation signal.

Optical amplitude modulators can be configured to overcome this power relationship and achieve modulator power as shown in Figure 42, which assigns low power modulator states to near zero modulation values. For example, as shown in Figure 43, the MZI modulator 4300 can be configured with asymmetric arms that provide built-in passive relative phase shift (e.g., phase near 180 degrees). shift) such that only a small active relative phase shift (and therefore low modulator power) is required for destructive light interference. MZI modulator 4300 includes an input optical splitter 4302 that splits the incoming optical signal to provide 50% of the power to the first arm and 50% of the power to the second arm. The active phase shifter 4304 in the first arm provides a way to use a variable phase shift to vary within a range of possible values (in this embodiment, for unsigned modulation values between 0 and 1) Method of modulating values. The variable phase shift is determined based on the magnitude of the applied electrical signal, which requires a certain amount of supplied electrical power (e.g., a diode-based phase shifter formed from doped semiconductor material in or near the waveguide of the first arm ). Even if no power is applied to the MZI modulator 4300, the passive phase shifter 4306 in the second arm provides relative phase shift between the first and second arms. For example, an optical material with a high refractive index can be configured to impose a relative phase shift of 180 degrees between the arms such that the output light combiner 4308 provides light interference such that No significant optical power is coupled to its output. Various alternative configurations of active and passive phase shifters can be implemented, including (but not limited to): both the active and passive phase shifters can be in one arm without a modulator in the other arm or Phase shifters; both arms can have active phase shifters and passive phase shifters (in a push-pull arrangement); or both arms can have active phase shifters and one arm can Has a passive phase shifter.

Alternatively, MZI modulators configured according to the symmetric differential configuration described herein can be used to provide near-zero coefficients using only small active relative phase shifts (and therefore low modulator power). For example, Figure 22A shows an optical amplitude modulator built using MZI configured according to a symmetric differential configuration, where the optical output is detected as shown in Figure 22B. Low modulation power is used to perform multiplication of modulation values with low amplitude (i.e. absolute value) (using optical amplitude modulation). Specifically, low power applied to phase modulator 2204 corresponds to modulation of low amplitude modulation values, resulting in a corresponding near-equal splitting in the output of coupler 2206 and a low amplitude current at junction 2216, representing multiplication result. The symmetrical differential configuration also has the advantage of being able to provide signed modulation values between -1 and +1 (as described in more detail below). Although this implementation uses a phase modulator in a single arm of the MZI, other implementations may have other configurations, such as a push-pull arrangement with phase modulators in both arms to provide opposite signs. phase shift.

The power distribution shown in Figure 42 shows that zero modulation power is used to achieve the zero modulation value, but in other embodiments there may be residual low but non-zero modulation power at the zero modulation value. For these low coefficient weight data sets, this can usually be achieved by using Reduced power consumption is achieved using a modulator that is designed such that the modulator modulates the optical signal by the modulation value using an increase in power relative to the absolute value of the modulation value. As the magnitude of the modulation value increases, the precise shape of the modulation power as a function of the modulation value may vary for different implementations and does not necessarily increase linearly. There may be different power consuming components in the optical amplitude modulator, which contribute to the overall power consumption. In some embodiments, the modulators are designed such that they modulate the optical signal by the modulation value using monotonically increasing power relative to the absolute value of the modulation value.

In some cases, the modulators and/or OMM cells 150 of the modulator array 144 may have nonlinear transfer functions. For example, MZI optical modulators may have a non-linear relationship (eg, sinusoidal dependence) between the applied control voltage and its transmission. In this case, the first DAC control signal can be adjusted or compensated based on the nonlinear transfer function of the modulator so that the linear relationship between the digital input vector and the generated light input vector can be maintained. Maintaining this linearity is generally important to ensure that the input to the OMM unit 150 is an accurate representation of the digit input vector. In some embodiments, the compensation of the first DAC control signal may be performed by the controller 110 through a lookup table, which maps the value of the digital input vector to the value to be output by the DAC unit 130, so that the resulting modulated optical signal Linearly proportional to the elements of the numeric input vector. The lookup table can be generated by characterizing the nonlinear transfer function of the modulator and calculating the inverse function of the nonlinear transfer function.

In some embodiments, the nonlinearity of the modulator and the resulting nonlinearity in the resulting optical input vector can be compensated for by an ANN computational algorithm.

The optical input vector generated by the modulator array 144 is input to the OMM unit 150 . The optical input vector may be N spatially separated optical signals, each of which has an optical power corresponding to an element of the digital input vector. For example, the optical power of an optical signal is usually in the range of 1 μW to 10 mW. The OMM unit 150 receives the optical input vector and performs N×N matrix multiplication based on its internal configuration. The internal configuration is controlled by electrical signals generated by the DAC unit 130 . For example, the DAC unit 130 receives a second DAC control signal from the controller 110, and the second DAC control signal corresponds to the neural network weights to be implemented by the OMM unit 150. The DAC unit 130 generates a weight control signal based on the second DAC control signal. The weight control signal is an analog signal suitable for controlling the reconfigurable elements within the OMM unit 150 . For example, the analog signal may be a voltage or a current, depending on the type of reconfigurable components of the OMM unit 150 . The voltage can have an amplitude ranging from 0.1V to 10V, and the current can have an amplitude ranging from 100μA to 10mA.

Modulator array 144 may operate at a modulation rate that is different from the reconfiguration rate of reconfigurable OMM unit 150 . The optical input vector generated by the modulator array 144 propagates through the OMM cell at approximately a proportion of the speed of light (e.g., 80%, 50%, or 25% of the speed of light), depending on the optical properties of the OMM cell 150 (e.g., effective refractive index) (effective index)). For a typical OMM unit 150, the propagation time of the optical input vector is in the range of 1 to tens of picoseconds, which corresponds to processing rates of tens to tens of GHz. As such, the rate at which optical processor 140 can perform matrix multiplication operations is limited in part by the rate at which optical input vectors can be generated. Modulators with bandwidths of several 10 GHz are readily available, and with bands in excess of 100 GHz Wide modulators are under development. As a result, the modulation rate of the modulator array 144 can be in the range of 5 GHz, 8 GHz, or several 10 GHz to several 100 GHz. In order to maintain operation of the modulator array 144 at such modulation rates, the circuitry of the controller 110 may be configured to output to the DAC unit at a rate greater than or equal to 5 GHz, 8 GHz, 10 GHz, 20 GHz, 25 GHz, 50 GHz or 100 GHz. 130 control signal.

Depending on the type of reconfigurable elements implemented by the OMM unit 150, the reconfiguration rate of the OMM unit 150 may be significantly slower than the modulation rate. For example, the reconfigurable elements of the OMM unit 150 may be of the thermo-optical type, which uses microheaters to adjust the temperature of the optical waveguides of the OMM unit 150, which in turn affects the phase of the optical signals within the OMM unit 150 and results in matrix multiplication. Due to thermal time constants associated with heating and cooling of the structure, the reconfiguration rate can be limited to tens of 100kHz to tens of MHz. As such, the modulator control signals used to control the modulator array 144 and the weight control signals used to reconfigure the OMM cells 150 may have significantly different speed requirements. Furthermore, the electrical characteristics of modulator array 144 may be significantly different from the electrical characteristics of the reconfigurable elements of OMM cell 150 .

In order to adapt to different characteristics of the modulator control signal and the weight control signal, in some embodiments, the DAC unit 130 may include a first DAC sub-unit 132 and a second DAC sub-unit 134. The first DAC sub-unit 132 may be specifically configured to generate the modulator control signal, and the second DAC sub-unit 134 may be specifically configured to generate the weight control signal. For example, the modulation rate of the modulator array 144 may be 25 GHz, and the first DAC subunit 132 may have 25 giga-samples per second. Second; GSPS) per-channel output update rate (per-channel output update rate) and 8-bit or higher resolution. The reconfiguration rate of the OMM unit 150 may be 1 MHz, and the second DAC sub-unit 134 may have an output update rate of 1 mega-samples per second (MSPS) and a resolution of 10 bits. Implementing separate first DAC sub-unit 132 and second DAC sub-unit 134 allows the DAC sub-units to be independently optimized for respective signals, which may reduce the overall power consumption, complexity, cost, or a combination thereof of the DAC unit 130 . It is worth noting that although the first DAC sub-unit 132 and the second DAC sub-unit 134 are described as sub-elements of the DAC unit 130, generally speaking, the first DAC sub-unit 132 and the second DAC sub-unit 134 can be integrated into On a common chi, or can be implemented as separate dies.

Based on the different characteristics of the first DAC sub-unit 132 and the second DAC sub-unit 134, in some embodiments, the memory unit 120 may include a first memory sub-unit and a second memory sub-unit. The first memory subunit may be a memory dedicated to storing the input data set and the digital input vector, and may have an operating speed sufficient to support the modulation rate. The second memory subunit may be a memory dedicated to storing the neural network weights and may have an operating speed sufficient to support the reconfiguration rate of the OMM unit 150 . In some embodiments, the first memory subunit may be implemented using Static Random Access Memory (SRAM), and the second memory subunit may be implemented using Dynamic Random Access Memory (Dynamic Random Access Memory). Memory; DRAM) to implement. In some embodiments, the first memory sub-unit and the second memory sub-unit may be implemented using DRAM. In some implementations For example, the first memory unit may be implemented as part of the controller 110 or as a cache. In some embodiments, the first and second memory subunits may be implemented as different address spaces by a single physical memory device.

The OMM unit 150 outputs a light output vector of length N, which corresponds to the result of an N×N matrix multiplication of the light input vector and the neural network weights. The OMM unit 150 is coupled to the detection unit 146, which is configured to generate N output voltages corresponding to N optical signals of the optical output vector. For example, the detection unit 146 may include an array of N photodetectors configured to absorb optical signals and generate photocurrent, and an array of N transimpedance amplifiers configured to convert light Current is converted into output voltage. The bandwidth of the photodetector and transimpedance amplifier may be set based on the modulation rate of the modulator array 144. The light detector can be formed from a variety of materials based on the wavelength of the detected light output vector. Examples of materials for photodetectors include germanium, silicon-germanium alloys, and indium gallium arsenide (InGaAs).

The detection unit 146 is coupled to the ADC unit 160 . The ADC unit 160 is configured to convert N output voltages into N digital light outputs, which are quantized digital representations of the output voltages. For example, the ADC unit 160 may be an N-channel ADC. The controller 110 may obtain N digital light outputs corresponding to the light output vectors of the light matrix multiplication unit 150 from the ADC unit 160 . The controller 110 may form a digital output vector of length N from the N digital light outputs, which corresponds to the result of an N×N matrix multiplication of the input digital vector of length N.

The various electronic components of ANN computing system 100 can be integrated in various ways. For example, the controller 110 may be a special application fabricated on a semiconductor die. Use integrated circuits. Other electronic components (eg, memory unit 120, DAC unit 130, ADC unit 160, or combinations thereof) may be monolithically integrated on the semiconductor die on which controller 110 is fabricated. As another embodiment, two or more electronic components may be integrated into a System-on-Chip (SoC). In a system-on-chip embodiment, the controller 110, the memory unit 120, the DAC unit 130, and the ADC unit 160 may be fabricated on corresponding dies, and the corresponding dies may be integrated to provide electrical connections between integrated components. on a common platform for sexual connection (e.g. inserter). Relative to the method of separately arranging and routing components on a printed circuit board (PCB), this system-on-chip approach may allow faster data transfer between the electronic components of the ANN computing system 100, thereby improving ANN The operating speed of system 100 is calculated. Furthermore, the system-on-chip approach may allow the use of different manufacturing techniques optimized for different electronic components, which may improve the performance of different components and reduce the overall cost of the monolithic integration approach. Although the integration of the controller 110, the memory unit 120, the DAC unit 130 and the ADC unit 160 has been described, generally speaking, a subset of the components may be integrated, while other components may be implemented for various reasons, such as power or cost. for separate parts. For example, in some embodiments, the memory unit 120 may be integrated with the controller 110 as a functional block within the controller 110 .

The various optical components of the ANN computing system 100 may also be integrated in various ways. Examples of the optical components of the ANN computing system 100 include the laser unit 142 , the modulator array 144 , the OMM unit 150 , and the light detector of the detection unit 146 . These optical components can be integrated in various ways to improve performance and/or reduce cost. For example, the laser unit 142, the modulator array 144, the OMM unit 150 and And the photodetector can be monolithically integrated on a common semiconductor substrate as a photonic integrated circuit (PIC). On photonic integrated circuits based on compound semiconductor material systems (such as III-V compound semiconductors (such as indium phosphide (InP))), lasers, modulators (such as electroabsorption modulators), waveguides, and Photodetectors can be monolithically integrated on a single die. This monolithic integration approach can reduce the complexity of aligning the inputs and outputs of various discrete optical components, which can require alignment accuracy from sub-micron to several microns. As another embodiment, the laser source of the laser unit 142 can be fabricated on a compound semiconductor die, and the optical power splitter, modulator array 144, OMM unit 150 and detection unit 146 of the laser unit 142 Detectors can be fabricated on silicon dies. PICs manufactured on silicon wafers (which can be called silicon photonics technology) generally have higher integration density, higher lithography resolution and lower cost than PICs based on III-V family. This greater integration density may be beneficial in the fabrication of the OMM unit 150 because the OMM unit 150 typically includes tens to hundreds of optical components, such as power splitters and phase shifters. In addition, the higher lithography resolution of silicon photonics technology can reduce process variations of the OMM unit 150 , thereby improving the accuracy of the OMM unit 150 .

ANN computing system 100 may be implemented in a variety of form factors. For example, the ANN computing system 100 may be implemented as a co-processor plugged into a host computer. This ANN computing system 100 may have the form factor of a Peripheral Component Interconnect Express (PCI Express) card and communicate with the host computer through the PCIe bus. The host computer can host multiple co-processor type ANN calculations. The computing system 100 is connected to the computer 102 through the network. This type of embodiment may be applicable to cloud data centers, where server racks may be dedicated to processing ANN calculation requests received from other computers or servers. As another example, a co-processor type ANN computing system 100 may be plugged directly into the computer 102 issuing the ANN computing request.

In some embodiments, the ANN computing system 100 may be integrated into physical systems requiring real-time ANN computing capabilities. For example, systems that rely heavily on real-time artificial intelligence tasks (such as self-driving vehicles, autonomous drones, object or facial recognition security cameras, and various Internet-of-Things) of-Things (IoT) devices) may benefit from having the ANN computing system 100 directly integrated with other subsystems of such systems. The ANN computing system 100 with direct integration can perform real-time artificial intelligence in devices with poor or no network connectivity and enhance the reliability and availability of mission-critical artificial intelligence systems.

Although DAC unit 130 and ADC unit 160 are shown coupled to controller 110 , in some embodiments, DAC unit 130 , ADC unit 160 , or both may alternatively or additionally be coupled to memory unit 120 . For example, the direct memory access (DMA) operation of the DAC unit 130 or the ADC unit 160 can reduce the computational load on the controller 110 and reduce the latency of reading and writing the memory unit 120, thereby The operation speed of the ANN calculation unit 100 is further improved.

Figure 2 shows the flow of a program 200 for performing ANN calculations. Figure. The steps of program 200 may be performed by controller 110 . In some embodiments, the various steps of process 200 may be run in parallel, combined, in a loop, or in any order.

At step 210, an artificial neural network (ANN) calculation request including an input data set and a first plurality of neural network weights is received. The input data set includes the first digit input vector. The first digit input vector is a subset of the input data set. For example, it could be a subregion of the image. ANN calculation requests may be generated by various entities (eg, computer 102). Computers may include one or more of various types of computing devices, such as personal computers, server computers, vehicle computers, and flight computers. The ANN calculation request generally refers to an electrical signal that notifies or informs the ANN calculation system 100 that the ANN calculation is to be performed. In some embodiments, an ANN calculation request may be split into two or more signals. For example, the first signal may query the ANN computing system 100 to check whether the ANN computing system 100 is ready to receive the input data set and the first plurality of neural network weights. In response to an affirmative response from the ANN computing system 100, the computer may send a second signal including the input data set and the first plurality of neural network weights.

In step 220, the input data set and the first plurality of neural network weights are stored. The controller 110 may store the input data set and the first plurality of neural network weights in the memory unit 120 . Storing the input data set and the first plurality of neural network weights in the memory unit 120 may allow flexibility in the operation of the ANN computing system 100 , which may, for example, improve the overall performance of the system. For example, by retrieving a desired portion of the input data set from memory unit 120, the input data set may be divided into numeric input vectors of a set size and format. Different parts of the input data set can be Various sequential processes, or shuffled, allow various types of ANN calculations to be performed. For example, shuffling can allow matrix multiplication to be performed via block matrix multiplication techniques when the input and output matrices are of different sizes. As another example, storing the input data set and the first plurality of neural network weights in the memory unit 120 may allow multiple ANN calculation requests to be queued by the ANN computing system 100 , which may allow the ANN computing system 100 Operation is maintained at its full speed without periods of inactivity.

In some embodiments, the input data set may be stored in a first memory subunit and the first plurality of neural network weights may be stored in a second memory subunit.

In step 230, a first plurality of modulator control signals are generated based on the first digital input vector, and a first plurality of weight control signals are generated based on the first plurality of neural network weights. The controller 110 may send the first DAC control signal to the DAC unit 130 to generate a first plurality of modulator control signals. DAC unit 130 generates a first plurality of modulator control signals based on the first DAC control signal, and modulator array 144 generates an optical input vector representing a first digital input vector.

The first DAC control signal may include a plurality of digital values to be converted by the DAC unit 130 into a first plurality of modulator control signals. Multiple digit values typically correspond to the first digit input vector and can be related through various mathematical relationships or lookup tables. For example, the plurality of digit values may be linearly proportional to the values of the elements of the first digit input vector. As another example, multiple digit values can be input through a lookup table and the first digit Associating elements of the input vector, the lookup table is configured to maintain a linear relationship between the digital input vector and the light input vector produced by the modulator array 144 .

The controller 110 may send the second DAC control signal to the DAC unit 130 to generate the first plurality of weight control signals. The DAC unit 130 generates a first plurality of weight control signals based on the second DAC control signal, and reconfigures the OMM unit 150 according to the first plurality of weight control signals to implement a matrix corresponding to the first plurality of neural network weights.

The second DAC control signal may include a plurality of digital values to be converted by the DAC unit 130 into the first plurality of weighted control signals. Multiple numerical values typically correspond to a first plurality of neural network weights, and can be related through various mathematical relationships or lookup tables. For example, the plurality of numerical values may be linearly proportional to the first plurality of neural network weights. As another example, the plurality of digital values may be calculated by performing various mathematical operations on the first plurality of neural network weights to generate weight control signals. The weight control signals may configure the OMM unit 150 to perform operations corresponding to the first plurality of neural network weights. Matrix multiplication of neural network weights.

In some embodiments, the first plurality of neural network weights representing the matrix M may be decomposed into M=USV* by a singular value decomposition (SVD) method, where U is an M×M positive matrix, S is an MxN diagonal matrix with nonnegative real numbers on the diagonal, and V* is the complex conjugate of the N×N positive matrix V. In this case, the first plurality of weight control signals may include a first plurality of OMM unit control signals corresponding to the matrix V, and a second plurality of OMM unit control signals corresponding to the matrix S. Further, OMM unit 150 may be configured to have configured A first OMM subunit is configured to implement matrix V, a second OMM subunit is configured to implement matrix S, and a third OMM subunit is configured to implement matrix U, such that the OMM unit 150 as a whole implements matrix M. SVD methods are further described in US Patent Publication No. US 2017/0351293A1 entitled "Apparatus and Methods for Optical Neural Networks," which is fully incorporated herein by reference.

In step 240, a first plurality of digital light outputs corresponding to the light output vector of the light matrix multiplication unit are obtained. The optical input vectors generated by the modulator array 144 are processed by the OMM unit 150 and converted into optical output vectors. The light output vector is detected by the detection unit 146 and converted into an electrical signal, which can be converted into a digital value by the ADC unit 160 . The controller 110 may send a conversion request to the ADC unit 160 to start converting the voltage output by the detection unit 146 into a digital light output. Once the conversion is completed, the ADC unit 160 may send the conversion result to the controller 110 . Alternatively, the controller 110 may retrieve the conversion results from the ADC unit 160 . The controller 110 may form a digital output vector from the digital light output, the digital output vector corresponding to the result of matrix multiplication of the input digital vector. For example, digital light output may be organized or concatenated to have a vector format.

In some embodiments, the ADC unit 160 may be set or controlled to perform ADC conversion based on the DAC control signal sent by the controller 110 to the DAC unit 130 . For example, the ADC conversion may be set to start at a preset time after the DAC unit 130 generates the modulation control signal. Such control of ADC conversion can simplify the operation of the controller 110 and reduce the number of necessary control operations.

In step 250, a non-linear transformation is performed on the first digital output vector to produce a first transformed digital output vector. An ANN's nodes, or artificial neurons, operate by first performing a weighted summation of signals received from nodes in previous layers, and then performing a nonlinear transformation ("activation") of the weighted summation to produce an output. Various types of ANN can implement various types of differentiable nonlinear transformations. Examples of nonlinear transformation functions include a rectified linear unit (RELU) function, a sigmoid function, a hyperbolic tangent function, an X^2 function, and an |X| function. This non-linear conversion is performed on the first digital output by controller 110 to produce a first converted digital output vector. In some embodiments, the nonlinear conversion may be performed by dedicated digital integrated circuits within controller 110 . For example, controller 110 may include one or more modules or circuit blocks that are particularly suitable for accelerating the calculation of one or more types of nonlinear transformations.

In step 260, the first converted digital output vector is stored. The controller 110 may store the first converted digital output vector in the memory unit 120 . In the case where the input data set is divided into a plurality of digit input vectors, the first converted digit output vector corresponds to the ANN calculation result of a portion of the input data set, such as the first digit input vector. As such, storing the first transformed digital output vector allows the ANN computing system 100 to perform and store additional calculations on other digital input vectors of the input data set to be later aggregated into a single ANN output.

In step 270, the artificial neural network output generated based on the first converted digital output vector is output. The controller 110 generates an ANN output that is a result of processing the input data set through an ANN defined by the first plurality of neural network weights. exist Where the input data set is divided into multiple digital input vectors, the resulting ANN output is an aggregate output that includes the first transformed digital output, but may further include additional transformed digital outputs corresponding to other portions of the input data set. Once the ANN output is generated, the generated output is sent to the computer that initiated the ANN calculation request (eg, computer 102).

Various performance metrics may be defined for the ANN computing system 100 implementing program 200. Defining performance metrics may allow the performance of the ANN computing system 100 implementing the optical processor 140 to be compared with the performance of other systems used to implement ANN computing instead of an electronic matrix multiplication unit. In one view, the rate at which ANN computations can be performed may be dictated in part by the first loop period, which is defined as the step 220 of storing the input data set and the first plurality of neural network weights in a memory unit versus The time elapsed between steps 260 of storing the first converted digital output vector in the memory unit. Therefore, the first cycle includes the time spent converting the electrical signal to the optical signal (eg, step 230), performing the matrix multiplication in the optical domain, and converting the result back to the electrical domain (eg, step 240). Both steps 220 and 260 involve storing data into memory unit 120, which is a step shared between ANN computing system 100 and conventional ANN computing systems without optical processor 140. In this way, measuring the memory-to-memory transaction time of the first cycle can allow the ANN computing system 100 to be compared with an ANN computing system without an optical processor 140 (such as implementing an electrical matrix multiplication unit). systems) to perform actual or fair comparisons of ANN calculations in circulation.

Due to the rate at which the modulator array 144 can generate optical input vectors (eg, at 25 GHz) and the processing rate of the OMM unit 150 (eg, >100 GHz), the ANN computing system 100 for performing a single ANN calculation of a single digit input vector has The first cycle period may be close to the reciprocal of the speed of modulator array 144 (eg, 40 ps). The first cycle period may be less than or equal to 100 ps, less than or equal to 200 ps, less than or equal to 500 ps, less than or equal to 1 ns, less than or equal to 2ns, less than or equal to 5ns, or less than or equal to 10ns.

For comparison, the multiplication execution time of M×1 vectors and M×M matrices of the electrical matrix multiplication unit is usually proportional to M^2-1 processor clock cycles. For M=32, this multiplication will take approximately 1024 cycles, which results in an execution time of over 300ns at a 3GHz clock speed, which is several orders of magnitude slower than the first loop period of the ANN computing system 100.

In some embodiments, the process 200 further includes the step of generating a second plurality of modulator control signals based on the first converted digital output vector. In some types of ANN calculations, a single digit input vector can be repeatedly propagated through the same ANN or processed by the same ANN. An ANN that implements multi-pass processing can be called a recurrent neural network (RNN). RNNs are neural networks in which the output of the network during the (k)th pass through the neural network is recycled back to the input of the neural network and used as input during the (k+1)th pass. RNNs can have various applications in pattern recognition tasks, such as speech or handwriting recognition. Once the second plurality of modulator control signals are generated, process 200 may Proceed from step 240 to step 260 to complete the second pass of the first digit input vector through the ANN. Generally speaking, the recycling of the converted digital output as a digital input vector can be repeated for a predetermined number of cycles depending on the characteristics of the RNN received in the ANN calculation request.

In some embodiments, the process 200 further includes the step of generating a second plurality of weight control signals based on the second plurality of neural network weights. In some cases, the artificial neural network calculation request further includes a second plurality of neural network weights. Generally speaking, in addition to the input layer and output layer, ANN also has one or more hidden layers. For an ANN with two hidden layers, the second plurality of neural network weights may correspond to the connectivity between the first layer of the ANN and the second layer of the ANN. In order to process the first digital input vector through the two hidden layers of the ANN, the first digital input vector may first be processed according to the process 200 until step 260, wherein the result of processing the first digital input vector through the first hidden layer of the ANN is stored in step 260. in memory unit 120. The controller 110 then reconfigures the OMM unit 150 to perform matrix multiplication corresponding to the second plurality of neural network weights associated with the second hidden layer of the ANN. Once the OMM unit 150 is reconfigured, the process 200 can generate a plurality of modulator control signals based on the first converted digital output vector, which generates an updated light input vector corresponding to the output of the first hidden layer. The updated light input vector is then processed by the reconfigured OMM unit 150, which corresponds to the second hidden layer of the ANN. In general, the steps described can be repeated until the numeric input vector has been processed through all hidden layers of the ANN.

As discussed above, in some embodiments of OMM unit 150 , the reconfiguration rate of OMM unit 150 may be significantly slower than the modulation rate of modulator array 144 rate. In this case, the throughput of the ANN computing system 100 may be adversely affected by the amount of time spent reconfiguring the OMM unit 150 during the period that it is unable to perform ANN calculations. To mitigate the impact of the relatively slow reconfiguration time of the OMM unit 150, batch processing techniques may be utilized, in which two or more digital input vectors are propagated through the OMM unit 150 without configuration changes to amortize the reconfiguration. Configuration time increases with larger number of digit input vectors.

Figure 2B shows a schematic diagram 290 illustrating the process 200 of Figure 2A. For an ANN with two hidden layers, instead of processing the first digit input vector through the first hidden layer, reconfiguring the OMM unit 150 for the second hidden layer, processing the first digit input vector through the reconfigured OMM unit 150, and The same operation is repeated for the remaining digit input vectors, and all digit input vectors of the input data set may first be processed by the OMM unit 150 configured for the first hidden layer (configuration #1), as shown in the upper part of the schematic diagram 290 . Once all the digit input vectors have been processed by the OMM unit 150 with configuration #1, the OMM unit 150 is reconfigured into configuration #2, which corresponds to the second hidden layer of the ANN. This reconfiguration may be significantly slower than the rate at which the OMM unit 150 can process the input vectors. Once the OMM unit 150 is reconfigured for a second hidden layer, the output vectors from the previous hidden layer may be batch processed by the OMM unit 150 . For large input data sets with tens or hundreds of thousands of digit input vectors, the impact of reconfiguration time can be reduced by approximately the same factor, which can significantly reduce the portion of time the ANN computing system 100 spends in reconfiguration.

In order to achieve batch processing, in some embodiments, the process 200 further includes the step of generating a second plurality of modulator control signals based on the second digital input vector through the DAC unit; obtaining the light output vector corresponding to the optical matrix multiplication unit from the ADC unit the steps of a second plurality of digital light outputs, the second plurality of digital light outputs forming a second digital output vector; the step of performing a nonlinear conversion on the second digital output vector to generate a second converted digital output vector; and in the memory The step of storing the second converted digital output vector in the unit. For example, generating the second plurality of modulator control signals may occur after step 260 . Furthermore, the ANN output of step 270 in this case is now based on the first converted digit output vector and the second converted digit output vector. The acquisition, execution and storage steps are similar to steps 240 to 260.

Batch processing technology is one of several technologies used to increase throughput of the ANN computing system 100. Another technique for increasing the throughput of the ANN computing system 100 is by processing multiple digital input vectors in parallel using wavelength division multiplexing (WDM). WDM is a technology that simultaneously propagates multiple optical signals of different wavelengths through a common propagation channel (such as the waveguide of the OMM unit 150). Unlike electrical signals, optical signals of different wavelengths can propagate through a common channel without affecting other optical signals of different wavelengths on the same channel. Additionally, light may be added (multiplexed) or dropped (demultiplexed) from a common propagation channel using well-known structures such as optical multiplexers and demultiplexers. signal.

In the context of the ANN computing system 100, multiple optical input vectors of different wavelengths may be independently generated, propagated simultaneously through the OMM unit 150, and independently On-site detection to enhance the throughput of the ANN computing system 100. 1F, a schematic diagram of a wavelength division multiplexing (WDM) artificial neural network (ANN) computing system 104 is shown. Unless otherwise described, WDM ANN computing system 104 is similar to ANN computing system 100. To implement WDM technology, in some embodiments of the WDM ANN computing system 104, the laser unit 142 is configured to generate multiple wavelengths, such as λ1, λ2, and λ3. Multiple wavelengths may preferably be separated by a sufficiently large wavelength spacing to allow easy multiplexing and demultiplexing onto a common propagation channel. For example, wavelength spacing greater than 0.5 nm, 1.0 nm, 2.0 nm, 3.0 nm, or 5.0 nm may allow for simple multiplexing and demultiplexing. On the other hand, the range between the shortest and longest wavelengths of the plurality of wavelengths (the "WDM bandwidth") may preferably be small enough such that the characteristics or performance of the OMM unit 150 remain substantially the same across the plurality of wavelengths. Optical components are often dispersive, meaning their optical properties vary with wavelength. For example, the power separation ratio of an MZI can vary with wavelength. However, by designing the OMM unit 150 to have a sufficiently large operating wavelength window, and by limiting the wavelengths within the operating wavelength window, the light output vector output by the OMM unit 150 at each wavelength can be is the sufficiently accurate result of the matrix multiplication implemented by the OMM unit 150. The operating wavelength window can be 1nm, 2nm, 3nm, 4nm, 5nm, 10nm or 20nm.

Figure 39A shows a schematic diagram of a Mach-Zehnder modulator 3900 that can be used to modulate the amplitude of an optical signal. Mach-Zehnder modulator 3900 includes two 1×2 port multi-mode interference couplers (MMI_1x2) 3902a and 3902b, two balanced arms (arms) 3904a and 3904b, and a phase shifter 3906 in one arm (or each one a phase shifter in each arm). When a voltage is applied to the phase shifter in one arm through signal line 3908, there will be a phase difference between the two arms 3904a and 3904b that will be converted into amplitude modulation. The 1x2 port multi-mode interference couplers 3902a and 3902b and the phase shifter 3906 are configured with broadband photonic components, and the optical path lengths of the two arms 3904a and 3904b are configured to be equal. This enables the Mach-Zehnder Modulator 3900 to operate over a wide wavelength range.

Figure 39B is a graph 3910 showing intensity-voltage graphs for the Mach-Zehnder modulator 3900 using the configuration shown in Figure 39A for wavelengths 1530 nm, 1550 nm, and 1570 nm. Graph 3910 shows that Mach-Zehnder modulator 3900 has similar intensity-voltage characteristics for different wavelengths in the range of 1530 nm to 1570 nm.

Referring back to FIG. 1F , the modulator array 144 of the WDM ANN computing system 104 includes banks of optical modulators configured to generate a plurality of optical input vectors, each group corresponding to one of a plurality of wavelengths. one and produces corresponding light input vectors with corresponding wavelengths. For example, for a system with optical input vectors of length 32 and 3 wavelengths (eg, λ1, λ2, and λ3), modulator array 144 may have 3 groups of 32 modulators each. In addition, the modulator array 144 further includes an optical multiplexer configured to combine multiple optical input vectors into a combined optical input vector including multiple wavelengths. For example, an optical multiplexer can combine the outputs of three modulator banks at three different wavelengths into a single propagation channel (eg, waveguide) for each element of the optical input vector. Thus, returning to the above embodiment, the combined optical input vector will have 32 optical signals, each signal including 3 wavelengths.

In addition, the detection unit 146 of the WDM ANN computing system 104 is further configured to demultiplex multiple wavelengths and generate multiple demultiplexed output voltages. For example, the detection unit 146 may include a demultiplexer configured to demultiplex three wavelengths included in each of the 32 signals of the multi-wavelength optical output vector and demultiplex the three single wavelengths The light output vector is routed to three sets of photodetectors coupled to three sets of transimpedance amplifiers.

Furthermore, the ADC unit 160 of the WDM ANN computing system 104 includes an ADC bank configured to convert a plurality of demultiplexed output voltages of the detection unit 146 . Each group corresponds to one of the plurality of wavelengths and produces a corresponding digital demultiplexed optical output. For example, the ADC bank may be coupled to the transimpedance amplifier bank of the detection unit 146 .

Controller 110 may implement a method similar to procedure 200, but extended to support multi-wavelength operation. For example, the method may include the step of obtaining a plurality of digital demultiplexed light outputs from the ADC unit 160, the plurality of digital demultiplexed light outputs forming a plurality of first digital output vectors, wherein one of the plurality of first digital output vectors is Each of corresponds to one of the plurality of wavelengths; performing a nonlinear transformation on each of the plurality of first digital output vectors to generate a plurality of steps of converting the first digital output vector; and in the memory unit Stores multiple steps for converting the first digital output vector.

In some cases, the ANN can be specifically designed and the digital input vectors can be specifically formed such that multi-wavelength light output vectors can be detected without demultiplexing. In this case, the detection unit 146 may be a wavelength-insensitive detection unit that does not demultiplex multiple wavelengths. Multiple wavelengths of light output vector. In this way, each photodetector of the detection unit 146 effectively adds multiple wavelengths of the optical signal to a single photocurrent, and each voltage output by the detection unit 146 corresponds to a matrix multiplication of multiple digital input vectors. The element-by-element sum of the result.

So far, the nonlinear transformation of the weight sum performed as part of the ANN calculation has been performed in the digital domain by the controller 110 . In some cases, nonlinear transformations may be computationally intensive or power consuming, significantly increasing the complexity of the controller 110, or limiting the performance of the ANN computing system 100 in terms of throughput or power efficiency. As a result, in some embodiments of the ANN computing system, nonlinear transformations can be performed in the analog domain through analog electronic devices.

Figure 3A shows a schematic diagram of the ANN computing system 300. ANN computing system 300 is similar to ANN computing system 100 except that an analog nonlinear unit 310 is added. The analog nonlinear unit 310 is disposed between the detection unit 146 and the ADC unit 160 . Analog nonlinear unit 310 is configured to receive the output voltage from detection unit 146 , apply a nonlinear transfer function, and output the converted output voltage to ADC unit 160 .

When the ADC unit 160 receives the voltage that has been nonlinearly converted by the analog nonlinear unit 310, the controller 110 may obtain a converted digital output voltage corresponding to the converted output voltage from the ADC unit 160. Because the digital output voltage obtained from the ADC unit 160 has already been nonlinearly converted ("activated"), the nonlinear conversion step of the controller 110 can be omitted, thereby reducing the computational burden of the controller 110 . Then, okay The first converted voltage obtained directly from the ADC unit 160 is stored in the memory unit 120 as a first converted digital output vector.

Analog nonlinear unit 310 may be implemented in various ways. For example, high-gain amplifiers in feedback configurations, with adjustable reference voltages, nonlinear IV characteristics of diodes, breakdown behavior of diodes, nonlinear CV characteristics of variable capacitors, or variable resistors. A comparator with nonlinear IV characteristics may be used to implement the analog nonlinear unit 310 .

The use of the analog nonlinear unit 310 can improve the performance of the ANN computing system 300, such as throughput or power efficiency, by reducing the number of steps performed in the digital domain. Moving the nonlinear transformation step out of the digital domain can allow additional flexibility and improvements in the operation of ANN computing systems. For example, in a recurrent neural network, the output of OMM unit 150 is activated and recycled back to the input of OMM unit 150 . The activation step is performed by the controller 110 in the ANN computing system 100, which requires digitizing the output voltage of the detection unit 146 at each pass through the OMM unit 150. However, since the activation step is now performed before digitization of the ADC unit 160, the number of ADC conversions required in performing the recursive neural network calculations can be reduced.

In some embodiments, analog nonlinear unit 310 may be integrated into ADC unit 160 as a nonlinear ADC unit. For example, the nonlinear ADC unit may be a linear ADC unit with a nonlinear lookup table that maps the linear digital output of the linear ADC unit to a desired nonlinear converted digital output.

Figure 3B shows a schematic diagram of the ANN computing system 302. ANN computing system 302 is similar to ANN computing system 300 of Figure 3A except that It further includes an analog memory unit 320 . The analog memory unit 320 is coupled to the DAC unit 130 (eg, through the first DAC subunit 132 ), the modulator array 144 and the analog nonlinear unit 310 . The analog memory unit 320 includes a multiplexer having a first input coupled to the DAC unit 130 and a second input coupled to the analog non-linear unit 310 . This allows the analog memory unit 320 to receive signals from the DAC unit 130 or the analog nonlinear unit 310 . The analog memory unit 320 is configured to store analog voltages and output the stored analog voltages.

Analog memory unit 320 may be implemented in various ways. For example, capacitor arrays can be used as analog voltage storage elements. The capacitor of the analog memory cell 320 can be charged to the input voltage by a charging circuit. The storage of the input voltage may be controlled based on a control signal received from the controller 110 . Capacitors can be electrically isolated from the surrounding environment to reduce charge leakage that can lead to undesired discharge of the capacitor. Additionally (or alternatively), a feedback amplifier may be used to maintain the voltage stored on the capacitor. The stored voltage of the capacitor can be read out by a buffer amplifier, which allows the charge stored by the capacitor to be maintained while outputting the stored voltage. These aspects of analog memory cell 320 may be similar to the operation of a sample and hold circuit. The buffer amplifier may function as a modulator driver for driving the modulator array 144 .

The operation of ANN computing system 302 will now be described. The first plurality of modulator control signals output by the DAC unit 130 (eg, by the first DAC sub-unit 132 ) are first input to the modulator array 144 through the analog memory unit 320 . In this step, the analog memory unit 320 may simply pass or buffer the first plurality of modulator control signals. The modulator array 144 generates light input based on the first plurality of modulator control signals. vector, which propagates through the OMM unit 150 and is detected by the detection unit 146. The output voltage of the detection unit 146 is nonlinearly converted by the analog nonlinear unit 310 . At this time, instead of being digitized by the ADC unit 160 , the output voltage of the detection unit 146 is stored by the analog memory unit 320 , which is then output to the modulator array 144 to be converted into the next light to be propagated through the OMM unit 150 Input vector. Under the control of the controller 110, the recursive processing may be performed for a preset amount of time or a preset number of cycles. Once the recursive processing is completed for a given digital input vector, the converted output voltage of the analog nonlinear unit 310 is converted by the ADC unit 160 .

The use of the analog memory unit 320 can significantly reduce the number of ADC conversions during the computation of the recurrent neural network, such as down to a single ADC conversion for each time the RNN computes a given digital input vector. Each ADC conversion takes a while and consumes a certain amount of energy. In this way, the circulation calculated by the RNN of the ANN computing system 302 may be higher than the circulation calculated by the RNN of the ANN computing system 100 .

The execution of the recursive neural network calculations can be controlled by controlling the analog memory unit 320 . For example, the controller can control the analog memory unit 320 to store the voltage at a specific time and output the stored voltage at different times. In this way, the signal loop from the analog memory 320 to the modulator array 144 through the analog nonlinear unit 310 and back to the analog memory unit 320 can be controlled by the controller controlling the storage and reading of the analog memory unit 320 .

Thus, in some embodiments, the controller 110 of the ANN computing system 302 may perform the following steps: based on generating the first plurality of modulator control signals and the first plurality of weight control signals, store the analog signal through the analog memory unit. A plurality of conversion output voltages of the nonlinear unit; outputting the stored conversion output voltage through the analog memory unit; obtaining a second plurality of conversion digital output voltages from the ADC unit, and the second plurality of conversion digital output voltages forming a second conversion digital output vector ; and storing the second converted digital output vector in the memory unit.

Input data sets processed by ANN computing systems typically include data with resolution greater than 1 bit. For example, a typical pixel of a grayscale digital image can have a resolution of 8 bits, or 256 different levels. One way to represent and process this data in the optical domain is to encode 256 different intensity levels of pixels as 256 different power levels of the optical signal input to the OMM unit 150 . Optical signals are analog in nature and therefore susceptible to noise and detection errors. Referring back to Figure 1A, in order to maintain the 8-bit resolution of the digital input vector throughout the ANN computing system 100 and produce a true 8-bit digital light output at the output of the ADC unit 160, each step of the signal chain Each part may be preferably designed for reproduction and 8-bit resolution.

For example, the DAC unit 130 may preferably be designed to support converting an 8-bit digital input vector into a modulator control signal with at least 8-bit resolution such that the modulator array 144 can generate a faithful representation of the digital input vector. 8-bit light input vector. Generally speaking, the modulator control signal may need to have additional resolution beyond the 8 bits of the digital input vector to compensate for the non-linearity of the modulator array 144. sexual response. Furthermore, the internal configuration of the OMM unit 150 may preferably be sufficiently stable to ensure that the value of the light output vector is not corrupted by any fluctuations in the configuration of the OMM unit 150 . For example, the temperature of the OMM unit 150 may need to be stabilized within 5 degrees, 2 degrees, 1 degree, or 0.1 degrees. Furthermore, the detection unit 146 may preferably have low enough noise to not destroy the 8-bit resolution of the light output vector, and the ADC unit 160 may preferably be designed to support analog voltages with at least 8-bit resolution. Digitization.

Power consumption and design complexity of various electronic components typically increase with bit resolution, operating speed, and bandwidth. For example, as a first-order approximation, the power consumption of the ADC unit 160 can scale linearly with the sampling rate, and the scaling factor is 2^N, where N is the bit resolution of the conversion result. In addition, the design consideration of the DAC unit 130 and the ADC unit 160 is usually a trade-off between sampling rate and bit resolution. As such, in some cases it can be expected that the ANN computing system will operate internally at a lower bit resolution than the resolution of the input data set, while maintaining the resolution of the ANN computing output.

Referring to Figure 4A, a schematic diagram of an artificial neural network (ANN) computing system 400 with 1-bit internal resolution is shown. ANN computing system 400 is similar to ANN computing system 100 except that DAC unit 130 is now replaced by driver unit 430 and ADC unit 160 is now replaced by comparator unit 460 .

The driver unit 430 is configured to generate a 1-bit modulator control signal and a multi-bit weight control signal. For example, the driving circuit of the driver unit 430 may directly receive a binary digital output from the controller 110 and convert the binary The signal is conditioned to a two-level voltage or current output suitable for driving the modulator array 144 .

The comparator unit 460 is configured to convert the output voltage of the detection unit 146 into a digital 1-bit light output. For example, the comparison circuit of the comparator unit 460 may receive a voltage from the detection unit 146, compare the voltage with a preset threshold voltage, and output a digital 0 or 1 when the received voltage is less than or greater than the preset threshold voltage, respectively.

Referring to Figure 4B, a mathematical representation of the operation of the ANN computing system 400 is shown. The operation of the ANN computing system 400 will now be described with reference to Figure 4B. For a given ANN calculation to be performed by the ANN calculation system 400, there are corresponding digital input vectors V and neural network weight matrices U. In this embodiment, the input vector V is a vector of length 4 with elements V ₀ to V ₃ , and the matrix U is a 4×4 matrix with weights U ₀₀ to U ₃₃ . Each element of vector V has a resolution of 4 bits. Each 4-bit vector element has bits 0 (bit 0) to bit 3 (bit 3) corresponding to 2^0 to 2^3 positions respectively. In this way, the decimal (base 10) of the 4-bit vector element is calculated by the sum of 2^0 ^* bit ₀ +2^ ^{1 *} bit ₁ +2^2 * bit ₂ +2^3 ^* bit ₃ )value. Therefore, as shown, the input vector V may be similarly decomposed into V _bit0 through V _bit3 by controller 110 .

Specific ANN calculations can then be performed by performing a series of matrix multiplications of 1-bit vectors, followed by summing the individual matrix multiplication results. For example, by generating a sequence of four 1-bit modulator control signals corresponding to four 1-bit input vectors through the driver unit 430, each of the decomposed input vectors V _bit0 to V _bit3 can be combined with the matrix Multiply U. This in turn produces a sequence of four 1-bit optical input vectors, which propagate through the OMM unit 150, which is configured to perform matrix multiplication of the matrix U through the driver unit 430. Then, the controller 110 can obtain a sequence of four digital 1-bit light outputs corresponding to a sequence of four 1-bit modulator control signals from the comparator unit 460 .

With the 4-bit vector decomposed into four 1-bit vectors, each vector should be processed by the ANN computing system 400 at a rate that other ANN computing systems (eg, ANN computing system 100) can process a single 4-bit vector. The vector is four times faster to maintain the same effective ANN calculation throughput. This increased internal processing speed can be thought of as time-division multiplexing four 1-bit vectors into a single timeslot for processing 4-bit vectors. The required increase in processing speed may be achieved at least in part by an increased operating speed of driver unit 430 and comparator unit 460 relative to DAC unit 130 and ADC unit 160, as the reduction in resolution of the signal conversion process typically results in achievable The signal conversion rate increases.

Although the signal conversion rate in 1-bit operation is increased by four times, the power consumption results can be significantly reduced compared to 4-bit operation. As mentioned above, the power consumption of signal conversion processing generally scales exponentially with bit resolution and linearly with conversion rate. Thus, a 16x reduction in power per conversion may be due to a 4x reduction in bit resolution, followed by a 4x increase in conversion rate. In summary, through the ANN computing system 400, the ANN computing system 100 can This achieves a 4x reduction in operating power while maintaining the same effective ANN computational throughput.

The controller 110 can then construct a 4-bit digital output vector from the four digital 1-bit light outputs by multiplying each digital 1-bit light output by the corresponding weight 2^0 to 2^3. Once the 4-bit digital output vector is constructed, ANN calculations can be performed by performing a non-linear transformation on the constructed 4-bit digital output vector to produce a transformed 4-bit digital output vector; and in memory unit 120 Store the converted 4-bit digital output vector.

Alternatively (or additionally), in some embodiments, each of the 4 digital 1-bit light outputs may be converted non-linearly. For example, a step-function nonlinear function can be used for nonlinear transformation. The converted 4-bit digital output vector can then be constructed from the nonlinearly converted digital 1-bit light output.

Although individual ANN computing systems 400 have been shown and described, generally speaking, the ANN computing system 100 of FIG. 1A may be designed to perform functions similar to those of the ANN computing system 400. For example, the DAC unit 130 may include a 1-bit DAC subunit configured to generate a 1-bit modulator control signal, and the ADC unit 160 may be designed to have a 1-bit resolution. This 1-bit ADC can be similar to or effectively equivalent to a comparator.

Furthermore, although the operation of an ANN computing system with an internal resolution of 1 bit has been described, in general, the internal resolution of an ANN computing system Can be reduced to an intermediate level below the N-bit resolution of the input data set. For example, the internal resolution can be reduced to 2^Y bits, where Y is an integer greater than or equal to 0.

The embodiments and functional operations described in the present disclosure may be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures in the disclosure and their structural equivalents, or one or more thereof. combination. The embodiments and functional operations described in the present disclosure may be implemented using one or more computer program instruction modules encoded on a computer-readable medium for execution by a data processing device or to control the operation of the data processing device. The computer-readable medium may be a manufactured product (such as a hard drive in a computer system or an optical disc sold through retail channels) or an embedded system. The computer-readable medium may be individually retrieved and subsequently encoded using one or more modules of computer program instructions, such as one or more modules of computer program instructions transmitted over a wired or wireless network. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of them.

A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including compiled or literal languages, declarative or procedural languages, and can be configured in In any form, including as a stand alone program or as a module, component, subroutine or other unit suitable for use in a computing environment. Computer programs do not necessarily correspond to files in a file system. Programs may be stored as part of a file that holds other programs or data (for example, one or more scripts stored in a markup language document), in a single file dedicated to the program in question, Or stored in multiple coordinated files (multiple coordinated file) (for example: a file that stores one or more modules, subroutines, or code portions). A computer program may be configured to execute on one computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communications network.

The processes and logic flows described in this disclosure may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and producing output. Processing and logic flows may also be performed by special purpose logic circuitry, and devices may also be implemented as special purpose logic circuitry, such as field programmable gate arrays (FPGAs) or application special integrated circuits (ASICs). .

Although this disclosure contains many implementation details, these should not be construed as limitations on the scope of the disclosure or the scope of the application, but rather as descriptions of specific features of particular embodiments of the disclosure. Certain features of this disclosure that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. In contrast, various features that are described in the context of a single embodiment can also be implemented separately in multiple embodiments or in any suitable subcombination. Furthermore, although the features above are described as operating in certain combinations and even originally claimed as such, in some cases one or more features may be removed from the claimed combination and the claimed combination may Points to a subcombination or a change in a subcombination.

Similarly, although operations are described in the drawings in a specific order, this should not be understood as requiring that these operations be performed in the specific order shown, or sequentially, or that all operations shown be performed to achieve desirable results. In some cases, multitasking Processing and parallel processing may be advantageous. Furthermore, the separation of various system components in the above embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described may generally be integrated in a single software product or packaged into multiple software products.

Thus, specific embodiments of the present disclosure have been described. Other embodiments are within the scope of the following claims. Additionally, the actions contained in the request can be performed in a different order and still achieve the desired results. For example, the optical matrix multiplication unit 150 in Figure 1A includes an optical interference unit 154 that includes a plurality of interconnected Mach-Zehnder interferences. In some embodiments, the optical interference unit can be implemented using one-, two- or three-dimensional passive diffractive optical elements that consume almost no power. Compared to an optical interference unit including a Mach-Zehnder interferometer, an optical interference unit using passive diffraction optics can have a smaller size or can be processed for the same wafer size if the number of inputs/outputs remains the same Larger number of inputs/outputs. Passive diffractive optical elements can be manufactured at lower cost than Mach-Zehnder interferometers.

Referring to FIG. 5 , in some embodiments, the artificial neural network computing system 500 includes a controller 110 , a memory unit 120 , a DAC unit 506 , an optical processor 504 and an ADC unit 160 . Memory unit 120 and ADC unit 160 are similar to corresponding components of artificial neural network computing system 100 in Figure 1A. Light processor 504 is configured to perform matrix calculations using optical components. In the artificial neural network computing system 500, the weight of the optical matrix multiplication unit 502 is fixed. DAC unit type 506 Similar to the first DAC subunit 132 of the artificial neural network computing system 100 of Figure 1A.

In an example operation of the ANN computing system 500, the computer 102 may send an artificial neural network calculation request to the ANN computing system 500. The ANN calculation request may include a set of input data to be processed by the provided ANN. The controller 110 receives the ANN calculation request and stores the input data set in the memory unit 120 .

In some embodiments, a hybrid approach is used in which part of the light matrix multiplication unit 150 includes a Mach-Zehnder interferometer and another part of the light matrix multiplication unit 150 includes passive diffractive elements.

The internal operation of ANN computing system 500 will now be described. The optical processor 504 includes a laser unit 142, a modulator array 144, a detection unit 146, and an optical matrix multiplication (OMM) unit 502. The laser unit 142, the modulator array 144, and the detection unit 146 are similar to corresponding components of the artificial neural network computing system 100 in FIG. 1A. In this embodiment, the OMM unit 502 includes two-dimensional diffractive optical elements and may be implemented as a passive integrated silicon photonic chip. The optical matrix multiplication unit 502 can be configured to implement a diffraction neural network, and can perform matrix multiplication with almost zero power consumption.

The optical processor 504 operates by encoding a digital input vector of length N onto an optical input vector of length N and propagating the optical input vector through the OMM unit 502 . The OMM unit 502 receives an optical input vector of length N and performs N×N matrix multiplication on the received optical input vector in the optical domain. The N×N matrix multiplication performed by OMM unit 502 is determined by the internal configuration of OMM unit 502 . OMM unit The internal configuration of 502 includes the size, location, and geometry of the diffractive optical elements, as well as the doping, if any, of impurities.

OMM unit 502 may be implemented in various ways. Figure 6 shows a schematic diagram of an OMM unit 502 using a two-dimensional array of diffractive elements. OMM unit 502 may include an array of input waveguides 602 for receiving optical input vectors, a two-dimensional optical interference unit 600 in optical communication with the array of input waveguides 602, and an array of output waveguides 604 in optical communication with optical interference unit 600. The optical interference unit 600 includes a plurality of diffractive optical elements and performs conversion (eg, linear conversion) of the light input vector to the second optical signal array. The array of output waveguides 604 guides the second array of optical signals output by the optical interference unit 600. At least one input waveguide in the array of input waveguides 602 is in optical communication with each output waveguide in the array of output waveguides 604 through the optical interference unit 600 . For example, for an optical input vector of length N, OMM unit 502 may include N input waveguides 602 and N output waveguides 604.

In some embodiments, the optical interference unit 600 includes a substrate with diffractive elements arranged in two dimensions (eg, in a 2D array). For example, multiple circular holes may be drilled or etched into the substrate. These holes have dimensions on the same order of magnitude as the wavelength of the input light, such that the light is diffracted by the holes (or the structure that defines the holes). For example, the size of the pores can range from 100 nm to 2 μm. The holes can be of the same or different sizes. The holes can also have other cross-sectional shapes, such as triangles, squares, rectangles, hexagons or irregular shapes. The substrate may be made of a material that is transparent or translucent to the input light, for example having a transmittance of 1% to 99% with respect to the input light. For example, the substrate can be modified from silicon, silicon oxide, silicon nitride, quartz, crystal (such as lithium niobate (LiNbO ₃ )), III-V materials (such as gallium arsenide or indium phosphide), erbium Made of erbium modified semiconductor or polymer.

In some embodiments, a holographic method can be used to form two-dimensional diffractive optical elements in a substrate. The substrate can be made of glass, crystal or photorefractive material.

When designing the OMM unit 502, we consider the size and position of the diffraction element in two dimensions (eg, X direction and Y direction), without considering the relative position of the diffraction element in the third dimension (eg, Z direction). Each diffraction element may be a three-dimensional structure formed in the substrate, such as a hole, a pillar or a stripe with a certain depth.

In Figure 6, the diffractive optical element is represented by a circle. Diffractive optical elements may also have other shapes, such as triangular, square, rectangular or irregular shapes. Diffractive optical elements can be of various sizes. Diffractive optical elements do not have to be located at grid points; their position can be changed. The schematic diagram in Figure 6 is for illustration purposes only. Actual diffractive optical elements may differ from those shown in the figures. Different arrangements of diffractive optical elements can be used to implement different matrix calculations, such as different matrix multiplication functions.

An optimization process can be used to determine the configuration of the diffractive optical elements. For example, the substrate may be divided into an array of pixels, and each pixel may be filled with substrate material (no holes) or filled with air (holes). The configuration of pixels can be modified iteratively, and for each configuration of pixels the output can be evaluated by passing light through a diffractive optical element. to perform the simulation. After performing simulations of all possible configurations of pixels, the configuration that provides the closest result to the desired matrix processing is selected as the diffractive optical element configuration of the OMM unit 502 .

As another example, the diffraction element is initially configured as an array of holes. The location, size, and shape of the holes can vary slightly from their original configuration. The parameters of each hole can be adjusted iteratively, and simulations can be performed to find the optimal configuration of the holes.

In some embodiments, machine learning processes are used to design diffractive optical elements. An analytical function that determines how pixels affect input light to produce output light, and an optimization process (such as a gradient descent method) is used to determine the best configuration of pixels.

In some embodiments, the OMM unit 502 may be implemented as a user-changeable component, and different OMM units 502 with different optical interference units 600 may be installed for different applications. For example, the ANN computing system 500 may be configured as an optical character recognition system, and the optical interference unit 600 may be configured to implement a neural network for performing optical character recognition. For example, the first OMM unit may have a first optical interference unit including a passive diffractive optical element configured to implement a first optical character recognition engine for a first set of written languages and fonts. neural network. The second OMM unit may have a second optical interference unit including a passive diffraction optical element configured to implement a second neural network for an optical character recognition engine for a second set of written languages and fonts, etc. . When the user wants to use the ANN computing system 500 to convert optical characters When recognition is applied to the first set of written languages and fonts, the user can insert the first OMM unit into the system. When the user wants to use the ANN computing system 500 to apply optical character recognition to a second set of written languages and fonts, the user can swap out the first OMM unit and insert a second OMM unit into the system.

For example, the ANN computing system 500 may be configured as a speech recognition system, and the optical interference unit 600 may be configured to implement a neural network for performing speech recognition. For example, the first OMM unit may have a first optical interference unit including a passive diffraction optical element configured to implement a first neural network for a speech recognition engine for a first spoken language. The second OMM unit may have a second optical interference unit including a passive diffractive optical element configured to implement a second neural network for the speech recognition engine for a second spoken language or the like. When the user wants to use the ANN computing system 500 to recognize speech in a first spoken language, the user can insert the first OMM unit into the system. When the user wants to use the ANN computing system 500 to recognize speech in a second spoken language, the user can swap out the first OMM unit and insert the second OMM unit into the system.

For example, the ANN computing system 500 may be part of a control unit of an autonomous vehicle, and the optical interference unit 600 may be configured to implement a neural network for performing road condition recognition. For example, the first OMM unit may have a first optical interference unit including a passive diffractive optical element configured to implement a first neural network for identifying road conditions (including road signs) in the United States road. The second OMM unit may have a second optical interference unit including a passive diffractive optical element configured to identify road conditions in Canada, including Second neural network including road signs). The third OMM unit may have a third optical interference unit including a passive diffractive optical element configured to implement a third neural network for identifying road conditions (including road signs) in Mexico and the like. When using autonomous vehicles in the United States, the first OMM unit is inserted into the system. When the autonomous vehicle crosses the border and enters Canada, the first OMM unit is swapped out and the second OMM unit is inserted into the system. On the other hand, when the autonomous vehicle crosses the border and enters Mexico, the first OMM unit is swapped out and the third OMM unit is inserted into the system.

For example, the ANN computing system 500 can be used for genetic sequencing. DNA sequences can be classified using convolutional neural networks, which is implemented using an ANN computing system 500 that includes passive diffraction optics. For example, the ANN computing system 500 can implement neural networks for distinguishing tumor types, predicting tumor grade, and predicting patient survival from gene expression patterns. For example, the ANN computing system 500 may implement a neural network for identifying the subset of genes or traits that are most predictive of the characteristic being analyzed. For example, the ANN computing system 500 may implement a neural network for predicting or inferring expression levels of all genes from a profile of a subset of genes. For example, the ANN computing system 500 can implement a neural network for epigenomic analyses, such as predicting transcription factor binding sites, enhancer regions, and genes from gene sequences. The chromatin can Chromatin accessibility. For example, ANN computing system 500 may implement a neural network for capturing structure within a genetic sequence.

For example, the ANN computing system 500 may be configured as a medical diagnostic system, and the OMM unit 502 may be configured to implement a neural network for analyzing physiological parameters to perform screening for diseases. For example, the ANN computing system 500 can be configured as a bacterial detection system, and the OMM unit 502 can be configured to implement a multiplicative function for analyzing DNA sequences to detect certain bacterial strains.

In some embodiments, OMM unit 502 includes a housing (eg, a cartridge) that protects a substrate with diffractive optical elements. The housing supports an input interface coupled to input waveguide 602 and an output interface coupled to output waveguide 604 . The input interface is configured to receive the output from the modulator array 144 and the output interface is configured to send the output of the OMM unit 502 to the detection unit 146 . The OMM units 502 may be designed as modules suitable for handling by ordinary consumers, allowing the user to easily switch from one OMM unit 502 to another. Machine learning techniques improve over time. The user can upgrade the ANN computing system 500 by swapping out the old OMM unit 502 and inserting the new upgraded version.

Similar to how optical compact discs can store digital information that can be retrieved by a CD player, OMM cells can store neural network configurations that can be used in optical processors. Just as optically compressed discs are low-cost media used to distribute digital information (including audio, video, and software programs) to consumers - In this way, the OMM unit can be a low-cost medium for distributing preconfigured neural networks or matrix processing functions (such as multiplication, convolution, or any other linear operation) to consumers.

In some embodiments, ANN computing system 500 is an optical computing platform configured to operate together with OMM units provided by different companies. This allows different companies to develop different passive optical neural networks for various applications. Passive optical neural networks are sold to end users in standardized packages, which can be installed in optical computing platforms to allow the ANN computing system 500 to perform various intelligent functions.

In some embodiments, the system may have a holder mechanism for supporting multiple OMM units 502 and a mechanical handling mechanism may be provided for automatically exchanging OMM units 502 . The system determines which OMM unit is required for the current application 502 and uses a mechanical handling mechanism to automatically retrieve the appropriate OMM unit from the holder mechanism and insert it into the light processor 504 .

For an optical wafer of a certain size, more passive diffraction elements can be assembled on the substrate than using an active interferometer (such as a Mach-Zehnder interferometer). For example, optical interference unit 154 in Figure 1B using a Mach-Zehnder interferometer can be configured to process 200×200 matrix multiplications, while optical interference unit 600 of the same overall size and using passive diffraction elements can be configured to process 5000×5000 matrix multiplication.

Passive diffractive optical elements consume almost no power, so the OMM unit 502 can be used in low power devices, such as battery operated devices. The OMM unit 502 is suitable for edge computing. For example, the OMM unit 502 may be used in smart sensors where the raw data from the sensor is processed using an OMM unit. Yuan 502 optical processor to handle. Smart sensors can be configured to send processed data to a central computer server, thereby reducing the amount of raw data sent to the central computer server. By placing smart processing capabilities on smart sensors, faults and anomalies can be detected earlier and dealt with more efficiently. OMM unit 502 is suitable for applications that need to handle large matrix multiplications. The OMM unit 502 is suitable for applications where the neural network has been trained and the weights have been determined, and does not require modification.

The substrate in which the diffractive optical element is formed may be planar or curved. In the embodiment of Figure 6A, input light enters the optical interference unit 600 from the left side, and output light exits the optical interference unit 600 from the right side (the terms "left", "right", "upper" and "lower" refer to the drawings direction shown"). In some embodiments, the passive diffractive optical element may be configured such that some output light exits the optical interference unit from the upper or lower portion, or any combination of left, right, upper, and lower 600. The substrate for the optical interference unit 600 may have various shapes, such as square, rectangular, triangular, circular, or elliptical. The optical interference unit 600 may include reflective elements or mirrors to redirect light propagation direction.

In some embodiments, the artificial neural network computing system 500 can be modified by adding an analog nonlinear unit 310 between the detection unit 146 and the ADC unit 160 . Analog nonlinear unit 310 is configured to receive the output voltage from detection unit 146 , apply a nonlinear transfer function, and output the converted output voltage to ADC unit 160 . The controller 110 may obtain the converted digital output voltage corresponding to the converted output voltage from the ADC unit 160 . Because the digital output voltage obtained from the ADC unit 160 has already been nonlinearly converted ("activated"), the nonlinear conversion step of the controller 110 can be omitted. steps, thereby reducing the computational burden on the controller 110. Then, the first conversion voltage directly obtained from the ADC unit 160 may be stored in the memory unit 120 as the first conversion digital output vector.

The light interference unit can be implemented using passive diffraction optical elements arranged in three dimensions. Referring to FIG. 7 , in some embodiments, the artificial neural network computing system 700 has an optical processor 702 that includes a three-dimensional OOM unit 708 . The artificial neural network computing system 700 includes a memory unit 120 and an ADC unit 160, which are similar to corresponding components of the artificial neural network computing system 500 in FIG. 5 . Optical processor 702 is configured to perform matrix calculations using diffractive optical elements arranged in three dimensions.

The optical processor 702 includes a laser unit 704 configured to output a two-dimensional beam array 714, and a two-dimensional modulator array 706 configured to modulate the two-dimensional beam array 714 to generate a modulated two-dimensional beam array. 716. The light processor 702 includes an optical matrix multiplication (OMM) unit 708 having a three-dimensional arrangement of diffractive optical elements and configured to process the modulated two-dimensional array of beams 716 and produce a two-dimensional array of output beams 718 . The light processor 702 includes a detection unit 710 having a two-dimensional photo sensor array to detect the two-dimensional output beam array 718. The ADC unit 160 converts the output of the detection unit 710 into a digital signal.

For example, the 3D OMM unit 708 can be implemented as a passively integrated silicon photonic column or cube. The optical matrix multiplication unit 708 can be configured to implement a diffraction neuron network and can perform matrix multiplication with almost zero power consumption.

There are many ways to encode input data for use with optical processor 702. For example, a numeric input vector of length N×N can be encoded into a vector of size The N×N light input matrix is propagated through the OMM unit 708 . OMM unit 708 performs (NxN)x(NxN) matrix multiplication in the optical domain on the received optical input matrix. The (N×N)×(N×N) matrix multiplication performed by the OMM unit 708 is determined by the internal configuration of the OMM unit 708, which includes the size, location, and geometry of the diffractive optical elements, and the doping of impurities ( if any).

OMM unit 708 may be implemented in various ways. Figure 8 shows a schematic diagram of an OMM unit 708 using a three-dimensional arrangement of diffraction elements. The OMM unit 708 may include an input waveguide matrix for receiving the optical input matrix 802, a three-dimensional optical interference unit 804 in optical communication with the input waveguide matrix, and an output waveguide matrix in optical communication with the optical interference unit 804 for providing an optical output matrix. 806. The optical interference unit 804 includes a plurality of diffractive optical elements and performs conversion (eg, linear conversion) of optical input (eg, N×N vector or matrix) to optical output (eg, N×N vector or matrix). The output waveguide matrix guides the optical signal output by the optical interference unit 804. At least one input waveguide in the input waveguide matrix is in optical communication with each output waveguide in the output waveguide matrix through optical interference unit 804. For example, for an optical input vector of length N×N, OMM unit 708 may include N×N input waveguides and N×N output waveguides.

In some embodiments, the optical interference unit 804 includes a substrate block having diffraction elements arranged in three dimensions (eg, in a 3D matrix). For example, a plurality of holes may be drilled or etched in each of a plurality of substrate slices, and the plurality of substrate slices may be combined to form a substrate block. These holes have dimensions on the same order of magnitude as the wavelength of the input light, such that the light is diffracted by the holes (or the structure that defines the holes). The holes can be of the same or different sizes. Holes can also have other cross-sectional shapes, For example triangles, squares, rectangles, hexagons or irregular shapes. In some embodiments, holographic methods can be used to form three-dimensional diffractive optical elements throughout the entire substrate block. The substrate may be made of a material that is transparent or translucent to the input light, for example having a transmittance of 1% to 99% with respect to the input light.

When designing the OMM unit 708, we consider the size and position of the diffractive elements in the x, y, and z directions. An optimization process can be used to determine the configuration of the diffractive optical elements. For example, a substrate block can be divided into a three-dimensional matrix of pixels, and each pixel can be filled with substrate material (no holes) or filled with air (holes). The configuration of pixels can be modified iteratively, and for each configuration of pixels, a simulation can be performed by passing light through a diffractive optical element and evaluating the output. After performing simulations of all possible configurations of pixels, the configuration that provides the closest result to the desired matrix processing is selected as the diffractive optical element configuration of the OMM unit 708 .

As another example, the diffractive element is initially configured as a three-dimensional matrix of holes. The location, size, and shape of the holes can vary slightly from their original configuration. The parameters of each hole can be adjusted iteratively, and simulations can be performed to find the optimal configuration of the holes.

In some embodiments, machine learning processes are used to design three-dimensional diffractive optical elements. An analytical function that determines how pixels affect input light to produce output light, and gradient descent methods are used to determine the optimal configuration of pixels.

In some embodiments, the OMM unit 708 may be implemented as a user variable component, and different OMM units 708 with different optical interference units 804 may be installed for different applications. For example, the artificial neural network computing system 700 may be configured is configured as a medical diagnostic system, and the optical interference unit 804 may be configured to implement a neural network for analyzing physiological parameters to perform screening for diseases. For example, the first OMM unit may have a first optical interference unit including a 3D passive diffractive optical element configured to implement a first neural network for screening a first group of diseases. The second OMM unit may have a second optical interference unit including a 3D passive diffraction optical element configured to implement a second neural network for screening a second group of diseases, or the like. The first and second OMM units may be developed by different companies that specialize in developing technologies for screening different diseases. When the user wants to use the artificial neural network computing system 700 to identify the first group of diseases, the user can insert the first OMM unit into the system. When the user wants to use the artificial neural network computing system 700 to identify a second group of diseases, the user can swap out the first OMM unit and insert the second OMM unit into the system.

For example, artificial neural network computing system 700 may be configured as an optical character recognition system, and optical interference unit 804 may be configured to implement a neural network for performing optical character recognition. For example, artificial neural network computing system 700 may be configured as a speech recognition system, and optical interference unit 804 may be configured to implement a neural network for performing speech recognition. For example, the artificial neural network computing system 700 may be part of a control unit of an autonomous vehicle, and the optical interference unit 804 may be configured to implement a neural network for performing road condition recognition.

For example, the artificial neural network computing system 700 can be used for gene sequencing. DNA sequences can be classified using convolutional neural networks, which is implemented using an artificial neural network computing system 700 that includes passive diffractive optical elements. For example Said, the artificial neural network computing system 700 can implement a neural network for distinguishing tumor types, predicting tumor grade, and predicting patient survival from gene expression patterns. For example, artificial neural network computing system 700 may implement a neural network for identifying a subset of genes or traits that are most predictive of a characteristic being analyzed. For example, artificial neural network computing system 700 can implement a neural network for predicting or inferring expression levels of all genes from a data graph of a subset of genes. For example, the artificial neural network computing system 700 can implement neural networks for epigenetic analysis, such as predicting transcription factor binding sites, enhancer regions, and chromatin accessibility from gene sequences. For example, the artificial neural network computing system 700 can implement a neural network for capturing structure within a genetic sequence. For example, the artificial neural network computing system 700 can be configured as a bacterial detection system, and the optical interference unit 804 can be configured to implement a multiplicative function for analyzing DNA sequences to detect certain bacterial strains.

In some embodiments, the OMM unit 708 includes a housing (eg, a cassette) that protects the substrate with the 3D diffractive optical elements. The housing supports an input interface coupled to input waveguide 602 and an output interface coupled to output waveguide 604 . The input interface is configured to receive the output from the modulator array 706 and the output interface is configured to send the output of the OMM unit 708 to the detection unit 710 . The OMM units 708 may be designed as modules suitable for handling by ordinary consumers, allowing the user to easily switch from one OMM unit 708 to another. Machine learning techniques improve over time. The user can upgrade the artificial neural network computing system 700 by swapping out the old OMM unit 708 and inserting a new upgraded version.

In some embodiments, the artificial neural network computing system 700 is an optical computing platform configured to operate together with OMM units provided by different companies. This allows different companies to develop different 3D passive optical neural networks for various applications. The 3D passive optical neural network is sold to end users in standardized packages, which can be installed in an optical computing platform to allow the artificial neural network computing system 700 to perform various intelligent functions.

In some embodiments, the system may have a holder mechanism for supporting multiple OMM units 708 , and a mechanical handling mechanism may be provided for automatically exchanging OMM units 708 . The system determines which OMM unit 708 is required for the current application and uses a mechanical handling mechanism to automatically retrieve the appropriate OMM unit 708 from the holder mechanism and insert it into the light processor 702 .

In some embodiments, the artificial neural network computing system 700 can be modified by adding an analog nonlinear unit between the detection unit 710 and the ADC unit 160 . The analog nonlinear unit is configured to receive the output voltage from the detection unit 710 , apply the nonlinear transfer function, and output the converted output voltage to the ADC unit 160 . The controller 110 may obtain the converted digital output voltage corresponding to the converted output voltage from the ADC unit 160 . Because the digital output voltage obtained from the ADC unit 160 has already been nonlinearly converted ("activated"), the nonlinear conversion step of the controller 110 can be omitted, thereby reducing the computational burden of the controller 110 . Then, the first conversion voltage directly obtained from the ADC unit 160 may be stored in the memory unit 120 as the first conversion digital output vector.

Light can be achieved using passive diffraction optical elements arranged in one dimension interference unit. Referring to Figure 9, in some embodiments, the artificial neural network computing system 900 has a light processor 906 that includes a one-dimensional light multiplication unit 916. Artificial neural network computing system 900 includes a memory unit 120, which is similar to corresponding components of artificial neural network computing system 100 in Figure 1A. The light processor 906 is configured to perform multiplication calculations using diffractive optical elements arranged in one dimension (along the axis of light propagation).

The optical processor 906 includes a laser unit 908 configured to output a laser beam 910 and a modulator 912 configured to modulate the laser beam 910 to generate a modulated beam 914 . The light processor 906 includes a one-dimensional light multiplication unit 916 having a one-dimensional arrangement of diffractive optical elements and configured to process the modulated beam 914 and generate an output beam 918 . The light processor 906 includes a detection unit 920 having a light sensor for detecting the output beam 916 . The output of the detection unit 920 is converted into a digital signal by the ADC unit 930.

For example, the light multiplication unit 916 may be implemented as a passively integrated silicon photonic waveguide with diffractive optical elements (eg, gratings or holes). The optical multiplication unit 916 may be configured to perform multiplication operations with almost zero power consumption.

There are many ways to encode input data for use by optical processor 906. For example, a digital input vector may be encoded as an optical input propagated through optical multiplication unit 916. Optical multiplication unit 916 performs multiplication of received optical input in the optical domain. The multiplication performed by the optical multiplication unit 916 is determined by the internal configuration of the optical multiplication unit 916 . The internal configuration of the OMM cell 502 includes the size, position, and geometry of the diffractive optical elements arranged in one dimension along the light propagation path, as well as the doping, if any, of impurities.

Light multiplication unit 916 may be implemented in various ways. Figure 10 shows a schematic diagram of a light multiplying unit 916 using a one-dimensional arrangement of diffractive elements. Optical multiplication unit 916 may include an input waveguide for receiving optical input 1002, a one-dimensional optical interference unit 1004 in optical communication with the input waveguide, and an output waveguide in optical communication with optical interference unit 1004 for providing optical output 1006. The optical interference unit 1004 includes a plurality of diffractive optical elements and performs conversion of optical input to optical output (eg, linear conversion). The output waveguide guides the optical signal output by the optical interference unit 1004.

In some embodiments, the optical interference unit 1004 includes an elongated substrate having diffraction elements arranged in one dimension along the light propagation path. For example, multiple holes may be drilled or etched into the substrate. These holes have dimensions on the same order of magnitude as the wavelength of the input light, such that the light is diffracted by the holes (or the structure that defines the holes). The holes can be of the same or different sizes. The substrate may be made of a material that is transparent or translucent to the input light, for example having a transmittance of 1% to 99% with respect to the input light. In some embodiments, holographic methods may also be used to form diffractive optical elements in substrates.

When designing the optical interference unit 1004, we consider the size and position of the diffractive elements along the propagation path of the light beam. An optimization process can be used to determine the configuration of the diffractive optical elements. For example, the substrate may be divided into a series of pixels, and each pixel may be filled with substrate material (no holes) or filled with air (holes). The configuration of pixels can be modified iteratively, and for each configuration of pixels, a simulation can be performed by passing light through a diffractive optical element and evaluating the output. Performing simulations of all possible configurations of pixels After that, the configuration that provides the result closest to the desired multiplication process is selected as the diffractive optical element configuration of the optical interference unit 1004 .

As another example, the diffraction element is initially configured as a series of holes. The location and size of the holes can vary slightly from their original configuration. The parameters of each hole can be adjusted iteratively, and simulations can be performed to find the optimal configuration of the holes.

In some embodiments, machine learning processes are used to design one-dimensional diffractive optical elements. An analytical function that determines how pixels affect input light to produce output light, and gradient descent methods are used to determine the optimal configuration of pixels.

In some embodiments, the light multiplication unit 916 may be implemented as a user variable component, and different light multiplication units 916 with different light interference units 1004 may be installed for different applications. For example, the artificial neural network computing system 900 can be configured as a bacterial detection system, and the optical interference unit 1004 can be configured to implement a multiplicative function for analyzing DNA sequences to detect certain bacterial strains. For example, the first light multiplication unit may have a first light interference unit including a 1D passive diffractive optical element configured to implement a first multiplication function for detecting the first group of bacteria. The second light multiplication unit may have a second light interference unit including a 1D passive diffraction optical element configured to implement a second multiplication function for detecting the second group of bacteria, and the like. The first and second light multiplying units may be developed by different companies that specialize in developing technologies for detecting different bacteria. When the user wants to use the artificial neural network computing system 900 to detect the first group of bacteria, the user can insert the first light multiplication unit into the system. When the user wants to use the artificial neural network computing system 900 To identify a second group of bacteria, the user can swap out the first light multiplication unit and insert a second light multiplication unit into the system. By using one-dimensional diffractive optical elements, the laser unit 908, the modulator 912, the detection unit 920 and the ADC unit 930 can be manufactured at low cost.

In some embodiments, the light multiplying unit 916 includes a housing (eg, a cassette) that protects the substrate with the ID diffractive optical element. The housing supports an input interface coupled to the input waveguide and an output interface coupled to the output waveguide. The input interface is configured to receive the output from the modulator 912 and the output interface is configured to send the output of the light multiplication unit 916 to the detection unit 920 . The light multiplication units 916 may be designed as modules suitable for handling by ordinary consumers, allowing the user to easily switch from one light multiplication unit 916 to another. Machine learning techniques improve over time. The user can upgrade the artificial neural network computing system 900 by swapping out the old optical multiplication unit 916 and inserting a new upgraded version.

In some embodiments, the artificial neural network computing system 900 is an optical computing platform configured to operate together with optical multiplication units provided by different companies. This allows different companies to develop different 1D passive optical multiplication functions for various applications. The 1D passive optical multiplication function is sold to end users in standardized packages, which can be installed in optical computing platforms to allow the artificial neural network computing system 900 to perform various intelligent functions.

In some embodiments, the system may have a holder mechanism for supporting a plurality of light multiplying units 916 and a mechanical handling mechanism may be provided for automatically exchanging the light multiplying units 916. The system determines which light multiplication unit is required for the current application 916, and uses a mechanical handling mechanism to automatically retrieve the appropriate light multiplication unit 916 from the holder mechanism and insert it into the light processor 906.

In some embodiments, the artificial neural network computing system 900 can be modified by adding an analog nonlinear unit between the detection unit 920 and the ADC unit 930 . The analog nonlinear unit is configured to receive the output voltage from the detection unit 920 , apply the nonlinear transfer function, and output the converted output voltage to the ADC unit 930 . The controller 902 may obtain the converted digital output voltage corresponding to the converted output voltage from the ADC unit 930 . Because the digital output voltage obtained from the ADC unit 930 has already been nonlinearly converted ("activated"), the nonlinear conversion step of the controller 902 can be omitted, thereby reducing the computational burden on the controller 902. Then, the first conversion voltage directly obtained from the ADC unit 930 may be stored in the memory unit 120 as the first conversion digital output vector.

Passive wafers with passive diffractive optics offer many advantages. First, because active components (usually the bulkiest components) have been eliminated, any given size of chip can contain larger neural networks. Typically useful neural networks can include millions of weights, which are challenging to implement on active chips and may require multiple data runs through the chip and reprogramming of the chip. In contrast, a single passive chip might be able to support an entire neural network. Second, the very low power consumption of passive chips is important for "edge" applications, since such applications may require a small footprint and low power consumption. Third, passive wafers can be manufactured at very low cost because they contain no active components.

Optical matrix multiplication units with passive diffractive optical elements can also be used in wavelength division multiplexing artificial neural network computing systems. For example, the OMM unit 150 of the WDM ANN computing system 104 in Figure 1F may be replaced by an OMM unit using passive diffractive optical elements. In this embodiment, the second DAC sub-unit 134 may be removed.

In some embodiments, a light processor (eg, 504, 702) may perform matrix processing in addition to matrix multiplication. The optical matrix multiplication units 502 and 708 may be replaced by optical matrix processing units that perform other types of matrix processing.

Figure 25 shows a flow diagram of a method 2500 for performing ANN calculations using an ANN calculation system 500, 700 or 900 that includes one or more light matrix multiplication units or light multiplication with passive diffraction elements Units, such as 2D OMM unit 502, 3D OMM unit 708, or 1D OM unit 916. The steps of method 2500 may be performed, at least in part, by controller 110 or 902. In some embodiments, the various steps of method 2500 may be run in parallel, combined, in a loop, or in any order.

In step 2510, an artificial neural network (ANN) calculation request including an input data set is received. The input data set includes the first digit input vector. The first digit input vector is a subset of the input data set. For example, it could be a subregion of the image. ANN calculation requests may be generated by various entities (eg, computer 102). Computers may include one or more of various types of computing devices, such as personal computers, server computers, vehicle computers, and flight computers. An ANN calculation request generally refers to an electrical signal that notifies or informs the ANN calculation system 500, 700 or 900 that the ANN calculation is to be performed. in some In embodiments, the ANN calculation request may be divided into two or more signals. For example, the first signal may query the ANN computing system 500, 700 or 900 to check whether the ANN computing system 500, 700 or 900 is ready to receive the input data set. In response to a positive response from the ANN computing system 500, 700, or 900, the computer may send a second signal including the input data set.

In step 2520, the input data set is stored. The controller 110 may store the input data set in the memory unit 120 . Storing the input data set in the memory unit 120 may allow flexibility in the operation of the ANN computing system 500, 700, or 900, which may, for example, improve the overall performance of the system. For example, by retrieving desired portions of the input data set from memory unit 120, the input data set may be divided into numeric input vectors of a set size and format. Different parts of the input data set can be processed in various orders, or shuffled, to allow various types of ANN calculations to be performed. For example, shuffling can allow matrix multiplication to be performed via block matrix multiplication techniques when the input and output matrices are of different sizes. As another example, storing the input data set in the memory unit 120 may allow multiple ANN calculation requests to be queued by the ANN computing system 500, 700, or 900, which may allow the ANN computing system 500, 700, or 900 to It maintains operation at full speed without periods of inactivity.

In step 2530, a first plurality of modulator control signals are generated based on the first digital input vector. Controller 110 may send the first DAC control signal to DAC unit 506, 712, or 904 to generate a first plurality of modulator control signals. The DAC unit 506, 712 or 904 generates a first plurality of modulations based on the first DAC control signal. The modulator control signal is applied and modulator array 144, 706 or modulator 912 generates an optical input vector representing the first digital input vector.

The first DAC control signal may include a plurality of digital values to be converted by the DAC unit 506, 712, or 904 into a first plurality of modulator control signals. Multiple digit values typically correspond to the first digit input vector and can be related through various mathematical relationships or lookup tables. For example, the plurality of digit values may be linearly proportional to the values of the elements of the first digit input vector. As another example, a plurality of digital values may be associated with elements of a first digital input vector via a lookup table configured to maintain the relationship between the digital input vector and the values generated by modulator array 144, 706 or modulator 912. Linear relationship between light input vectors.

In some embodiments, the 2D OMM unit 502, the 3D OMM unit 708, or the ID OM unit 916 are configured to perform light matrix processing based on the optical input vector and a plurality of neural network weights implemented using passive diffraction elements or Light multiplication. Multiple neural network weights representing a matrix M can be decomposed into M = USV* by the singular value decomposition (SVD) method, where U is an M×M positive matrix and S is an MxN diagonal with non-negative real numbers on the diagonal. matrix, and V* is the complex conjugate of the N×N positive matrix V. In this case, the passive diffraction element may be configured to implement matrix V, matrix S, and matrix U, such that OMM unit 502 or 708 as a whole implements matrix M.

In step 2540, the light output vector corresponding to the light matrix multiplication unit or the first plurality of digital light outputs of the light multiplication is obtained. The light input vector generated by the modulator array 144, 706 or the modulator 912 is processed by the 2D OMM unit 502, the 3D OMM unit 708 or the ID OM unit 916 and converted into a light output vector. quantity. The light output vector is detected by the detection unit 146, 710 or 920 and converted into an electrical signal, which may be converted into a digital value by the ADC unit 160 or 930. The controller 110 or 902 may send a conversion request to the ADC unit 160 or 930 to start converting the voltage output by the detection unit 146, 710 or 920 into a digital light output. Once the conversion is completed, the ADC unit 160 or 930 may send the conversion result to the controller 110 or 902. Alternatively, the controller 110 or 902 may retrieve the conversion results from the ADC unit 160 or 930. Controller 110 or 902 may form a digital output vector from the digital light output, the digital output vector corresponding to the result of matrix multiplication of the input digital vector. For example, digital light output can be organized or connected to have a vector format.

In some embodiments, the ADC unit 160 or 930 may be configured or controlled to perform ADC conversion based on the DAC control signal being sent by the controller 110 or 902 to the DAC unit 506, 712, or 904. For example, the ADC conversion may be set to start at a preset time after the DAC unit 506, 712 or 904 generates the modulation control signal. Such control of ADC conversion can simplify the operation of the controller 110 or 902 and reduce the number of necessary control operations.

In step 2550, a non-linear transformation is performed on the first digital output vector to produce a first transformed digital output vector. An ANN's nodes, or artificial neurons, operate by first performing a weighted summation of signals received from nodes in previous layers, and then performing a nonlinear transformation ("activation") of the weighted summation to produce an output. Various types of ANN can implement various types of differentiable nonlinear transformations. Examples of nonlinear transfer functions include modified linear unit (RELU) functions, sigmoid functions, hyperbolic tangent functions, X^2 functions, and |X| functions. This non-linear transformation is performed on the first digital output by controller 110 or 902 to produce a first transformed digital output vector. In some embodiments, the nonlinear conversion may be performed by dedicated digital integrated circuits within controller 110 or 902. For example, controller 110 or 902 may include one or more modules or circuit blocks that are particularly suitable for accelerating the calculation of one or more types of nonlinear transformations.

In step 2560, the first converted digital output vector is stored. The controller 110 or 902 may store the first converted digital output vector in the memory unit 120 . In the case where the input data set is divided into a plurality of digit input vectors, the first converted digit output vector corresponds to the ANN calculation result of a portion of the input data set, such as the first digit input vector. As such, storing the first transformed digital output vector allows the ANN computing system 500, 700, or 900 to perform and store additional calculations on other digital input vectors of the input data set to be later aggregated into a single ANN output.

In step 2570, the artificial neural network output generated based on the first transformed digital output vector is output. The controller 110 or 902 generates an ANN output that is the result of processing the input data set through an ANN defined by the first plurality of neural network weights. Where the input data set is divided into multiple digital input vectors, the resulting ANN output is an aggregate output that includes the first transformed digital output, but may further include additional transformed digital outputs corresponding to other portions of the input data set. Once the ANN output is generated, the generated output is sent to the computer that initiated the ANN calculation request (eg, computer 102).

The 2D OMM unit 502, the 3D OMM unit 708, or the 1D OM unit 916 may represent the weight coefficient of a hidden layer of the neural network. If nerves If the network has multiple hidden layers, additional 2D OMM units 502, 3D OMM units 708, or 1D OM units 916 may be coupled in series. Figure 26 shows an embodiment of an ANN computing system 2600 for implementing a neural network with two hidden layers. The first 2D light matrix multiplication unit 2604 represents the weight coefficient of the first hidden layer, and the second 2D light matrix multiplication unit 2606 represents the weight coefficient of the second hidden layer. ANN computing system 2600 includes controller 110, memory unit 120, DAC unit 506, and optoelectronic processor 2602. Memory unit 120 and DAC unit 506 are similar to corresponding components of artificial neural network computing system 500 in FIG. 5 . Optoelectronic processor 2602 is configured to perform matrix calculations using optical and electronic components.

The optoelectronic processor 2602 includes a first laser unit 142a, a first modulator array 144a, a first 2D optical matrix multiplication unit 2604, a first detection unit 146a, a first analog nonlinear unit 310a, an analog memory unit 320, The second laser unit 142b, the second modulator array 144b, the second 2D optical matrix multiplication unit 2606, the second detection unit 146b, the second analog nonlinear unit 310b and the ADC unit 160. The operations of the first laser unit 142, the first modulator array 144a, the first detection unit 146a, the first analog nonlinear unit 310a, and the analog memory unit 320 are similar to the corresponding components shown in Section 3B. The first 2D OMM unit 2604 is similar to the 2D OMM 502 of FIG. 5 . The output of the analog memory unit 320 drives the second modulator array 144b, which modulates the laser light from the second laser unit 142b to generate a light vector. The light vectors from the second modulator array 144b are processed by the second 2D OMM unit 2606, which performs matrix multiplication and generates light output vectors, which are detected by the second detection unit 246b. No. The second detection unit 246b is configured to generate an output voltage corresponding to the optical signal from the optical output vector of the second 2D OMM unit 2606. ADC unit 160 is configured to convert the output voltage into a digital output voltage. The controller 110 may obtain a digital output corresponding to the light output vector of the second 2D OMM unit 2606 from the ADC unit 160 . The controller 110 may form a digital output vector from the digital output, the digital output vector corresponding to a result of a second matrix multiplication of a nonlinear transformation of a result of a first matrix multiplication of the input digital vector. The second laser unit 142b is combined with the first laser unit 142a by using an optical splitter to divert some of the light from the first laser unit 142a to the second modulator array 144b.

The above principles can be applied to implement neural networks with three or more hidden layers, where the weight coefficient of each hidden layer is represented by the corresponding 2D OMM unit.

Figure 27 shows an embodiment of an ANN computing system 2700 for implementing a neural network with two hidden layers. The first 3D light matrix multiplication unit 2704 represents the weight coefficient of the first hidden layer, and the second 3D light matrix multiplication unit 2706 represents the weight coefficient of the second hidden layer. ANN computing system 2700 includes controller 110, memory unit 120, DAC unit 712, and optoelectronic processor 2702. Memory unit 120 and DAC unit 712 are similar to corresponding components of artificial neural network computing system 700 in FIG. 7 . Optoelectronic processor 2702 is configured to perform matrix calculations using optical and electronic components.

The optoelectronic processor 2402 includes a first laser unit 704a, a first modulator array 706a, a first 3D light matrix multiplication unit 2704, a first detection unit 710a, a first analog nonlinear unit 310a, an analog memory unit 320, The second thunder The emitter unit 704b, the second modulator array 706b, the second 3D light matrix multiplication unit 2706, the second detection unit 710b, the second analog nonlinear unit 310b and the ADC unit 160. The operations of the first laser unit 704a, the first modulator array 706a, the first detection unit 710a, the first analog nonlinear unit 310a, and the analog memory unit 320 are similar to the corresponding components shown in Section 3B. The first 3D OMM unit 2704 is similar to the 3D OMM 708 of FIG. 7 . The output of the analog memory unit 320 drives the second modulator array 706b, which modulates the laser light from the second laser unit 704b to generate a light vector. The light vectors from the second modulator array 706b are processed by the second 3D OMM unit 2706, the second 2D OMM unit 2706 performs matrix multiplication and generates light output vectors, and the light output vectors are detected by the second detection unit 710b. The second detection unit 710b is configured to generate an output voltage corresponding to the optical signal from the optical output vector of the second 3D OMM unit 2706. ADC unit 160 is configured to convert the output voltage into a digital output voltage. The controller 110 may obtain a digital output corresponding to the light output vector of the second 3D OMM unit 2406 from the ADC unit 160 . The controller 110 may form a digital output vector from the digital output, the digital output vector corresponding to a result of a second matrix multiplication of a nonlinear transformation of a result of a first matrix multiplication of the input digital vector. The second laser unit 704b is combined with the first laser unit 704a by using an optical splitter to divert some of the light from the first laser unit 704a to the second modulator array 706b.

The above principles can be applied to implement neural networks with three or more hidden layers, where the weight coefficient of each hidden layer is represented by the corresponding 3D OMM unit.

The 2D OMM unit 502 and the 3D OMM unit 708 with passive diffraction optics are suitable for Recurrent Neural Networks (RNN), where the output of the network during the (k)th pass through the neural network is recycled back to The input of the neural network is used as input during the (k+1)th pass so that the weight coefficients of the neural network remain the same during multiple passes.

Figure 28 shows an embodiment of a neural network computing system 2800 that can be used to implement recursive neural networks. Neural network computing system 2800 includes optical processor 2802, which operates in a manner similar to optical processor 140 of Figure 3B, except that OMM unit 150 is replaced by a 2D OMM unit 2804, which may be similar to the 2D OMM of Figure 6 Unit 502. The neural network weights of the 2D OMM unit 2804 are fixed, so the neural network computing system 2800 does not require the second DAC subunit 134 used in the artificial neural network computing system 302 of Figure 3B.

Figure 29 shows an embodiment of a neural network computing system 2900 that can be used to implement recursive neural networks. Neural network computing system 2900 includes optical processor 2902 that operates in a manner similar to optical processor 140 of Figure 3B except that laser unit 142, modulator array 144, OMM unit 150, and detection unit 146 are individually Replaced by the laser unit 704, modulator array 706, 3D OMM unit 2904 and detection unit 710 in Figure 7. The neural network weights of the 3D OMM unit 2904 are fixed, so the neural network computing system 2900 does not require the second DAC subunit 134 used in the artificial neural network computing system 302 of Figure 3B.

Figure 30 shows a schematic diagram of an artificial neural network computing system 3000 with 1-bit internal resolution. ANN computing system 3000 is similar to that of Figure 4A ANN computing system 400, except that the OMM unit 150 is replaced by a 2D OMM unit 3004 (which is similar to the 2D OMM unit 502 of Figure 5) and the second driver sub-unit 434 is omitted. ANN computing system 3000 operates in a manner similar to ANN computing system 400 in that the input vector is decomposed into a plurality of 1-bit vectors and the individual matrix multiplication results can then be calculated by performing a series of matrix multiplications of the 1-bit vectors. Sum, to perform some ANN calculations.

Figure 31 shows a schematic diagram of an artificial neural network computing system 3100 with 1-bit internal resolution. The ANN computing system 3100 is similar to the ANN computing system 400 of Figure 4A, except that the OMM unit 150 is replaced by a 3D OMM unit 3104 (which is similar to the 3D OMM unit 708 of Figure 7), and the second driver sub-unit 434 is omitted. In the embodiment of FIG. 31 , the laser unit 142 , the modulator array 144 and the detection unit 143 of FIG. 4A are respectively composed of the laser unit 704 , the modulator array 706 and the detection unit 710 of FIG. 7 replace. The ANN computing system 3100 operates in a manner similar to the ANN computing system 400 in that the input vector is decomposed into a plurality of 1-bit vectors and the individual matrix multiplication results can then be calculated by performing a series of matrix multiplications of the 1-bit vectors. Sum, to perform some ANN calculations.

這裡，指數l和i表示第l層神經網路中的第i個神經元，λ是光的波長，r是距離，其中：

來自每個二次光源的輸出可以被寫為輸入乘以光源的相位和強度調變：

這裡，t是傳輸調變(transmission modulation)，其是包括幅度和相位調變的複數項(complex term)，並且

是來自所有先前光源的輸入的總和。總體來說，輸出可以合併為遠場繞射時間w和幅度|A|和額外相位項(phase term)。因此，每一層中的每個點可以被認為是從前一層獲取來自多個神經元的輸入，並且在輸出到下一層之前加入額外的相位和強度調變的神經元。 The principle of optical diffraction neural network is described below. Optical diffractive neural networks can be implemented as several layers of diffractive or transmissive optical media. Based on the Huygens-Fresnel principle, each point in the diffraction medium can be considered as a secondary light source. For each light source, far field diffraction can be described by the following equation:

Here, the indices l and i represent the i-th neuron in the l-th layer neural network, λ is the wavelength of light, and r is the distance, where:

The output from each secondary light source can be written as the input times the phase and intensity modulation of the light source:

Here, t is the transmission modulation, which is a complex term including amplitude and phase modulation, and

is the sum of input from all previous light sources. Overall, the output can be combined into far-field diffraction time w and amplitude |A| and an additional phase term. Therefore, each point in each layer can be thought of as taking input from multiple neurons from the previous layer and adding additional phase and intensity modulated neurons before outputting to the next layer.

The following describes a compact design of a compact photonic matrix multiplier unit that can implement general positive matrix multiplication. Referring to FIG. 11 , the photon matrix multiplier unit 1100 includes a modulator 1102 , a plurality of interconnected interferometers 1104 and an attenuator 1106 . Interconnect interferometer 1104 includes directional coupler layers (or groups or sets of directional couplers) 1108a, 1108b, 1108c, 1108d, and 1108e (collectively 1108) and phase shifter layers 1110a, 1110b, 1110c, and 1110d (collectively for 1110). Each directional coupler layer (or group or set of directional couplers) may include one or more directional couplers. In this example , interconnect interferometer 1104 includes five layers of directional couplers 1108 and four layers of phase shifters. In other embodiments, the photonic matrix multiplier unit 1100 may have different directional coupler and phase shifter layers. Compared to conventional matrix multiplier units using interconnected Mach-Zehnder interferometers, the photonic matrix multiplier unit 1100 has directional couplers 1108 positioned in such a way that the number of layers of directional couplers 1108 is reduced.

Here, the term "layer" in the terms "directional coupler layer" and "phase shifter layer" refers to a group or a collection of orientations based on their position relative to the input port and the output port in the photonic matrix multiplier unit 1100 coupler or phase shifter. In the embodiment of Figure 11, the input optical signal is processed by the first layer directional coupler 1108a, then by the second layer phase shifter 1110a, then by the third layer directional coupler 1108b, and then by the fourth layer phase shifter 1110a. Shifter 1110b processing, etc.

For example, a conventional matrix multiplier unit using interconnected Mach-Zehnder interferometers may require 2N layers of directional couplers, whereas the photonic matrix multiplier unit 1100 only requires N+2 layers of directional couplers. N represents the number of input signals, or the number of digits in the input vector. The mesh architecture used in the photonic matrix multiplier unit 1100 has perhaps the most compact geometry of a photonic interconnect interferometer that can perform general matrix calculations.

Figure 12A shows a schematic diagram comparing the interconnection interferometer 1104 of the photon matrix multiplier unit 1100 with various numbers of input signals compared to conventional designs. When there are four input signals, the interconnected Mach-Zehnder interferometer 1200 according to the conventional design requires 8 layers of directional couplers, while the interconnected interferometer 1202 according to the new compact design only requires 6 layers of directional couplers. When there are three input signals, the design based on common knowledge The interconnected Mach-Zehnder interferometer 1204 requires 6 layers of directional couplers, while the interconnected interferometer 1206 according to the new compact design requires only 5 layers of directional couplers. When there are 8 input signals, the interconnected Mach-Zehnder interferometer 1208 according to the conventional design requires 16 layers of directional couplers, while the interconnected interferometer 1210 according to the new compact design only requires 10 layers of directional couplers.

Generally speaking, when there are n input signals, the interconnected Mach-Zehnder interferometer according to the conventional design requires 2n layers of directional couplers, while the interconnected interferometer according to the new compact design only requires n+2 layers of directional couplers. .

In a conventional design, there are n layers of Mach-Zehnder interferometers for n input signals, and each Mach-Zehnder interferometer includes a directional coupler, followed by a pair of phase shifters, followed by another directional coupler. Therefore, an n-layer Mach-Zehnder interferometer has a 2n-layer directional coupler. As a result, in the conventional design, for n input signals, n layers of phase shifters and 2n layers of directional couplers are required.

In contrast, in the new compact design, a layer of directional couplers is followed by a first layer of phase shifters, then a layer of directional couplers, then a second layer of phase shifters, then a layer of directional couplers, then is the third layer phase shifter, and so on. After the last layer of phase shifters, there are two layers of directional couplers. As a result, for n input signals, there are n layers of phase shifters and n+2 layers of directional couplers.

Because directional couplers occupy a large amount of space, compared with the conventional design, the number of directional couplers is reduced from 2. Reducing n to n+2 can result in a significant reduction in the size of the photon matrix multiplier unit 1100.

Figure 12B shows a schematic diagram of a compact interconnection interferometer 1212 according to the new design, in which the number of input signals is 5.

The following describes a compact design decomposition using gradient descent. The compact design of the photon matrix multiplier described above can take any positive matrix U and use an analytic decomposition algorithm to determine which phases need to be achieved using phase shifters and therefore implement the matrix U. For example, we can extract the phase from a given matrix U by using gradient descent. The gradient descent process is as follows. We start with a fixed matrix U and initialize random weights θ for a compactly designed phase shifter. We use compact design to construct the matrix U', that is, U'=CompactDesign(θ). Next we look at the loss function L=|U-U'|^2 (which is the Frobenius norm of the matrix) and minimize this function using gradient descent (i.e. borrow By using gradient updates to update θ).

Referring to Figure 13, we use homodyne detection (i.e. we take out the real part at the output), so an additional attenuator layer 1302 is provided before detection to simulate an orthogonal matrix. This means that along with θ, we also need to learn the diagonal weight of the attenuator (diagonal weight) x. This way we know the required phase and diagonal weights of U and can obtain the decomposition numerically.

The following describes an optical generative adversarial network (OGAN), which includes a generator configured to efficiently generate faithful data. Figure 14 shows an embodiment of an optical generative adversarial network 1400, where the generator 1404 includes a generator configured or trained to generate something like A neural network is a synthetic image 1410 of a real image, and the discriminator 1402 includes a neural network trained to determine whether an input image is real or synthetic. An initial training image set 1406 is provided to train the discriminator 1402 so that the discriminator 1402 learns the characteristics of real images. Similarly, the generator 1404 is trained using a set of training images (not shown) so that the generator 1404 can generate a synthetic image 1410 with characteristics similar to real images.

In some embodiments, training of the discriminator 1402 is performed electronically, for example, using a transistor-based data processor such as a central processing unit or a general purpose graphic processor unit. Calculate the weights of the neural layers of the discriminator 1402. Similarly, training of the generator 1404 is also performed electronically to calculate the weights of the neural layers of the generator 1404 .

The synthetic image 1410 generated by the generator 1404 may be provided to the discriminator 1402 to further train the discriminator 1402 so that the discriminator 1402 can detect real images more accurately. The detection results of the discriminator 1402 can also be used to further train the generator 1404, so that the generator 1404 can generate a more realistic synthetic image 1410, that is, closer to a real image.

Photogenerated adversarial network 1400 has many applications. For example, in some applications, obtaining a large number of real images for training the discriminator 1402 may be difficult or expensive. In order to train the discriminator 1402 to detect, for example, cancer cells, a large number of cancer cell images are required during the training phase. It may be difficult and expensive to obtain large numbers of cancer cell images from cancer patients, so there may not be enough samples to train the discriminator 1402 with sufficient accuracy. To improve the discriminator 1402, the generator 1404 is trained to generate cancer Real images of cells, and real synthetic images 1410 of cancer cells are used to further train the discriminator 1402, thereby improving the ability of the discriminator 1402 to detect cancer cells.

In some embodiments, the generator 1404 may be an optical chip that contains active components, such as active phase shifters for modifying the weights of the neural network. After training the generator 1404, the active elements are fixed so that the weights are fixed. The random noise 1408 is fed to the generator 1404, which then generates a synthetic image 1410 based on the random noise 1408, where the synthetic image 1410 resembles a real image of a cancer cell.

In some embodiments, generator 1404 is implemented using optical matrix multiplication units as shown in Figure 5, Figure 7, and/or Figure 9. After determining the weights of the neural network, the optical matrix multiplication unit is configured based on the determined weights to implement the neural network. Because the input to the generator 1404 is random noise 1408, there is no need to have a modulator array, allowing the generator 1404 to have a small footprint.

Regardless of whether the generator 1404 is implemented using a passive optical chip or an optical chip with active components, the trained generator 1404 can produce realistic images (e.g., similar to real images of cancer cells), which can then be provided to discriminator 1402 to further train and improve the discriminator 1402. Generator 1404 has high throughput and can generate composite images 1410 at rates that are orders of magnitude faster than using conventional electronic data processors (eg, general purpose graphics processing units). The generator 1404 has low power consumption compared to conventional electronic data processors, possibly by several orders of magnitude.

Generator 1404 has a variety of applications. For example, the synthesized images produced by generator 1404 may have many applications in the medical field. generator 1404 may be configured to synthesize images of tissue associated with certain diseases, and the synthesized images may be used to train discriminator 1402 to identify tissue associated with the disease. For example, the synthetic image generated by the generator 1404 may have many applications in the field of autonomous driving or navigation. For example, the generator 1404 may be configured to generate synthetic images of various traffic conditions, and the synthetic images may be used to train the discriminator 1402 to identify traffic conditions. For example, the composite image generated by generator 1404 may have many applications in the field of manufacturing quality control. For example, the generator 1404 may be configured to generate synthetic images of defective products, and the synthetic images may be used to train the discriminator 1402 to detect defective products.

In some embodiments, the light generation adversarial network 1400 includes a coherent light source, a filter for inputs of random amplitude and phase, where both amplitude and phase follow a known distribution. The photogenerated countermeasure network 1400 includes a mesh of interferometers for rapid information processing. The photogenerated countermeasure network 1400 can be designed with an architecture that does not require shuffling the weights, ie, no reprogramming of the interferometer. The optically generated countermeasures network 1400 may also be designed to include fast phase shifters with operating frequencies greater than 1 GHz. For example, it can have (i) nonlinearity in the analog electronics domain, (ii) simple optical nonlinearity, or (iii) nonlinearity in the digital electronics domain.

The following describes a novel photonic circuit with interconnected Mach-Zehnder interferometers configured to implement a logic gate. Referring to Figure 15, the Mach-Zehnder interferometer 1500 includes a phase shifter 1502 configured to cause the Mach-Zehnder interferometer 1500 to achieve the following rotation:

偵測器1602產生表示偵測訊號的絕對值的輸出，因此偵測器1602的輸出為：

比較器1604的類比電子閾值被偏置(biased)為1/2，以移除1/

因子，因此比較器1604的輸出是：

Referring to Figure 16, the photonic circuit 1600 can implement XOR gates and OR gates. Photonic circuit 1600 includes a Mach-Zehnder interferometer 1500, a detector 1602, and a comparator 1604 with analog electronic thresholds. When input signals x ₁ and x ₂ are provided to photonic circuit 1600, Mach-Zehnder interferometer 1500 performs the following operations:

Detector 1602 produces an output representing the absolute value of the detection signal, so the output of detector 1602 is:

The analog electronic threshold of comparator 1604 is biased by 1/2 to remove 1/2

factor, so the output of comparator 1604 is:

The photonic circuit 1600 produces the following results for various combinations of the input signals x ₁ and x ₂ : 0,0 → 0,0 → 0,0

0,1 → 1/sqrt(2),1/sqrt(2) → 1,1

1,0 → 1/sqrt(2),1/sqrt(2) → 1,1

,1/

)，偵測器1602輸出(1/

,1/

)，並且比較器1604產生結果(1,1)。當輸入(x₁,x₂)=(1,0)時，馬赫曾德爾干涉儀1500執行乘法，產生結果(1/

,1/

)，偵測器1602輸出(1/

,1/

)，並且比較器1604產生結果(1,1)。當輸入(x₁,x₂)=(1,1)時，馬赫曾德爾干涉儀1500執行乘法，產生結果(0,

)，偵測器1602輸出(0,

)，並且比較器1604產生結果(0,1)。上述結果指示出偵測器1602在第一輸出1606a產生1/

．|x₁-x₂|，並且在第二輸出1606b產生1/

．|x₁+x₂|。比較器1604移除1/

因子以在第一輸出1608a產生XOR(x₁,x₂)並且在第二輸出1608b產生OR(x₁,x₂)。 1,1 → 0,sqrt(2) → 0,1 In the above content, the first pair of numbers is the input signal, the second pair of numbers is the output of the detector 1602 and the third pair of numbers is the output of the comparator 1604. When the input (x ₁ , x ₂ )=(0,0), the Mach-Zehnder interferometer 1500 performs multiplication, producing the result (0,0), the detector 1602 outputs (0,0), and the comparator 1604 produces Result(0,0). When the input (x ₁ ,x ₂ )=(0,1), the Mach-Zehnder interferometer 1500 performs multiplication, producing the result (-1/

,1/

), the detector 1602 outputs (1/

,1/

), and comparator 1604 produces the result (1,1). When the input (x ₁ ,x ₂ )=(1,0), the Mach-Zehnder interferometer 1500 performs multiplication, producing the result (1/

,1/

), the detector 1602 outputs (1/

,1/

), and comparator 1604 produces the result (1,1). When the input (x ₁ ,x ₂ )=(1,1), the Mach-Zehnder interferometer 1500 performs multiplication, producing the result (0,

), the detector 1602 outputs (0,

), and comparator 1604 produces the result (0,1). The above result indicates that the detector 1602 generates 1/

． |x ₁ -x ₂ |, and 1/ is produced at the second output 1606b

． |x ₁ +x ₂ |. Comparator 1604 removes 1/

Factor to produce XOR(x ₁ ,x ₂ ) at the first output 1608a and OR(x ₁ ,x ₂ ) at the second output 1608b.

偵測器1602的輸出被再循環回到光子電路1700的輸入，並且在訊號第二次通過馬赫曾德爾干涉儀1500和偵測器1602之後，偵測器1602產生最終輸出：

Referring to Figure 17A, the photonic circuit 1700 can implement AND gates and OR gates. Photonic circuit 1700 includes a Mach-Zehnder interferometer 1500 and a detector 1602, where the output of detector 1602 is recycled once. When input signals x ₁ and x ₂ are provided to photonic circuit 1700, Mach-Zehnder interferometer 1500 and detector 1602 produce outputs:

The output of detector 1602 is recycled back to the input of photonic circuit 1700, and after the signal passes a second time through Mach-Zehnder interferometer 1500 and detector 1602, detector 1602 produces the final output:

在上述內容中，第一對數字是輸入訊號，第二對數字是第一次通過之後偵測器1602的輸出，並且第三對數字是第二次通過之後偵測器1602的輸出。當輸入(x₁,x₂)=(0,0)時，在第一次通過馬赫曾德爾干涉儀1500之後，偵測器1602輸出(0,0)，並且在第二次通過馬赫曾德爾干涉儀1500之後，偵測器1602輸出(0,0)。當輸入(x₁,x₂)=(0,1)時，在第一次通過馬赫曾德爾干涉儀1500之後，偵測器1602輸出(1/

,1/

)，並且在第二次通過馬赫曾德爾干涉儀1500之後，偵測器1602輸出(0,1)。當輸入(x₁,x₂)=(1,0)時，在第一次通過馬赫曾德爾干涉儀1500之後，偵測器1602輸出 (1/

,1/

)，並且在第二次通過馬赫曾德爾干涉儀1500之後，偵測器1602輸出(0,1)。當輸入(x₁,x₂)=(1,1)時，在第一次通過馬赫曾德爾干涉儀1500之後，偵測器1602輸出(0,

)，並且在第二次通過馬赫曾德爾干涉儀1500之後，偵測器1602輸出(1,1)。上述結果指示出在兩次通過之後，檢測器1602在第一輸出1704產生表示AND(x₁,x₂)的訊號，並且在第二輸出1706產生表示OR(x₁,x₂)的訊號。 The photonic circuit 1700 produces the following results for various combinations of the input signals x ₁ and x ₂ :

In the above, the first pair of numbers is the input signal, the second pair of numbers is the output of the detector 1602 after the first pass, and the third pair of numbers is the output of the detector 1602 after the second pass. When the input (x ₁ , x ₂ )=(0,0), after the first pass through the Mach-Zehnder interferometer 1500, the detector 1602 outputs (0,0), and after the second pass through the Mach-Zehnder interferometer 1500 After interferometer 1500, detector 1602 outputs (0,0). When the input (x ₁ ,x ₂ )=(0,1), after passing through the Mach-Zehnder interferometer 1500 for the first time, the detector 1602 outputs (1/

,1/

), and after the second pass through the Mach-Zehnder interferometer 1500, the detector 1602 outputs (0,1). When the input (x ₁ ,x ₂ )=(1,0), after passing through the Mach-Zehnder interferometer 1500 for the first time, the detector 1602 outputs (1/

,1/

), and after the second pass through the Mach-Zehnder interferometer 1500, the detector 1602 outputs (0,1). When the input (x ₁ , x ₂ )=(1,1), after the first pass through the Mach-Zehnder interferometer 1500 , the detector 1602 outputs (0,

), and after the second pass through the Mach-Zehnder interferometer 1500, the detector 1602 outputs (1,1). The above results indicate that after two passes, the detector 1602 generates a signal representing AND(x ₁ ,x ₂ ) at the first output 1704 and a signal representing OR(x ₁ ,x ₂ ) at the second output 1706.

Figure 17B shows another embodiment of a photonic circuit 1710 that includes a first Mach-Zehnder interferometer 1712, a first detector 1714, a second Mach-Zehnder interferometer 1716, and a second detector 1718. The second detector 1718 generates a first output 1720 representing AND(x ₁ ,x ₂ ) and a second output 1722 representing OR(x ₁ ,x ₂ ).

The above describes the use of photonic circuits including Mach-Zehnder interferometers, directional couplers, planar optical waveguides, and photodetectors to implement logic gates (eg, AND, OR, and XOR gates). Logic gates can be used to generate comparators used in sorting algorithms, for example similar to the bitonic sorter described in the link URL <https://en.wikipedia.org/wiki/Bitonic_sorter> algorithm. As another example, logic gates can be used to construct a hashing algorithm similar to SHA-2, described in the link URL <https://en.wikipedia.org/wiki/SHA-2>, which is A standard recommended by NIST and has many applications, including Bitcoin mining and the creation of Bitcoin addresses. Because most of the logic circuits implemented using the above photonic circuits are dynamic, so they can have less latency and lower power consumption than CMOS logic gates. Optical logic gates are designed without optical nonlinearity. The non-linear response comes from the detection signal using a light detector.

Incoherent or low-coherence optical computing systems.

The following describes an optoelectronic computing system that handles incoherent or low-coherence optical signals when performing matrix calculations. The optical processor 140 of the ANN computing system 100 in Figure 1 includes a laser unit 142 that generates N optical outputs having the same wavelength and being optically coherent. The optical matrix multiplication unit 150 performs N×N matrix multiplication in the optical domain, where the optical signal remains coherent from the input of the OMM unit 150 to the output of the OMM unit 150 . The advantages of the OMM unit 150 performing matrix multiplication in the optical domain have been described above. The following describes an optoelectronic computing system that does not require the optical signal to be coherent throughout the matrix multiplication process, in which some parts of the computation are performed in the optical domain, and some parts of the computation are performed in the electrical domain. The advantages of optoelectronic computing systems have been described in the Summary of the Invention above.

Optoelectronic computing systems use different types of operations to produce computational results. Each operation is performed on a signal (e.g., electrical or optical) that is most suitable for the underlying physics of the operation (e.g., in terms of energy consumption and/or speed). of. For example, replication can be performed using optical power splitting, summation can be performed using current-based summation, and multiplication can be performed using optical amplitude modulation. An example of a calculation that can be performed using these three types of operations is multiplying a vector by a matrix (eg, as used in artificial neural network calculations). A variety of other calculations can be performed using these operations, which represent a general set of linear operations that can perform a variety of calculations, including: Vector-vector inner product, vector-vector element-wise multiplication, vector-scalar element-wise multiplication, or matrix-matrix element-wise multiplication, but is not limited thereto.

Referring to Figure 18, an embodiment of an optoelectronic computing system 1800 includes a set of optical ports/light sources 1802A, 1802B, etc. that provide optical signals. For example, in some embodiments, optical port/light source 1802A may include an optical input coupler that provides optical signals coupled to optical path 1803 . In other embodiments, optical port/light source 1802A may include a modulated light source, such as a laser (e.g., for coherent-sensitive embodiments) or a light emitting diode (LED) (e.g., for coherence-insensitive embodiments). embodiment), which generates an optical signal coupled to optical path 1803. Some embodiments may include a combination of coupling an optical signal to a port in the optoelectronic computing system 1800 and a light source that generates the optical signal within the optoelectronic computing system 1800 . An optical signal may include any light wave that has or is modulating information using any of various forms of modulation. Optical path 1803 may be defined based on the guidance pattern of an optical waveguide (eg, a waveguide embedded in a photonic integrated circuit (PIC) or optical fiber), or based on the relationship between optical port/light source 1802A and another module of optoelectronic computing system 1800 defined by a given free space path.

In some embodiments, optoelectronic computing system 1800 is configured to perform calculations on arrays of input values encoded on corresponding optical signals provided by optical ports/light sources 1802A, 1802B, etc. For example, for various machine learning applications based on neural networks, the computation can implement vector-matrix multiplication (or vector-by-matrix multiplication), where an input vector is multiplied by a matrix to produce an output vector as the result. An optical signal may represent elements of a vector, possibly including only a selected subset of the vector's elements. For example, for some neural network models, when calculating The size of the matrices used in the calculation can be larger than the size of the matrix that can be loaded into the hardware system that performs the vector-matrix multiplication portion of the calculation. Therefore, performing part of the computation may involve dividing matrices and vectors into smaller segments that can be individually provided to the hardware system.

乘以 矩陣

以產生輸出向量

。對於該向量矩陣乘法

，輸出向量

的兩個元素中的每一個可以由不同的等式表示，如下所示。 The module shown in Figure 18 may be part of a larger system that performs vector matrix multiplication on relatively large matrices (or sub-matrices), such as a 64x64 element matrix. However, for illustrative purposes, the module will be described in the context of an example calculation that performs vector-matrix multiplication using a 2×2 element matrix. The modules referenced in this example will include two replication modules 1804A and 1804B, four multiplication modules 1806A, 1806B, 1806C, and 1806D, and two summation modules, only one of which is summation module 1808. 18 is shown in Figure 1. These modules will make the input vector

multiply matrix

to produce the output vector

. For this vector matrix multiplication

, the output vector

Each of the two elements of can be represented by a different equation, as shown below.

y _A = M _A x _A + M _B x _B

y _B = M _C x _A + M _D x _B

These equations can be broken down into separate steps that can be performed in the optoelectronic computing system 1800 using a set of basic operations: copy operations, multiplication operations, and summation operations. In these equations, each element of the input vector appears twice, so there are two copy operations. There are also four multiplication operations, and there are two summation operations. For systems that use larger matrices to implement vector-matrix multiplication, the number of operations performed will be larger, and The corresponding number of instances of each operation using matrices whose shape is not square will be different (i.e. the number of columns will be different than the number of rows).

In this embodiment, copying operations are performed by copying modules 1804A and 1804B. The elements of the input vectors x _A and x _B are respectively represented by values encoded on the optical signals from optical ports/light sources 1802A and 1802B. Each of these values is used in two equations, so copy each value to feed the two resulting copies to different corresponding multiplication modules. A value may be encoded in a specific time slot, for example, using a light wave that has been modulated to have a power from a set of multiple power levels, or a light wave that has a duty cycle from a set of multiple duty cycles, As described in more detail below. The value is copied by copying the light signal on which the value is encoded. The optical signal encoded with a value representative of element x _A is replicated by replication module 1804A, and the optical signal encoded with a value representative of element x _B is replicated by replication module 1804B. Each replica module can be implemented using an optical power splitter, such as a waveguide optical splitter, which couples the guided mode in the input waveguide to each of the two output waveguides on a Y-splitter. Gradually split power, or, for example, a free-space beam splitter, which uses a dielectric interface or film with one or more layers to individually transmit and reflect the two output beams from the input beam.

In this disclosure, when we say that a light signal encoded with a value representing element x _A is replicated by replication module 1804A, we mean that multiple copies of the signal representing element x _A are generated based on the input signal, and the replication module 1804A The output signal of group 1804A does not necessarily have the same amplitude as the input signal. For example, if replica module 1804A splits the input signal power evenly between two output signals, then each of the two output signals will have a power equal to or less than 50% of the input signal power. The two output signals are copies of each other, and the amplitude of each output signal of replica module 1804A is different from the amplitude of the input signal. Furthermore, in some embodiments that have a set of multiple replica modules for replicating a given optical signal or subset of optical signals, each individual replica module does not necessarily split power evenly among the replicas it produces, But the set of replica modules can be collectively configured to provide replicas with approximately equal power as the input of the downstream module (eg, downstream multiplication module).

In this embodiment, multiplication operations are performed by four multiplication modules 1806A, 1806B, 1806C, and 1806D. For each copy of an optical signal, a multiplication module multiplies the copy of the optical signal by the matrix element value, which can be performed using optical amplitude modulation. For example, multiplication module 1806A multiplies the input vector element x _A by the matrix element M _A . The value of the vector element x _A can be encoded on the optical signal, and the value of the matrix element M _A can be encoded as the amplitude modulation level of the optical amplitude modulator.

An optical signal encoded with a vector element x _A can be encoded using different forms of amplitude modulation. The amplitude of the optical signal may correspond to a specific instantaneous power level P _A of the physical light wave in a specific time slot, or may correspond to a specific energy E _A (the power integrated over time) of the physical light wave on a specific time slot. over time) generates total energy). For example, the power of the laser source may be modulated to have a specific power level from a predetermined set of multiple power levels. In some embodiments, it may be useful to operate an electronic circuit near an optimized operating point, so instead of varying the power at many possible power levels, an optimized "on" power level and a signal are used, where the signal are tuned to be "on" and "off" (at zero power) for specific parts of the time slot. The portion of time the power is at the "on" level corresponds to a specific energy level. Any of these specific values of power or energy can be mapped to a specific value of element x _A (using a linear or non-linear mapping relationship). After the signal is in the electrical domain, actual integration over time to produce a specific total energy level can occur downstream of the optoelectronic computing system 1800, as described in greater detail below.

調變電磁場幅度的調變器也可以被認為是藉由對應的數值M調變基於功率的訊號幅度(因為光功率與電磁場幅度的平方成比例)。 Additionally, the term "amplitude" may refer to the amplitude of a signal represented by the instantaneous or integrated power in a light wave, or equivalently may refer to the "electromagnetic field amplitude" of the light wave. This is because the electromagnetic field amplitude has a well-defined relationship with the signal amplitude (e.g., instantaneous power is generated by integrating the electromagnetic field intensity (proportional to the square of the electromagnetic field amplitude) over the lateral dimension of a guided mode or free space beam). This results in a relationship between modulation values, because by a specific value

A modulator that modulates the amplitude of an electromagnetic field can also be thought of as modulating the power-based signal amplitude by a corresponding value M (since optical power is proportional to the square of the electromagnetic field amplitude).

Optical amplitude modulators used to encode matrix elements MA by multiplication modules can operate by changing the amplitude of _the optical signal (i.e., the power in the optical signal) using any of a variety of physical interactions. For example, the modulator may include a ring resonator, an electroabsorption modulator, a thermal electro-optical modulator, or a Mach-Zehnder interference (MZI) modulator. In some technologies, a portion of the power is absorbed as part of a physical interaction, and in other technologies, power is transferred using physical interactions that modify another property of the light wave than its power, such as its polarization or phase, Or modify the coupling of optical power between different optical structures (e.g. using tunable resonators). For optical amplitude modulators that operate using interference (eg, destructive and/or constructive interference) between light waves that have traveled on different paths, a coherent light source (eg, a laser) may be used. For light amplitude modulators that operate using absorption, either coherent or incoherent or low-coherence light sources may be used, such as LEDs.

In one embodiment of a waveguide 1x2 optical amplitude modulator, a phase modulator is used to modulate power in an optical wave by placing the phase modulator in one of the plurality of waveguides of the modulator. For example, a waveguide 1x2 optical amplitude modulator may split light waves guided by an input optical waveguide into a first arm and a second arm. The first arm includes a phase shifter which imparts a corresponding phase shift relative to the phase delay of the second arm. The modulator then combines the light waves from the first and second arms. In some embodiments, different values of phase retardation are provided in light waves guided by the input optical waveguide through constructive interference or destructive interference to multiply the power by a value between 0 and 1. In some embodiments, the first arm and the second arm are combined into each of the two output waveguides, and the difference between the photocurrents generated by respective photodetectors receiving light waves from the two output waveguides is provided by The result of multiplying a number (e.g., by a number between -1 and 1), as described in more detail below. By appropriately choosing the amplitude scaling of the encoded optical signal, the range of matrix element values can be mapped to any range of positive values (0 to M) or signed values (-M to M).

In this embodiment, the summation operation is performed by two summation modules, where summation module 1808 (shown in Figure 18) is used to perform the summation in the equation used to calculate the output vector element y _B . The corresponding summation module (not shown) is used to perform the summation in the equation used to calculate the output vector element y _A . The summation module 1808 generates an electrical signal that represents the sum of the results of the two multiplication modules 1806C and 1806D. In this embodiment, the electrical signals are in _the form of current isum , each of which is proportional to the sum of the powers in the output optical signals generated by multiplication modules 1806C and 1806D. The summation operation that produces this current i _sum is performed in the optoelectronic domain in some embodiments, and in the electrical domain in other embodiments. Alternatively, some embodiments may use optoelectronic domain summing for some summing modules and electrical domain summing for other summing modules.

In embodiments where the summation is performed in the electrical domain, the summation module 1808 may be implemented using: (1) Two or more input conductors, each carrying the input current and the magnitude of its output current representing the multiplication module The result of one of the group, and (2) at least one output conductor carrying a current that is the sum of the input currents. This would occur, for example, if the conductors were wires that met at a junction. For example (without being bound by theory), based on Kirchhoff's current law, which states that the current flowing into a contact is equal to the current flowing out of the contact. For these embodiments, signals 1810A and 1810B provided to summing module 1808 are input currents that may be generated by a photodetector that is part of a multiplication module that generates a corresponding photocurrent, Its amplitude is proportional to the power in the received optical signal. The summation module 1808 then provides the output current i _sum . The instantaneous value of the output current or the integrated value of the output current can then be used to represent the quantitative value of the sum.

In embodiments where the summation is performed in the optoelectronic domain, the summation module 1808 can be implemented using a photodetector (eg, a photodiode) that receives optical signals generated by different corresponding multiplication modules. For these embodiments, signals 1810A and 1810B provided to summation module 1808 are input optical signals, each input optical signal including a light wave whose power represents the result of one of the multiplication modules. The output current isum in this embodiment is _the photocurrent generated by the photodetector. Because the wavelengths of the light waves are different (i.e., different enough so that no significant constructive or destructive interference occurs between them), the photocurrent will be proportional to the sum of the powers of the received optical signals. The photocurrent is also approximately equal to the sum of the individual currents that will result in individual detection optical powers detected by separate equivalent photodetectors. The wavelengths of the light waves are different, but close enough that the light detectors have approximately the same response (eg, wavelengths within the approximately flat detection bandwidth of the light detector). As discussed above, summation in the electrical domain using current summation enables simpler system architectures by avoiding the need for multiple wavelengths.

，並且矩陣是

。輸入向量的每個元素在不同的光訊號上編碼。兩個不同的複製模組1902執行光複製操作以在不同的路徑(例如：“上”路徑和“下”路徑)上分離計算。存在四個乘法模組1904，每個乘法模組1904使用光幅度調變乘以不同的矩陣元素。在每個乘法模組1904的輸出，存在光偵測模組1906，其將光訊號轉換為電流形式的電訊號。使用求和模組1908將不同輸入向量元素的兩個上路 徑組合，並且使用求和模組1908將不同輸入向量元素的兩個下路徑組合，求和模組1908在電域中執行求和。因此，輸出向量的每個元素都在不同的電訊號上編碼。如第19A圖所示，隨著計算的進行，遞增地產生輸出向量的每個分量，以個別地產生上路徑和下路徑的以下結果。 Figure 19A shows an embodiment of a system configuration 1900 for an implementation of a system that performs vector matrix multiplication using a 2x2 element matrix, where the summation operation is performed in the electrical domain. In this example, the input vector is

, and the matrix is

. Each element of the input vector is encoded on a different optical signal. Two different replication modules 1902 perform optical replication operations to separate computations on different paths (eg, "upper" path and "lower" path). There are four multiplying modules 1904, each multiplying a different matrix element using optical amplitude modulation. At the output of each multiplication module 1904, there is a light detection module 1906, which converts the optical signal into an electrical signal in the form of a current. The two upper paths of different input vector elements are combined using summation module 1908, and the two lower paths of different input vector elements are combined using summation module 1908, which performs the summation in the electrical domain. Therefore, each element of the output vector is encoded on a different electrical signal. As shown in Figure 19A, as the calculation proceeds, each component of the output vector is generated incrementally to produce the following results for the upper path and the lower path individually.

M ₁₁ v ₁ + M ₁₂ v ₂

M ₂₁ v ₁ + M ₂₂ v ₂

System configuration 1900 may be implemented using any of a variety of optoelectronic technologies. In some embodiments, there is a common substrate (eg, a semiconductor (eg, silicon)) that can support integrated optical and electronic components. The optical path may be implemented in a waveguide structure having a material with a higher optical index surrounded by a material with a lower optical index that defines a waveguide for propagating light waves carrying the optical signal. Electrical paths can be made of conductive materials and are used to propagate the current carrying the electrical signal. (In Figures 19A to 20A and 21A to 24E, unless otherwise stated, the thickness of the lines representing paths is used to distinguish between optical paths (represented by thicker lines) and electrical paths (represented by thinner lines). Indicated by lines or dashed lines).) Optical devices, such as splitters and optical amplitude modulators, and electronic devices, such as photodetectors and operational amplifiers (op-amps), can be fabricated on a common substrate. Alternatively, different devices with different substrates can be used to implement different parts of the system, and those devices can communicate over communication channels. For example, fiber optics can be used to provide communication channels to send optical signals between multiple devices used to implement the overall system. Those optical signals can represent the different input vectors provided when performing vector matrix multiplication. The same subset, and/or a different subset of the intermediate results computed when performing vector matrix multiplication, as described in more detail below.

In the present disclosure, the drawings may show optical waveguides passing through the electrical signal lines, and it should be understood that the optical waveguides do not intersect the electrical signal lines. Electrical signal lines and optical waveguides can be placed on different layers of the device.

Figure 19B shows an embodiment of a system configuration 1920 for an implementation of a system that performs vector matrix multiplication using a 2×2 element matrix, where the summation operation is performed in the optoelectronic domain. In this embodiment, different input vector elements are encoded on the optical signal using two different corresponding wavelengths λ ₁ and λ ₂ . Furthermore, the optical output signals of the multiplication module 1904 are combined in the optical combiner module 1910 so that the optical waveguide guides the two optical signals at two wavelengths to each optoelectronic summing module 1912, which can use optical detection It is implemented with a detector, such as the light detection module 1906 used in the embodiment of Figure 19A. However, in this embodiment, the sum is represented by the photocurrent representing the power of the two wavelengths, rather than by the current leaving the junction between different conductors.

In the present disclosure, when a drawing shows two optical waveguides crossing each other, it will be clear from the description whether the two optical waveguides are actually optically coupled to each other. For example, two waveguides crossing each other shown from a top view of the device may be implemented in different layers and therefore do not cross each other. For example, the optical path providing optical signal λ ₂ as input to replication module 1902 and the optical path providing optical signal M ₁₁ V ₁ from multiplication module 1904 to optical combiner module 1910 are not optically coupled to each other, although They may appear to cross each other in the diagram. Similarly, the optical path providing the optical signal λ ₂ from the replica module 1902 to the multiplication module 1904 and the optical path providing the optical signal M ₂₁ V ₁ from the multiplication module 1904 to the optical combiner module 1910 are not optically coupled to each other, Although in the diagram they may appear to cross each other.

，並且矩陣是

。舉例來說，輸入向量元素v ₁ 至v _n 由n個波導提供，並且每個輸入向量元素由一或多個複製模組處理，以將m個副本的輸入向量元素提供至m個相應路徑。存在m×n個乘法模組，每個乘法模組使用光幅度調變乘以不同的矩陣元素以產生表示M _ij ．v _j (i=1...m,j=1...n)的電訊號或光訊號。使用第i個求和模組(i=1...m)組合表示M _ij ．v _j (j=1...n)的訊號，以個別產生m個路徑的以下結果。 The system configuration shown in Figures 19A and 19B can be extended to implement a system configuration for performing vector matrix multiplication using an m×n element matrix. In this implementation, the input vector is

, and the matrix is

. For example, input vector elements v ₁ to v _n are provided by n waveguides, and each input vector element is processed by one or more replication modules to provide m copies of the input vector element to m corresponding paths. There are m × n multiplication modules, and each multiplication module uses optical amplitude modulation to multiply different matrix elements to generate the representation M _ij . v _j ( i =1... m, j =1... n ) electrical or optical signal. Use the i-th summation module ( i =1... m ) combination to represent M _ij . The signals of v _j ( j =1... n ) can produce the following results of m paths individually.

M ₁₁ v ₁ + M ₁₂ v ₂ +…+ M _{1 n} v _n

M ₂₁ v ₁ + M ₂₂ v ₂ +…+ M _{2 n} v _n

...

M _{m 1} v ₁ + M _{m 2} v ₂ +…+ M _mn v _n

Since optical amplitude modulation can reduce the power in an optical signal from its full value to a lower value, to zero (or close to zero) power, multiplication of any value between 0 and 1 can be achieved. However, some calculations may require multiplication by values greater than 1 and/or multiplication by signed (positive or negative) values. First, in order to extend the range from 0 to M _max (where M _max >1), the original modulation of the optical signal may include an explicit or implicit scaling of the original vector element amplitude M _max (or equivalent Ground, by 1/M _max scaling (the value mapped to the amplitude of a specific vector element in the linear map) so that the range of matrix element amplitudes 0 to 1 quantitatively corresponds to the range 0 to M _max in the calculation. Second, to extend the positive range 0 to M _max of matrix element values to the signed range -M _max to M _max , a symmetric differential configuration can be used, as described in more detail below. Similarly, symmetric differential configurations can also be used to extend the positive range of values encoded on various signals to the signed range of values.

和

，其中假設每個數值在0(例如：對應接近零的光功率)與V_max(例如：對應最大功率準位的光功率)之間變化。兩個光訊號之間的關係是，當一個光訊號用“主要(main)”值

編碼時，另一個光訊號用對應的“反對稱(anti-symmetric)”值

編碼，使得當在一個光訊號上編碼的主要值

從0單調增加(monotonically increase)到V_max時，在配對光訊號上編碼的反對稱值

從V_max單調減少(monotonically decrease)到0。或者，相反地，當在一個光信號上編碼的主要值

從V_max單調減少到0時，在配對光訊號上編碼的反對稱值

從0單調增加到V_max。在上路徑和下路徑中的光訊號被相應光偵測模組1906轉換為電流訊號之後，電流訊號之間的差異可以由電流減法模組(current subtraction module)2002產生。編碼

和

的當前訊號之間的差異導致用有號值V₁編碼的電流，給定為：

其中有號值V₁在-V_max與V_max之間單調增加，因為無號主要值

從0單調增加到V_max並且與其成對的反對稱值

從V_max單調減少到0。存在可用於實現第20圖A的對稱差分配置的各種技術，如第20B圖和第20C圖所示。 Figure 20A shows an embodiment of a symmetric differential configuration 2000 for providing a signed range of values for values encoded on an optical signal. In this embodiment, there are two related optical signals, encoded as unsigned values, designated as

and

, where each value is assumed to vary between 0 (eg, corresponding to optical power close to zero) and V _max (eg, corresponding to optical power at the maximum power level). The relationship between two optical signals is that when an optical signal uses the "main" value

When encoding, another optical signal uses the corresponding "anti-symmetric" value

Encoding such that when encoding the primary value on an optical signal

The antisymmetric value encoded in the paired optical signal as it monotonically increases from 0 to V _max

Monotonically decreases from V _max to 0. Or, conversely, when the primary value encoded on an optical signal

The antisymmetric value encoded in the paired optical signal as V _max decreases monotonically from V max to 0

Monotonically increases from 0 to V _max . After the optical signals in the upper path and the lower path are converted into current signals by the corresponding light detection modules 1906, the difference between the current signals can be generated by the current subtraction module 2002. encoding

and

The difference between the current signals results in a current encoded with _signed value V, given by:

where the signed value V ₁ increases monotonically between -V _max and V _max , because the unsigned main value

Antisymmetric value that increases monotonically from 0 to V _max and is paired with it

Decreases monotonically from V _max to 0. There are various techniques that can be used to implement the symmetric differential configuration of Figure 20A, as shown in Figures 20B and 20C.

In Figure 20B, an optical signal is detected in a common-terminal configuration, where two photodiode detectors are connected to the common terminal 2032 (eg, inverting terminal) of the operational amplifier 2030 ). In this configuration, current 2010 generated from the first photodiode detector 2012 and current 2014 generated from the second photodiode detector 2016 combine at the junction 2018 between the three conductors to produce current 2010 The difference between current 2014 and current 2020. Current 2010 and current 2014 are supplied from opposite sides of corresponding photodiodes, which are connected at the other end to a voltage source (not shown) providing a bias voltage of the same magnitude V _bias but with opposite sign, as shown in Figure 20B Show. In this configuration, the difference occurs due to the behavior of the currents meeting at common junction 2018. The difference current 2020 represents the difference between the signaled value encoded in the electrical signal and the electrical signal corresponding to the unsigned value encoded in the detected optical signal. The op amp 2030 may be configured in a transimpedance amplifier (TIA) configuration in which the other terminal 2024 is connected to ground and the output 2026 is fed back to the common 2032 using a resistive element 2028 that provides a current proportional to the difference current 2020 voltage. This TIA configuration will provide the resulting value as an electrical signal in the form of a voltage signal.

In Figure 20C, the optical signal is detected in a differential terminal configuration, where two photodiode detectors are connected to different terminals of operational amplifier 2050. In this configuration, current 2040 generated from the first photodiode detector 2042 is connected to the inverting terminal 2052, and current 2044 generated from the second photodiode detector 2046 is connected to the non-inverting terminal 2054. Currents 2040 and 2044 are supplied from the same ends of corresponding photodiodes, which are connected at the other end to a voltage source (not shown) providing the same magnitude V _bias and the same sign of the bias voltage, as shown in Figure 20C. The output 2056 of the operational amplifier 2050 in this configuration provides a current proportional to the difference between the current 2040 and the current 2044. In this configuration, differences occur due to the behavior of the circuitry of op amp 2050. The difference current flowing out of output 2056 represents the difference between the signaled value encoded in the electrical signal and the electrical signal corresponding to the unsigned value encoded in the detected optical signal.

和

的無號值來進行調變，其中假設每個數值在0(例如：對應被調變降低至接近零的光功率)與M_max(例如：對應保持在最大功率準位附近的光功率)之間變化。兩個調變準位之間的關係是，當一個調變準位被配置在“主要”值

時，另一個調變準位被配置在對應的“反對稱”值

，使得當一個調變器的主要值

從0單調增加到M_max時，另一個調變器的反對稱值

從M_max單調減少到0。或者，相反地，當一個調變器的主要值

從M_max單調減少到0時，另一個調變器的反對稱值

從0單調增加到M_max。在複製模組1902複製編碼了數值V的輸入光訊號之後，每個調變器將調變的輸出光訊號提供給對應的光偵測模組1906。 在上路徑中的乘法模組1904包括與

相乘的調變器，並且提供以數值

V編碼的光訊號。在下路徑中的乘法模組1904包括與

相乘的調變器，並且提供以數值

V編碼的光訊號。在光訊號被相應光偵測模組1906轉換為電流訊號之後，它們之間的差異可以由電流減法模組2102產生。編碼

V和

V的電流訊號之間的差異導致用V編碼的電流乘以有號值M₁₁，給定為：

其中有號值M₁₁在-M_max與M_max之間單調增加，因為無號主要值

從0單調增加到M_max並且與其成對的反對稱值

從M_max單調減少到0。 Figure 21A shows an embodiment of a symmetric differential configuration 2100 for providing a signed range of values for the values encoded to implement the modulation level of the optical amplitude modulator of the multiplication module 1904. In this embodiment, there are two related modulators, configured to be

and

Modulation is performed with unsigned values, where each value is assumed to be between 0 (for example: corresponding to the optical power that is modulated to be reduced to close to zero) and M _max (for example: corresponding to the optical power maintained near the maximum power level) changes between times. The relationship between two modulation levels is that when a modulation level is configured at the "main" value

, the other modulation level is configured at the corresponding "antisymmetric" value

, such that when the main value of a modulator

The antisymmetric value of another modulator when increasing monotonically from 0 to M _max

Monotonically decreases from M _max to 0. Or, conversely, when the primary value of a modulator

The antisymmetric value of the other modulator as it decreases monotonically from M _max to 0

Increases monotonically from 0 to M _max . After the replicating module 1902 replicates the input optical signal encoded with the value V, each modulator provides the modulated output optical signal to the corresponding light detection module 1906 . The multiplication module 1904 in the upper path consists of

multiplied modulator and provides the numerical

V-encoded optical signal. The multiplication module 1904 in the lower path consists of

multiplied modulator and provides the numerical

V-encoded optical signal. After the light signals are converted into current signals by the corresponding light detection modules 1906, the difference between them can be generated by the current subtraction module 2102. encoding

V and

The difference between the current signals of V results in the current encoded in V multiplied by the signed value M ₁₁ , given by:

Among them, the signed value M ₁₁ increases monotonically between -M _max and M _max , because the unsigned main value

antisymmetric value that increases monotonically from 0 to M _max and is paired with it

Monotonically decreases from M _max to 0.

和

的無號值，並且對於第二有號輸入向量元素值V₂，有兩個指定為

和

的無號值。在光訊號上編碼的每個無號值由複製模組2112接收，複製模組2112執行一或多個光複製操作，其操作在四個相應光路徑上產生四個光訊號副本。在複製模組2112的一些實施例中，存在三個不同的Y形波導分離器，每個Y形波導分離器被配置以使用不同的功率比進行分離(這可以使用各種光子裝置中的任何一種來實現)。舉例來說，第一分離器可以使用 1：4的功率比進行分離，以將25%(1/4)的功率轉移到第一路徑，第二分離器可以使用1：3的功率比進行分離，以將25%(1/4=1/3×3/4)的功率轉移到第二路徑，以及第三分離器可以使用1：2的功率比進行分離，以將25%(1/4=1/2×2/3×3/4)的功率轉移到第三路徑，並且剩餘的25%的功率到第四路徑。作為複製模組2112的一部分的個別分離器可以佈置在基板的不同部分中，以將不同副本適當地分配到系統內的不同路徑。在複製模組2112的其他實施例中，可以適當地以不同的分離率分離不同數量的路徑。舉例來說，第一分離器可以使用1：2的功率比分離以提供兩個實質上有相同功率之中間光訊號(例如：輸入光波之50%功率給兩個輸出端口)。接著，可以使用具有1：2的功率比的第二分離器來分離這些中間光訊號中的一個，以將25%的輸入光波功率轉移到第一路徑和第二路徑中的每一個，並且可以使用具有1：2功率比的第三分離器來分離那些中間光訊號中的另一個，以將25%的輸入光波功率轉移到第三路徑和第四路徑中的每一個。 Figure 21B shows an example embodiment of a system configuration 2110 for an implementation of an optoelectronic computing system 1800 that performs vector matrix multiplication using a 2×2 element matrix, where the summation operations are performed in the electrical domain and have signed elements and signed elements of matrices. In this embodiment, for each signed element of the input vector, there are two associated optical signals encoding unsigned values. For the first signed input vector element value V ₁ , there are two specified as

and

unsigned value of , and for the second signed input vector element value V ₂ , there are two specified as

and

The unsigned value. Each unsigned value encoded on the optical signal is received by replication module 2112, which performs one or more optical replication operations that produce four copies of the optical signal on four corresponding optical paths. In some embodiments of replication module 2112, there are three different Y-shaped waveguide splitters, each Y-shaped waveguide splitter configured to use a different power ratio for splitting (this can use any of a variety of photonic devices to achieve). For example, the first splitter can split using a power ratio of 1:4 to transfer 25% (1/4) of the power to the first path, and the second splitter can split using a power ratio of 1:3 , to transfer 25% (1/4=1/3×3/4) of the power to the second path, and the third splitter can separate using a power ratio of 1:2 to transfer 25% (1/4 =1/2×2/3×3/4) of the power is transferred to the third path, and the remaining 25% of the power is transferred to the fourth path. Individual splitters that are part of the replication module 2112 may be arranged in different portions of the substrate to appropriately distribute different replicas to different paths within the system. In other embodiments of replication module 2112, it may be appropriate to separate different numbers of paths at different separation rates. For example, the first splitter can split using a power ratio of 1:2 to provide two intermediate optical signals with substantially the same power (for example: 50% of the power of the input light wave to the two output ports). Then, one of these intermediate optical signals can be split using a second splitter with a power ratio of 1:2 to transfer 25% of the input optical wave power to each of the first and second paths, and can Another one of those intermediate optical signals is split using a third splitter with a 1:2 power ratio to transfer 25% of the input optical wave power to each of the third and fourth paths.

This type of optical replication distribution network with binary tree topology offers certain advantages. For example, because a binary tree optical replication distribution network can use a symmetrical design on a uniform 1:2 power splitter for all wavelengths (such as a Y-shaped adiabatic waveguide taper), the network will be consistent with Wavelength independence facilitates its use at multiple wavelengths. In addition, non-uniform power splitters may require precise length control to convert different power ratios (for example: 1/n, 1/(n-1),...etc. for n-branch networks) coupling part. However, such accuracy can be difficult with existing process variations. This two Yuanshu's optical replication distribution network also enables the shortening of electrical paths in some compact die layouts, as described below with reference to Figures 45A-45G for more details.

The system configuration 2110 also includes other modules arranged as shown in FIG. 21B to provide two different output electrical signals representing output vectors that are the results of vector matrix multiplication performed by the artificial neural network computing system 100 . There are 16 different multiplication modules 1904 that modulate different copies of the light signal representing the input vector, and there are 16 different light detection modules 1906 that provide electrical signals that represent the intermediate results of the calculations. There are also two different summing modules 2114A and 2114B that calculate the overall sum of each output electrical signal. In the figure, the signal lines electrically coupling the light detection module 1906 to the summing module 2114B are shown as dotted lines. Since each overall summation may include some anti-symmetric terms subtracted from the paired main terms from any symmetric differential configuration for vector elements and/or matrix elements, So summation modules 2114A and 2114B may include mechanisms for adding some terms of the summation after being transformed (equivalently, subtracting from non-inverted terms). For example, in some embodiments, summing modules 2114A and 2114B include an inverting input port and a non-inverting input port such that terms to be added to the overall summation can be connected to the non-inverting input port, and The terms to be subtracted from the overall sum can be connected to the inverting input port. One embodiment of such a summing module is an operational amplifier in which the non-inverting terminal is connected to a wire conducting a current representing the signal to be added, and the inverting terminal is connected to a wire conducting a current representing the signal to be subtracted. of wires. Alternatively, if the inverse of the antisymmetric term is performed by other means rotation, the inverting input port may not be needed on the summing module. Summing modules 2114A and 2114B individually generate the following summation results to complete vector matrix multiplication.

In this disclosure, when a diagram shows two electrical signal lines crossing each other, it will be clearly described whether the two electrical signal lines are electrically coupled to each other. For example, a signal line carrying an M ₂₁ ⁺ V1 ⁺ signal is not electrically coupled to a signal line carrying an M ₁₁ ⁺ V ₁ ^- signal or a signal line carrying an M ₁₁ ^- V ₁ ^- signal.

The system configuration shown in Figure 21B can be extended to implement a system configuration that performs vector matrix multiplication using an m×n element matrix, where the input vectors and matrices include signed elements.

There are various techniques that can be used to implement the symmetric differential configuration of Figure 21B. Some of these techniques utilize 1×2 optical amplitude modulators to implement the multiplication module 1904, and/or provide pairs of optical signals associated with primary and antisymmetric pairings. Figure 22A shows an embodiment of a 1x2 optical amplitude modulator 2200. In this embodiment, the 1×2 optical amplitude modulator 2200 includes an input optical splitter 2202 that splits the input optical signal to provide 50% of the power to a phase modulator including a phase modulator 2204 (also known as a phase shifter). first path, and 50% power is provided to the second path that does not include the phase modulator. The path can be defined in different ways, depending on whether the optical amplitude modulator is implemented as a free-space interferometer or as a waveguide interferometer. For example, in a free-space interferometer, one path is defined by the transmission of a beam through a beam splitter, and another path is defined by the reflection of the wave from the beam splitter. In a waveguide interferometer, each Each path is defined by different optical waveguides that have been coupled to the incoming waveguide (eg in a Y-splitter). Phase modulator 2204 may be configured to impart a phase shift such that the total phase delay of the first path differs from the total phase delay of the second path by a configurable phase shift value (e.g., may be set to between 0 degrees and 180 degrees). the value of the phase shift somewhere between).

1×2 optical amplitude modulator 2200 includes a 2×2 coupler 2206 that uses optical interference or optical coupling in a specific manner to combine light waves from the first and second input paths to transfer power to the first and second input paths at different ratios. one and two output paths, which depend on the phase shift. For example, in a free space interferometer, a phase shift of 0 degrees results in constructive interference of approximately all input power split between the two paths to exit one output path of the beam splitter implementing coupler 2206, And a phase shift of 180 degrees results in constructive interference of substantially all input power split between the two paths to exit the other output path of the beam splitter implementing coupler 2206. In a waveguide interferometer, a phase shift of 0 degrees results in separation between the two paths with approximately all input power coupled to one output waveguide of the coupler 2206, and a phase shift of 180 degrees results in separation between the two paths. Approximately all input power is coupled to the other output waveguide of coupler 2206. The phase shift between 0 degrees and 180 degrees can then multiply the power in the light wave (and the value encoded on the light wave) by a value between 0 and 1 through partial constructive or destructive interference or partial waveguide coupling. Multiplication by any value between 0 and 1 can then be mapped to multiplication by any value between 0 and M _max as described above.

Additionally, the relationship between the power in the two light waves emitted from the modulator 2200 follows the relationship between the power of the primary and antisymmetric pairs described above. As the amplitude of the optical power of one signal increases, the amplitude of the optical power of the other signal decreases, so the difference between the detected photocurrents can be generated by a signed vector element, or multiplied by a signed matrix element, like this stated here. For example, the pair of correlated optical signals may be provided from two output ports of the modulator 2200 such that the difference between the amplitudes of the correlated optical signals corresponds to the result of multiplying the input value by the signed matrix element value. Figure 22B shows a symmetric differential configuration 2210 of a 1x2 optical amplitude modulator 2200 with an optical signal arranged at the output to be detected in the common-terminal version of the symmetric differential configuration of Figure 20B . Current signals corresponding to the photocurrents generated by a pair of photodetectors 2212 and 2214 are combined at junction 2216 to provide an output current signal whose amplitude corresponds to the difference between the amplitudes of the associated optical signals. In other embodiments, such as in the symmetric differential configuration of Figure 20C, different circuit combinations can be used to combine the photocurrent detected from the two output optical signals.

Other techniques may be used to construct a 1×2 optical amplitude modulator for implementing the multiplication module 1904, and/or to provide pairs of optical signals associated with primary and antisymmetric pairings. Figure 22C shows another embodiment of a symmetric differential configuration 2220 of another type of 1x2 optical amplitude modulator. In this embodiment, the 1×2 optical amplitude modulator includes a ring resonator 2222 configured to split the optical power of the optical signal at the input port 2221 to two output ports. The ring resonator 2222 (also called a "microring") can be fabricated by forming a circular waveguide on a substrate, where the circular waveguide is coupled to a straight waveguide corresponding to the input port 2221. When the light signal wave Near the resonant wavelength associated with ring resonator 2222, a light wave coupled into the ring circulates around the ring on a clockwise path 2226 and interferes destructively at the coupling location, causing the reduced power light wave to exit through path 2224 to the first output port. The circulating light wave also couples out of the ring, allowing another light wave to exit on path 2228 through the curved waveguide, which directs the light wave out of the second output port.

Since the time scale of the optical power circulating around the ring resonator 2222 is small compared to the time scale of the amplitude modulation of the optical signal, an antisymmetric power relationship is quickly established between the two output ports, allowing detection by the optical detector 2212 The light waves of and the light waves detected by the light detector 2214 form principal and antisymmetric pairs. The resonant wavelength of ring resonator 2222 can be adjusted to monotonically decrease/increase the primary/antisymmetric signal to achieve signed results, as described above. When the ring is completely unresonant, all power leaves the first output port via path 2224, and when it is fully resonant, all power leaves via path 2228 with appropriate adjustments to certain other parameters (e.g., quality factor and coupling coefficient) Second output port. Specifically, in order to achieve complete power transfer, the coupling coefficients characterizing the coupling efficiency between the waveguide and the ring resonator should match. In some embodiments, it is useful to have a relatively shallow tuning curve, which can be achieved by lowering the quality factor of ring resonator 2222 (e.g., by increasing losses) and correspondingly increasing coupling into and out of the ring. coefficients to achieve. Shallow tuning curves provide smaller amplitude sensitivity to the resonant wavelength. Techniques such as temperature control may also be used for resonant wavelength tuning and/or stabilization.

Figure 22D shows another embodiment of a symmetric differential configuration 2230 of another type of 1x2 optical amplitude modulator. In this example, 1×2 light amplitude modulation The converter includes two ring resonators 2232 and 2234. The optical power of the optical signal at input port 2231 is divided into two ports. When the wavelength of the optical signal approaches the resonance wavelength associated with the two ring resonators 2232 and 2234, the reduced power optical wave exits the first output port through path 2236. A portion of the light wave is also coupled into ring resonator 2232, which circulates around the ring on clockwise path 2238, and into ring resonator 2234, which circulates around the ring on counterclockwise path 2240. The circulating light wave is then coupled out of the loop, allowing another light wave to exit the second output port through path 2242. In this embodiment, the light waves detected by light detector 2212 and the light waves detected by light detector 2214 also form a primary and antisymmetric pairing.

和

，並且光幅度調變器2302B提供一對光訊號，其對第二有號向量元素編碼一對數值

和

。向量矩陣乘法器(VMM)子系統2310A接收輸入光訊號，執行分離操作、乘法操作以及如上面所述的一些求和操作，並且提供將由額外電路處理的輸出電流訊號。在一些實施例中，輸出電流訊號表示被進一步處理以產生最終總和的部分總和，其導致輸出向量的有號向量元素。在此實施例中，一些最終求和操作被執行作為由運算放大器2306A和2306B的反相 和非反相端子的電流訊號表示的不同部分總和之間的減法。減法用以提供有號值，如上面所述(例如：參考第21B圖)。此實施例還說明了某些元素如何成為多個模組的一部分。具體來說，由波導分離器2303執行的光學複製可以被認為是複製模組(例如：第21B圖中的複製模組2112的一個)的一部分和乘法模組(例如：第21B圖中的乘法模組1904的一個)的一部分。在VMM子系統2310A內使用的光幅度調變器被配置用於在第20B圖中所示的公共端子配置(common-terminal configuration)中進行偵測。 Figures 23A and 23B show different embodiments using optical amplitude modulators, such as a 1x2 optical amplitude modulator 2200, for implementing an optoelectronic computing system 1800 that performs vector matrix multiplication in a 2x2 element matrix. Figure 23A shows an embodiment of an optoelectronic system configuration 2300A including optical amplitude modulators 2302A and 2302B that provide values representing signed vector elements of an input vector. Optical amplitude modulator 2302A provides a pair of optical signals that encode a pair of values for the first signed vector element

and

, and optical amplitude modulator 2302B provides a pair of optical signals that encode a pair of values for the second signed vector element

and

. Vector matrix multiplier (VMM) subsystem 2310A receives an input optical signal, performs separation operations, multiplication operations, and some summing operations as described above, and provides an output current signal that will be processed by additional circuitry. In some embodiments, the output current signal representation is a partial sum that is further processed to produce a final sum, which results in signed vector elements of the output vector. In this embodiment, some final summation operations are performed as subtractions between different partial sums represented by the current signals at the inverting and non-inverting terminals of operational amplifiers 2306A and 2306B. Subtraction is used to provide signed values, as described above (e.g. see Figure 21B). This example also illustrates how certain elements can be part of multiple mods. Specifically, the optical replication performed by waveguide splitter 2303 can be considered to be part of a replication module (eg, one of replication modules 2112 in Figure 21B) and a multiplication module (eg, multiplication in Figure 21B Part of Module 1904). The optical amplitude modulator used within VMM subsystem 2310A is configured for detection in the common-terminal configuration shown in Figure 20B.

Figure 23B shows an embodiment of an optoelectronic system configuration 2300B similar to the optoelectronic system configuration 2300A shown in Figure 23A. However, VMM subsystem 2310B includes an optical modulator configured for detection in the differential terminal configuration shown in Figure 20C. In this embodiment, the output current signal of VMM subsystem 2310B also represents the signed vector elements that are further processed to produce a final sum, which results in the output vector. The final summation operation performed as a subtraction between different partial sums represented by the current signals at the inverting and non-inverting terminals of operational amplifiers 2306A and 2306B is different from the embodiment of Figure 23A. However, as mentioned above (eg with reference to Figure 21B), the final subtraction still results in providing a signed value.

Figure 23C shows an embodiment of an optoelectronic system configuration 2300C that uses an alternative arrangement of VVM subsystem 2310C with detection in a common terminal configuration, such as the VVM subsystem 2310A shown in Figure 23A, but with load multiplication The resulting optical signal from the multiplication module is routed through a subsystem within the waveguide to a portion of the substrate that includes a detector configured to convert the optical signal into an electrical signal. In a In some embodiments, this grouping of detectors allows for shortened electrical paths, possibly reducing electrical crosstalk or other damage caused by otherwise long electrical paths that would be used. Optical waveguides may be routed within one layer of the substrate, or to prevent waveguide crossovers (and associated losses) that may be encountered within one layer, waveguides may be routed within multiple layers of the substrate to Allows greater flexibility in routing paths that cross in two dimensions of the substrate but not in the third dimension (depth in the substrate). Various other changes can be made in the system configuration, including changes to the components included in the VMM subsystem. For example, optical amplitude modulators 2302A and 2302B may be included as part of the VMM subsystem. Alternatively, the VMM subsystem may include optical input ports for receiving pairs of primary and antisymmetric optical signals generated by modules other than optical amplitude modulators, or for interfacing with other types of subsystems. In some embodiments, instead of grouping detectors and using multiple layers on the substrate for waveguides, an alternative way to avoid waveguide crossover losses and still limit electrical path lengths involves rearranging the waveguides and circuits on the photonic integrated circuit die. Component layout. For example, in order to provide multiple waveguide layers in a substrate, some production processes may bring additional cost and/or complexity. Conversely, optical routing may include optical replication distribution networks that facilitate shortening of electrical paths in some compact die layouts, as explained below with reference to Figures 45A-45G.

A long wire between a given photodetector and a downstream port has an associated parasitic capacitance that causes increased power dissipation to drive the signal along the wire. To limit power losses in the system, the layout of the components on the die containing photonic integrated circuits (PICs) implemented in the optical processor can be optimized to allow compact electrical routing. For example, some PICs implement distributed photonics processing Processing, for example, the vector matrix multiplier subsystem 2310A or the vector matrix multiplier subsystem 2310B can be configured to produce a relatively narrow "optical ribbon" that includes: an optical waveguide that carries the light input optical signals (e.g., from light modulators that provide elements of the input vector); optoelectronic nodes (e.g., including MZI modulators and detectors); and wires that carry electrical signals that provide electrical output (e.g., feeds that provide output transimpedance amplifier of vector elements). In some embodiments, the transimpedance amplifier (eg, TIA 2306A and 2306B) is part of an electronic integrated circuit (EIC) with a flip-chip connected photonic integrated circuit (PIC). Optical cabling consists of multiple "strands" that contain optical replication distribution networks and optoelectronic "nodes" corresponding to specific multiplication matrix fields, intersecting "tiles" , the tile contains elements corresponding to specific columns of the multiplication matrix. These tiles in the PIC also overlap with corresponding tiles in the EIC, as described in more detail below.

Figure 45A shows an example of a strand 4500 in the above-mentioned optical cable. The unit 4500 includes: a binary tree photoconductive network, light distribution corresponding to the input vector element, using a 1:2 splitter 4502 (or two binary trees each assigning the primary and antisymmetric values of the element); and the optoelectronic node 4504 , perform optoelectronic calculations. In addition, the PIC will include wires (not shown) extending from node 4504, connecting the other strands of wire at the junction. Each optical replication distribution network root may be fed by a root modulator (not shown) that modulates the light wave according to elements of the input vector (eg, an MZI modulator such as the 2302A or 2302B). In some embodiments, the optoelectronic node 4504 at each leaf node of the optical replication distribution network includes an MZI modulator 4505 for multiplication by matrix elements, and a pair of MZI modulator outputs for optoelectronic conversion. Light Detector 4507. circuit The length of wires used for electrical signals depends in part on the width of the entire optical cable. For N×N array elements (for example: N×N matrix multiplication), there are sets of N strands in the cable, each set having its own optical replication distribution network. Because the length of the longest wire may need to traverse as many as N strands, each optical replication distribution network needs to occupy a narrow area. For simplicity and clarity of the diagram, the example diagram is depicted with elements of a 4×4 array, but in some embodiments, the value of N may be significantly increased (eg, 32, 64, 128, or larger).

As explained above, an error-tolerant and wavelength-independent optical replication distribution network can be fabricated using the above binary tree topology, and the optical replication distribution network is assigned to nodes with assigned values. As a motivation for considering the asymmetric setup of binary trees in stock 4500, consider the possible sizes of symmetric binary trees under N×N matrix multiplication. Because the width (N) of a row of N-element trees is greater than its depth (log2(N)), the tree can be set up so that its narrowest dimension exceeds its width. However, the last level of the binary tree, at the leaf nodes, may need to be distributed symmetrically across the width of the tree, so the waveguides in the tree need to be turned 90 degrees to expand enough width. Based on the need to support a minimum radius of waveguide curvature at each level of the tree (to limit bending losses), there may be restrictions on how narrow this depth dimension can be, resulting in a minimum width at each level of the tree (e.g. about 40 microns) . So, in this example, the total width is proportional to log2(N) times 40 microns. Instead, consider using a symmetrical setup of binary trees in stock 4500. The width of the 1:2 Y-shaped separator can be limited to about 1 micron per arm (that is, about 2 microns in total) without changing the orientation, and there is no need to make a 90-degree turn that consumes about 10 microns. The widest part of the strand is at the top node, which is the width of the rectangular waveguide adjacent to node +log2(N). The width of each node is large enough to accommodate the width of a 2-arm MZI modulator (that is, 20 microns or less). The width of the adjacent waveguide is approximately 2.5 microns (the distance between the waveguide itself and its neighbor). Therefore, the total strand width is proportional to 20 microns plus log2(N) times 2.5 microns, which is probably much narrower than in the case of a symmetric binary tree.

Figure 45B shows how the ribbon 4510 can be arranged on the PIC die. The wiring 4510 includes a first line 4512A of the tile 4514 provided on one side of the die; and a second line 4512B of the tile 4514 provided on the other side of the die. Connection portion 4515 is provided by extending one or more waveguides in each strand. The tiles are distributed into two or more straight lines spread over the die area, connected by the waveguides of the distribution network replicated by the light in the strand, making a more compact setup possible. Extending the waveguide in this manner will gradually increase the total optical insertion loss (for example, by approximately 1 dB/cm of additional waveguide length), however this additional loss is usually tolerated. The number of tile wires connected by the extended waveguide (e.g., 2 wires, 3 wires, 4 wires, or more) can be selected to jointly optimize die area and total power loss in the overall system. Likewise, the total number of waveguide extensions can be limited by computational constraints, such that the strand length is significantly less than the clock cycle propagation time, resulting in a limit on the total length of a strand (e.g., less than 10 cm).

Figure 45C shows the arrangement of ribbon cables 4510 (tile boundaries not shown), superimposed on the arrangement of bumps 4516 used to connect pads (e.g., made of conductive material such as metal) on the PIC that provide electrical input and output ports. or alloy composition) is electrically coupled to the corresponding pads on the EIC that provide electrical output and input ports. For example, signals are provided through the output port of the EIC to control MZI modulators (eg, two bumps per MZI in a given optoelectronic node). In some embodiments, there are one or more additional bumps per optoelectronic node (e.g., for temperature control of a given MZI modulator). bumps), and additional pads for various other electrical signal exchanges between the PIC and the EIC. In order to convert electrical signals from the EIC to the PIC for control, and to receive electrical signals from the PIC to the EIC, the pads in the PIC are aligned with the corresponding pads at the bump locations in the EIC. An example bump connecting a PIC output port to an EIC input port is a bump (not shown) that connects a pad in the tile that provides the sum current of the wires for the multiple optoelectronic nodes of the tile to a pad in the EIC. TIA input pad. A typical bump diameter may be approximately 100 microns, although bumps may be smaller (eg, 50 microns). Therefore, in some embodiments, the spacing between bumps (eg, 100 microns) may be greater than the space required for the tiles in the strand, in which case the tiles may be spread out to provide approximately even tile spacing.

Figure 45D shows another example of a cable 4520, depicting an example of a tile 4522 that includes a root modulator 4524 that modulates data values onto one of the optical waves that feeds one of the optical replication distribution networks. . Each strand (including the strand fed by root modulator 4524) also has an array of optoelectronic nodes 4526 (4 nodes in this example). Within node 4526, there is a set of 4528 bumps for transmitting the phase modulation values from the EIC to the PIC MZI modulator arm (eg: modulation weights as matrix multiplication). Tile 4522 also contains wires ending in pads that are connected via bumps 4530 to the pads of the inputs of TIA 4532 in the EIC. In Figure 45D, bumps 4528, 4530 and TIA 4532 are shown superimposed on tile 4522, but are not part of tile 4522. Because the root modulator 4524 of the tile 4522 is placed at a different location on the die than the node of the optical replication distribution network, the modulator 4524 connected to the waveguide portion contains the optical delay portion of the waveguide (or other form of optical delay) , so that there is always The effective optical distance and corresponding time delay match other tile root modulators. Therefore, the waveguide portion 4534 is longer than the waveguide portion 4536 in this example.

Figure 45E shows another optical cable 4540 of a different optoelectronic computing system. This optoelectronic computing system uses EIC instead of PIC to perform more calculations. In this example, there are still four similar configurations of tiles 4542, 4544, 4546 and 4548 in the PIC for 4x4 matrix multiplication. However, the light wave carrying the modulation data value is detected and coupled to the EIC through the bump in the EIC connected to the TIA. The multiplications and additions that are part of the VMM operation are then performed electrically using digital values by digital circuitry in the EIC. For this purpose, possible time differences due to different waveguide lengths can be compensated for in the context of synchronous communications in the digital realm, so no optical delays are required.

Figure 45F shows another example of optical routing 4550 and the type of optical processing that can occur in tile 4552, which performs various types of data processing in any PIC. Typically, photodiodes are used to convert encoded optical signals onto different strands of light waves distributed across the cable into electrical signals. These electrical signals are fed to data processing circuitry 4560 in the PIC. The PIC also contains a data upload circuit 4570 for uploading the results of any operation to a flip chip connected EIC or any other form of integrated electronic circuit.

Figure 45G shows a view of the optoelectronic computing system 4580, depicting examples of the setup of various functions in the system, including weight values (W#,#) for multiplying matrix elements, photodiodes (PD) for optical or electrical addition ), and analog-to-digital converter modules used to convert analog electrical signals into digital electrical signals. The PIC or EIC in the system 4580 may contain functions of different parts.

In some settings, matrix multiplication may have a different number of columns or rows. For example, for an M×N matrix multiplier, there are M electrical tiles (one for each column) in the EIC, and M tiles in the PIC, each of which has N-weighted modulators, which are the same as those in the optical cabling. Corresponds to one of the N shares. As mentioned above, in order to be more consistent in experience, there may be a plurality of straight lines: the first straight line of M/2 tiles and the second straight line of M/2 tiles, or four straight lines of M/4 tiles, etc. etc., instead of arranging M tiles in a straight line. In some cases, four lines may be sufficient because there may be decreasing collections in a particular allocation, whereas in some cases the number of lines may be more but less than M.

In some embodiments, the EIC includes circuits of components such as weight drivers, data drivers, memories (for example, to store matrix weights and accumulation results), digital-to-analog converters, analog-to-digital converters, digital logic (for example, to accumulate ), and part of the digital data bus used to communicate with other tiles. In most cases, the need for communication between different tiles (for example, different columns in a matrix) is limited due to the limited dependence of the calculation data between different tiles. Therefore, this layout can allow (short) columns to be summed (via current) to a given TIA (and the corresponding elements of the output vector) to become relatively independent of others in the layout. Most of the time, there is no relationship between a given output vector and the input vector for the next iteration. However, in some iterations of a computation (such as a neural network calculation), the output vector is related to the input vector used to perform the next iteration. There are dependencies between corresponding elements of the input vector. Very rarely, there may be more dependencies between other elements, such as when all elements are accumulated into the normalized operation As part of the calculation, the normalization operation divides each element by the cumulative sum. Therefore, components that need to communicate with each other more frequently can be placed closer to each other in this layout.

Figure 24A shows an embodiment of a system configuration 2400A for an implementation of an optoelectronic computing system 1800, where there are multiple devices 2410 hosting different multiplication modules (eg, multiplication modules 1806A, 1806B, 1806C, and 1806D) , each multiplication module is configured as a VMM subsystem (hereinafter collectively referred to as VMM subsystem 2410) to perform vector matrix multiplication on different subsets of vector elements by different sub-matrices of a larger matrix. For example, each multiplication module can be configured similar to system configuration 2110 (Figure 21B), but instead of using a 2×2 element matrix to implement the VMM subsystem, each multiplication module can be configured to use a matrix to implement the VMM. The subsystem, matrix, has the same dimensions that can be efficiently fabricated on a single device with a common substrate for the modules within the device. For example, each multiplication module can use a 64×64 element matrix to implement the VMM subsystem.

The different VMM subsystems are arranged so that the results of each submatrix are appropriately combined to produce the results of the larger combined matrix (for example: multiplying the elements of a 128-element vector resulting from a 128×128-element matrix). Each set of optical ports or light sources 2402 provides a set of optical signals that represent a different subset of the vector elements of the larger input vector. The replication module 2404 is configured to replicate all optical signals within the received optical signal group (encoded on the light waves guided in the group 2403 of 64 optical waveguides) and provide the optical signal group to two different optical waveguide groups In each group, the optical waveguide group in this embodiment is the group 2405A of 64 optical waveguides and the group 2405B of 64 optical waveguides. For example, by using an array of waveguide splitters to perform this replication operation, each splitter in the array Replicate one of the subset of input vector elements by dividing the light waves in the group of optical waveguides 2403 into a first corresponding light wave in the group of optical waveguides 2405A and a second corresponding light wave in the group of optical waveguides 2405B element. If multiple wavelengths (eg, W wavelength) are used in some embodiments, the number of separate waveguides (and therefore the number of optical ports or light sources 2402) may be reduced by 1/W. Each VMM subsystem 2410 performs a vector-matrix multiplication, providing its partial result as a set of electrical signals (a subset of the elements for the output vector). The corresponding partial results from the different VMM subsystems 2410 are expressed as shown in Figure 24A The summation module 2414 is summed together, using any of the techniques described here, such as current summation at junctions between conductors. In some embodiments, vector matrices using the desired matrix can be performed recursively by combining the results from smaller submatrices, for any number of recursion levels, ending with a single element optical amplitude modulator at the root level of the recursion. The multiplication is completed by using a single-element optical amplitude modulator at the root level of the recursion. At different levels of recursion, VMM subsystem devices can be more compact (e.g., different data centers connected by long-distance fiber optic networks at one level, different multi-chips connected by fiber optics in another data center) devices, different chips within the device connected by fiber optics at another level, and different parts of modules on the same chip connected by on-chip waveguides at another level).

Figure 24B shows another embodiment of a system configuration 2400B in which additional devices are used for optical transmission and reception of each VMM subsystem 2410. At the output of each VMM subsystem 2410, an optical transmitter array 2420 is used to couple each optical signal to a channel within an optical transmission line (e.g., an optical fiber between VMM subsystems 2410 Optical fiber in a fiber bundle, whose VMM subsystem 2410 may be hosted by a separate device and/or distributed at a remote location, or a waveguide within a set of waveguides on an integrated device, whose integrated device Host the VMM subsystem 2410) on a common substrate. Optical receiver array 2422 is used to output each subset of vector elements to convert the optical signal into an electrical signal before corresponding pairs of partial results are summed by summation module 2414 .

Figure 24C shows another embodiment of a system configuration 2400C in which the VMM subsystem 2410 can be reconfigured such that different vector matrix multiplications for different sub-matrices can be rearranged differently. For example, the shape of a larger matrix formed by combining different sub-matrices may be configurable. In this embodiment, two different subsets of optical signals are provided to optical switch 2430 from each set of optical ports or light sources 2402 . There is also an electrical switch 2440 capable of rearranging the subset of electrical signals representing the partial results to be summed by the summation module 2414 to provide one output vector or separate output vectors for the desired calculation. . For example, instead of using vector matrix multiplication of a matrix of size 2mx2n consisting of four submatrices of size mxn , the VMM subsystem 2410 may be rearranged to use a matrix of size 2mxn or a matrix of size mx2n matrix.

Figure 24D shows another embodiment of a system configuration 2400D in which the VMM subsystem 2410 may be reconfigured in other ways. Optical switch 2430 can receive up to four separate sets of optical signals and can be configured to provide different sets of optical signals to different VMM subsystems 2410 or to duplicate any set of optical signals to multiple VMM subsystems 2410 . Furthermore, electrical switch 2440 may be configured to provide any combination of received sets of electrical signals to summing module 2414. This greater reconfigurability enables a wider variety of vector matrix multiplication calculations, including multiplication using matrices of size m×3n , 3m×n , m×4n , 4m×n .

Figure 24E shows another embodiment of a system configuration 2400E that includes additional circuitry that can perform various operations (e.g., digital logic operations) to enable the system configuration 2400E (e.g., for a complete optoelectronic computing system, or for a larger Optoelectronic systems for large computing platforms) can be used to implement computing techniques such as artificial neural networks or other forms of machine learning. The data storage subsystem 2450 may include volatile storage media (eg, SRAM and/or DRAM) and/or non-volatile storage media (eg, solid state drives and/or hard drives). The data storage subsystem 2450 may also include a hierarchical cache module. The stored data may include training data, intermediate result data, or production data for feeding to an online computational system. Data storage subsystem 2450 may be configured to provide concurrent access to input data for modulation on different optical signals provided by optical port or light source 2402. Conversion of data stored in digital form to an analog form available for modulation may be performed by circuitry (digital-to-analog converter) included at the output of data storage subsystem 2450, or at the input of optical port or light source 2402, Or split between the two. The auxiliary processing subsystem 2460 may be configured to perform auxiliary operations (e.g., nonlinear operations, data shuffling, etc.) on the data, which may be processed multiple times through vector matrix multiplication using the VMM subsystem 2410 Iterative loop. The resulting data 2462 from those auxiliary operations can be The bit form is sent to the data storage subsystem 2450. The data retrieved by the data storage subsystem 2450 can be used to modulate the optical signal using the appropriate input vector and to provide control signals used to set the modulation level of the optical amplitude modulator in the VMM subsystem 2410 (not shown ). Conversion of data encoded in an electrical signal in analog form to digital form may be performed by circuitry within auxiliary processing subsystem 2460 (eg, an analog-to-digital converter).

In some embodiments, a digital controller (not shown in the figures) is provided to control the data storage subsystem 2450, the hierarchical cache module, various circuits (such as digital-to-analog converters and analog-to-digital converters), VMM subsystems Operation of system 2410 and optical port or light source 2402. For example, the digital controller is configured to execute program code to implement a neural network with multiple hidden layers. The digital controller iteratively performs matrix processing associated with each layer of the neural network. The digital controller performs a first iteration of matrix processing by retrieving the first matrix data from the data storage subsystem 2450 and setting the modulation quasi-future of the optical amplitude modulator in the VMM subsystem 2410 based on the retrieved data, wherein the first The matrix data represents the coefficients of the first layer of the neural network. The digital controller retrieves a set of input data from the data storage subsystem and sets the modulation levels for the optical port or light source 2402 to generate a set of optical input signals representing elements of the first input vector.

The VMM subsystem 2410 performs matrix processing based on the first input vector and the first matrix data, representing the processing of the signal by the first layer of the neural network. After the auxiliary processing subsystem 2450 generates the first set of result data 2462, the digital controller retrieves the second matrix information representing the coefficients of the second layer of the neural network from the data storage subsystem. The second iteration of matrix processing is performed by setting the modulation quasi-future of the optical amplitude modulator in the VMM subsystem 2410 based on the second matrix data. The first set of result data 2462 is used as a second input vector to set the modulation level of the optical port or light source 2402. The VMM subsystem 2410 performs matrix processing based on the second input vector and the second matrix data, representing the processing of the signal by the second layer of the neural network, and so on. In the last iteration, the output of the signal processed by the last layer of the neural network is produced.

In some embodiments, when performing computations associated with the hidden layers of the neural network, the resulting data 2462 is not sent to the data storage subsystem 2450 but is used by the digital controller to directly control the digital-to-analog converter, whose digital The analog converter generates a control signal for setting the modulation level of the optical amplitude modulator in the VMM subsystem 2410. This reduces the time required to store data to and access data from the data storage subsystem 2450.

Other processing techniques may be incorporated into other embodiments of system configurations. For example, various techniques used with other kinds of subsystems like vector matrix multiplication (e.g., subsystems that do not have electrical summation or signed multiplication as described here but instead use optical interference) may be incorporated into some system configurations , such as some of the techniques described in U.S. Patent Publication No. US 2017/0351293, which is fully incorporated herein by reference.

Referring to Figure 32A, the artificial neural network (ANN) computing system 3200 includes a photoelectric matrix multiplication unit 3220. The photoelectric matrix multiplication unit 3220 includes a copy module, a multiplication module and a summation module as shown in Figures 18 to 24D. set to be able to handle incoherent or low-coherence optical signals when performing matrix calculations. The artificial neural network computing system 3200 includes a controller 110, a memory unit 120, a DAC unit 130 and an ADC. Units 160 are similar to those of the artificial neural network computing system 100 of Figure 1A. Controller 110 receives requests from computer 102 and sends computational output to computer 102, similar to that shown in Figure 1A.

The optoelectronic processor 3210 includes a light source 3230, which may be similar to the laser unit 142 of FIG. 1A, where multiple output signals of the light source 3230 are coherent. The light source 3230 may also use light emitting diodes to generate multiple output signals that are incoherent or have low coherence. The optoelectronic matrix multiplication unit 3220 includes a modulator array 144 that receives the modulator control signal generated by the first DAC subunit 132 based on the input vector and the operations performed by the optical processor 140 of FIG. 1A resemblance. The output of modulator array 144 may be compared to the output of optical port or light source 1802 in Figure 18. The optical matrix multiplication unit 3220 processes the optical signal from the modulator array 144 in the same manner as the replica module 1804, the multiplication module 1806, and the summation module 1808 in FIG. 18 process the optical signal from the optical port or light source 1802. In a similar way.

，並且將輸入向量乘以矩陣

，以產生 輸出向量

。 Referring to Figure 32B, the photoelectric matrix multiplication unit 3220 receives the input vector

, and multiply the input vector by the matrix

, to produce the output vector

.

The optoelectronic matrix multiplication unit 3220 includes m optical paths 1803_1, 1803_2, ..., 1803_m (collectively 1803), which carry optical signals representing input vectors. The copy module 1804_1 provides a copy of the input optical signal v ₁ to the multiplication modules 1806_11, 1806_21, ..., 1806_ m 1 . The copy module 1804_2 provides a copy of the input optical signal v ₂ to the multiplication modules 1806_12, 1806_22, ..., 1806_m2 . The copy module 1804_n provides a copy of _the input optical signal vn to the multiplication modules 1806_1n , 1806_2n ,..., 1806_mn .

As described above, the amplitudes of the replicas of optical signal v ₁ provided by replica module 1804_1 are the same (or substantially the same) relative to each other, but are different from the optical signal v ₁ provided by modulator array 144 . For example, if replication module 1804_1 evenly divides the signal power of optical signal v ₁ provided by modulator array 144 among m signals, then each of the m signals will have a modulator power equal to or less than The array 144 provides a power of 1/ m of the power of the optical signal v ₁ .

The multiplication module 1806_11 multiplies the input signal v ₁ and the matrix element M ₁₁ to generate M ₁₁ . v1 _. The multiplication module 1806_21 multiplies the input signal v ₁ and the matrix element M ₂₁ to generate M ₂₁ . v1 _. The multiplication module 1806_ m 1 multiplies the input signal v ₁ and the matrix element M _{m 1} to generate M _{m 1} . v1 _. The multiplication module 1806_12 multiplies the input signal v ₂ and the matrix element M ₁₂ to generate M ₁₂ . v2 _. The multiplication module 1806_22 multiplies the input signal v ₂ and the matrix element M ₂₂ to generate M ₂₂ . v2 _. The multiplication module 1806_ m 2 multiplies the input signal v ₂ and the matrix element M _{m 2} to generate M _{m 2} . v2 _. The multiplication module 1806_1 n multiplies the input signal v _n and the matrix element M _{1 n} to generate M _{1 n} . vn _. The multiplication module 1806_2 n multiplies the input signal v _n and the matrix element M _{2 n} to generate M _{2 n} . vn _. The multiplication module 1806_mn multiplies the input signal v _n and the matrix element M _mn to generate M _mn . v _n , and so on.

The second DAC subunit 134 generates a control signal based on the value of the matrix element and sends the control signal to the multiplication module 1806 so that the multiplication module 1806 can multiply the value of the input vector element by the matrix element by using optical amplitude modulation. value. For example, the multiplication module 1806_11 may include an optical amplitude modulator, and by encoding the numerical value of the matrix element M ₁₁ as the amplitude modulation level applied to the input optical signal representing the input vector element v ₁ , the The input vector element v ₁ is multiplied by the matrix element M ₁₁ .

The summation module 1808_1 receives the outputs of the multiplication modules 1806_11, 1806_12, ..., 1806_1 n and produces a sum y ₁ equal to M ₁₁ v ₁ + M ₁₂ v ₂ + ... + M _{1 n} v _n . The summation _{module 1808_2} receives the outputs of _the multiplication modules 1806_21 , _{1806_22} , ... , 1806_2n and produces _{a sum y2 equal to M21v1 +} M22v2 + _... + M2nvn . The summation module 1808_n receives the outputs _{of the} multiplication modules 1806_m1, _{1806_m2} , ... _, 1806_mn and produces _a sum equal to Mm1v1 + Mm2v2 + _... + Mmnvn y _n . In the artificial neural network computing system 3200, the output of the photoelectric matrix multiplication unit 3220 is provided to the ADC unit 160 without passing through the detection unit 146 as in the artificial neural network computing system 100 of FIG. 1A. This is because the multiplication module 1806 or the summation module 1808 has already converted the optical signal into an electrical signal, so there is no need for a separate detection unit 146 in the artificial neural network computing system 3200 .

Figure 33 shows a flow diagram of a method 3300 of performing ANN calculations using the ANN calculation system 3200 of Figure 32A. The steps of method 3300 may be performed by controller 110 of ANN computing system 3200. In some embodiments, the various steps of method 3300 may be run in parallel, combined, in a loop, or in any order.

In step 3310, an artificial neural network (ANN) calculation request including an input data set and a first plurality of neural network weights is received. The input data set includes the One-digit input vector. The first digit input vector is a subset of the input data set. For example, it could be a subregion of the image. ANN calculation requests may be generated by various entities (eg, computer 102 of Figure 32A). Computers may include one or more of various types of computing devices, such as personal computers, server computers, vehicle computers, and flight computers. The ANN calculation request generally refers to an electrical signal that notifies or informs the ANN calculation system 100 that the ANN calculation is to be performed. In some embodiments, an ANN calculation request may be split into two or more signals. For example, the first signal may query the ANN computing system 3300 to check whether the ANN computing system 3300 is ready to receive the input data set and the first plurality of neural network weights. In response to a positive response from the ANN computing system 3300, the computer 102 may send a second signal including the input data set and the first plurality of neural network weights.

In step 3320, the input data set and the first plurality of neural network weights are stored. The controller 110 may store the input data set and the first plurality of neural network weights in the memory unit 120 . Storing the input data set and the first plurality of neural network weights in the memory unit 120 may allow flexibility in the operation of the ANN computing system 3300, which may, for example, improve the overall performance of the system. For example, by retrieving a desired portion of the input data set from memory unit 120, the input data set may be divided into numeric input vectors of a set size and format. Different parts of the input data set can be processed in various orders, or shuffled, to allow various types of ANN calculations to be performed. For example, shuffling can allow matrix multiplication to be performed via block matrix multiplication techniques when the input and output matrices are of different sizes. As another embodiment, the input data set and the first plurality of neural network weights may be stored in the memory unit 120 To allow multiple ANN computing requests to be queued by the ANN computing system 3300, this may allow the ANN computing system 3300 to maintain operation at its full speed without periods of inactivity.

In some embodiments, the input data set may be stored in a first memory subunit and the first plurality of neural network weights may be stored in a second memory subunit.

In step 3330, a first plurality of modulator control signals are generated based on the first digital input vector, and a first plurality of weight control signals are generated based on the first plurality of neural network weights. The controller 110 may send the first DAC control signal to the DAC unit 130 to generate a first plurality of modulator control signals. DAC unit 130 generates a first plurality of modulator control signals based on the first DAC control signal, and modulator array 144 generates an optical input vector representing a first digital input vector.

The first DAC control signal may include a plurality of digital values to be converted by the DAC unit 130 into a first plurality of modulator control signals. Multiple digit values typically correspond to the first digit input vector and can be related through various mathematical relationships or lookup tables. For example, the plurality of digit values may be linearly proportional to the values of the elements of the first digit input vector. As another example, a plurality of digital values may be associated with elements of the first digital input vector via a lookup table configured to maintain linearity between the digital input vector and the light input vector generated by the modulator array 144 relation.

The controller 110 may send the second DAC control signal to the DAC unit 130 to generate the first plurality of weight control signals. The DAC unit 130 generates a first plurality of weight control signals based on the second DAC control signal, and The heavy control signal reconfigures the photoelectric matrix multiplication unit 3220 to implement matrices corresponding to the first plurality of neural network weights.

The second DAC control signal may include a plurality of digital values to be converted by the DAC unit 130 into the first plurality of weighted control signals. Multiple numerical values typically correspond to a first plurality of neural network weights, and can be related through various mathematical relationships or lookup tables. For example, the plurality of numerical values may be linearly proportional to the first plurality of neural network weights. As another embodiment, a plurality of digital values may be calculated by performing various mathematical operations on the first plurality of neural network weights to generate a weight control signal. The weight control signal may configure the photoelectric matrix multiplication unit 3220 to perform corresponding to the first Matrix multiplication of multiple neural network weights.

In step 3340, a first plurality of digital outputs corresponding to the electrical output vector of the photoelectric matrix multiplication unit 3220 are obtained. The optical input vector generated by the modulator array 144 is processed by the optoelectronic matrix multiplication unit 3220 and converted into an electrical output vector. The electrical output vector is converted into a digital value by ADC unit 160. The controller 110 may send a conversion request to the ADC unit 160 to start converting the voltage output by the photoelectric matrix multiplication unit 3220 into a digital output. Once the conversion is completed, the ADC unit 160 may send the conversion result to the controller 110 . Alternatively, the controller 110 may retrieve the conversion results from the ADC unit 160 . Controller 110 may form a digital output vector from the digital output, the digital output vector corresponding to the result of matrix multiplication of the input digital vector. For example, digital output can be organized or concatenated to have a vector format.

In some embodiments, the ADC unit 160 may be set or controlled to perform based on the DAC control signal being sent by the controller 110 to the DAC unit 130 ADC conversion. For example, the ADC conversion may be set to start at a preset time after the DAC unit 130 generates the modulation control signal. Such control of ADC conversion can simplify the operation of the controller 110 and reduce the number of necessary control operations.

In step 3350, a non-linear transformation is performed on the first digital output vector to produce a first transformed digital output vector. An ANN's nodes, or artificial neurons, operate by first performing a weighted summation of signals received from nodes in previous layers, and then performing a nonlinear transformation ("activation") of the weighted summation to produce an output. Various types of ANN can implement various types of differentiable nonlinear transformations. Examples of nonlinear transformation functions include a rectified linear unit (RELU) function, a sigmoid function, a hyperbolic tangent function, an X^2 function, and an |X| function. This non-linear conversion is performed on the first digital output by controller 110 to produce a first converted digital output vector. In some embodiments, the nonlinear conversion may be performed by dedicated digital integrated circuits within controller 110 . For example, controller 110 may include one or more modules or circuit blocks that are particularly suitable for accelerating the calculation of one or more types of nonlinear transformations.

In step 3360, the first converted digital output vector is stored. The controller 110 may store the first converted digital output vector in the memory unit 120 . In the case where the input data set is divided into a plurality of digit input vectors, the first converted digit output vector corresponds to the ANN calculation result of a portion of the input data set, such as the first digit input vector. As such, storing the first transformed digital output vector allows the ANN calculation system 3200 to perform and store additional calculations on other digital input vectors of the input data set to be later aggregated into a single ANN output.

In step 3370, the artificial neural network output generated based on the first converted digital output vector is output. The controller 110 generates an ANN output that is a result of processing the input data set through an ANN defined by the first plurality of neural network weights. Where the input data set is divided into multiple digital input vectors, the resulting ANN output is an aggregate output that includes the first transformed digital output, but may further include additional transformed digital outputs corresponding to other portions of the input data set. Once the ANN output is generated, the generated output is sent to the computer that initiated the ANN calculation request (eg, computer 102).

Various performance metrics may be defined for the ANN computing system 3200 implementing the method 3300. Defining performance metrics may allow the performance of the ANN computing system 3200 implementing the optoelectronic processor 3210 to be compared with the performance of other systems used to alternatively implement ANN computing of the electronic matrix multiplication unit. In one view, the rate at which ANN computations can be performed can be dictated in part by the first loop period, which is defined as the step 3320 of storing the input data set and the first plurality of neural network weights in a memory unit versus The time elapsed between steps 3360 of storing the first converted digital output vector in the memory unit. Therefore, the first cycle includes the time spent converting the electrical signal into an optical signal (eg, step 3330) and performing matrix multiplication in the optical and electrical domains (eg, step 3340). Both steps 3320 and 3360 involve storing data into the memory unit 120, which is a step shared between the ANN computing system 3200 and conventional ANN computing systems without the optoelectronic processor 3210. In this way, measuring the memory-to-memory transaction time of the first cycle may allow the ANN computing system 3200 to be used with or without the optoelectronic processor 3210. Actual or fair comparison of ANN calculation flows between ANN calculation systems (such as systems implementing electrical matrix multiplication units).

Due to the rate at which the modulator array 144 can generate optical input vectors (e.g., at 25 GHz) and the processing rate of the optoelectronic matrix multiplication unit 3220 (e.g., >25 GHz), an ANN computing system for performing a single ANN calculation of a single digit input vector The first cycle period of 3200 may be close to the reciprocal of the speed of modulator array 144 (eg, 40 ps). The first cycle period may be less than or equal to 100 ps, less than or equal to 200 ps, less than or equal to 500 ps, less than or equal to 1 ns, less than or equal to 2ns, less than or equal to 5ns, or less than or equal to 10ns.

For comparison, the multiplication execution time of M×1 vectors and M×M matrices of the electrical matrix multiplication unit is usually proportional to M^2-1 processor clock cycles. For M=32, this multiplication will take approximately 1024 cycles, which results in an execution time of over 300ns at a 3GHz clock speed, which is several orders of magnitude slower than the first cycle period of the ANN computing system 3200.

In some embodiments, method 3300 further includes the step of generating a second plurality of modulator control signals based on the first converted digital output vector. In some types of ANN calculations, a single digit input vector can be repeatedly propagated through the same ANN or processed by the same ANN. As mentioned above, an ANN that implements multi-pass processing can be called a recurrent neural network (RNN). RNN is a neural network in which the output of the network during the (k)th pass through the neural network is recycled back to the input of the neural network and during the (k+1)th pass The passing period is used as input. RNNs can have various applications in pattern recognition tasks, such as speech or handwriting recognition. Once the second plurality of modulator control signals are generated, method 3300 may proceed from step 3340 to step 3360 to complete the second pass of the first digital input vector through the ANN. Generally speaking, the recycling of the converted digital output as a digital input vector can be repeated for a predetermined number of cycles depending on the characteristics of the RNN received in the ANN calculation request.

In some embodiments, the method 3300 further includes the step of generating a second plurality of weight control signals based on the second plurality of neural network weights. In some cases, the artificial neural network calculation request further includes a second plurality of neural network weights. As mentioned above, generally speaking, in addition to the input layer and output layer, ANN also has one or more hidden layers. For an ANN with two hidden layers, the second plurality of neural network weights may correspond to the connectivity between the first layer of the ANN and the second layer of the ANN. In order to process the first digital input vector through the two hidden layers of the ANN, the first digital input vector may first be processed according to the method 3300 until step 3360, where the result of processing the first digital input vector through the first hidden layer of the ANN is stored in step 3360. in memory unit 120. The controller 110 then reconfigures the optoelectronic matrix multiplication unit 3220 to perform matrix multiplication corresponding to the second plurality of neural network weights associated with the second hidden layer of the ANN. Once the opto-matrix multiplication unit 3220 is reconfigured, the method 3300 can generate a plurality of modulator control signals based on the first converted digital output vector, which generates an updated optical input vector corresponding to the output of the first hidden layer. Then the updated optical input vector is processed by the reconfigured photoelectric matrix multiplication unit 3220, which corresponds to the second phase of the ANN. Hidden layer. In general, the steps described can be repeated until the numeric input vector has been processed through all hidden layers of the ANN.

In some embodiments of opto-matrix multiplication unit 3220, the reconfiguration rate of opto-matrix multiplication unit 3220 may be significantly slower than the modulation rate of modulator array 144. In this case, the throughput of the ANN computing system 3200 may be adversely affected by the amount of time spent reconfiguring the optoelectronic matrix multiplication unit 3220 during the period that it is unable to perform ANN calculations. To mitigate the impact of the relatively slow reconfiguration time of the opto-matrix multiplication unit 3220, batch processing techniques may be utilized, in which two or more digital input vectors are propagated through the opto-matrix multiplication unit 3220 without configuration changes to amortize (amortize) reconfiguration time on a larger number of digit input vectors.

Figure 34 shows a schematic diagram 3290 illustrating the method 3300 of Figure 33. For an ANN with two hidden layers, instead of processing the first digital input vector through the first hidden layer, the photoelectric matrix multiplication unit 3220 is reconfigured for the second hidden layer, and the first digital input is processed through the reconfigured photoelectric matrix multiplication unit 3220 vectors, and repeating the same operation for the remaining digital input vectors, all digital input vectors of the input data set can be processed first through the photoelectric matrix multiplication unit 3220 configured for the first hidden layer (configuration #1), as shown in the upper part of schematic diagram 3290 shown. Once all digital input vectors have been processed by the opto-matrix multiplication unit 3220 with configuration #1, the opto-matrix multiplication unit 3220 is reconfigured into configuration #2, which corresponds to the second hidden layer of the ANN. This reconfiguration may be significantly slower than the rate at which the opto-matrix multiplication unit 3220 can process the input vectors. Once the optoelectronic matrix multiplication unit 3220 is reconfigured Set for the second hidden layer, the output vectors from the previous hidden layer can be batch processed by the opto-matrix multiplication unit 3220. For large input data sets with tens or hundreds of thousands of digit input vectors, the impact of reconfiguration time can be reduced by approximately the same factor, which can significantly reduce the portion of time the ANN computing system 3200 spends in reconfiguration.

In order to achieve batch processing, in some embodiments, the method 3300 further includes the step of generating a second plurality of modulator control signals based on the second digital input vector through the DAC unit; obtaining from the ADC unit the output vector corresponding to the photoelectric matrix multiplication unit. The steps of a second plurality of digital outputs, the second plurality of digital outputs forming a second digital output vector; the steps of performing a nonlinear transformation on the second digital output vector to generate a second transformed digital output vector; and storing in a memory unit The second step is to convert the digital output vector. For example, generating the second plurality of modulator control signals may occur after step 3360. Furthermore, the ANN output of step 3370 in this case is now based on the first converted digit output vector and the second converted digit output vector. The acquisition, execution and storage steps are similar to steps 3340 to 3360.

Batch processing technology is one of several technologies used to increase throughput of the ANN computing system 3200. Another technique used to increase the throughput of the ANN computing system 3200 is by processing multiple digital input vectors in parallel using wavelength division multiplexing (WDM). As mentioned above, WDM is a technology that simultaneously propagates multiple optical signals of different wavelengths through a common propagation channel (such as the waveguide of the photoelectric matrix multiplication unit 3220). Unlike electrical signals, optical signals of different wavelengths can propagate through a common channel without affecting other optical signals of different wavelengths on the same channel. Additionally, one can use light such as Known structures of multiplexers and demultiplexers add (multiplexed) or drop (demultiplexed) optical signals from a common propagation channel.

In the context of the ANN computing system 3200, multiple optical input vectors of different wavelengths may be independently generated, propagated simultaneously through the optical path and optical processing components (e.g., optical amplitude modulators) of the optoelectronic matrix multiplication unit 3220, and electronically Processing components (eg, detectors and/or summation modules) are processed independently to enhance the throughput of the ANN computing system 3200.

Referring to Figure 35A, in some embodiments, a wavelength division multiplexing (WDM) artificial neural network (ANN) computing system 3500 includes an optoelectronic processor 3510, which includes an optoelectronic matrix multiplication unit 3520, and an optoelectronic matrix multiplication unit 3520. Unit 3520 has a replication module, a multiplication module, and a summation module as shown in Figures 18 to 24D to be able to process incoherent or low-coherence optical signals at multiple wavelengths when performing matrix calculations. Encoding. WDM ANN computing system 3500 is similar to ANN computing system 3200, except that WDM technology is used. For some embodiments of ANN computing system 3500, light source 3230 is configured to generate multiple wavelengths, such as λ1, λ2, and λ3, as shown in Figure 1F The WDM ANN computing system 104 is similar.

Multiple wavelengths may preferably be separated by a sufficiently large wavelength spacing to allow easy multiplexing and demultiplexing onto a common propagation channel. For example, wavelength spacing greater than 0.5 nm, 1.0 nm, 2.0 nm, 3.0 nm, or 5.0 nm may allow for simple multiplexing and demultiplexing. On the other hand, the range between the shortest wavelength and the longest wavelength of the plurality of wavelengths (the "WDM bandwidth") may preferably be small enough such that the characteristics or performance of the optoelectronic matrix multiplication unit 3520 remains substantially the same across the plurality of wavelengths. Optical components are often dispersive, meaning their optical properties vary with wavelength. For example, the power separation ratio of an MZI can vary with wavelength. However, by designing the photoelectric matrix multiplication unit 3520 to have a sufficiently large operating wavelength window, and by limiting the wavelength within the operating wavelength window, the photoelectric matrix multiplication unit 3520 outputs corresponding to each wavelength. The electrical output vector may be a sufficiently accurate result of matrix multiplication implemented by the optoelectronic matrix multiplication unit 3520. The operating wavelength window can be 1nm, 2nm, 3nm, 4nm, 5nm, 10nm or 20nm.

Modulator array 144 of WDM ANN computing system 3500 includes banks of optical modulators configured to generate a plurality of optical input vectors, each group corresponding to one of a plurality of wavelengths and generating a corresponding The corresponding light input vector of the wavelength. For example, for a system with optical input vectors of length 32 and 3 wavelengths (eg, λ1, λ2, and λ3), modulator array 144 may have 3 groups of 32 modulators each. In addition, the modulator array 144 further includes an optical multiplexer configured to combine multiple optical input vectors into a combined optical input vector including multiple wavelengths. For example, an optical multiplexer can combine the outputs of three modulator banks at three different wavelengths into a single propagation channel (eg, waveguide) for each element of the optical input vector. Thus, returning to the above embodiment, the combined optical input vector will have 32 optical signals, each signal including 3 wavelengths.

The optoelectronic processing component of the WDM ANN computing system 3500 is further configured to demultiplex multiple wavelengths and generate multiple demultiplexed output electrical signals. Referring to FIG. 35B , the optoelectronic matrix multiplication unit 3520 includes an optical path 1803 configured to receive a combined optical input vector including a plurality of wavelengths from the modulator array 144 . For example, optical path 1803_1 receives combined optical input vector element v ₁ at wavelengths λ1, λ2, and λ3. Copies of the light input vector elements v ₁ at wavelengths λ1, λ2 and λ3 are provided to the multiplication modules 3530_11, 3530_21, ..., and 3530_m 1 . In some embodiments in which the multiplication module 3530 outputs an electrical signal, the multiplication module 3530_11 outputs a representation of M ₁₁ . The three electrical signals v ₁ correspond to the input vector elements v ₁ at wavelengths λ1, λ2 and λ3. The output electrical signals of the multiplication module 3530_11 corresponding to the input vector element v ₁ at wavelengths λ1, λ2 and λ3 are shown as (λ1), (λ2) and (λ3) respectively. Similar notation applies to the outputs of other multiplication modules. The output of the multiplication module 3530_21 represents M ₂₁ . The three electrical signals v ₁ respectively correspond to the input vector elements v ₁ at wavelengths λ1, λ2 and λ3. The output of the multiplication module 3530_ m 1 represents M _{m 1} . The three electrical signals v ₁ correspond to the input vector elements v ₁ at wavelengths λ1, λ2 and λ3.

Copies of the optical input vector element v ₂ at wavelengths λ1, λ2 and λ3 are provided to the multiplication modules 3530_12, 3530_22, ..., and 3530_m2 . The output of the multiplication module 3530_12 represents M ₁₂ . The three electrical signals v ₂ correspond to the input vector elements v ₂ at wavelengths λ1, λ2 and λ3. The output of multiplication module 3530_22 represents M ₂₂ . The three electrical signals v ₂ correspond to the input vector elements v ₂ at wavelengths λ1, λ2 and λ3. The multiplication module 3530_m 2 outputs three electrical signals representing M _{m 2} _ v ₂ , which correspond to the input vector elements v ₂ at wavelengths λ1, λ2 and λ3.

Copies of the optical input vector elements v _n including wavelengths λ1, λ2 and λ3 are provided to the multiplication modules 3530_1 n , 3530_2 n , . . . , and 3530_ mn . The output of multiplication module 3530_1 n represents M _{1 n} . The three electrical signals v _n correspond to the input vector elements v _n at wavelengths λ1, λ2, and λ3. The output of multiplication module 3530_2 n represents M _{2 n} . The three electrical signals v _n correspond to the input vector elements v _n at wavelengths λ1, λ2, and λ3. The output of the multiplication module 3530_ mn represents M _mn . The three electrical signals v _n correspond to the input vector elements v _n at wavelengths λ1, λ2, and λ3, and so on.

For example, each multiplication module 3530 may include a demultiplexer configured to demultiplex three wavelengths contained in each of the 32 signals of the multi-wavelength light vector and demultiplex the 3 The single wavelength light output vector is routed to three sets of photodetectors (e.g., operational amplifiers 2030 (Fig. 20B) or 2050 (Fig. 20C)) coupled to three sets of operational amplifiers or transimpedance amplifiers (e.g., op amps 2030 (Fig. 20B) or 2050 (Fig. 20C)). Light detectors 2012, 2016 (Fig. 20B) or 2042, 2046 (Fig. 20C)).

The three summation modules 1808 receive the output from the multiplication module 3530 and generate the sum y corresponding to the input vectors at various wavelengths. For example, three summation modules 1808_1 receive the outputs of the multiplication modules 3530_11, 3530_12, ..., 3530_1 n and generate sums y ₁ corresponding to input vector elements v ₁ at wavelengths λ1 , λ2 and λ3 , respectively. (λ1), y ₁ (λ2), y ₁ (λ2), where the sum y ₁ at each wavelength is equal to M ₁₁ v ₁ + M ₁₂ v ₂ +…+ M _{1 n} v _n . The three summation modules 1808_2 receive the outputs of the multiplication modules 3530_21, 3530_22, ..., 3530_2n and generate sums y2 _{( λ1} ) , y ₂ (λ2), y ₂ (λ2), where the sum y ₂ at each wavelength is equal to M ₂₁ v ₁ + M ₂₂ v ₂ +…+ M _{2 n} v _n . The three summation modules 1808_n receive the outputs of the multiplication modules 3530_m1 , 3530_m2 , ..., 3530_mn and generate the sum y of the input vector elements vn respectively corresponding to the _wavelengths λ1, λ2 and λ3 _n (λ1), y _n (λ2), y _n (λ2), where the sum y _n at each wavelength is equal to M _{m 1} v ₁ + M _{m 2} v ₂ +…+ M _mn v _n .

Referring again to FIG. 35A , the ADC unit 160 of the WDM ANN computing system 3500 includes banks of ADCs configured to convert multiple demultiplexed output voltages of the optoelectronic matrix multiplication unit 3520 . Each group corresponds to one of a plurality of wavelengths and produces a corresponding digitized demultiplexed output. For example, a bank of ADCs 160 may be coupled to a bank of summing modules 1808 .

Controller 110 may implement a similar method to method 3300 (Fig. 33), but extended to support multi-wavelength operation. For example, the method may include the step of obtaining a plurality of digital demultiplexed outputs from the ADC unit 160, the plurality of digital demultiplexed outputs forming a plurality of first digital output vectors, wherein each of the plurality of first digital output vectors corresponding to one of the plurality of wavelengths; performing a nonlinear transformation on each of the plurality of first digital output vectors to generate a plurality of steps of converting the first digital output vector; and storing the plurality of steps in the memory unit. Steps to convert the first digit of the output vector.

In some cases, the ANN can be specifically designed and the digital input vectors can be specifically formed such that the multi-wavelength product of the multiplication module 3530 can be added without demultiplexing. In this case, the multiplication module 3530 may be a wavelength-insensitive multiplication module that does not demultiplex the multiple wavelengths of the multi-wavelength product. In this way, each photodetector of the multiplication module 3530 effectively adds multiple wavelengths of the optical signal into a single photocurrent, and each of the outputs of the multiplication module 3530 The voltage corresponds to the sum of the products of vector elements and matrix elements for multiple wavelengths. The summation module 1808 (requiring only one group) outputs an element-by-element sum of the matrix multiplication results of multiple numeric input vectors.

，並且矩陣是

。在此實施例中，輸入向量具有多個波長λ1、λ2以及λ3，並且輸入向量的每個元素在不同的光訊號上編碼。兩個不同的複製模組1902執行光複製操作以在不同的路徑(例如：“上”路徑和“下”路徑)上分離計算。存在四個乘法模組1904，每個乘法模組1904使用光幅度調變乘以不同的矩陣元素。每個乘法模組1904的輸出被提供給多路分解器和一組光偵測模組3310，光偵測模組3310將波長分波多路複用光訊號轉換成與波長λ1、λ2以及λ3相關的電流形式的電訊號。使用與波長λ1、λ2以及λ3相關的一組求和模組3320來組合不同輸入向量元素的兩個上路徑，並且使用與波長λ1、λ2以及λ3相關的一組求和模組3320來組合不同輸入向量元素的兩個下路徑，其中求和模組3320在電域中執行求和。因此，對每個波長的輸出向量的每個元素都在不同的電訊號上編碼。如第35A圖所示，隨著計算的進行，遞增地產生輸出向量的每個分量，以對每個波長個別地產生上路徑和下路徑的以下結果。 Figure 35C shows an embodiment of a system configuration 3500 for the implementation of a demultiplexed optoelectronic matrix multiplication unit 3520 using a 2x2 element matrix to perform vector matrix multiplication, where the summation operation is performed in the electrical domain. In this example, the input vector is

, and the matrix is

. In this embodiment, the input vector has multiple wavelengths λ1, λ2, and λ3, and each element of the input vector is encoded on a different optical signal. Two different replication modules 1902 perform optical replication operations to separate computations on different paths (eg, "upper" path and "lower" path). There are four multiplying modules 1904, each multiplying a different matrix element using optical amplitude modulation. The output of each multiplication module 1904 is provided to a demultiplexer and a set of optical detection modules 3310, which convert the wavelength demultiplexed optical signal into wavelengths λ1, λ2, and λ3 An electrical signal in the form of an electric current. A set of summation modules 3320 associated with wavelengths λ1, λ2, and λ3 are used to combine the two upper paths of different input vector elements, and a set of summation modules 3320 associated with wavelengths λ1, λ2, and λ3 are used to combine different Two lower paths of input vector elements, where summation module 3320 performs the summation in the electrical domain. Therefore, each element of the output vector for each wavelength is encoded in a different electrical signal. As shown in Figure 35A, as the calculation proceeds, each component of the output vector is generated incrementally to produce the following results for the upper path and lower path individually for each wavelength.

M ₁₁ v ₁ + M ₁₂ v ₂

M ₂₁ v ₁ + M ₂₂ v ₂

System configuration 3500 may be implemented using any of a variety of optoelectronic technologies. In some embodiments, there is a common substrate (eg, a semiconductor (eg, silicon)) that can support integrated optical and electronic components. The optical path may be implemented in a waveguide structure having a material with a higher optical index surrounded by a material with a lower optical index that defines a waveguide for propagating light waves carrying the optical signal. Electrical paths can be made of conductive materials and are used to propagate the current carrying the electrical signal. (In Figure 35C, the thickness of the lines representing paths is used to differentiate between optical paths (represented by thicker lines) and electrical paths (represented by thinner or dashed lines).) Optical devices can be fabricated on a common substrate ( such as splitters and optical amplitude modulators), and electronic devices such as photodetectors and operational amplifiers (op-amps). Alternatively, different devices with different substrates can be used to implement different parts of the system, and those devices can communicate over communication channels. For example, fiber optics can be used to provide communication channels to send optical signals between multiple devices used to implement the overall system. Those optical signals may represent different subsets of the input vectors provided when performing vector matrix multiplication, and/or different subsets of the intermediate results computed when performing vector matrix multiplication, as described in more detail below.

The non-linear transformation of the weight sum performed so far as part of the ANN calculation is performed by the controller 110 in the digital domain. In some cases, nonlinear transformations may be computationally intensive or power-consuming, significantly increase the complexity of the controller 110, or limit the ANN computing system 3200 in terms of throughput or power efficiency (Fig. 32A ) performance. In this way, in In some embodiments of the ANN computing system, nonlinear transformations can be performed in the analog domain through analog electronics.

Figure 36 shows a schematic diagram of the ANN computing system 3600. ANN computing system 3600 is similar to ANN computing system 3200 except that an analog nonlinear unit 310 is added. The analog nonlinear unit 310 is provided between the photoelectric matrix multiplication unit 3220 and the ADC unit 160 . The analog nonlinear unit 310 is configured to receive an output voltage from the optoelectronic matrix multiplication unit 3220 , apply a nonlinear transfer function, and output the converted output voltage to the ADC unit 160 .

When the ADC unit 160 receives the voltage that has been nonlinearly converted by the analog nonlinear unit 310, the controller 110 may obtain a converted digital output voltage corresponding to the converted output voltage from the ADC unit 160. Because the digital output voltage obtained from the ADC unit 160 has already been nonlinearly converted ("activated"), the nonlinear conversion step of the controller 110 can be omitted, thereby reducing the computational burden of the controller 110 . Then, the first conversion voltage directly obtained from the ADC unit 160 may be stored in the memory unit 120 as the first conversion digital output vector.

The analog nonlinear unit 310 may be implemented in various ways, as discussed above with respect to the analog nonlinear unit 310 of Figure 3A. The use of the analog nonlinear unit 310 can improve the performance of the ANN computing system 3600, such as throughput or power efficiency, by reducing the number of steps performed in the digital domain. Moving the nonlinear transformation step out of the digital domain can allow additional flexibility and improvements in the operation of ANN computing systems. For example, in a recurrent neural network, the output of the optoelectronic matrix multiplication unit 3220 is activated and recycled back to the input of the optoelectronic matrix multiplication unit 3220. Activation steps are calculated by ANN The controller 110 in the system 3200 performs this by digitizing the output voltage of the opto-electronic matrix multiplication unit 3220 on each pass through the opto-electronic matrix multiplication unit 3220 . However, since the activation step is now performed before digitization of the ADC unit 160, the number of ADC conversions required in performing the recursive neural network calculations can be reduced.

In some embodiments, analog nonlinear unit 310 may be integrated into ADC unit 160 as a nonlinear ADC unit. For example, the nonlinear ADC unit may be a linear ADC unit with a nonlinear lookup table that maps the linear digital output of the linear ADC unit to a desired nonlinear converted digital output.

Figure 37 shows a schematic diagram of the ANN computing system 3700. The ANN computing system 3700 is similar to the ANN computing system 3600 of FIG. 36 except that it further includes an analog memory unit 320 . The analog memory unit 320 is coupled to the DAC unit 130 (eg, through the first DAC subunit 132 ), the modulator array 144 and the analog nonlinear unit 310 . The analog memory unit 320 includes a multiplexer having a first input coupled to the first DAC subunit 132 and a second input coupled to the analog nonlinear unit 310 . This allows the analog memory unit 320 to receive signals from the first DAC sub-unit 132 or the analog non-linear unit 310 . The analog memory unit 320 is configured to store analog voltages and output the stored analog voltages. Analog memory unit 320 may be implemented in various ways, as discussed above with respect to analog memory unit 320 of Figure 3B.

The operation of ANN computing system 3700 will now be described. The first plurality of modulator control signals output by the DAC unit 130 (eg, by the first DAC sub-unit 132 ) are first input to the modulator array 144 through the analog memory unit 320 . at this step In this step, the analog memory unit 320 may simply pass or buffer the first plurality of modulator control signals. The modulator array 144 generates an optical input vector based on the first plurality of modulator control signals, which is propagated through the optoelectronic matrix multiplication unit 3220 . The output voltage of the photoelectric matrix multiplication unit 3220 is nonlinearly converted by the analog nonlinear unit 310 . At this time, instead of being digitized by the ADC unit 160 , the output voltage of the photoelectric matrix multiplication unit 3220 is stored by the analog memory unit 320 , which is then output to the modulator array 144 to be converted into a voltage to be propagated through the photoelectric matrix multiplication unit 3220 The next light input vector. Under the control of the controller 110, the recursive processing may be performed for a preset amount of time or a preset number of cycles. Once the recursive processing is completed for a given digital input vector, the converted output voltage of the analog nonlinear unit 310 is converted by the ADC unit 160 .

The advantages of using analog memory unit 320 in ANN computing system 3700 are similar to the advantages of using analog memory unit 320 in ANN computing system 302 of Figure 3B. Similarly, execution of recursive neural network computations using ANN computing system 3700 may be similar to that of ANN computing system 302 of Figure 3B.

As discussed above with respect to the ANN computing system 400 of Figure 4A, there are advantages to using an ANN computing system that internally operates at a lower bit resolution than the resolution of the input data set, while maintaining the resolution of the ANN computing output. Referring to Figure 38, a schematic diagram of an artificial neural network (ANN) computing system 3800 with 1-bit internal resolution is shown. ANN computing system 3800 is similar to ANN computing system 3200 (FIG. 32A), except that DAC unit 130 is now replaced by driver unit 430, and ADC unit 160 is now replaced by comparator unit 460.

The driver unit 430 and comparator unit 460 in the ANN computing system 3800 of Figure 38 operate in a similar manner to the driver unit 430 and comparator 460 in the ANN computing system 400 of Figure 4A. The mathematical representation of the operation of the ANN computing system 3800 in Figure 38 is similar to the mathematical representation of the operation of the ANN computing system 400 shown in Figure 4A.

The ANN calculation system 3800 performs certain ANN calculations by performing a series of matrix multiplications of 1-bit vectors and then summing the individual matrix multiplication results. Using the embodiment shown in Figure 4A, by generating a sequence of four 1-bit modulator control signals corresponding to the four 1-bit input vectors through the driver unit 430, the decomposed input vectors V _bit0 to V _bit3 can be Each of is multiplied by the matrix U. This in turn produces a sequence of four 1-bit optical input vectors, which are processed by the optoelectronic matrix multiplication unit 3220 configured through the driver unit 430 to achieve matrix multiplication of the matrix U. Then, the controller 110 can obtain a sequence of four digital 1-bit light outputs corresponding to a sequence of four 1-bit modulator control signals from the comparator unit 460 .

With the 4-bit vector decomposed into four 1-bit vectors, each vector should be processed by the ANN computing system 3800 at a speed that other ANN computing systems, such as the ANN computing system 3200 (Fig. 32A), can process Four times the speed of a single 4-bit vector to maintain the same effective ANN computational throughput. This increased internal processing speed can be thought of as time-division multiplexing four 1-bit vectors into a single timeslot for processing 4-bit vectors. The required increase in processing speed may be achieved, at least in part, by the increase in driver unit 430 and comparator unit 460 relative to DAC unit 130 and ADC unit 160. This is achieved by increasing the operating speed, since a reduction in the resolution of the signal conversion process usually results in an increase in the achievable signal conversion rate.

In this embodiment, although the signal conversion rate in 1-bit operation is increased by four times, the power consumption results can be significantly reduced compared to 4-bit operation. As mentioned above, the power consumption of signal conversion processing generally scales exponentially with bit resolution and linearly with conversion rate. Thus, a 16x reduction in power per conversion may be due to a 4x reduction in bit resolution, followed by a 4x increase in conversion rate. In summary, a 4x reduction in operating power can be achieved with the ANN computing system 3800 over the ANN computing system 3200 while maintaining the same effective ANN computing throughput.

The controller 110 can then construct a 4-bit digital output vector from the four digital 1-bit light outputs by multiplying each digital 1-bit light output by the corresponding weight 2^0 to 2^3. Once the 4-bit digital output vector is constructed, ANN calculations can be performed by performing a non-linear transformation on the constructed 4-bit digital output vector to produce a transformed 4-bit digital output vector; and in memory unit 120 Store the converted 4-bit digital output vector.

Alternatively (or additionally), in some embodiments, each of the 4 digital 1-bit light outputs may be converted non-linearly. For example, a step-function nonlinear function can be used for nonlinear transformation. The converted 4-bit digital output vector can then be constructed from the nonlinearly converted digital 1-bit light output.

Although individual ANN computing systems 3800 have been shown and described, generally speaking, the ANN computing system 3200 of Figure 32A can be designed to perform functions similar to the ANN computing system 3800. For example, the DAC unit 130 may include a 1-bit DAC subunit configured to generate a 1-bit modulator control signal, and the ADC unit 160 may be designed to have a 1-bit resolution. This 1-bit ADC can be similar to or effectively equivalent to a comparator.

Furthermore, although the operation of an ANN computing system has been described with an internal resolution of 1 bit, in general the internal resolution of an ANN computing system can be reduced to an intermediate level below the N-bit resolution of the input data set. For example, the internal resolution can be reduced to 2^Y bits, where Y is an integer greater than or equal to 0.

Various alternative system configurations or signal processing techniques may be used with the various embodiments of the different systems, subsystems, and modules described herein.

In some embodiments, it may be useful to replace some or all VMM subsystems with alternative subsystems, including subsystems that use different embodiments of various copy modules, multiplication modules, and/or sum modules. For example, a VMM subsystem may include an optical replication module as described herein and an electrical summation module as described herein, but the multiplication module may be replaced with a subsystem that performs multiplication operations in the electrical domain rather than the optical domain. In this embodiment, the optical amplitude modulator array can be replaced by a detector array to convert optical signals into electrical signals, followed by an electronic subsystem (eg, ASIC, processor, or SoC). Alternatively, if optical signal routing is to be used for a summation module configured to detect optical signals, the electronic subsystem may include the use of electrical Electro-optical conversion of an array of electrically-modulated optical sources.

In some embodiments, it may be useful to be able to use a single wavelength for some or all optical signals for some or all VMM calculations. Alternatively, in some embodiments, to help reduce the number of optical input ports that may be needed, the input ports may receive multiplexed optical signals with different values encoded on different light waves at different wavelengths. Those light waves can then be separated at appropriate locations in the system, depending on whether any of the replication module, multiplication module, and/or summation module are configured to operate at multiple wavelengths. However, even in multi-wavelength embodiments, it may be useful to use the same wavelength for different subsets of optical signals used in the same VMM subsystem.

In some embodiments, accumulators can be used to implement time domain encoding of optical and electrical signals received by various modules, thereby alleviating the need for circuitry to operate efficiently at a large number of different power levels. For example, a signal using binary (on-off) amplitude modulation coding with a specific duty cycle on N slots of each symbol can be passed through an accumulator (Integrated Telecommunications The signal's current or voltage is then converted into a signal with N amplitude levels per symbol. Therefore, if optical devices (e.g., phase modulators in optical amplitude modulators) can operate at a symbol bandwidth B, they can instead operate at a symbol bandwidth B/100, using N per symbol value =100 time slots. An integration amplitude of 50% has a duty cycle of 50% (for example: the first 50 slots are at a non-zero "on" level, followed by 50 slots at a zero or near-zero "off" level), while 10% points margin There is a 10% duty cycle (for example: the first 10 slots are at a non-zero "on" level, followed by 90 slots at a zero "off" level). In the embodiments described herein, such accumulators may be positioned consistent with each electrical signal at any location in the path of each electrical signal within the VMM subsystem, such as in the VMM subsystem for before the summation module of all electrical signals or after the summation module of all electrical signals in the VMM subsystem. The VMM subsystem can also be configured so that there is no significant relative time shift between different electrical signals maintaining the alignment of different symbols.

Referring to Figure 40, in some embodiments, homodyne detection can be used to obtain the phase and amplitude of the modulation signal. Homodyne detector 4000 includes a beam splitter 4002 including a 2×2 multi-mode interference (MMI) coupler, two photodetectors 4004a and 4004b, and a subtractor 4006. Beam splitter 4002 receives input signals E ₁ and E ₂ , and the output of beam splitter 402 is detected by photodetectors 4004a and 4004b. For example, the input signal E ₁ may be the signal to be detected, and the input signal E ₂ may be generated by a local oscillator with constant laser power. The local oscillator signal E _{2 is mixed with the input signal E 1 by} the beam splitter 4002 before the signal is detected by the photodetectors 4004a and 4004b. Subtractor 4006 outputs the difference between the outputs of photodetectors 4004a and 4004b. The output 4008 of the subtractor 4006 is proportional to | E ₁ | | E ₂ | sin(θ), where | E ₁ | and E ₂ | are the amplitudes of the two input optical fields and θ is their relative phase. Because the output is related to the product of two light fields, extremely weak light signals can be detected even at a single photon level.

For example, the homodyne detector 4000 can be used in Figures 1A, 1F, 3A to 4A, 5, 7, 9, 18 to 24E Figure 26 to Figure 32B and Figure 35A to Figure 38 in the system shown. The homodyne detector 4000 provides gain on the signal and therefore provides a better signal to noise ratio. For coherent systems, the null detection detector 4000 provides the added benefit of revealing the phase information of the signal through the polarity of the detection result.

In the embodiment of Figure 19B, system configuration 1920 includes a 2x2 element matrix in which two input vector elements are encoded on two optical signals using two different corresponding wavelengths λ ₁ and λ ₂ . Two optical fibers can be used to provide two optical signals to the system configuration 1920. For example, a system that performs matrix processing on a 4×4 matrix can receive four input optical signals carried on four optical fibers. Although more optical fibers can be used to carry more input optical signals for systems that handle larger matrices, it is difficult to couple large numbers of optical fibers to an optoelectronic chip because the coupling between the optical fiber and the optoelectronic chip takes up considerable space. .

One way to reduce the number of optical fibers required to carry optical signals to an optoelectronic die is to use wavelength division multiplexing. Multiple optical signals with different wavelengths can be multiplexed and transmitted using a single optical fiber. For example, referring to Figure 41, in computing system 4100, first optical signal 4102 having wavelength λ ₁ is modulated by first modulator 4104 to produce a first modulation representing first input vector element V ₁ Light Signal 4120. The second optical signal 4106 having wavelength λ ₂ is modulated by the second modulator 4108 to produce a second modulated optical signal 4122 representing the second input vector element V ₂ . The first and second modulated optical signals are combined by a multiplexer 4110 to generate a wavelength division multiplexed signal, which is transmitted to the optoelectronic chip 4114 through the optical fiber 4112. The optoelectronic chip 4114 includes a plurality of matrix multiplication modules 4116a, 4116b, 4116c, and 4116d (collectively, 4116) and 4118a, 4118b, 4118c, and 4118d (collectively, 4118).

Inside the optoelectronic chip 4114, the wavelength demultiplexing signal is demultiplexed by a demultiplexer 4118 to separate the first modulated optical signal 4120 and the second modulated optical signal 4122. In this embodiment, the first modulated optical signal 4120 is replicated by replication module 4124 to generate a replica of the optical signal sent to matrix multiplication modules 4116a and 4118a. The second modulated optical signal 4122 is replicated by replication module 4126 to produce a replica of the optical signal sent to matrix multiplication modules 4116b and 4118b. The outputs of matrix multiplication units 4116a and 4116b are combined using optical coupler 4120a, and the combined signal is detected by photodetector 4122a.

The third optical signal 4124 having wavelength λ ₁ is modulated by the third modulator 4128 to produce a third modulated optical signal 4132 representing the third input vector element V ₃ . The fourth optical signal 4126 having wavelength λ ₂ is modulated by the fourth modulator 4130 to generate a fourth modulated optical signal 4134 representing the fourth input vector element V ₄ . The third and fourth modulated optical signals are combined by a multiplexer 4136 to generate a wavelength division multiplexed signal, which is transmitted to the optoelectronic chip 4114 through the optical fiber 4138.

Inside the optoelectronic chip 4114, the wavelength division multiplexing signal provided by the optical fiber 4138 is demultiplexed by a demultiplexer 4140 to separate the optical signals 4132 and 4134. In this embodiment, the third modulated optical signal 4132 is replicated by replication module 4142 to generate a replica of the optical signal sent to matrix multiplication modules 4116c and 4118c. The fourth modulated optical signal 4134 is replicated by replication module 4144 to produce a replica of the optical signal sent to matrix multiplication modules 4116d and 4118d. Matrix multiplication unit 4116c and The outputs of 4116d are combined using optical coupler 4120b, and the combined signal is detected by photodetector 4122b. The outputs of matrix multiplication units 4118a and 4118b are combined using an optical coupler, and the combined signal is detected by a photodetector. The outputs of matrix multiplication units 4118c and 4118d are combined using an optical coupler, and the combined signal is detected by a photodetector.

In some embodiments, a multiplexer may multiplex optical signals with three or more (eg, 10 or 100) wavelengths to produce a wavelength-demultiplexed signal transmitted over a single optical fiber, and The demultiplexer inside the photoelectric transceiver can demultiplex the wavelength division multiplexing signal to separate signals with different wavelengths. This allows more optical signals to be transmitted in parallel through the optical fiber to the optoelectronic chip, increasing the data processing throughput of the optoelectronic chip.

In some embodiments, laser unit 142 of Figure 1A includes a single laser that provides light waves that can be modulated with different optical signals. In that case, the light waves in the various waveguides of the system have approximately the same common wavelength as each other within the resolution of the linewidth of the laser. For example, the light waves may have wavelengths within 1 nm of each other. However, the laser unit 142 may also include multiple lasers capable of wavelength demultiplexing using different optical signals modulated onto different corresponding optical waves (eg, each having a linewidth of 1 nm or less). Use operation. Different light waves may have peak wavelengths that are separated from each other by a wavelength distance greater than the linewidth of the individual lasers (eg, greater than 1 nm). In some embodiments, a wavelength division multiplexing system may use optical signals that are modulated onto light waves with wavelengths of several nanometers (eg, 3 nm or greater). However, if the demultiplexer has better resolution, the difference between different wavelengths in a WDM system can also be less than 3nm.

The digital controller (for example, used to control the components shown in Figure 24E) and functional operations described in the present disclosure can be implemented in digital electronic circuits, or in computer software, firmware or hardware, including the present disclosure The structures in and their structural equivalents, or one or more combinations thereof. The embodiments and functional operations described in the present disclosure may be implemented using one or more computer program instruction modules encoded on a computer-readable medium for execution by a data processing device or to control the operation of the data processing device. The computer-readable medium may be a manufactured product (such as a hard drive in a computer system or an optical disc sold through retail channels) or an embedded system. The computer-readable medium may be individually retrieved and subsequently encoded using one or more modules of computer program instructions, such as one or more modules of computer program instructions transmitted over a wired or wireless network. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of them.

A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including compiled or literal languages, declarative or procedural languages, and can be configured in In any form, including as a stand alone program or as a module, component, subroutine or other unit suitable for use in a computing environment. Computer programs do not necessarily correspond to files in a file system. Programs may be stored as part of a file that holds other programs or data (for example, one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or stored in multiple coordinated files (e.g., files that store one or more modules, subroutines, or code portions). A computer program may be configured to execute on one computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communications network.

The processes and logic flows described in this disclosure may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and producing output. Processing and logic flows may also be performed by special purpose logic circuitry, and devices may also be implemented as special purpose logic circuitry, such as field programmable gate arrays (FPGAs) or application special integrated circuits (ASICs). .

While the present disclosure has been described in connection with certain embodiments, it is to be understood that the disclosure is not limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims. The scope shall be given the broadest interpretation to include all such modifications and equivalent constructions permitted by law.

For example, Figure 42 shows the probability distribution function for a data set in which small coefficients occur more frequently. In another embodiment, it is assumed that the data set has properties such that the probability distribution function (PDF) of the coefficients yields higher probabilities (and therefore more frequent instances) for larger coefficients (ie coefficients with relatively larger absolute values). For such data sets ("high coefficient weight data sets"), reduced power consumption can be achieved by designing the modulator such that the modulator operates in a lower power state to use larger coefficients (in the data set occur more frequently), while operating in a higher power state allows calculations using smaller coefficients (occur less frequently in the data set).

Some background information on the various systems described in this disclosure is available at U.S. Provisional Application 62/680,944 filed on June 5, 2018, U.S. Provisional Application 62/744,70 filed on October 12, 2018, and U.S. Application 16/431,167 filed on June 4, 2019. The entire disclosures of the above applications are incorporated herein by reference.

Although the present disclosure is defined in the appended claims, it should be understood that the present disclosure may also be defined in accordance with the following embodiments:

Embodiment 1: A computing system, including: a memory unit configured to store a data set and complex neural network weights; a digital-to-analog converter (DAC) unit configured to generate a complex modulator control signal and generate a complex weighted control signal; the optical processor includes: a laser unit configured to generate a complex optical output; a complex optical modulator coupled to the laser unit and the DAC unit, the optical modulator is configured Based on the modulator control signal, the light output generated by the laser unit is modulated to generate the light input vector; the light matrix multiplication unit is coupled to the light modulator and the DAC unit, and the light matrix multiplication unit is configured based on The weight control signal converts the light input vector into the light output vector; and the light detection unit is coupled to the light matrix multiplication unit and is configured to generate a complex output voltage corresponding to the light output vector; an analog-to-digital converter (analog-to- digital converter (ADC) unit, coupled to the light detection unit, and configured to convert the output voltage into a complex digital light output; The controller, including the integrated circuit, is configured to perform the following operations: receive an artificial neural network calculation request from the computer including an input data set and a first plurality of neural network weights, wherein the input data set includes a first digital input vector; storing the input data set and the first plurality of neural network weights in the memory unit; and generating, through the DAC unit, a first plurality of modulator control signals based on the first digital input vector and based on the first plurality of neural networks The weight generates a first plurality of weight control signals.

Embodiment 2: The computing system of Embodiment 1, wherein the operation further includes: obtaining a first plurality of digital light outputs corresponding to the light output vector of the light matrix multiplication unit from the ADC unit, and the first plurality of digital light outputs form the first digit. output vector; perform a non-linear transformation on the first digital output vector to generate a first transformed digital output vector; and store the first transformed digital output vector in the memory unit

Embodiment 3: The computing system of Embodiment 2, wherein the computing system has a first cycle, which is defined as the step of storing the input data set and the first plurality of neural network weights in the memory unit and the steps of storing the input data set and the first plurality of neural network weights in the memory unit. The time elapsed between steps of storing the first converted digital output vector in , and wherein the first cycle period is less than or equal to 1 ns.

Embodiment 4: The computing system of Embodiment 2, wherein the operation further includes: outputting an artificial neural network output generated based on the first converted digital output vector.

Embodiment 5: The computing system of Embodiment 2, wherein the operation further includes: generating a second plurality of modulator control signals based on the first converted digital output vector through the DAC unit.

Embodiment 6: The computing system of Embodiment 2, wherein the artificial neural network calculation request further includes a second plurality of neural network weights, and the operation further includes: based on obtaining the first plurality of digital light outputs, through a digital analog The converter unit generates a second plurality of weight control signals based on the second plurality of neural network weights.

Embodiment 7: The computing system of Embodiment 6, wherein the first plurality of neural network weights and the second plurality of neural network weights correspond to different layers of the artificial neural network.

Embodiment 8: The computing system of Embodiment 2, wherein the input data set further includes a second digital input vector, and the operation further includes: generating a second plurality of modulator control signals based on the second digital input vector through the DAC unit ; Obtain a second plurality of digital light outputs corresponding to the light output vector of the light matrix multiplication unit from the ADC unit, and the second plurality of digital light outputs form a second digital output vector; perform a non-linear transformation on the second digital output vector to generate a second transformed digital output vector; store the second transformed digital output vector in a memory unit; and output a result generated based on the first transformed digital output vector and the second transformed digital output vector Artificial neural network output, wherein the optical output vector of the optical matrix multiplication unit is generated by a second optical input vector generated based on the second plurality of modulator control signals, and the second optical input vector is generated by the optical matrix multiplication unit based on the first The above weight control signal is obtained to convert.

Embodiment 9: The computing system of Embodiment 1 further includes: an analog nonlinear unit disposed between the light detection unit and the ADC unit, the analog nonlinear unit is configured to receive the output voltage from the light detection unit and apply The nonlinear transfer function, and outputting the complex conversion output voltage to the ADC unit, wherein the operation further includes: obtaining a first plurality of conversion digital output voltages corresponding to the conversion output voltage from the ADC unit, and the first plurality of conversion digital output voltages form a first conversion a digital output vector; and storing the first converted digital output vector in the memory unit.

Embodiment 10: The computing system of Embodiment 1, wherein the integrated circuit of the controller is configured to generate a first plurality of modulator control signals with a frequency greater than or equal to 8 GHz.

Embodiment 11: The computing system of Embodiment 1 further includes: An analog memory unit is disposed between the DAC unit and the light modulator. The analog memory unit is configured to store a complex analog voltage and output the stored analog voltage; and an analog nonlinear unit is disposed in the light detection unit. With the ADC unit, the analog nonlinear unit is configured to receive an output voltage from the light detection unit, apply a nonlinear transfer function, and output a complex converted output voltage.

Embodiment 12: The computing system of Embodiment 11, wherein the analog memory unit includes a complex capacitor.

Embodiment 13: The computing system of Embodiment 11, wherein the analog memory unit is configured to receive and store the conversion output voltage of the analog nonlinear unit, and output the stored conversion output voltage to the light modulator, and wherein the operation is more The method includes: storing the conversion output voltage of the analog nonlinear unit in the analog memory unit based on generating the first plurality of modulator control signals and the first plurality of weight control signals; and outputting the stored conversion output voltage through the analog memory unit; Obtaining a second plurality of converted digital output voltages from the ADC unit, the second plurality of converted digital output voltages forming a second converted digital output vector; and storing the second converted digital output vector in the memory unit.

Embodiment 14: The computing system of Embodiment 1, wherein the input data set of the artificial neural network calculation request includes a complex digital input vector, wherein the laser unit is configured to generate a complex wavelength, and the light modulator includes: a complex optical modulator bank configured to generate a complex optical input vector, each optical modulator bank corresponding to a wavelength and generating a corresponding optical input vector having a corresponding wavelength; and an optical multiplexer, Configured to combine the optical input vectors into a combined optical input vector including wavelengths, wherein the light detection unit is further configured to demultiplex the wavelengths and generate a complex demultiplexed output voltage, and wherein the operations include: obtaining the complex digital bits from the ADC unit The demultiplexed light output is digitally demultiplexed to form a complex first digital output vector, where each first digital output vector corresponds to a wavelength; a nonlinear conversion is performed on each first digital output vector to generate the complex converted first a digital output vector; and storing the converted first digital output vector in the memory unit, wherein each digital input vector corresponds to a light input vector.

Embodiment 15: The computing system of Embodiment 1, wherein the artificial neural network calculation request includes a complex digital input vector, wherein the laser unit is configured to generate a complex wavelength, and the optical modulator includes: a complex optical modulator group, configured to generate a plurality of optical input vectors, one for each optical modulator group corresponding to a wavelength, and generate corresponding optical input vectors having corresponding wavelengths; and An optical multiplexer configured to combine the optical input vectors into a combined optical input vector including wavelengths, and wherein the operations include: obtaining a first plurality of digital optical outputs from the ADC unit corresponding to the optical output vector, the optical output vector including wavelength, the first plurality of digital light outputs form a first digital output vector; perform a nonlinear transformation on the first digital output vector to generate a first converted digital output vector; and store the first converted digital output vector in a memory unit.

Embodiment 16: The computing system of Embodiment 1, wherein the DAC unit includes: a 1-bit DAC subunit configured to generate a complex 1-bit modulator control signal, wherein the resolution of the ADC unit is 1 bit, where The resolution of the first digit input vector is N bits, and the operation includes: decomposing the first digit input vector into N 1-bit input vectors, each N 1-bit input vector corresponding to the first digit input vector. One of N bits; generate a sequence of N 1-bit modulator control signals corresponding to N 1-bit input vectors through the 1-bit DAC subunit; obtain the corresponding N 1-bit modulators from the ADC unit A sequence of N digital 1-bit light outputs of a sequence of control signals; constructing an N-bit digital output vector from a sequence of N digital 1-bit light outputs; Perform a non-linear transformation on the constructed N-bit digital output vector to generate a transformed N-bit digital output vector; and store the transformed N-bit digital output vector in a memory unit.

Embodiment 17: The computing system of Embodiment 1, wherein the memory unit includes: a digital input vector memory configured to store the first digital input vector and including at least one static random access memory; and neural network weights The memory is configured to store the neural network weights and includes at least one dynamic random access memory.

Embodiment 18: The computing system of Embodiment 1, wherein the DAC unit includes: a first DAC subunit configured to generate a modulator control signal; and a second DAC subunit configured to generate a weight control signal, wherein the One DAC subunit and the second DAC subunit are different.

Embodiment 19: The computing system of Embodiment 1, wherein the laser unit includes: a laser source configured to generate light; and an optical power splitter configured to split the light generated by the laser source into light output, wherein Each light output has approximately the same power.

Embodiment 20: The computing system of Embodiment 1, wherein the light modulator includes one of an MZI modulator, a ring resonance modulator, or an electroabsorption modulator.

Embodiment 21: The computing system of Embodiment 1, wherein the light detection unit includes: a complex photodetector; and a complex amplifier configured to convert the photocurrent generated by the photodetector into an output voltage.

Embodiment 22: The computing system of Embodiment 1, wherein the integrated circuit is a special application integrated circuit.

Embodiment 23: The computing system of Embodiment 1, wherein the optical matrix multiplication unit includes: an input waveguide array, used to receive the light input vector; an optical interference unit, in optical communication with the input waveguide array, used to perform conversion of the light input vector into Linear conversion of the second optical signal array; and an output waveguide array in optical communication with the optical interference unit for guiding the second optical signal array, wherein at least one input waveguide in the input waveguide array passes through the optical interference unit and the output waveguide array Each of the output waveguide optical communications.

Embodiment 24: The computing system of Embodiment 23, wherein the optical interference unit includes: complex interconnected Mach-Zehnder Interference (MZI), and each MZI of the interconnected MZIs includes: a first phase shifter configured to change the MZI a separation ratio; and a second phase shifter configured to shift a phase of one output of the MZI, wherein the first phase shifter and the second phase shifter are coupled to the weight control signal.

Embodiment 25: A computing system, including: a memory unit configured to store a data set and a complex neural network weight; a driver unit configured to generate a complex modulator control signal and generate a complex weight control signal; an optical processor, It includes: a laser unit configured to generate a plurality of light outputs; a plurality of light modulators coupled to the laser unit and a driver unit, the light modulator being configured to control a signal based on the modulator, and the modulation is performed by the laser unit the generated optical output to generate an optical input vector; an optical matrix multiplication unit coupled to the optical modulator and the driver unit, the optical matrix multiplication unit configured to convert the optical input vector into an optical output vector based on the weight control signal; and The light detection unit is coupled to the light matrix multiplication unit and is configured to generate a complex output voltage corresponding to the light output vector; the comparator unit is coupled to the light detection unit and is configured to convert the output voltage into a complex digital number. 1-bit light output; and a controller, including an integrated circuit, configured to: receive an artificial neural network calculation request from the computer including an input data set and a first plurality of neural network weights, wherein the input data set including a first digital input vector having N-bit resolution; storing an input data set and a first plurality of neural network weights in a memory unit; The first digital input vector is decomposed into N 1-bit input vectors, each N 1-bit input vector corresponds to one of the N bits of the first digital input vector; the corresponding N 1-bit inputs are generated through the driver unit A sequence of N 1-bit modulator control signals of the vector; a sequence of N digital 1-bit light output corresponding to the sequence of N 1-bit modulator control signals is obtained from the comparator unit; from the N digital 1 A sequence of bit light outputs constructs an N-bit digital output vector; performs a nonlinear transformation on the constructed N-bit digital output vector to produce a transformed N-bit digital output vector; and stores the transformed N-bit digital output vector in a memory unit Output vector.

Embodiment 26: The calculation method is used to perform artificial neural network calculations in a computing system. The computing system has a light matrix multiplication unit configured to convert a light input vector into a light output vector based on a complex weight control signal. Calculate The method includes: receiving an artificial neural network calculation request including an input data set and a first plurality of neural network weights from a computer, wherein the input data set includes a first digital input vector; and storing the input data set and the first plurality of neural network weights in a memory unit. a plurality of neural network weights; generating a first plurality of modulator control signals based on a first digital input vector through a digital-to-analog converter (DAC) unit, and generating a first plurality of weights based on the first plurality of neural network weights control signal; The first plurality of digital light outputs corresponding to the light output vector of the light matrix multiplication unit are obtained from the analog-to-digital converter (ADC) unit, and the first plurality of digital light outputs form the first digital output vector; performing a nonlinear conversion on the output vector to generate a first converted digital output vector; storing the first converted digital output vector in the memory unit; and outputting an artificial neural network output generated based on the first converted digital output vector by the controller .

Embodiment 27: Computing method, including: providing input information in an electronic format; converting at least a portion of the electronic input information into a light input vector; optically converting the light input vector into a light output vector based on light matrix multiplication; converting the light output vector into an electronic format; and electronically applying a nonlinear transformation to the electronically converted light output vector to provide output information in an electronic format.

Embodiment 28: The calculation method is as in Embodiment 27, further including: repeating electro-optical converting (electronic-to-optical converting), optical transforming (optical transforming), and photoelectric conversion for new electronic input information corresponding to the output information provided in the electronic format. Optical-to-electronic converting and nonlinear conversion for electrical applications.

Embodiment 29: The calculation method as in Embodiment 28, wherein the light matrix multiplication used for the initial light conversion and the light matrix multiplication used for the repeated light conversion are the same and correspond to the same layer of the artificial neural network.

Embodiment 30: The calculation method as in Embodiment 28, wherein the light matrix multiplication used for the initial light conversion and the light matrix multiplication used for the repeated light conversion are different and correspond to different layers of the artificial neural network.

Embodiment 31: The calculation method is as in Embodiment 27, further including: repeating electro-optical conversion, optical conversion, photoelectric conversion and electrically applied nonlinear conversion for different parts of the electronic input information, wherein the optical matrix multiplication is used for the initial optical conversion. It is the same as light matrix multiplication for repeated light conversion, and corresponds to the first layer of artificial neural networks.

Embodiment 32: The calculation method of Embodiment 31, further comprising: providing electronic intermediate information in an electronic format based on the electronic output information for the plurality of parts of the electronic input information generated by the first layer of the artificial neural network; and for each different part of the electronic intermediate information, repeated electro-optical conversions, optical conversions, photoelectric conversions, and electrically applied nonlinear conversions, where the optical matrix multiplication for the initial optical conversion and the repeated optical conversions associated with the different parts of the electronic intermediate information The converted light matrix multiplication is the same and corresponds to the second layer of the artificial neural network.

Embodiment 33: A computing system, comprising: a light processor comprising a passive diffractive optical element, wherein the passive diffractive optical element is configured to convert a light input vector or matrix into a light output vector or matrix representing an application The result of matrix processing of a light input vector or matrix and a given vector defined by the arrangement of diffractive optical elements.

Embodiment 34: The computing system of Embodiment 33, wherein matrix processing includes matrix multiplication between a light input vector or matrix and a given vector defined by an arrangement of diffractive optical elements.

Embodiment 35: The computing system of Embodiment 33, wherein the optical processor includes an optical matrix processing unit, which includes: an input waveguide array for receiving the optical input vector, and an optical interference unit including a passive diffraction optical element, wherein the optical interference unit a unit in optical communication with the input waveguide array and configured to perform a linear conversion of the optical input vector into a second array of optical signals; and an output waveguide array in optical communication with the optical interference unit for guiding the second array of optical signals, wherein At least one input waveguide of the input waveguide array is in optical communication with each output waveguide of the output waveguide array through the optical interference unit.

Embodiment 36: The computing system of Embodiment 35, wherein the optical interference unit includes a substrate having at least one of holes or strips, the size of the holes is in the range of 100 nm to 10 μm, and the width of the strips is in the range of 100 nm to 10 μm. within the range.

Embodiment 37: The computing system of Embodiment 35, wherein the optical interference unit includes a substrate having passive diffractive optical elements disposed in a two-dimensional configuration, and the substrate includes at least one of a planar substrate or a curved substrate.

Embodiment 38: The computing system of Embodiment 37, wherein the substrate includes a planar substrate parallel to a direction of light propagation from the input waveguide array to the output waveguide array.

Embodiment 39: The computing system of Embodiment 33, wherein the optical processor includes an optical matrix processing unit, which includes: an input waveguide array for receiving the optical input matrix, and an optical interference unit including a passive diffraction optical element, wherein the optical interference unit a unit in optical communication with the input waveguide array and configured to perform linear conversion of the optical input matrix into a second optical signal array; and an output waveguide array in optical communication with the optical interference unit for guiding the second optical signal array, wherein At least one input waveguide of the input waveguide array is in optical communication with each output waveguide of the output waveguide array through the optical interference unit.

Embodiment 40: The computing system of Embodiment 39, wherein the optical interference unit includes a substrate having at least one of holes or stripes, the size of the holes is in the range of 100 nm to 10 μm, and the width of the strip is 100 nm to the range of 10μm.

Embodiment 41: The computing system of Embodiment 39, wherein the optical interference unit includes a substrate having passive diffraction optical elements arranged in a three-dimensional configuration.

Embodiment 42: The computing system of Embodiment 41, wherein the substrate has a shape of at least one of a cube, a column, a prism, or an irregular volume.

Embodiment 43: The computing system of Embodiment 39, wherein the optical processor includes an optical interference unit including a hologram with passive diffraction optical elements (hologram), the light processor is configured to receive modulated light representing the light input matrix and to continuously convert the light as it passes through the hologram until the light exits the hologram as a light output matrix.

Embodiment 44: The computing system of Embodiment 35 or 39, wherein the optical interference unit includes a substrate with passive diffraction optical elements, and the substrate includes silicon, silicon oxide, silicon nitride, quartz, lithium niobate, phase change material or at least one of the polymers.

Embodiment 45: The computing system of embodiment 35 or 39, wherein the optical interference unit includes a substrate having a passive diffraction optical element, and the substrate includes at least one of a glass substrate or an acrylic substrate.

Embodiment 46: The computing system of Embodiment 33, wherein the passive diffractive optical element is formed in part from a dopant.

Embodiment 47: The computing system of Embodiment 33, wherein the matrix processing represents the processing of input data by the neural network, and the input data is represented by a light input vector.

Embodiment 48: The computing system of Embodiment 33, wherein the light processor includes: a laser unit configured to generate a complex light output; a complex light modulator coupled to the laser unit and the DAC unit and configured to Based on the complex modulator control signal, the light output generated by the laser unit is modulated to generate a light input vector; A light matrix processing unit coupled to the light modulator, the light matrix processing unit including a passive diffractive optical element configured to convert a light input vector into a light output vector based on a complex weight defined by the passive diffractive optical element ; And a light detection unit, coupled to the light matrix processing unit, and configured to generate a complex output electrical signal corresponding to the light output vector.

Embodiment 49: The computing system of Embodiment 48, wherein the passive diffractive optical element is arranged in a three-dimensional configuration, the light modulator includes a two-dimensional light modulator array, and the light detection unit includes a two-dimensional light detector array .

Embodiment 50: The computing system of Embodiment 48, wherein the optical matrix processing unit includes a housing module to support and protect the input waveguide array, the optical interference unit, and the output waveguide array, and the optical processor includes a receiving module, The receiving module is configured to receive the optical matrix processing unit. The receiving module includes a first interface that enables the optical matrix processing unit to receive the optical input vector from the optical modulator, and a second interface that enables the optical matrix processing unit to The light output vector is transmitted to the light detection unit.

Embodiment 51: The computing system of Embodiment 48, wherein the output electrical signal includes at least one of a complex voltage signal or a complex current signal.

Embodiment 52: The computing system of Embodiment 48, further comprising: a memory unit; a digital-to-analog converter (DAC) unit configured to generate a modulator control signal; an analog-to-digital converter (ADC) unit coupled to a light detection unit and configured to convert the output electrical signal into a complex digital output; and A controller, including an integrated circuit, configured to perform the following operations: receive an artificial neural network calculation request including an input data set from a computer, wherein the input data set includes a first digital input vector; and store the input data set in a memory unit ; and generating a first plurality of modulator control signals based on the first digital input vector through the DAC unit.

Embodiment 53: Computing method, comprising: 3D printing a light matrix processing unit including a passive diffractive optical element, wherein the passive diffractive optical element is configured to convert a light input vector or matrix into a light output vector or matrix, which represents an application The result of matrix processing of a light input vector or matrix and a given vector defined by the arrangement of diffractive optical elements.

Embodiment 54: A computational method comprising: using one or more laser beams to generate a hologram including a passive diffractive optical element, wherein the passive diffractive optical element is configured to convert a light input vector or matrix into a light output vector or A matrix that represents the result of matrix processing applied to a light input vector or matrix and a given vector defined by the arrangement of diffractive optical elements.

Embodiment 55: A computing system, comprising: a light processor comprising a passive diffractive optical element disposed in a one-dimensional manner, wherein the passive diffractive optical element is configured to convert light input into light output, a representation of which is applied to the light input and The result of matrix processing of a given vector defined by an arrangement of diffractive optical elements.

Embodiment 56: The computing system of Embodiment 55, wherein matrix processing includes matrix multiplication between the optical input and a given vector defined by the arrangement of diffractive optical elements.

Embodiment 57: The computing system of Embodiment 55, wherein the optical processor includes an optical matrix processing unit, which includes: an input waveguide for receiving optical input, and an optical interference unit including a passive diffraction optical element, wherein the optical interference unit and an input waveguide in optical communication and configured to perform linear conversion of the light input; and an output waveguide in optical communication with the optical interference unit for directing the light output.

Embodiment 58: The computing system of Embodiment 57, wherein the optical interference unit includes a substrate having at least one of holes or grating elements, the holes or grating elements having a size in the range of 100 nm to 10 μm.

Embodiment 59: A computing system including: a memory unit; a digital-to-analog converter (DAC) unit configured to generate a complex modulator control signal; an optical processor including: a laser unit configured to generate a complex optical output ; A plurality of optical modulators, coupled to the laser unit and the DAC unit, the optical modulator is configured to modulate the optical output generated by the laser unit based on the modulator control signal to generate an optical input vector; A light matrix processing unit coupled to the light modulator, the light matrix processing unit including a passive diffractive optical element configured to convert a light input vector into a light output vector based on a complex weight defined by the passive diffractive optical element ; and a light detection unit, coupled to the light matrix processing unit, and configured to generate a complex output electrical signal corresponding to the light output vector; an analog-to-digital converter (ADC) unit, coupled to the light detection unit, and configured to convert the output electrical signal into a complex digital light output; the controller, including the integrated circuit, is configured to perform the following operations: receive an artificial neural network calculation request from the computer including an input data set, wherein the input data set includes a first digit input vector; store the input data set in the memory unit; and generate a first plurality of modulator control signals based on the first digital input vector through the DAC unit.

Embodiment 60: The computing system of Embodiment 59, wherein the matrix processing unit includes a passive diffraction optical element configured to convert a light input vector into a light output vector that represents the difference between the light input vector and the light output vector generated by the passive diffraction optical element. The product of matrix multiplications between defined vectors.

Embodiment 61: The computing system of Embodiment 59, wherein the operation further includes: obtaining a first plurality of digital light outputs corresponding to the light output vector of the light matrix processing unit from the ADC unit, and the first plurality of digital light outputs form the first digital output vector; Perform a non-linear transformation on the first digital output vector to generate a first transformed digital output vector; and store the first transformed digital output vector in a memory unit.

Embodiment 62: The computing system of Embodiment 61, wherein the computing system has a first cycle defined as the steps of storing the input data set in the memory unit and storing the first converted digital output vector in the memory unit. The time elapsed between steps, and where the first cycle period is less than or equal to 1 ns.

Embodiment 63: The computing system of Embodiment 61, wherein the operation further includes: outputting an artificial neural network output generated based on the first converted digital output vector.

Embodiment 64: The computing system of Embodiment 61, wherein the operation further includes: generating a second plurality of modulator control signals based on the first converted digital output vector through the DAC unit.

Embodiment 65: The computing system of Embodiment 61, wherein the input data set further includes a second digital input vector, and the operation further includes: generating a second plurality of modulator control signals based on the second digital input vector through the DAC unit ; Obtain a second plurality of digital light outputs corresponding to the light output vector of the light matrix processing unit from the ADC unit, and the second plurality of digital light outputs form a second digital output vector; perform a non-linear transformation on the second digital output vector to generate a second transformed digital output vector; store the second transformed digital output vector in a memory unit; and output a result generated based on the first transformed digital output vector and the second transformed digital output vector The artificial neural network output, wherein the optical output vector of the optical matrix processing unit is generated by a second optical input vector generated based on the second plurality of modulator control signals, and the second optical input vector is generated by the optical matrix multiplication unit based on the passive Diffractive optical elements are converted by weights defined by them.

Embodiment 66: The computing system of Embodiment 59 further includes: an analog nonlinear unit disposed between the light detection unit and the ADC unit, the analog nonlinear unit is configured to receive an output electrical signal from the light detection unit, Apply a nonlinear transfer function, and output the complex conversion output electrical signal to the ADC unit. The operation further includes: obtaining the first plurality of conversion digital output electrical signals corresponding to the conversion output electrical signal from the ADC unit, and the first plurality of conversion digital output electrical signals. forming a first converted digital output vector; and storing the first converted digital output vector in the memory unit.

Embodiment 67: The computing system of Embodiment 59, wherein the integrated circuit of the controller is configured to generate a first plurality of modulator control signals with a frequency greater than or equal to 8 GHz.

Embodiment 68: The computing system of Embodiment 59, further comprising: An analog memory unit is disposed between the DAC unit and the light modulator. The analog memory unit is configured to store a complex analog voltage and output the stored analog voltage; and an analog nonlinear unit is disposed in the light detection unit. Between the ADC unit and the analog nonlinear unit, the analog nonlinear unit is configured to receive the output electrical signal from the light detection unit, apply the nonlinear transfer function, and output the complex conversion output electrical signal.

Embodiment 69: The computing system of Embodiment 68, wherein the analog memory unit includes a complex capacitor.

Embodiment 70: The computing system of Embodiment 68, wherein the analog memory unit is configured to receive and store the conversion output electrical signal of the analog nonlinear unit, and output the stored conversion output electrical signal to the light modulator, and wherein The operation further includes: based on generating the first plurality of modulator control signals, storing the conversion output electrical signal of the analog nonlinear unit in the analog memory unit; outputting the stored conversion output electrical signal through the analog memory unit; obtaining from the ADC unit The second plurality of conversion digital output electrical signals forms a second conversion digital output vector; and the second conversion digital output vector is stored in the memory unit.

Embodiment 71: The computing system of Embodiment 59, wherein the artificial neural network calculation request includes a complex digital input vector, wherein the laser unit is configured to generate a complex wavelength, and wherein the light modulator includes: a plurality of optical modulator groups configured to generate a plurality of optical input vectors, each optical modulator group corresponding to a wavelength and generating a corresponding optical input vector having a corresponding wavelength; and an optical multiplexer configured to combine The optical input vectors are combined into a combined optical input vector including wavelengths, wherein the light detection unit is further configured to demultiplex the wavelengths and generate a complex demultiplexed output electrical signal, and the operation includes: obtaining the complex digital demultiplexer from the ADC unit Decompose the light output, and digitally demultiplex the light output to form a complex first digital output vector, where each first digital output vector corresponds to a wavelength; perform a nonlinear conversion on each first digital output vector to generate a complex converted first digital output vector; and store the converted first digital output vector in the memory unit, wherein each digital input vector corresponds to a light input vector.

Embodiment 72: The computing system of Embodiment 59, wherein the artificial neural network calculation request includes a complex digital input vector, wherein the laser unit is configured to generate a complex wavelength, and the optical modulator includes: a complex optical modulator group, configured to generate a plurality of optical input vectors, one for each optical modulator group corresponding to a wavelength, and generate corresponding optical input vectors having corresponding wavelengths; and An optical multiplexer configured to combine the optical input vectors into a combined optical input vector including wavelengths, and wherein the operations include: obtaining a first plurality of digital optical outputs from the ADC unit corresponding to the optical output vector, the optical output vector including wavelength, the first plurality of digital light outputs form a first digital output vector; perform a nonlinear transformation on the first digital output vector to generate a first converted digital output vector; and store the first converted digital output vector in a memory unit.

Embodiment 73: The computing system of Embodiment 59, wherein the DAC unit includes: a 1-bit DAC unit configured to generate a complex 1-bit modulator control signal, wherein the resolution of the ADC unit is 1 bit, wherein the The resolution of the one-digit input vector is N bits, and the operation includes: decomposing the first-digit input vector into N 1-bit input vectors, each N 1-bit input vector corresponding to N of the first-digit input vector One of the bits; generate a sequence of N 1-bit modulator control signals corresponding to N 1-bit input vectors through the 1-bit DAC unit; obtain corresponding N 1-bit modulator control signals from the ADC unit The sequence of N digits of the sequence of 1-bit light output; Construct an N-bit digital output vector from a sequence of N digital 1-bit light outputs; perform a nonlinear transformation on the constructed N-bit digital output vector to produce a converted N-bit digital output vector; and store in a memory unit Convert an N-bit digital output vector.

Embodiment 74: The computing system of Embodiment 59, wherein the memory unit includes a digital input vector memory configured to store the first digital input vector and includes at least one static random access memory.

Embodiment 75: The computing system of Embodiment 59, wherein the laser unit includes: a laser source configured to generate light; and an optical power splitter configured to split the light generated by the laser source into light output, wherein Each light output has approximately the same power.

Embodiment 76: The computing system of embodiment 59, wherein the light modulator includes one of an MZI modulator, a ring resonance modulator, or an electroabsorption modulator.

Embodiment 77: The computing system of Embodiment 59, wherein the light detection unit includes: a plurality of light detectors; and a plurality of amplifiers configured to convert the photocurrent generated by the light detector into an output electrical signal.

Embodiment 78: The computing system of Embodiment 59, wherein the integrated circuit includes an application special integrated circuit.

Embodiment 79: The computing system of Embodiment 59, wherein the light matrix processing unit includes: an input waveguide array, used to receive the light input vector; an optical interference unit, in optical communication with the input waveguide array, used to perform conversion of the light input vector into Linear conversion of the second optical signal array, wherein the optical interference unit includes a passive diffraction optical element; and an output waveguide array in optical communication with the optical interference unit for guiding the second optical signal array, wherein at least one of the input waveguide arrays The input waveguide is in optical communication with each output waveguide in the output waveguide array through the optical interference unit.

Embodiment 80: A computing system, including: a memory unit; a driver unit configured to generate a complex modulator control signal; a light processor including: a laser unit configured to generate a complex light output; a complex light modulator , coupled to the laser unit and the driver unit, the optical modulator is configured to modulate the light output generated by the laser unit based on the modulator control signal to generate an optical input vector; the optical matrix processing unit is coupled To the light modulator and driver unit, the light matrix processing unit includes a passive diffractive optical element configured to convert the light input vector into a light output vector based on a complex weighted control signal defined by the passive diffractive optical element; and The light detection unit is coupled to the light matrix processing unit and is configured to generate a complex output electrical signal corresponding to the light output vector; the comparator unit is coupled to the light detection unit and is configured to convert the output electrical signal into a complex digital 1-bit light output; and a controller, including an integrated circuit, configured to perform the following operations: receive an artificial neural network calculation request from the computer including an input data set, wherein the input data set includes an N-bit resolution The first digit input vector of One of the bits; generate a sequence of N 1-bit modulator control signals corresponding to N 1-bit input vectors through the driver unit; obtain a sequence of N 1-bit modulator control signals corresponding to the comparator unit a sequence of N digital 1-bit light outputs; construct an N-bit digital output vector from the sequence of N digital 1-bit light outputs; perform a nonlinear transformation on the constructed N-bit digital output vector to produce a converted N-bit the N-bit digital output vector; and storing the converted N-bit digital output vector in the memory unit.

Embodiment 81: The computing system of Embodiment 80, wherein the light matrix processing unit includes a passive diffractive optical element configured to convert a light input vector into a light output vector that represents an input vector represented by the light input vector and The product of matrix multiplications between given vectors defined by passive diffractive optical elements.

Embodiment 82: A computing method for performing artificial neural network calculations in a computing system having a light matrix processing unit. The computing method includes: receiving an artificial neural network calculation request including an input data set from a computer, the input data set including a digital input vector; storing the input data set in the memory unit; generating a first plurality of modulator control signals based on the first digital input vector through a digital-to-analog converter (DAC) unit; by using passive diffraction optics including A light matrix processing unit of an arrangement of elements that converts a light input vector into a light output vector, where the light output vector represents the result of matrix processing applied to the light input vector and a given vector defined by the arrangement of diffractive optical elements; from analogy The digital converter (ADC) unit obtains a first plurality of digital light outputs corresponding to the light output vector of the light matrix processing unit, and the first plurality of digital light outputs form a first digital output vector; the controller controls the first digital output vector performing a nonlinear transformation to generate a first transformed digital output vector; storing the first transformed digital output vector in a memory unit; and The controller outputs an artificial neural network output generated based on the first converted digital output vector.

Embodiment 83: The calculation method of Embodiment 82, wherein converting the light input vector into the light output vector includes converting the light input vector into a matrix representing a relationship between the digital input vector and a predetermined vector defined by the arrangement of the diffractive optical elements. Multiply the product of the light output vector.

Embodiment 84: Computing method, comprising: providing input information in an electronic format; converting at least a portion of the electronic input information into a light input vector; and converting the light input vector optics based on light matrix processing by a light processor including passive diffractive optical elements. ground into a light output vector; convert the light output vector into an electronic format; and electronically apply a nonlinear transformation to the electronically converted light output vector to provide output information in an electronic format.

Embodiment 85: The calculation method of Embodiment 84, wherein optically converting the light input vector into the light output vector includes based on a relationship between the input vector represented by the light input vector and a predetermined vector defined by the passive diffractive optical element. The light matrix multiplication optically converts the light input vector into the light output vector.

Embodiment 86: The calculation method of Embodiment 84 further includes: repeating electro-optical conversion, optical conversion, photoelectric conversion and electrically applied nonlinear conversion for new electronic input information corresponding to the output information provided in an electronic format.

Embodiment 87: The calculation method of Embodiment 86, wherein the light matrix processing for initial light conversion and the light matrix processing for repeated light conversion are the same and correspond to the same layer of the artificial neural network.

Embodiment 88: The calculation method is as in Embodiment 84, further including: repeating electro-optical conversion, optical conversion, photoelectric conversion and electrically applied nonlinear conversion for different parts of the electronic input information, wherein the optical matrix processing is used for the initial optical conversion. The light matrix processing is the same as that of repeated light conversion, and corresponds to a layer of the artificial neural network.

Embodiment 89: The computing system includes: an optical matrix processing unit configured to process an input vector of length N, wherein the optical matrix processing unit includes an N+2 layer of directional coupler and an N layer of phase shifter, And N is a positive integer.

Embodiment 90: The computing system of embodiment 89, wherein the optical matrix processing unit includes no more than N+2 layers of directional couplers.

Embodiment 91: The computing system of embodiment 89, wherein the light matrix processing unit includes a light matrix method unit.

Embodiment 92: The computing system of Embodiment 89, wherein the optical matrix processing unit includes: a substrate, and interconnected interferometers disposed on the substrate, wherein each interferometer includes an optical waveguide disposed on the substrate, and a directional coupler and phase shifters are part of the interconnected interferometer.

Embodiment 93: The computing system of embodiment 89, wherein the light matrix processing unit includes an attenuator layer after the last layer of directional couplers.

Embodiment 94: The computing system of Embodiment 93, wherein one layer of attenuators includes N attenuators.

Embodiment 95: The computing system of Embodiment 93, including one or more homodyne detectors for detecting the output from the attenuator.

Embodiment 96: The computing system of Embodiment 89, wherein N=3, and the optical matrix processing unit includes: an input terminal (terminal) configured to receive an input vector; a first-layer directional coupler coupled to the input terminal; The first layer of phase shifters is coupled to the first layer of directional couplers; the second layer of directional couplers is coupled to the first layer of phase shifters; the second layer of phase shifters is coupled to the second layer of directional couplers ; The third layer directional coupler, coupled to the second layer phase shifter; the third layer phase shifter, coupled to the third layer directional coupler; the fourth layer directional coupler, coupled to the third layer phase shifter and a fifth layer directional coupler coupled to the fourth layer directional coupler.

Embodiment 97: The computing system of embodiment 89, wherein N=4, and the light matrix processing unit includes: an input terminal configured to receive an input vector; The first, second, third and fourth layers of directional couplers, each layer of directional couplers is followed by a layer of phase shifters, where the first layer of directional couplers is coupled to the input; the penultimate layer (second to last) -to-last layer) directional coupler, coupled to the fourth layer phase shifter; and a last layer directional coupler, coupled to the penultimate layer directional coupler.

Embodiment 98: The computing system of Embodiment 89, wherein N=8, and the optical matrix processing unit includes: an input terminal configured to receive an input vector; eight layers of directional couplers, each layer of directional couplers is followed by a layer of phase shift. device, wherein the first layer of directional couplers is coupled to the input end; the penultimate layer of directional couplers is coupled to the eighth layer of phase shifters; and the final layer of directional couplers is coupled to the penultimate layer of directional couplers. .

Embodiment 99: The computing system of Embodiment 89, wherein the optical matrix processing unit includes: an input terminal configured to receive an input vector; N layers of directional couplers, each layer of directional couplers is followed by a layer of phase shifters, wherein the first The layer directional coupler is coupled to the input terminal; the penultimate layer directional coupler is coupled to the Nth layer directional coupler; and the final layer directional coupler is coupled to the penultimate layer directional coupler.

Embodiment 100: The computing system of embodiment 99, wherein N is an even number.

Embodiment 101: The computing system of Embodiment 100, wherein each i-th layer directional coupler includes N/2 directional couplers, where i is an odd number, and each j-th layer directional coupler includes N/2-1 directional couplers, where j is an odd number.

Embodiment 102: The computing system of Embodiment 100, wherein for each i-th layer directional coupler where i is an odd number, the k-th directional coupler is coupled to the (2k-1)-th and 2k-th directional couplers of the previous layer. Output, k is an integer from 1 to N/2.

Embodiment 103: The computing system of Embodiment 100, wherein for each j-th layer of directional couplers where j is an even number, the m-th directional coupler is coupled to the (2m)-th and (2m+) of the previous layer. 1) output, m is an integer from 1 to N/2-1.

Embodiment 104: The computing system of Embodiment 100, wherein each i-th layer phase shifter includes N phase shifters, where i is an odd number, and each j-th layer phase shifter includes N-2 phase shifters. , where j is an even number.

Embodiment 105: The computing system of embodiment 99, wherein N is an odd number.

Embodiment 106: The computing system of embodiment 105, wherein each layer of directional couplers includes (N-1)/2 directional couplers.

Embodiment 107: The computing system of Embodiment 105, wherein each layer of phase shifters includes N-1 phase shifters.

Embodiment 108: Computing system, including: a generator (generator) configured to generate a first data set, wherein the generator includes a light matrix processing unit; and a discriminator (discriminator) configured to receive data including data from the first data set and data from a third data set A second data set of data, the data in the first data set has similar characteristics to the data in the third data set, and the data in the second data set is classified as data from the first data set or from the third data set. Data set information.

Embodiment 109: The computing system of Embodiment 108, wherein the light matrix processing unit includes at least one of the following: (i) the light matrix multiplication unit of any one of Embodiments 1 to 25, (ii) Embodiments 32 to 52, The passive diffractive optical element of any one of Embodiments 55 to 81, or (iii) the light matrix processing unit of any one of Embodiments 89 to 107.

Embodiment 110: The computing system of embodiment 108, wherein the third data set includes real data, the generator is configured to generate synthetic data similar to the real data, and the discriminator is configured to classify the data as real data or synthetic data.

Embodiment 111: The computing system of embodiment 108, wherein the generator is configured to generate a data set for training an autonomous vehicle (vehicle), a medical diagnostic system, a fraud detection system, a weather forecast system, a financial forecast system, At least one of a facial recognition system, a speech recognition system, or a product defect detection system.

Embodiment 112: The computing system of embodiment 108, wherein the generator is configured to generate an image that resembles at least one of a real object or a real scene images, and the discriminator is configured to classify the received images as (i) images of real objects or real scenes, or (ii) synthetic images produced by the generator.

Embodiment 113: The computing system of embodiment 112, wherein the real object includes at least one of a person, an animal, a cell, a tissue, or a product, and the real scene includes a scene encountered by the vehicle.

Embodiment 114: The computing system of embodiment 113, wherein the discriminator is configured to classify the received image as (i) a real person, a real animal, a real cell, a real tissue, a real product, or a real object encountered by the vehicle scene, or (ii) a composite image produced by a generator.

Embodiment 115: The computing system of Embodiment 113, wherein the vehicle includes at least one of a motorcycle, a car, a truck, a train, a helicopter, an airplane, a submarine, a ship, or a drone.

Embodiment 116: The computing system of embodiment 113, wherein the generator is configured to generate images of tissues or cells associated with at least one of human disease, animal disease, or plant disease.

Embodiment 117: The computing system of embodiment 116, wherein the generator is configured to generate images of tissues or cells associated with a human disease, and the disease includes cancer, Parkinson's disease, sickle cell anemia, heart disease, cardiovascular disease disease, diabetes, chest disease, or skin disease.

Embodiment 118: The computing system of embodiment 116, wherein the generator is configured to generate images of tissues or cells associated with cancer, and the cancer includes at least one of skin cancer, breast cancer, lung cancer, liver cancer, prostate cancer, or brain cancer. By.

Embodiment 119: The computing system of Embodiment 108 further includes a random noise generator configured to generate random noise input to the generator, and the generator is configured to generate the first data set based on the random noise.

Embodiment 120: A computing system includes: a random noise generator configured to generate random noise; and a generator configured to generate data based on the random noise, wherein the generator includes an optical matrix processing unit.

Embodiment 121: The computing system of Embodiment 120, wherein the light matrix processing unit includes at least one of the following: (i) the light matrix multiplication unit of any one of Embodiments 1 to 25, (ii) Embodiments 33 to 52, The passive diffractive optical element of any one of Embodiments 55 to 81, or (iii) the light matrix processing unit of any one of Embodiments 89 to 107.

Embodiment 122: A computing system including: an optical circuit configured to perform a logic function on two input signals, the optical circuit including: a first directional coupler having two input terminals and two output terminals, Two input terminals are configured to receive two input signals; a first pair of phase shifters is configured to modify the phase of signals at the two output terminals of the first directional coupler; the second directional coupler has two input terminals and two output terminals, the two input terminals being configured to receive signals from the first pair of phase shifters; and The second pair of phase shifters is configured to modify the phase of the signal at the two output terminals of the second directional coupler.

Embodiment 123: The computing system of embodiment 122, wherein the phase shifter is configured to cause the optical circuit to perform rotation:

Embodiment 124: The computing system of embodiment 122, wherein when the input signals x ₁ and x ₂ are provided to the two input terminals of the first directional coupler, the phase shifter is configured to cause the optical circuit to perform an operation:

Embodiment 125: The computing system of Embodiment 124, wherein the optical circuit includes a first photodetector configured to generate an absolute value of the signal from the second pair of phase shifters to cause the optical circuit to perform:

Embodiment 126: The computing system of Embodiment 125, wherein the optical circuit includes a comparator configured to compare the output signal of the first photodetector with a threshold to generate a binary value to cause the optical circuit to generate Output:

其產生輸出AND(x₁,x₂)和OR(x₁,x₂)。 Embodiment 127: The computing system of Embodiment 125, wherein the optical circuit includes a feedback mechanism configured such that the output signal of the photodetector is fed back to the input of the first directional coupler, and Through the first directional coupler, the first pair of phase shifters, the second directional coupler and the second pair of phase shifters, and detected by the photodetector, the optical circuit performs an operation:

It produces the outputs AND(x ₁ ,x ₂ ) and OR(x ₁ ,x ₂ ).

其產生輸出AND(x₁,x₂)和OR(x₁,x₂)。 Embodiment 128: The computing system of Embodiment 125, wherein the optical circuit includes: a third directional coupler having two input terminals and two output terminals, the two input terminals being configured to receive from the second pair of phase shifters. signal; the third pair of phase shifters is configured to modify the phase of the signal at the two output terminals of the third directional coupler; the fourth directional coupler has two input terminals and two output terminals, the two input terminals are configured to receive signals from a third pair of phase shifters; a fourth pair of phase shifters configured to modify the phase of signals at the two output terminals of the fourth directional coupler; and a second photodetector configured to The absolute value of the signal from the fourth pair of phase shifters is generated to cause the optical circuit to perform:

It produces the outputs AND(x ₁ ,x ₂ ) and OR(x ₁ ,x ₂ ).

Embodiment 129: The computing system of embodiment 122, including a bitonic sorter configured to perform a sorting function of the bitonic sorter using optical circuits.

Embodiment 130: The computing system of embodiment 122, including a device configured to perform a hashing function using optical circuits.

Embodiment 131: The computing system of embodiment 130, wherein the hash function includes secure hash algorithm 2 (SHA-2).

Embodiment 132: A computing device comprising: a plurality of optical waveguides, wherein a set of a plurality of input values is encoded on a corresponding optical signal carried by the optical waveguide; a plurality of replica modules, and for at least two of the one or more optical signals each of the subsets, a corresponding set of one or more replication modules configured to divide the one or more subsets of the optical signal into two or more copies of the optical signal; the complex multiplication module, and For each of the at least two copies of the first subset of the one or more optical signals, a corresponding multiplication module is configured to multiply the one or more optical signals of the first subset using optical amplitude modulation. One or more matrix element values, wherein at least one multiplication module includes an optical amplitude modulator, the optical amplitude modulator includes an input port and two output ports, and provides a pair of correlated optical signals from the two output ports, such that The difference between the amplitudes of the correlated optical signals corresponds to the result of multiplying the input value by the signed matrix element value; and One or more summing modules, and for the results of the two or more multiplication modules, a corresponding summation module is configured to generate an electrical signal, the electrical signal representing the sum of the results of the two or more multiplication modules.

Embodiment 133: The computing device of Embodiment 132, wherein an input value encoded on a corresponding optical signal in a set of a plurality of input values represents an element of an input vector multiplied by a matrix including one or more matrix element values. .

Embodiment 134: The computing device of any one of embodiments 132 or 133, wherein a plurality of output values are encoded on corresponding electrical signals generated by one or more summing modules, and a plurality of The output value in the output value represents the element of the output vector, which is generated by multiplying the input vector by the matrix.

Embodiment 135: The computing device of embodiments 132 to 134, wherein each optical signal carried by the optical waveguide includes an optical wave having a common wavelength, the common wavelength being substantially the same for all optical signals.

Embodiment 136: The computing device of any one of Embodiments 132 to 135, wherein the replica module includes at least one replica module having an optical splitter that sends a predetermined proportion of the power of the light wave to the input port. an output port, and the remaining proportion of the power of the light wave is sent to the second output port at the input port.

Embodiment 137: The computing device of Embodiment 136, wherein the optical splitter includes a waveguide optical splitter that sends a determined proportion of the power of the optical wave guided by the input optical waveguide to the first output optical waveguide and is The remaining proportion of the power of the light wave guided by the optical waveguide is sent to the second output optical waveguide.

Embodiment 138: The computing device of Embodiment 137, wherein the guided mode of the input optical waveguide is adiabatically coupled to the plurality of guided modes of each of the first output optical waveguide and the second output optical waveguide.

Embodiment 139: The computing device of any one of Embodiments 136 to 138, wherein the optical splitter includes a beam splitter including at least one surface that transmits a determined proportion of the power of the light wave at the input port and at the input port. The remaining fraction of the power of the light wave reflected by the port.

Embodiment 140: The computing device of embodiment 139, wherein the at least one optical waveguide includes an optical fiber coupled to an optical coupler that couples a guided mode of the optical fiber to a free-space propagation mode.

Embodiment 141: The computing device of any one of embodiments 132 to 140, wherein the multiplication module includes at least one coherence-sensitive multiplication module, the coherence-sensitive multiplication module configured to Interference between, using optical amplitude modulation to multiply one or more optical signals of the first subset by one or more matrix element values, the light wave has a coherence length, and the coherence length is at least the same as the propagation distance through the coherence sensitive multiplication module long.

Embodiment 142: The computing device of Embodiment 141, wherein the coherence-sensitive multiplication module includes a Mach-Zehnder interferometer (MZI), which divides the light waves guided by the input optical waveguide into the first phase of the MZI. An optical waveguide arm and a second optical waveguide arm of a Mach-Zehnder interferometer. The first optical waveguide arm includes a phase shifter, and the phase shifter has a phase relative to the second optical waveguide arm. The delay creates a relative phase shift, and the Mach-Zehnder interferometer combines the complex light waves from the first optical waveguide arm and the second optical waveguide arm into at least one output optical waveguide.

Embodiment 143: The computing device of Embodiment 142, wherein the MZI combines the complex light waves from the first optical waveguide arm and the second optical waveguide arm into each of the first output optical waveguide and the second output optical waveguide, A photodetector receives a light wave from the first output optical waveguide to generate a first photocurrent, a second photodetector receives a light wave from the second output optical waveguide to generate a second photocurrent, and the result of the coherent sensitive multiplication module includes The difference between the first photocurrent and the second photocurrent.

Embodiment 144: The computing device of any one of embodiments 141 to 143, wherein the coherence sensitive multiplication module includes one or more ring resonators, the ring resonators including at least one coupled to the first optical waveguide. a ring resonator and at least one ring resonator coupled to the second optical waveguide.

Embodiment 145: The computing device of Embodiment 144, wherein the first photodetector receives the light wave from the first optical waveguide to generate the first photocurrent, and the second photodetector receives the light wave from the second optical waveguide. , to generate a second photocurrent, and the result of the coherent sensitive multiplication module includes the difference between the first photocurrent and the second photocurrent.

Embodiment 146: The computing device of any one of embodiments 132 to 145, wherein the multiplication module includes at least one coherence-insensitive multiplication module, the coherence-insensitive multiplication module configured to Energy absorption within the first subset of one or more optical signals is multiplied by one or more matrix element values using optical amplitude modulation.

Embodiment 147: The computing device of embodiment 146, wherein the coherent insensitive multiplication module includes an electro-absorption modulator.

Embodiment 148: The computing device of any one of Embodiments 132-147, wherein the one or more summing modules include at least one summing module having: (1) two or more input conductors, each an input conductor carrying an electrical signal in the form of an input current, the magnitude of which represents a corresponding result of a multiplication module, and (2) at least one output conductor carrying an electrical signal representing the sum of the corresponding results in the form of an output current , the output current is proportional to the sum of the input currents.

Embodiment 149: The computing device of Embodiment 148, wherein the two or more input conductors and the output conductors comprise a plurality of conductors meeting at one or more junctions between the conductors, and the output current is substantially equal to the sum of the input currents.

Embodiment 150: The computing device of embodiment 148 or 149, wherein at least a first input current of the input current is provided in the form of at least one photocurrent generated by at least one photodetector, the photodetector receiving The optical signal generated by the first multiplication module of the multiplication module.

Embodiment 151: The computing device of embodiment 150, wherein the first input current is provided in the form of a difference between two photocurrents generated by different corresponding photodetectors, and the photodetectors receive Different corresponding optical signals generated by the first multiplication module.

Embodiment 152: The computing device of any one of embodiments 132-151, wherein one of the copies of the first subset of the one or more optical signals consists of a single optical signal, and wherein an input value on the single optical signal is encoded.

Embodiment 153: The computing device of embodiment 152, wherein the multiplication module corresponding to the copy of the first subset multiplies the encoded input value by a single matrix element value.

Embodiment 154: The computing device of any one of Embodiments 132-153, wherein one of the copies of the first subset of the one or more optical signals includes more than one optical signal and less than all optical signals. A quantity in which multiple input values of an optical signal are encoded.

Embodiment 155: The computing device of embodiment 154, wherein the multiplication module corresponding to the copy of the first subset multiplies the encoded input value by a different corresponding matrix element value.

Embodiment 156: The computing device of Embodiment 152, wherein different multiplication modules corresponding to different corresponding copies of the first subset of the one or more optical signals are included in different devices, and the different devices perform optical communication between the different devices. one of the copies of the first subset of one or more optical signals.

Embodiment 157: The computing device of any one of Embodiments 132-156, wherein two or more of the optical waveguides, two or more of the replication modules, two or more of the multiplication modules, and one or more At least one of the summing modules is provided on the substrate of the common device.

Embodiment 158: The computing device of Embodiment 157, wherein the device performs vector matrix multiplication, wherein the input vectors are provided as a set of optical signals and the output vectors are provided as a set of electrical signals.

Embodiment 159: The computing device as in any one of embodiments 132 to 155, further comprising an accumulator, the accumulator integrates the input electrical signal corresponding to the output of the multiplication module or the summation module, wherein time domain coding is used. encoding) to encode the input electrical signal, the time domain encoding uses on-off amplitude modulation in each of a plurality of time slots, and the accumulator generates an output electrical signal, the output electrical signal is expressed in more than two The amplitude level is encoded by the amplitude level, which corresponds to the different duty cycles of the time domain encoding on multiple time slots.

Embodiment 160: The computing device of any one of embodiments 132-159, wherein each of the two or more multiplication modules corresponds to a different subset of the one or more optical signals.

Embodiment 161: The computing device of any one of Embodiments 132-160, further comprising each replica for the second subset of the one or more optical signals, and the first subset of the one or more optical signals. The multiplication module is configured to multiply one or more optical signals of the second subset by one or more matrix element values using optical amplitude modulation.

Embodiment 162: A calculation method, including: encoding a set of multiple input values on a corresponding optical signal; For each of the at least two subsets of the one or more optical signals, use a respective set of one or more replication modules to divide the one or more subsets of the optical signals into two or more replicas of the optical signals ; for each of the at least two copies of the first subset of the one or more optical signals, using a corresponding multiplication module to multiply the one or more optical signals of the first subset using optical amplitude modulation One or more matrix element values, wherein at least one multiplication module includes an optical amplitude modulator, the optical amplitude modulator includes an input port and two output ports, and provides a pair of correlated optical signals from the two output ports, such that The difference between the amplitudes of the correlated optical signals corresponds to the result of multiplying the input value by the element value of the signed matrix; and for the result of two or more multiplication modules, a summation module is used to generate an electrical signal, the electrical signal represents The sum of the results of two or more multiplication modules.

Embodiment 163: Calculation method, comprising: encoding a set of input values representing elements of an input vector on a corresponding optical signal; encoding a set of coefficients representing matrix elements as amplitudes of a set of optical amplitude modulators coupled to the optical signal Modulation level, wherein at least one optical amplitude modulator including one input port and two output ports provides a pair of correlated optical signals from the two output ports, such that the difference between the amplitudes of the correlated optical signals corresponds to the input the result of multiplying a value by an element value of a signed matrix; and a set of output values encoding elements representing an output vector on a corresponding electrical signal, at least one of which is in the form of a current whose amplitude corresponds to the corresponding element of the input vector multiplied by the matrix The sum of the corresponding elements of a row.

Embodiment 164: The calculation method of Embodiment 163, wherein at least one optical signal is provided by a first optical waveguide, and the first optical waveguide is coupled to an optical splitter, and the optical splitter determines the power of the optical wave guided by the first optical waveguide. A given proportion of the power of the light wave guided by the first optical waveguide is sent to the second output optical waveguide, and a remaining proportion of the power of the light wave guided by the first optical waveguide is sent to the third optical waveguide.

Embodiment 165: A computing device comprising: a plurality of optical waveguides encoding a set of input values representing elements of an input vector on a corresponding optical signal carried by the optical waveguide; a set of optical amplitude modulators coupled to the optical signal to convert A set of coefficients representing matrix elements is encoded as an amplitude modulation level, and at least one optical amplitude modulator including one input port and two output ports provides a pair of correlated optical signals from the two output ports, such that the correlated optical signals The difference between the amplitudes corresponds to the result of multiplying the input value by the element value of the signed matrix; and a complex summation module encoding a set of output values representing the elements of the output vector on the corresponding electrical signal, wherein at least one telecommunication signal The signal is the form of a current whose amplitude corresponds to the corresponding element of the input vector multiplied by the sum of the corresponding elements of a column (row) of the matrix.

Embodiment 166: A computational method for multiplying an input vector by a given matrix comprising: encoding a set of input values representing elements of the input vector on corresponding optical signals of a set of optical signals; coupling a first set of one or more devices to a first set of one or more waveguides, providing a first subset of the set of optical signals, and generating a first sub-matrix of a given matrix multiplied by results in values on the first subset; coupling a second set of one or more devices to a second set of one or more waveguides, providing a second subset of the set of optical signals, and producing a second set of the given matrix a result of multiplying the sub-matrix by a value on a second subset of the set of optical signals; coupling a third set of one or more devices to a third set of one or more waveguides to provide the output generated by the first optical splitter a copy of the first subset of the set of optical signals, and produce a result of multiplying the third submatrix of the given matrix by the value on the first subset of the set of optical signals; converting the fourth set of one or more devices coupled to a fourth set of one or more waveguides, providing a copy of a second subset of the set of optical signals produced by the second optical splitter, and producing a fourth submatrix of the given matrix multiplied by the set of optical signals a result of values on a second subset of the signal; wherein the first, second, third and fourth submatrices connected together form a given matrix; and wherein at least one output value representing an element of the output vector is encoded in On the electrical signal, the output vector corresponds to the input vector multiplied by the given matrix. The electrical signal is generated by the device communicating with the first group of one or more devices and the second group of one or more devices.

Embodiment 167: The computing device of embodiment 166, wherein the first group of one or more devices, the second group of one or more devices, the third group of one or more devices, and the fourth group of one or more devices Each pair of is mutually exclusive.

Embodiment 168: A computing device, comprising: A first group of one or more devices is configured to receive a first group of optical signals and generate a first matrix multiplied by a value encoded on the first group of optical signals; a second group of one or more devices is configured to to receive a second set of optical signals and generate a result of multiplying the second matrix by a value encoded on the second set of optical signals; and a third set of one or more devices configured to receive a third set of optical signals and generate a third set of optical signals. The result of three matrices multiplied by the values encoded on the third set of optical signals; a fourth set of one or more devices configured to receive the fourth set of optical signals and generate a fourth matrix multiplied by the fourth set of optical signals the result of an encoded numerical value; and a configurable connection path within a first group of one or more devices, a second group of one or more devices, a third group of one or more devices, or a fourth group of one or more devices. Between two or more, wherein a first configuration of configurable connection paths is configured to (1) provide a copy of a first set of optical signals as a second set of optical signals, a third set of optical signals, or a fourth set of optical signals at least one, and (2) providing one or more signals from the first set of one or more devices and one or more signals from the second set of one or more devices to the summing module, the summing module The group is configured to generate an electrical signal representing the sum of the values encoded on the signal received by the summation module.

Embodiment 169: A computing device, comprising: a first set of one or more devices configured to receive a first set of optical signals and generate a result based on optical amplitude modulation of the one or more optical signals of the first set of optical signals; a second set of one or more devices configured to receive a second set of optical signals and generate a result based on optical amplitude modulation of the one or more optical signals of the second set of optical signals; The third group of one or more devices is configured to receive the third group of optical signals and generate a result based on the optical amplitude modulation of the one or more optical signals of the third group of optical signals; the fourth group of one or more devices, configured to receive a fourth set of optical signals and produce an optical amplitude modulation result of one or more optical signals based on the fourth set of optical signals; and a configurable connection path between the first set of one or more devices, the second between two or more of a group of one or more devices, a third group of one or more devices, or a fourth group of one or more devices, wherein a first configuration of configurable connection paths is configured to (1) provide a a copy of one set of optical signals as a third set of optical signals, or (2) providing one or more signals from the first set of one or more devices and one or more signals from a second set of one or more devices To the summation module, the summation module is configured to generate an electrical signal representing the sum of the values encoded on the signal received by the summation module.

Embodiment 170: The computing device of embodiment 169, wherein the first group of one or more devices, the second group of one or more devices, the third group of one or more devices, and the fourth group of one or more devices Each pair of groups is mutually exclusive.

Embodiment 171: The computing device of embodiment 169 or 170, wherein the first configuration of configurable connection paths is configured to (1) provide a copy of the first set of optical signals as a third set of optical signals, and (2) convert from One or more signals from the first group of one or more devices and one or more signals from the second group of one or more devices are provided to a summing module configured to generate an electrical signal, the electrical signal Represents the sum of the values encoded on at least two different signals received by the summation module.

Embodiment 172: The computing device of any one of embodiments 169-171, wherein the first configuration of configurable connection paths is configured to provide a copy of the first set of optical signals as a third set of optical signals, and the configurable connection paths The second configuration is configured to provide one or more signals from the first set of one or more devices and one or more signals from the second set of one or more devices to the summing module, the summing module Configured to generate an electrical signal representing a sum of values encoded on the signal received by the summation module.

Embodiment 173: A computing device comprising: a plurality of optical waveguides, wherein a plurality of input values are encoded on corresponding optical signals carried by the optical waveguide; and a plurality of replica modules including at least two for the one or more optical signals. In each of the subsets, a corresponding set of one or more replication modules is configured to divide a subset of one or more optical signals into two or more replicas of the optical signal; a complex multiplication module, including for For each of the at least two copies of a first subset of one or more optical signals, a corresponding multiplication module is configured to multiply the one or more optical signals of the first subset by a or a plurality of numerical values; and one or more summation modules, including results for two or more multiplication modules, the summation module being configured to generate an electrical signal, the electrical signal representing a result of the two or more multiplication modules. A sum of results, wherein the result includes at least one result encoded on an electrical signal and the result is derived from a copy of the optical signal that propagates through no more than a single optical amplitude modulator before being converted into an electrical signal.

Embodiment 174: A computing system, comprising: a first unit configured to generate a complex modulator control signal; A processor including: a light source configured to provide a plurality of light outputs; a plurality of light modulators coupled to the light source and the first unit, the light modulator being configured to control a signal based on the modulator, the modulation being provided by the light source a light output to generate a light input vector, the light input vector including a complex light signal; and a matrix multiplication unit coupled to the light modulator and the first unit, the matrix multiplication unit being configured to control the light input based on the complex weight signal the vector is converted into an analog output vector; a second unit is coupled to the matrix multiplication unit, and the second unit is configured to convert the analog output vector into a digital output vector; and a controller, including an integrated circuit, is configured to perform the following operations : Receive an artificial neural network calculation request, the artificial neural network calculation request includes an input data set, the input data set includes a first digital input vector; receive a first plurality of neural network weights; and through the first unit, based on the first digital The input vector generates a first plurality of modulator control signals, and a first plurality of weight control signals are generated based on the first plurality of neural network weights.

Embodiment 175: The computing system of embodiment 174, wherein the first unit includes a digital-to-analog converter (DAC).

Embodiment 176: The computing system of embodiment 174 or 175, wherein the second unit includes an analog-to-digital converter (ADC).

Embodiment 177: The computing system of any one of Embodiments 174-176, including a memory unit configured to store a data set and a complex neural network weight.

Embodiment 178: The computing system of Embodiment 177, wherein the integrated circuit of the controller is further configured to perform operations including storing the input data set and the first plurality of neural network weights in the memory unit.

Embodiment 179: The computing system of any one of embodiments 174-178, wherein the first unit is configured to generate the weight control signal.

Embodiment 180: The computing system of any one of Embodiments 174 to 178, wherein the controller includes an application specific integrated circuit (ASIC), and the step of receiving the artificial neural network calculation request includes receiving the artificial neural network calculation request from the general purpose data processor. Network computing requests.

Embodiment 181: The computing system of any one of embodiments 174 to 178, wherein the first unit, the processing unit, the second unit, and the controller are disposed on at least one of a multi-chip module or an integrated circuit, And the step of receiving the artificial neural network calculation request includes receiving the artificial neural network calculation request from a second data processor, wherein the second data processor can be external to the multi-chip module or integrated circuit, and the second data processor can The processing unit is coupled to the multi-chip module or integrated circuit through a communication channel, and the processing unit can process data at a data rate that is at least one order of magnitude greater than the data rate of the communication channel.

Embodiment 182: The computing system of embodiment 174, wherein the first unit, the processing unit, the second unit, and the controller are operable for an optoelectronic processing cycle repeated in a plurality of iterations, and the optoelectronic processing cycle includes: (1) based on The modulator controls at least a first optical modulation operation of at least one of the signals and controls at least a second optical modulation operation of at least one of the signals based on the weight, and (2) (a) an electrical summation operation or (b) At least one of the electrical storage operations.

Embodiment 183: The computing system of Embodiment 182, wherein the photoelectric processing cycle includes electrical storage operations, and the electrical storage operations are performed using a memory unit coupled to the controller, wherein the operations performed by the controller further include The input data set and the first plurality of neural network weights are stored in the memory unit.

Embodiment 184: The computing system of Embodiment 182, wherein the photoelectric processing loop includes an electrical summing operation, and the electrical summing operation is performed using an electrical summing module within the matrix multiplication unit, wherein the electrical summing module is configured To produce a current corresponding to an element of the analog output vector, the current represents the sum of the corresponding element of the light input vector multiplied by the corresponding neural network weight.

Embodiment 185: The computing system of Embodiment 182, wherein the photoelectric processing loop includes at least one signal path on which no more than one first optical signal is executed in a single loop iteration based on at least one of the modulator control signals. modulation operations, and performing no more than one second light modulation operation in a single loop iteration based on at least one of the weighted control signals.

Embodiment 186: The computing system of Embodiment 185, wherein the first light modulation operation is performed by one of the light modulator coupled to the light source and the matrix multiplication unit of the light output, and the second light modulation operation It is performed by the light modulator included in the matrix multiplication unit.

Embodiment 187: The computing system of Embodiment 182, wherein the photoelectric processing loop includes at least one signal path on which no more than one electrical storage operation is performed in a single loop iteration.

Embodiment 188: The computing system of embodiment 174, wherein the light source includes a laser unit configured to generate a light output.

Embodiment 189: The computing system of Embodiment 174, wherein the matrix multiplication unit includes: an input waveguide array for receiving the light input vector, and the light input vector includes the first optical signal array; and an optical interference unit that is in optical communication with the input waveguide array. , for performing linear conversion of the light input vector into the second optical signal array; and an output waveguide array in optical communication with the optical interference unit for guiding the second optical signal array, wherein at least one input in the input waveguide array The waveguide is in optical communication with each output waveguide in the output waveguide array through the optical interference unit.

Embodiment 190: The computing system of Embodiment 189, wherein the optical interference unit includes: a plurality of interconnected MZIs, each of the interconnected MZIs including: a first phase shifter configured to change a separation ratio of the MZIs; and The second phase shifter is configured to shift the phase of an output of the MZI, wherein the first phase shifter and the second phase shifter are coupled to the weight control signal.

Embodiment 191: The computing system of Embodiment 174, wherein the matrix multiplication unit includes: complex replication modules, wherein each replication module corresponds to a subset of one or more optical signals of the optical input vector and is configured to convert a or a plurality of subsets of the optical signal into two or more copies of the optical signal; complex multiplication modules, wherein each multiplication module corresponds to one or more subsets of the optical signal and is configured to use optical amplitude modulation to one or more optical signals of a subset multiplied by one or more matrix element values; and one or more summation modules, wherein each summation module is configured to generate an electrical signal, the electrical signal represents a value of the multiplication module The sum of two or more results.

Embodiment 192: The computing system of Embodiment 191, wherein at least one multiplication module includes an optical amplitude modulator, the optical amplitude modulator includes an input port and two output ports, and provides a pair of correlation signals from the two output ports. optical signals such that the difference between the amplitudes of the associated optical signals corresponds to the result of multiplying the input value by the signed matrix element value.

Embodiment 193: The computing system of embodiment 191 or 192, wherein the matrix multiplication unit is configured to multiply the light input vector by a matrix including one or more matrix element values.

Embodiment 194: The computing system of Embodiment 193, wherein a set of multiple output values is encoded on a corresponding electrical signal generated by one or more summation modules, And the output values in a set of multiple output values represent elements of the output vector, which is generated by multiplying the light input vector by the matrix.

Embodiment 195: The computing system of any one of embodiments 174 to 194, wherein the computing system includes a memory unit configured to store the input data set and the neural network weights, the second unit including an analog-to-digital converter (ADC) unit, and the operation further includes: obtaining a first plurality of digital outputs corresponding to the analog output vector of the matrix multiplication unit from the analog-to-digital converter unit, and the first plurality of digital outputs form a first digital output vector; performing a non-linear transformation on the output vector to generate a first transformed digital output vector; and storing the first transformed digital output vector in the memory unit.

Embodiment 196: The computing system of Embodiment 195, wherein the computing system has a first cycle, and the first cycle is defined as the step of storing the input data set and the first plurality of neural network weights in the memory unit. The time elapsed between steps of storing the first converted digital output vector in the memory unit, and wherein the first cycle period is less than or equal to 1 ns.

Embodiment 197: The computing system of embodiment 195 or 196, wherein the operation further includes: outputting an artificial neural network output generated based on the first converted digital output vector.

Embodiment 198: The computing system of any one of embodiments 195 to 197, wherein the first unit includes a digital-to-analog converter (DAC) unit, and the operations further include: Through the digital-to-analog converter unit, a second plurality of modulator control signals are generated based on the first converted digital output vector.

Embodiment 199: The computing system of any one of embodiments 195 to 198, wherein the first unit includes a digital-to-analog converter (DAC) unit, the artificial neural network calculation request further includes a second plurality of neural network weights, and The operation further includes: based on obtaining the first plurality of digital outputs, generating a second plurality of weight control signals based on a second plurality of neural network weights through a digital-to-analog converter unit.

Embodiment 200: The computing system of Embodiment 199, wherein the first plurality of neural network weights and the second plurality of neural network weights correspond to different layers of the artificial neural network.

Embodiment 201: The computing system of any one of embodiments 195 to 200, wherein the first unit includes a digital-to-analog converter (DAC) unit, and the input data set further includes a second digital input vector, and wherein the operations further include: Generate a second plurality of modulator control signals based on the second digital input vector through the digital-to-analog converter unit; obtain a second plurality of digital outputs corresponding to the analog output vectors of the matrix multiplication unit from the analog-to-digital converter unit, and obtain the second plurality of modulator control signals based on the second digital input vector. digital outputs to form a second digital output vector; performing a nonlinear transformation on the second digital output vector to generate a second transformed digital output vector; storing the second converted digital output vector in the memory unit; and outputting an artificial neural network output generated based on the first converted digital output vector and the second converted digital output vector, wherein the analog output vector of the matrix multiplication unit can be generated based on the second converted digital output vector. A second light input vector generated by the plurality of modulator control signals is generated, and the second light input vector is converted by the matrix multiplication unit based on the first mentioned weight control signal.

Embodiment 202: The computing system of any one of embodiments 174 to 201, wherein the computing system includes a memory unit configured to store the input data set and the neural network weights, and the second unit includes an analog-to-digital conversion The converter (ADC) unit, and the computing system further includes: an analog nonlinear unit, which is disposed between the matrix multiplication unit and the analog-to-digital converter unit. The analog nonlinear unit can be configured to receive a complex output voltage from the matrix multiplication unit, apply non-linear linear transfer function, and outputting the complex conversion output voltage to the analog-to-digital converter unit, wherein the operations performed by the integrated circuit of the controller may further include: obtaining the first plurality of conversion digital outputs corresponding to the conversion output voltage from the analog-to-digital converter unit voltage, the first plurality of converted digital output voltages forming a first converted digital output vector; and storing the first converted digital output vector in the memory unit.

Embodiment 203: The computing system of any one of embodiments 174-202, wherein the integrated circuit of the controller is configured to generate a first plurality of modulator control signals at a frequency greater than or equal to 8 GHz.

Embodiment 204: The computing system of any one of embodiments 174 to 190, wherein the first unit includes a digital-to-analog converter (DAC) unit, the second unit may include an analog-to-digital converter (ADC) unit, and the matrix multiplication unit includes : the optical matrix multiplication unit, coupled to the optical modulator and the digital-to-analog converter unit, the optical matrix multiplication unit configured to convert the optical input vector into the optical output vector based on the weight control signal; and the optical detection unit, coupled to The light matrix multiplication unit is configured to generate a complex output voltage corresponding to the light output vector.

Embodiment 205: The computing system of Embodiment 204 further includes: an analog memory unit disposed between the DAC unit and the light modulator, the analog memory unit is configured to store a complex analog voltage and output the stored analog voltage. voltage; and an analog nonlinear unit disposed between the light detection unit and the ADC unit, the analog nonlinear unit configured to receive an output voltage from the light detection unit, apply a nonlinear transfer function, and output a complex conversion output voltage.

Embodiment 206: The computing system of embodiment 205, wherein the analog memory unit includes a complex capacitor.

Embodiment 207: The computing system of embodiment 205 or 206, wherein the analog memory unit is configured to receive and store the converted output voltage of the analog nonlinear unit, and output the stored converted output voltage to the light modulator, and operate May include: Based on generating the first plurality of modulator control signals and the first plurality of weight control signals, the conversion output voltage of the analog nonlinear unit is stored in the analog memory unit; the stored conversion output voltage is output through the analog memory unit; from the analog memory unit The digital converter unit obtains a second plurality of converted digital output voltages, and the second plurality of converted digital output voltages form a second converted digital output vector; and stores the second converted digital output vector in the memory unit.

Embodiment 208: The computing system of embodiment 204, wherein the computing system may include a memory unit configured to store an input data set and a neural network weight, and the input data set requested by the artificial neural network calculation may include a plurality of digit inputs. A vector, wherein the light source is configured to generate a complex number of wavelengths, wherein the light modulator includes: a complex number of light modulator groups configured to generate a complex number of light input vectors, each light modulator group corresponding to a wavelength, and generating a light having a corresponding wavelength corresponding optical input vectors; and an optical multiplexer configured to combine the optical input vectors into a combined optical input vector including wavelengths, wherein the light detection unit may be further configured to demultiplex the wavelengths and generate a complex multiplex path decomposes the output voltage, and operations therein include: The complex digital demultiplexed light output is obtained from the analog-to-digital converter unit, and the digital demultiplexed light output forms a complex first digital output vector, where each first digital output vector corresponds to a wavelength; execute for each first digital output vector nonlinear conversion to generate a complex converted first digital output vector; and storing the converted first digital output vector in a memory unit, wherein each digital input vector corresponds to an optical input vector.

Embodiment 209: The computing system of embodiment 174, wherein the computing system includes a memory unit configured to store the input data set and the neural network weights, the second unit includes an analog-to-digital converter (ADC) unit, and the artificial neural network The path calculation request may include a complex digital input vector, wherein the light source is configured to generate a complex number of wavelengths, and wherein the optical modulators include: a complex optical modulator group configured to generate a complex optical input vector, each optical modulator group corresponding to one wavelength, and generating a corresponding optical input vector having a corresponding wavelength; and an optical multiplexer configured to combine the optical input vectors into a combined optical input vector including the wavelength, and the operations may include: from the analog-to-digital converter The unit obtains a first plurality of digital light outputs corresponding to a light output vector, the light output vector includes a wavelength, and the first plurality of digital light outputs form a first digital output vector; Performing a non-linear transformation on the first digital output vector to generate a first transformed digital output vector; and the first transformed digital output vector in the memory unit.

Embodiment 210: The computing system of any one of embodiments 174 to 209, wherein the first unit includes a digital-to-analog converter (DAC) unit, the second unit may include an analog-to-digital converter (ADC) unit, and the digital-to-analog converter The converter unit may include: a 1-bit digital-to-analog converter subunit configured to generate a complex 1-bit modulator control signal, wherein the resolution of the analog-to-digital converter unit may be 1 bit, wherein the first digital input vector The resolution may be N bits, and the operation further includes: decomposing the first digit input vector into N 1-bit input vectors, each of the N 1-bit input vectors corresponding to N of the first digit input vector One of the bits; a sequence of N 1-bit modulator control signals corresponding to N 1-bit input vectors is generated through the 1-bit digital-to-analog converter subunit; the corresponding N 1s are obtained from the analog-to-digital converter unit A sequence of N digital 1-bit light outputs of a sequence of bit modulator control signals; constructing an N-bit digital output vector from a sequence of N digital 1-bit light outputs; Perform a non-linear transformation on the constructed N-bit digital output vector to generate a transformed N-bit digital output vector; and store the transformed N-bit digital output vector in a memory unit.

Embodiment 211: The computing system of any one of embodiments 174-210, wherein the computing system includes a memory unit configured to store the input data set and the neural network weights, and the memory unit includes: a digital input vector memory , is configured to store the first digital input vector and includes at least one static random access memory; and the neural network weight memory is configured to store the neural network weights and includes at least one dynamic random access memory.

Embodiment 212: The computing system of any one of embodiments 174 to 211, wherein the first unit includes a digital-to-analog converter (DAC) unit, the digital-to-analog converter unit includes: a first digital-to-analog converter subunit, configured to generate a modulator control signal; and a second digital-to-analog converter subunit configured to generate a weight control signal, wherein the first digital-to-analog converter subunit and the second digital-to-analog converter subunit are different.

Embodiment 213: The computing system of any one of embodiments 174-212, wherein the light source includes: a laser source configured to generate light; and An optical power splitter is configured to split light generated by the laser source into optical outputs, each optical output having approximately the same power.

Embodiment 214: The computing system of any one of embodiments 174 to 213, wherein the optical modulator includes one of an MZI modulator, a ring resonance modulator, or an electroabsorption modulator.

Embodiment 215: The computing system of Embodiment 204, wherein the light detection unit includes: a plurality of light detectors; and a plurality of amplifiers configured to convert the photocurrent generated by the light detector into an output voltage.

Embodiment 216: The computing system of any one of embodiments 174-215, wherein the integrated circuit is a special application integrated circuit.

Embodiment 217: The computing system of any one of embodiments 174 and 191 to 194, including a complex optical waveguide coupled between an optical modulator and a matrix multiplication unit, wherein the optical input vector includes a set of multiple Input value, a set of multiple input values is encoded on the corresponding optical signal carried by the optical waveguide, and each optical signal carried by an optical waveguide includes light waves with a common wavelength, and the common wavelength is approximately the same in all optical signals. same.

Embodiment 218: The computing system of any one of embodiments 191 to 194 and 217, wherein the replica module includes at least one replica module having an optical splitter that transmits a determined proportion of the power of the light wave at the input port. to the first output port, and the remaining proportion of the power of the light wave is sent to the second output port at the input port.

Embodiment 219: The computing system of Embodiment 218, wherein the optical splitter includes a waveguide optical splitter that sends a determined proportion of the power of the optical wave guided by the input optical waveguide to the first output optical waveguide and is The remaining proportion of the power of the light wave guided by the optical waveguide is sent to the second output optical waveguide.

Embodiment 220: The computing system of Embodiment 219, wherein the guided mode of the input optical waveguide is adiabatically coupled to the complex guided modes of each of the first and second output optical waveguides.

Embodiment 221: The computing system of any one of embodiments 218 to 220, wherein the optical splitter includes a beam splitter including at least one surface that transmits a determined proportion of the power of the light wave at the input port, and at the input port The remaining fraction of the power of the light wave reflected by the port.

Embodiment 222: The computing system of any one of embodiments 217-221, wherein at least one optical waveguide includes an optical fiber coupled to an optical coupler that couples a guided mode of the optical fiber to a free space propagation mode.

Embodiment 223: The computing system of any one of embodiments 174, 191 to 194, and 217 to 222, wherein the multiplication module includes at least one coherence sensitive multiplication module, the coherence sensitive multiplication module configured to interference using optical amplitude modulation to multiply one or more optical signals of a first subset by one or more matrix element values, the light waves having a coherence length that is at least as long as the propagation distance through the coherence sensitive multiplication module .

Embodiment 224: The computing system of Embodiment 223, wherein the coherence-sensitive multiplication module includes a Mach-Zehnder interferometer (MZI), which divides the light waves guided by the input optical waveguide into the first The optical waveguide arm and the second optical waveguide arm of the Mach-Zehnder interferometer, the first optical waveguide arm includes a phase shifter, the phase shifter generates a relative phase shift with respect to the phase delay of the second optical waveguide arm, and the Mach-Zehnder interferometer The plurality of light waves from the first optical waveguide arm and the second optical waveguide arm are combined into at least one output optical waveguide.

Embodiment 225: The computing system of Embodiment 224, wherein the MZI combines the complex light waves from the first optical waveguide arm and the second optical waveguide arm into each of the first output optical waveguide and the second output optical waveguide, A photodetector receives a light wave from the first output optical waveguide to generate a first photocurrent, a second photodetector receives a light wave from the second output optical waveguide to generate a second photocurrent, and the result of the coherent sensitive multiplication module includes The difference between the first photocurrent and the second photocurrent.

Embodiment 226: The computing system of any one of embodiments 223 to 225, wherein the coherence sensitive multiplication module includes one or more ring resonators, the ring resonators including at least one ring resonator coupled to the first optical waveguide and at least one ring resonator coupled to the second optical waveguide.

Embodiment 227: The computing system of Embodiment 226, wherein the first photodetector receives the light wave from the first optical waveguide to generate the first photocurrent, and the second photodetector receives the light wave from the second optical waveguide, to generate a second photocurrent, and the result of the coherent sensitive multiplication module includes a difference between the first photocurrent and the second photocurrent.

Embodiment 228: The computing system of any one of embodiments 174, 191 to 194, and 217 to 227, wherein the multiplication module includes at least one coherent insensitive multiplication module, the coherent insensitive multiplication module configured to For energy absorption, optical amplitude modulation is used to multiply one or more optical signals of the first subset by one or more matrix element values.

Embodiment 229: The computing system of embodiment 228, wherein the coherent insensitive multiplication module includes an electroabsorption modulator.

Embodiment 230: The computing system of any one of embodiments 174, 191-194, and 217-229, wherein the one or more summation modules include at least one summation module having the following components: (1) two or A plurality of input conductors, each input conductor carrying an electrical signal in the form of an input current, the amplitude of the input current representing a corresponding result of a multiplication module, and (2) at least one output conductor, the output conductor carrying a corresponding signal in the form of an output current The resulting sum of electrical signals, the output current is proportional to the sum of the input currents.

Embodiment 231: The computing system of embodiment 230, wherein the two or more input conductors and the output conductors include a plurality of conductors meeting at one or more junctions between the conductors, and the output current is substantially equal to the sum of the input currents.

Embodiment 232: The computing system of embodiment 230 or 231, wherein at least a first input current of the input current is provided in the form of at least one photocurrent, the photocurrent is generated by at least one photodetector, and the photodetector receives The optical signal generated by the first multiplication module of the multiplication module.

Embodiment 233: The computing system of embodiment 232, wherein the first input current is provided in the form of a difference between two photocurrents, the two photocurrents are generated by different corresponding photodetectors, and the photodetectors receive Different corresponding optical signals generated by the first multiplication module.

Embodiment 234: The computing system of any one of embodiments 174-233, wherein one of the copies of the first subset of the one or more optical signals consists of a single optical signal, wherein an input value on the single optical signal is encoded.

Embodiment 235: The computing system of Embodiment 234, wherein the multiplication module corresponding to the copy of the first subset multiplies the encoded input value by a single matrix element value.

Embodiment 236: The computing system of any of embodiments 174, 191-194, and 217-235, wherein one of the copies of the first subset of the one or more optical signals includes more than one optical signal, and Less than the number of all optical signals for which multiple input values are encoded.

Embodiment 237: The computing system of embodiment 236, wherein the multiplication module corresponding to the copy of the first subset multiplies the encoded input value by a different corresponding matrix element value.

Embodiment 238: The computing system of Embodiment 237, wherein different multiplication modules corresponding to different corresponding copies of the first subset of the one or more optical signals are included in different devices, and the different devices perform optical communication between the different devices. one of the copies of the first subset of one or more optical signals.

Embodiment 239: The computing system of any one of embodiments 174, 191 to 194, and 217 to 238, wherein two or more of the optical waveguides, two or more of the replica modules At least one of a plurality of, two or more multiplication modules, and one or more summation modules is disposed on a substrate of a common device.

Embodiment 240: The computing system of embodiment 239, wherein the device performs vector matrix multiplication, wherein the input vectors are provided as a set of optical signals and the output vectors are provided as a set of electrical signals.

Embodiment 241: The computing system as in any one of Embodiments 174, 191 to 194, and 217 to 240, further comprising an accumulator, the accumulator integrates the input electrical signal corresponding to the output of the multiplication module or the summation module, wherein The input electrical signal is encoded using time domain coding using switching amplitude modulation within each of a plurality of time slots, and the accumulator can generate an output electrical signal with more than two amplitude levels. Encoding, amplitude levels correspond to different duty cycles of time domain encoding on multiple time slots.

Embodiment 242: The computing system of any one of embodiments 174, 191-194, and 217-241, wherein each of the two or more multiplication modules corresponds to a different subset of the one or more optical signals.

Embodiment 243: The computing system of any one of embodiments 174, 191 to 194, and 217 to 242, further comprising each replica of the second subset of the one or more optical signals, and the one or more optical signals. The optical signals in the first subset of signals are different, and the multiplication module is configured to multiply the one or more optical signals in the second subset by one or more matrix element values using optical amplitude modulation.

Embodiment 244: A computing system, including: a memory unit configured to store a data set and a complex neural network weight; a driver unit configured to generate a complex modulator control signal; An optoelectronic processor, including: a light source configured to provide a plurality of light outputs; a plurality of light modulators coupled to the light source and the driver unit, the light modulator being configured to control a signal based on the modulator, the modulation generated by the light source a light output to generate a light input vector; a matrix multiplication unit coupled to the light modulator and the driver unit, the matrix multiplication unit being configured to convert the light input vector into an analog output vector based on the complex weight control signal; and a comparator unit , coupled to the matrix multiplication unit and configured to convert the analog output vector into a complex digital 1-bit output; and a controller, including an integrated circuit, configured to perform the following operations: receive an input data set and a first plurality of An artificial neural network calculation request for neural network weights, wherein the input data set includes a first digital input vector with N-bit resolution; storing the input data set and the first plurality of neural network weights in the memory unit; The first digital input vector is decomposed into N 1-bit input vectors, each N 1-bit input vector corresponds to one of the N bits of the first digital input vector; the corresponding N 1-bit inputs are generated through the driver unit A sequence of N 1-bit modulator control signals of the vector; Obtain a sequence of N digital 1-bit outputs corresponding to a sequence of N 1-bit modulator control signals from the comparator unit; construct an N-bit digital output vector from the sequence of N digital 1-bit light outputs; construct performing a nonlinear conversion on the N-bit digital output vector to generate a converted N-bit digital output vector; and storing the converted N-bit digital output vector in a memory unit.

Embodiment 245: The computing system of embodiment 244, wherein receiving the artificial neural network calculation request includes receiving the artificial neural network calculation request from a general purpose computer.

Embodiment 246: The computing system of embodiment 244, wherein the driver unit is configured to generate the weight control signal.

Embodiment 247: The computing system of Embodiment 244, wherein the matrix multiplication unit includes: an optical matrix multiplication unit coupled to the optical modulator and the driver unit, the optical matrix multiplication unit being configured to convert the light input vector based on the weight control signal into a light output vector; and a light detection unit coupled to the light matrix multiplication unit and configured to generate a complex output voltage corresponding to the light output vector.

Embodiment 248: The computing system of Embodiment 244, wherein the matrix multiplication unit includes: an input waveguide array for receiving light input vectors; an optical interference unit in optical communication with the input waveguide array for performing linear conversion of the optical input vector into the second optical signal array; and an output waveguide array in optical communication with the optical interference unit for guiding the second optical signal array, At least one input waveguide in the input waveguide array is in optical communication with each output waveguide in the output waveguide array through the optical interference unit.

Embodiment 249: The computing system of Embodiment 248, wherein the optical interference unit includes: a plurality of interconnected MZIs, each MZI of the interconnected MZIs including: a first phase shifter configured to change the separation ratio of the MZI; and A two-phase shifter is configured to shift a phase of an output of the MZI, wherein the first phase shifter and the second phase shifter can be coupled to the weight control signal.

Embodiment 250: The computing system of embodiment 244, wherein the matrix multiplication unit includes: a complex replication module including one or more for each of at least two subsets of the one or more optical signals of the optical input vector A respective set of replication modules configured to divide a subset of one or more optical signals into two or more replicas of the optical signal; a complex multiplication module including a first subset for the one or more optical signals In each of the at least two copies, a corresponding multiplication module is configured to multiply the one or more optical signals of the first subset by one or more matrix element values using optical amplitude modulation; and one or more A summation module includes results for two or more multiplication modules, the summation module is configured to generate an electrical signal, and the electrical signal represents the sum of the results of the two or more multiplication modules.

Embodiment 251: The computing system of embodiment 250, wherein at least one multiplication module includes an optical amplitude modulator, the optical amplitude modulator includes an input port and two output ports, and provides a pair of correlation signals from the two output ports. optical signals such that the difference between the amplitudes of the associated optical signals corresponds to the result of multiplying the input value by the signed matrix element value.

Embodiment 252: The computing system of embodiment 250 or 251, wherein the matrix multiplication unit is configured to multiply the light input vector by a matrix including one or more matrix element values.

Embodiment 253: The computing system of embodiment 252, wherein a set of a plurality of output values is encoded on a corresponding electrical signal generated by one or more summing modules, and the output value of the set of the plurality of output values is Represents the element of the output vector generated by multiplying the light input vector by a matrix.

Embodiment 254: A computing method for performing artificial neural network calculations in a system having a matrix multiplication unit configured to convert an optical input vector into an analog output vector based on a complex weight control signal, the computing method including: receiving An artificial neural network calculation request including an input data set and a first plurality of neural network weights, wherein the input data set includes a first digital input vector; storing the input data set and the first plurality of neural network weights in a memory unit ; Generate a first plurality of modulator control signals based on the first digital input vector, and generate a first plurality of weight control signals based on the first plurality of neural network weights; obtain a first plurality of output vectors corresponding to the matrix multiplication and generation unit; digital outputs, the first plurality of digital outputs forming a first digital output vector; performing a nonlinear conversion on the first digital output vector by the controller to generate the first converted digital output vector; storing the first converted digital output vector in the memory unit; and using the controller to output the first converted digital output vector based on The resulting artificial neural network output.

Embodiment 255: The calculation method of Embodiment 254, wherein receiving the artificial neural network calculation request includes receiving the artificial neural network calculation request from the computer through the communication channel.

Embodiment 256: The calculation method of embodiment 254 or 255, wherein generating the first plurality of modulator control signals includes generating the first plurality of modulator control signals through a digital-to-analog converter (DAC) unit.

Embodiment 257: The computing method of any one of embodiments 254-256, wherein obtaining the first plurality of digital outputs includes obtaining the first plurality of digital outputs from an analog-to-digital converter (ADC) unit.

Embodiment 258: The calculation method of Embodiment 257, including: applying a first plurality of modulator control signals to a plurality of optical modulators coupled to the light source and the DAC unit; and using the optical modulator based on the modulator The control signal modulates the light output generated by the laser unit to generate a light input vector.

Embodiment 259: The calculation method of Embodiment 258, wherein the matrix multiplication unit is coupled to the light modulator and the DAC unit, and the calculation method includes: A matrix multiplication unit is used to convert the optical input vector into an analog output vector based on the weighted control signal.

Embodiment 260: The calculation method of Embodiment 259, wherein the ADC unit is coupled to the matrix multiplication unit, and the calculation method includes: using the ADC unit to convert the analog output vector into the first plurality of digital outputs.

Embodiment 261: The calculation method of Embodiment 259 or 260, wherein the matrix multiplication unit includes an optical matrix multiplication unit coupled to the optical modulator and the DAC unit, and converting the optical input vector into the analog output vector includes using the optical matrix multiplication unit. The light input vector is converted into a light output vector based on the weight control signal, and the calculation method may include: using a light detection unit coupled to the light matrix multiplication unit to generate a complex output voltage corresponding to the light output vector.

Embodiment 262: The calculation method as in Embodiment 254, including: receiving the light input vector at the input waveguide array; using an optical interference unit in optical communication with the input waveguide array to perform linear conversion of the light input vector into the second optical signal array. ; and using an output waveguide array in optical communication with the optical interference unit to guide the second optical signal array, wherein at least one input waveguide in the input waveguide array is in optical communication with each output waveguide in the output waveguide array through the optical interference unit.

Embodiment 263: The calculation method of Embodiment 262, wherein the optical interference unit includes a complex interconnected Mach-Zehnder interferometer (MZI), and each interconnected MZI The MZI may include a first phase shifter and a second phase shifter, and the first phase shifter and the second phase shifter may be coupled to the weight control signal, wherein the calculation method includes: using the first phase shifter to change the MZI separation ratio, and shifting the phase of one output of the MZI using a second phase shifter.

Embodiment 264: The calculation method of Embodiment 258, comprising: for each of at least two subsets of the one or more optical signals of the optical input vector, using a corresponding set of one or more replica modules to or a subset of the optical signal is divided into two or more copies of the optical signal; for each of the at least two copies of the first subset of the one or more optical signals, a corresponding multiplication module is used to use the optical signal Amplitude modulation multiplies one or more optical signals of the first subset by one or more matrix element values; and for the results of the two or more multiplication modules, using a summation module to generate an electrical signal, the electrical signal represents The sum of the results of two or more multiplication modules.

Embodiment 265: The calculation method of Embodiment 264, wherein at least one multiplication module includes an optical amplitude modulator, the optical amplitude modulator includes an input port and two output ports, and provides a pair of correlation signals from the two output ports. optical signals such that the difference between the amplitudes of the associated optical signals corresponds to the result of multiplying the input value by the signed matrix element value.

Embodiment 266: The calculation method of embodiment 264 or 265, including using a matrix multiplication unit to multiply the light input vector by a matrix including one or more matrix element values.

Embodiment 267: The calculation method of Embodiment 266, including encoding a set of multiple output values on corresponding electrical signals generated by one or more summing modules, and using an output value from the set of multiple output values. Represents the element of the output vector. The light output vector is generated by multiplying the input vector by the matrix.

Embodiment 268: A computing method, comprising: providing input information in an electronic format; converting at least a portion of the electronic input information into an optical input vector; optically converting the optical input vector into an analog output vector based on matrix multiplication; and electronically converting the nonlinear Applied to analog output vectors to provide output information in electronic format.

Embodiment 269: The calculation method of Embodiment 268 further includes: repeating electro-optical conversion, photoelectric conversion, and electrically applied nonlinear conversion for new electronic input information corresponding to the output information provided in an electronic format.

Embodiment 270: The calculation method of Embodiment 269, wherein the matrix multiplication for the initial photoelectric conversion and the matrix multiplication for the repeated photoelectric conversion are the same and correspond to the same layer of the artificial neural network.

Embodiment 271: The calculation method of Embodiment 269, wherein the matrix multiplication for initial photoelectric conversion and the matrix multiplication for repeated photoelectric conversion are different and correspond to different layers of the artificial neural network.

Embodiment 272: The calculation method is the same as that of Embodiment 268, further including: For different parts of the electronic input information, repeated electro-optical conversion, photoelectric conversion, and nonlinear conversion of electrical applications, where the matrix multiplication for the initial photoelectric conversion and the matrix multiplication for the repeated photoelectric conversion are the same, and correspond to the artificial neural network. One layer.

Embodiment 273: The computing method of Embodiment 272, further comprising: providing electronic intermediate information in an electronic format based on electronic output information for multiple portions of electronic input information generated by the first layer of the artificial neural network; and for each different part of the electronic intermediate information, repeated electro-optical conversions, photoelectric conversions, and electrically applied nonlinear transformations, where matrix multiplications for the initial photoelectric conversions and matrix multiplications for the repeated photoelectric conversions associated with the different parts of the electronic intermediate information is the same and corresponds to the second layer of the artificial neural network.

Embodiment 274: A calculation method for performing artificial neural network calculations. The calculation method includes: a first unit configured to generate a complex vector control signal and a complex weight control signal; a second unit configured to control based on the vector The signal provides an optical input vector; a matrix multiplication unit coupled to the second unit and the first unit, the matrix multiplication unit configured to convert the optical input vector into an output vector based on the weight control signal; and a controller, including an integrated circuit, is Configure to do the following: receiving an artificial neural network calculation request including an input data set and a first plurality of neural network weights, wherein the input data set includes a first digital input vector; and generating, through the first unit, a first plurality of neural network weights based on the first digital input vector vector control signals and generating a first plurality of weight control signals based on a first plurality of neural network weights; wherein the first unit, the second unit, the matrix multiplication unit and the controller are used for an optoelectronic processing cycle repeated in a plurality of iterations , and the photoelectric processing cycle includes: (1) at least two optical modulation operations, and (2) at least one of (a) electrical summing operations or (b) electrical storage operations.

Embodiment 275: A method for performing artificial neural network calculations, the calculation method includes: providing input information in an electronic format; converting at least a portion of the electronic input information into optical input vectors; and using a set of neural network weights based on matrix multiplication Converting an optical input vector into an output vector; wherein operations and conversion operations are provided to be performed in a photoelectric processing loop using different corresponding sets of neural network weights and different corresponding input information, the photoelectric processing loop is repeated in a plurality of iterations, and the photoelectric processing loop includes (1) At least two optical modulation operations, and (2) at least one of (a) electrical summing operations or (b) electrical storage operations.

Embodiment 276: A computing system, including: a first unit configured to generate a complex modulator control signal; A processing unit including: a light source or port configured to provide a plurality of light outputs; a first set of light modulators coupled to the light source or port and the first unit, the plurality of light modulators in the first set of light modulators configured to modulate an optical output provided by the light source or port based on a plurality of digital input values corresponding to a first set of the modulator control signals to generate an optical input vector, the optical input vector comprising a complex optical signal; and a matrix multiplication unit including a second set of light modulators, wherein the matrix multiplication unit is coupled to the first unit, and the matrix multiplication unit is configured to perform based on modulation applied to the second set of light modulators The light input vector is converted into an analog output vector by using a complex digital weight value corresponding to the second group of modulator control signals in the modulator control signal, wherein at least one of the first group of light modulators or the second group of light modulators At least one optical modulator of one is configured to modulate the optical signal based on a first modulator control signal of the modulator control signals, and the first unit is configured to shape the first modulator control signal to Including bandwidth enhancement associated with amplitude changes associated with a corresponding change in a complex continuous digital value corresponding to the first modulator control signal.

Embodiment 277: The computing system of embodiment 276, further comprising: a second unit coupled to the matrix multiplication unit, and the second unit is configured to convert the analog output vector into a digital output vector; and a controller, including an integrated circuit circuit, configured to perform the following operations: Receive an artificial neural network calculation request, the artificial neural network calculation request includes an input data set, the input data set includes a first digital input vector; receive a first plurality of neural network weights; and through the first unit, based on the first digital input The vector generates a first plurality of modulator control signals and a first plurality of weight control signals based on the first plurality of neural network weights.

Embodiment 278: The computing system of embodiment 276 or 277, wherein the first unit includes a digital-to-analog converter (DAC).

Embodiment 279: The computing system of Embodiment 277, further comprising a memory unit configured to store the data set and the complex neural network weights.

Embodiment 280: The computing system of embodiment 279, wherein the integrated circuit of the controller is further configured to perform operations including storing the input data set and the first plurality of neural network weights in the memory unit.

Embodiment 281: The computing system of any one of embodiments 277 to 280, wherein the controller includes an application specific integrated circuit (ASIC), and the step of receiving the artificial neural network calculation request includes receiving the artificial neural network calculation request from the general purpose data processor. Network computing requests.

Embodiment 282: The computing system of any one of embodiments 277 to 281, wherein the first unit, the processing unit, the second unit, and the controller are disposed on at least one of a multi-chip module or an integrated circuit, and the step of receiving the artificial neural network calculation request includes receiving the artificial neural network calculation request from a second data processor, wherein the second data processor is in the multi-chip module or integrated circuit Outside the circuit, the second data processor is coupled to the multi-chip module or integrated circuit through the communication channel, and the processing unit can process data at a data rate that is at least one order of magnitude greater than the data rate of the communication channel.

Embodiment 283: The computing system of any one of embodiments 277-282, wherein the first unit, the processing unit, the second unit, and the controller are used for an optoelectronic processing loop repeated in a plurality of iterations, the optoelectronic processing loop comprising: (1) at least a first optical modulation operation based on at least one of the modulator control signals, and at least a second optical modulation operation based on at least one of the weight control signals, and (2) (a) electrical At least one of a summation operation or (b) an electrical storage operation.

Embodiment 284: The computing system of Embodiment 283, wherein the photoelectric processing cycle includes electrical storage operations, and the electrical storage operations are performed using a memory unit coupled to the controller, wherein the operations performed by the controller further include The input data set and the first plurality of neural network weights are stored in the memory unit.

Embodiment 285: The computing system of embodiment 283 or 284, wherein the photoelectric processing loop includes an electrical summation operation, and the electrical summation operation is performed using an electrical summation module within the matrix multiplication unit, wherein the electrical summation module is configured to produce a current corresponding to an element of the analog output vector, the current representing the sum of the corresponding element of the optical input vector multiplied by the corresponding neural network weight.

Embodiment 286: The computing system of any one of embodiments 276-285, wherein the first modulator control signal includes a class associated with a complex predetermined amplitude level. ratio signal, and each of the amplitude levels is associated with a different corresponding digital value.

Embodiment 287: The computing system of embodiment 286, wherein the first modulator control signal includes an analog signal associated with two predetermined amplitude levels, and each of the amplitude levels is associated with a different corresponding binary value.

Embodiment 288: The computing system of embodiment 287, wherein the consecutive digit values include complex consecutive binary values in a series of binary values.

Embodiment 289: The computing system of embodiment 288, wherein the controller is configured to increase the amplitude between a first predetermined amplitude level associated with the first time interval and a second predetermined amplitude level associated with the second time interval. The magnitude of the change in amplitude shapes the first modulator control signal to include bandwidth enhancement for an initial portion of the second time interval.

Embodiment 290: The computing system of embodiment 288 or 289, wherein a series of binary values is used to determine the amplitude of the first modulator control signal for modulating the optical signal according to a non-return to zero (NRZ) modulation mode. accurate position.

Embodiment 291: The computing system of any one of embodiments 288 to 290, wherein the first unit is configured to be connected in series by a diode structure of the first modulator in the second set of optical modulators Pumping current between the diode structure and the capacitance of the circuit providing the first modulator control signal shapes the first modulator control signal to include bandwidth enhancement and the amount of charge transferred by the pump current Determined, at least in part, based on a constant voltage over a time period that provides successive digital values.

Embodiment 292: A computing device comprising: a plurality of optical waveguides coupled to a first set of optical amplitude modulators, wherein the first set of optical an amplitude modulator encoding a set of a plurality of input values on a corresponding complex optical signal carried by an optical waveguide; a complex replication module and for each of at least two subsets of the one or more optical signals, replicating a corresponding group of one or more modules configured to divide one or more subsets of the optical signal into two or more copies of the optical signal; a plurality of multiplication modules, each of the multiplication modules including a second group an optical amplitude modulator, and for each of the at least two copies of the first subset of the one or more optical signals, a corresponding one of the multiplication modules is configured to use a second set The optical amplitude modulator of the optical amplitude modulator multiplies one or more optical signals of the first subset by one or more matrix element values; and one or more summation modules, and for two of the multiplication modules or the results of more than one, a corresponding one of the summing module is configured to generate an electrical signal, the electrical signal represents the sum of the results of two or more of the multiplication modules; wherein the first group of optical amplitude modulators or the second group At least one optical amplitude modulator of at least one of the optical amplitude modulators is configured to modulate the optical signal by the modulation value using monotonically increasing power relative to an absolute value of the modulation value.

Embodiment 293: The computing device of embodiment 292, wherein at least one optical amplitude modulator of at least one of the first set of optical amplitude modulators or the second set of optical amplitude modulators includes coherently sensitive optical amplitude modulation The coherent sensitive light amplitude modulator is configured to modulate the light signal by modulating the value based on the interference between complex light waves. The light waves have a coherence length that is at least as long as the propagation distance through the coherent sensitive light amplitude modulator. Same length.

Embodiment 294: The computing device of embodiment 293, wherein the coherently sensitive optical amplitude modulator includes a Mach-Zehnder interferometer (MZI) that separates light waves guided by the input optical waveguide into The first optical waveguide arm and the second optical waveguide arm of the Mach-Zehnder interferometer, the first optical waveguide arm includes an active phase shifter, the phase delay of the active phase shifter relative to the second optical waveguide arm generates a relative phase shift, and the Mach-Zehnder interferometer The Zehnder interferometer combines the plurality of light waves from the first optical waveguide arm and the second optical waveguide arm into at least one output optical waveguide.

Embodiment 295: The computing device of embodiment 294, wherein the power used to modulate the optical signal by the modulation value includes power applied to the active phase shifter.

Embodiment 296: The computing device of embodiment 292, wherein a complex input value in a set of a plurality of input values encoded on a corresponding optical signal represents a complex number of an input vector multiplied by a matrix including one or more matrix element values. element.

Embodiment 297: The computing device of embodiment 296, wherein a plurality of output values are encoded on a plurality of corresponding electrical signals generated by one or more summing modules, and the plurality of complex numbers in the plurality of output values are The output value represents the complex element of the output vector, which is produced by multiplying the input vector by the matrix.

Embodiment 298: The computing device of any one of embodiments 292-297, wherein each of the optical signals carried by the optical waveguide includes an optical wave having a common wavelength, the common wavelength being substantially the same for all optical signals.

Embodiment 299: The computing device of any one of embodiments 292 to 297, wherein the replica module includes at least one replica module having an optical splitter that converts a given amount of power of the light wave at an input port of the replica module. Proportion sent to copy module the first output port, and the remaining proportion of the power of the light wave is sent from the input port of the replication module to the second output port of the replication module.

Embodiment 300: The computing device of embodiment 299, wherein the optical splitter includes a waveguide optical splitter that sends a predetermined proportion of the power of the optical wave guided by the input optical waveguide of the replication module to the first optical waveguide of the replication module. an output optical waveguide, and sends a remaining proportion of the power of the light wave guided by the input optical waveguide of the replication module to a second output optical waveguide of the replication module.

Embodiment 301: The computing device of embodiment 300, wherein the guided mode of the input optical waveguide is adiabatically coupled to the plurality of guided modes of each of the first and second output optical waveguides.

Embodiment 302: The computing device of embodiment 299 or 230, wherein the light splitter includes a beam splitter, the beam splitter includes at least one surface that transmits a determined proportion of the power of the light wave at the input port and reflects a given proportion of the power of the light wave at the input port. remaining proportion of power.

Embodiment 303: The computing device of Embodiment 302, wherein at least one of the optical waveguides includes an optical fiber coupled to an optical coupler that couples a guided mode of the optical fiber to a free space propagation mode.

Embodiment 304: The computing device of any one of embodiments 292 to 303, wherein the multiplication module includes at least one coherently sensitive optical amplitude modulator configured to be based on interference between complex light waves , multiplying one or more optical signals of the first subset by one or more matrix element values, the light wave has a coherence length that is at least as long as the propagation distance through the coherently sensitive optical amplitude modulator.

Embodiment 305: The computing device of Embodiment 304, wherein the coherently sensitive light amplitude modulator includes a Mach-Zehnder interferometer, which divides the light waves guided by the input optical waveguide into the first light of the Mach-Zehnder interferometer. The waveguide arm and the second optical waveguide arm of the Mach-Zehnder interferometer, the first optical waveguide arm includes a phase shifter, the phase shifter generates a relative phase shift with respect to the phase delay of the second optical waveguide arm, and the Mach-Zehnder interferometer will The plurality of light waves from the first optical waveguide arm and the second optical waveguide arm are combined into at least one output optical waveguide.

Embodiment 306: The computing device of Embodiment 305, wherein the Mach-Zehnder interferometer combines the complex light waves from the first optical waveguide arm and the second optical waveguide arm into each of the first output optical waveguide and the second output optical waveguide. One, the first photodetector receives the light wave from the first output optical waveguide to generate the first photocurrent, the second photodetector receives the light wave from the second output optical waveguide to generate the second photocurrent, and is coherently sensitive to light amplitude. The result of the modulator includes the difference between the first photocurrent and the second photocurrent.

Embodiment 307: The computing device of any one of Embodiments 304-306, wherein the coherently sensitive optical amplitude modulator includes one or more ring resonators including at least one ring coupled to the first optical waveguide. The resonator and at least one ring resonator coupled to the second optical waveguide.

Embodiment 308: The computing device of Embodiment 307, wherein the first photodetector receives the light wave from the first optical waveguide to generate the first photocurrent, and the second photodetector receives the light wave from the second optical waveguide, to generate a second photocurrent, and the result of the coherently sensitive optical amplitude modulator includes a difference between the first photocurrent and the second photocurrent.

Embodiment 309: The computing device of any one of embodiments 292 to 308, wherein the multiplication module includes at least one coherent insensitive optical amplitude modulator configured to be based on energy within the light wave. Absorption multiplies a first subset of one or more optical signals by one or more matrix element values.

Embodiment 310: The computing device of embodiment 309, wherein the coherent insensitive optical amplitude modulator includes an electroabsorption modulator.

Embodiment 311: The computing device of any one of embodiments 292 to 310, wherein the one or more summing modules include at least one summing module having the following components: (1) two or more input conductors, an input each of the conductors carrying an electrical signal in the form of an input current, the magnitude of the input current representing a corresponding result of a corresponding one of the multiplication modules, and (2) at least one output conductor carrying a corresponding result in the form of an output current The sum of electrical signals, the output current is proportional to the sum of the input currents.

Embodiment 312: The computing device of Embodiment 311, wherein the two or more input conductors and the output conductors comprise a plurality of conductors meeting at one or more junctions between the conductors, and the output current is substantially equal to the sum of the input currents.

Embodiment 313: The computing device of embodiment 311 or 312, wherein at least a first input current of the input current is provided in the form of at least one photocurrent, the photocurrent is generated by at least one photodetector, and the photodetector receives The optical signal generated by the first multiplication module of the multiplication module.

Embodiment 314: The computing device of embodiment 313, wherein the first input current is provided in the form of a difference between two photocurrents, the two photocurrents are generated by different corresponding photodetectors, and the photodetectors receive The out-of-phase generated by the first multiplication module Respond to light signals.

Embodiment 315: The computing device of any one of embodiments 292-314, wherein one of the copies of the first subset of the one or more optical signals consists of a single optical signal, wherein the input value on the single optical signal One is coded.

Embodiment 316: The computing device of embodiment 315, wherein the multiplication module corresponding to the copy of the first subset multiplies the encoded input value by a single matrix element value.

Embodiment 317: The computing device of any one of embodiments 292-316, wherein one of the copies of the first subset of the one or more optical signals includes more than one optical signal and less than all optical signals. A quantity in which multiple input values of an optical signal are encoded.

Embodiment 318: The computing device of embodiment 317, wherein the multiplication module corresponding to the copy of the first subset multiplies the encoded input value by different corresponding matrix element values.

Embodiment 319: The computing device of Embodiment 318, wherein different multiplication modules corresponding to different corresponding copies of the first subset of the one or more optical signals are included in different devices, and the different devices perform optical communication between the different devices. one of the copies of the first subset of one or more optical signals.

Embodiment 320: The computing device of Embodiment 319, wherein two or more of the optical waveguides, two or more of the replication modules, two or more of the multiplication modules, and at least one of the one or more summing modules One is provided on the base plate of the common device.

Embodiment 321: The computing device of Embodiment 320, wherein the common device performs vector matrix multiplication, wherein the input vectors are provided as a set of optical signals, and Provides the output vector as a set of electrical signals.

Embodiment 322: The computing device of any one of embodiments 292 to 321, further comprising an accumulator integrating input electrical signals corresponding to outputs of one of the multiplication modules or one of the summation modules, wherein using Time domain encoding encodes the input electrical signal, the time domain encoding uses switching amplitude modulation within each of a plurality of time slots, and the accumulator generates an output electrical signal, the output electrical signal is encoded with more than two amplitude levels, The amplitude levels correspond to different duty cycles of the time domain encoding on multiple time slots.

Embodiment 323: The computing device of any one of embodiments 292-322, wherein each of the two or more multiplication modules corresponds to a different subset of the one or more optical signals.

Embodiment 324: The computing device of any one of embodiments 292-323, further comprising each replica of the second subset of the one or more optical signals and the first subset of the one or more optical signals. The multiplication module is configured to multiply one or more optical signals of the second subset by one or more matrix element values using optical amplitude modulation.

Embodiment 325: A method of operating a computing system, comprising: using a first set of optical amplitude modulators to encode a set of multiple input values on a corresponding optical signal; for each of at least two subsets of the one or more optical signals Or, using a corresponding set of one or more replication modules to divide a plurality of subsets of one or more optical signals into two or more copies of the optical signal; for at least one of the first subset of the one or more optical signals each of the two copies, using a corresponding multiplication module to multiply the one or more optical signals of the first subset by the one or more matrix element values using the optical amplitude modulators of the second set of optical amplitude modulators; and for two or more The result of the multiplication module uses a summation module to generate an electrical signal, and the electrical signal represents the sum of the results of two or more multiplication modules; wherein the first group of optical amplitude modulators or the second group of optical amplitude modulators At least one of the at least one optical amplitude modulators is configured to modulate the optical signal by the modulation value using monotonically increasing power relative to an absolute value of the modulation value.

100:Artificial neural network computing system

102:Computer

110:Controller

120: Memory unit

130: Digital to analog converter unit

132: First digital-to-analog converter subunit

134: Second digital-to-analog converter subunit

140: Optical processor

142:Laser unit

144:Modulator array

146:Detection unit

150:Light matrix multiplication unit

160:Analog-to-digital converter unit

Claims (34)

An optoelectronic computing system includes: a first semiconductor die, including a photonic integrated circuit (PIC), wherein: the photonic integrated circuit includes an optical cable, the optical cable includes a plurality of strands, each of the strands includes An optical replication distribution network and a plurality of optoelectronic nodes. Each of the optoelectronic nodes corresponds to a specific column of a multiplication matrix. The optoelectronic nodes are associated with a plurality of optical tiles. Each of the optical tiles includes one of the corresponding multiplication matrices. A plurality of elements in a specific column; the optical replication distribution network includes a plurality of optical splitters, each of which is configured to split an input light wave at an input port into two or more corresponding output ports. Two or more optical signals; a second semiconductor die including an electronic integrated circuit (EIC), wherein the electronic integrated circuit includes a plurality of electrical tiles, each of the electrical tiles including one or more electrical Input ports configured to receive corresponding electrical values; wherein the optical tiles in the photonic integrated circuit are at least partially identical to the corresponding electrical tiles in the electronic integrated circuit overlapping; and wherein the first semiconductor die is electrically coupled to the second semiconductor die, with each of at least some of the electrical output ports of the photonic integrated circuit electrically connected to the electronic integrated circuit. One of the people waiting for electricity input into the mouth. The photoelectric computing system of claim 1, wherein the optical cable includes N stocks, and N is a positive integer; Wherein, the photonic integrated circuit includes M optical tiles, M is a positive integer, the optical tiles include N weight modulators, and each weight modulator corresponds to one of the N stocks. ; wherein the electronic integrated circuit includes M electrical tiles; and wherein the photonic integrated circuit and the electronic integrated circuit are configured to perform an M×N matrix multiplication. The photoelectric computing system of claim 1 or 2, wherein at least one of M and N is equal to or greater than 4. The photoelectric computing system of claim 1 or 2, wherein at least one of M and N is equal to or greater than 32. The photoelectric computing system of claim 1 or 2, wherein at least one of M and N is equal to or greater than 128. The optoelectronic computing system of claim 1 or 2, wherein the M optical tiles are configured as two or more straight lines of optical tiles, and each of the straight lines of the optical tiles includes one or more optical tiles. , at least one straight line of the optical tiles includes two or more optical tiles. The optoelectronic computing system of claim 1 or 2, wherein the M optical tiles are configured as a first straight line of M/2 optical tiles and a second straight line of M/2 optical tiles. The optoelectronic computing system of claim 1 or 2, wherein the M optical tiles are configured as four straight lines of the optical tiles, and each of the straight lines of the optical tiles includes M/4 optical tiles. For example, the photoelectric computing system of claim 1 or 2, wherein the electronic integrated circuit includes a complex weight driver, a complex data driver, a memory, a complex digital-to-analog converter, a complex analog-to-digital converter, a complex digital logic, and a A circuit on at least one of the digital data buses that carries data between two or more electrical tiles. The photoelectric computing system of claim 1 or 2, wherein the electrical tiles that communicate with each other more frequently are arranged closer to each other than the electrical tiles that communicate with each other less frequently. The optoelectronic computing system of claim 1 or 2, wherein the photonic integrated circuit includes a plurality of pads providing a plurality of electrical input and output ports, the electronic integrated circuit includes a plurality of pads providing a plurality of electrical input and output ports, providing The pads of the electrical input ports of the photonic integrated circuit are electrically connected to the pads that provide the electrical output ports of the electronic integrated circuit, and the pads that provide the electrical output ports of the photonic integrated circuit The pads are electrically connected to the pads that provide the electrical input ports of the electronic integrated circuit; wherein the electronic integrated circuit includes a plurality of transimpedance amplifiers, and the transimpedance amplifiers process the electrical input ports a complex value received at; and wherein the photonic integrated circuit includes a first tile, the first tile includes a first pad configured to connect the plurality of optoelectronics of the first tile A summed current of one of the plurality of conductors of the node is provided to a second pad of the electronic integrated circuit, And the second pad of the electronic integrated circuit is electrically connected to an input of a transimpedance amplifier. The optoelectronic computing system of claim 1 or 2, wherein each optical splitter is configured to transmit half of the power of the input light wave at the input port to two of the output ports, and the optical splitters are configured to Complex nodes in a binary tree arrangement connected by complex optical waveguides that are complex links of the binary tree arrangement. An optoelectronic computing system including: a first semiconductor die including a photonic integrated circuit (PIC) including an array of a plurality of optoelectronic circuit segments, each of the optoelectronic circuit segments including a matrix multiplication unit, each of the matrix multiplication units includes at least one light detection unit that detects at least one light wave in an optoelectronic operation, and at least one wire in the photonic integrated circuit electrically couples the light detector and an electrical an output port; and a second semiconductor die including an electronic integrated circuit (EIC) including a plurality of electrical input ports receiving corresponding plural electrical values; wherein the electrical output of the photonic integrated circuit The port is connected to one of the electrical input ports of the electronic integrated circuit. The optoelectronic computing system of claim 13, wherein the photonic integrated circuit includes: a plurality of optical waveguides, wherein the corresponding plurality of optical signals carried by the optical waveguides A plurality of input values encoding a group on the signal; and an optical replication distribution network including a plurality of optical splitters, wherein each of the optical splitters is configured to divide an input light wave located at an input port into the input light wave two or more copies, and transmit the two or more copies of the input light wave to two or more output interfaces respectively; wherein each of the optoelectronic circuit sections is configured to receive data from the optical copy distribution network A light wave passing through one of the output interfaces; and wherein each of the optoelectronic circuit sections includes: an optoelectronic operation module that selects one of two values to perform an operation: (1) copying the distribution with the light an optical value on which one of the input values for network scaling is based, and (2) an electrical value provided by an electrical input port; at least one light detector that detects at least one light wave from the optoelectronic operation; and at least one integrated Wires in the photonic integrated circuit are electrically coupled to the light detector and an electrical output terminal. The optoelectronic computing system of claim 14, wherein the electronic integrated circuit further includes complex digital-to-analog converters (DACs) that provide complex electrical values to corresponding complex electrical output ports, and the electrical values of the electronic integrated circuit The output ports are electrically connected to the electrical input ports of the photonic integrated circuit. The optoelectronic computing system of claim 14 or 15, wherein the optical splitters are arranged as plural nodes in a binary tree arrangement by plural lights as complex links of the binary tree arrangement. waveguide connection. The optoelectronic computing system of claim 16, wherein the optical replication distribution network includes a plurality of binary tree arrangements, each of the binary tree allocations encoding a different one of the plurality of input values of the corresponding plurality of optical signals. The optoelectronic computing system of claim 14 or 15, wherein the optical waveguides in the optical replication distribution network are disposed in the first semiconductor die to avoid passing through any of the optical replication distribution network. Iso-optical waveguide. The optoelectronic computing system of claim 14 or 15, wherein the optoelectronic circuit sections are disposed on the first semiconductor die and are arranged in a plurality of straight lines as a whole. The optoelectronic computing system of claim 19, wherein each of the straight lines is optically coupled to each other straight line through one or more of the optical waveguides in the optical replication distribution network. The optoelectronic computing system of claim 14 or 15, wherein the optoelectronic operating module includes a Mach-Zehnder interferometer configured to select one of two values to perform the multiplication operation (1) to The optical replication distribution network scales one of the input values based on the optical value, and (2) the electrical value provided by the electrical input port. The optoelectronic computing system of claim 14 or 15, wherein the electronic integrated circuit further includes a transimpedance amplifier having an input electrically coupled to the electrical output port of the photonic integrated circuit. An optoelectronic computing system including: A first semiconductor die, including a photonic integrated circuit (PIC), the photonic integrated circuit including: a plurality of optical waveguides, wherein as many as one group is encoded on corresponding plurality of optical signals carried by the optical waveguides an input value; an optical replication distribution network including a plurality of optical splitters or directional couplers; and an array of a plurality of optoelectronic circuit segments, each of the optoelectronic circuit segments being connected from a plurality of output ports of the optical replication distribution network One receives an optical wave, wherein each of the optoelectronic circuit segments includes: (a) an optoelectronic operation module that selects one of two values to perform an optoelectronic operation: (a1) scaled with the optical replication distribution network One of the input values is an optical value based on, and (a2) an electrical value provided by the electrical input port; (b) at least one light detector that detects at least one light wave of the optoelectronic operation; and (c) at least At least one wire integrated into the photonic integrated circuit is electrically coupled to the photodetector and to an electrical output port, wherein a portion of the wire integrated into the photonic integrated circuit will The photodetector is connected to a contact between the conductors from different optoelectronic circuit segments; and a second semiconductor die includes an electronic integrated circuit (EIC), the electronic integrated circuit including A plurality of electrical input ports receive corresponding plural electrical values; Wherein, the electrical output port of the photonic integrated circuit is connected to one of the electrical input ports of the electronic integrated circuit. Such as the optoelectronic computing system of claim 23, wherein the electronic integrated circuit further includes complex digital-to-analog converters (DACs) that provide complex electrical values to corresponding complex electrical output ports, and the electrical inputs of the photonic integrated circuit The ports are electrically connected to the electrical output ports of the electronic integrated circuit. The optoelectronic computing system of claim 23, wherein the optoelectronic operating module includes a Mach-Zehnder interferometer configured to select one of two values to perform the multiplication operation (1) with the light Copy the optical value upon which one of these input values is scaled by the distribution network, and (2) the electrical value provided by the electrical input port. The optoelectronic computing system of claim 23, wherein the electronic integrated circuit further includes a transimpedance amplifier having an input electrically coupled to the electrical output port of the photonic integrated circuit. The optoelectronic computing system of any one of claims 23 to 26, wherein each optical splitter transmits half of the power of an input light wave at an input port to two output ports. The optoelectronic computing system of claim 23, wherein the optical replication distribution network includes a plurality of directional couplers in series. The optoelectronic computing system of any one of claims 23, 24, 25, 26, and 28, wherein the optical splitters are arranged as plural nodes in a binary tree arrangement, and the binary tree is configured as the binary tree The complex number of optical waveguides connected by the yuan tree is connection. The optoelectronic computing system of claim 29, wherein the optical replication distribution network includes a plurality of binary tree arrangements, each of the binary tree allocations encoding a different one of the plurality of input values of the corresponding plurality of optical signals. The optoelectronic computing system of claim 30, wherein the complex light propagation lengths differ between roots of the binary tree assignment and different complex optoelectronic circuit sections. The optoelectronic computing system of claim 31, wherein the optical waveguides in the optical replication distribution network are disposed in the first semiconductor die to avoid passing through any of the light in the optical replication distribution network. waveguide. The optoelectronic computing system of claim 32, wherein the optoelectronic circuit sections are disposed on the first semiconductor die and are arranged in a plurality of straight lines as a whole. The optoelectronic computing system of claim 33, wherein each of the straight lines is optically coupled to each other straight line through one or more of the optical waveguides in the optical replication distribution network.