CN112232504A

CN112232504A - Photon neural network

Info

Publication number: CN112232504A
Application number: CN202010956262.0A
Authority: CN
Inventors: 田野; 李强; 赵洋; 王玮; 刘胜平
Original assignee: United Microelectronics Center Co Ltd
Current assignee: United Microelectronics Center Co Ltd
Priority date: 2020-09-11
Filing date: 2020-09-11
Publication date: 2021-01-15
Anticipated expiration: 2040-09-11
Also published as: CN112232504B

Abstract

A photonic neural network, comprising: the optical modulation unit modulates the signal to be processed to the real amplitude of light, and introduces a nonlinear corresponding relation between the real amplitude and the phase of the light during modulation to obtain an optical signal of a first array; the photon matrix calculation unit receives the optical signals of the first array and carries out matrix calculation to obtain the optical signals of the second array, and matrix multiplication and nonlinear transformation are simultaneously carried out during matrix calculation; the projection calculation unit is used for receiving the optical signals of the second array and extracting the real part of the complex amplitude of the optical signals of the second array based on optical interference so as to obtain the optical signals of a third array, and the real part represents the operation result of matrix multiplication and nonlinear transformation; and the light receiving unit receives the optical signals of the third array to acquire processed signals. The scheme of the invention provides the photon artificial intelligence chip which can simultaneously carry out linear and nonlinear operation, and has the advantages of low power consumption, high operation speed, high neural network integration level and reconfigurable nonlinear operation.

Description

Photon neural network

Technical Field

The invention relates to the technical field of photonic artificial intelligence chips, in particular to a photonic neural network.

Background

In the most popular deep learning in the field of artificial intelligence at present, the operation process mainly relates to two parts: matrix multiplication and nonlinear activation functions. Specifically, the artificial intelligence algorithm has the characteristics that the processing content is unstructured data (such as video, images and voice), a large amount of linear algebraic operations are required in the processing process, and the processing process parameters are large. The computing hardware mainly including the central processing unit cannot meet the computing power requirement of Artificial intelligence, and can only be realized by depending on an Artificial Intelligence (AI) chip. Specifically, the AI chip is a chip specifically oriented to AI applications, and is an important physical basic carrier of AI technology.

Currently, AI chips are mainly implemented based on Complementary Metal Oxide Semiconductor (CMOS) technology. As the size of integrated circuit devices continuously approaches to the physical limit, moore's law shows a slow trend, and meanwhile, the microelectronic processor has problems of reduced energy efficiency ratio, limited clock frequency (difficult to exceed 6 gigahertz (GHz)), electronic crosstalk, high power consumption, heat generation and the like, and the continuous improvement of the performance of the conventional electronic AI chip is severely restricted.

In order to break through the problems of the electronic chip in the AI field, the photon artificial intelligence chip is produced. However, the current photonic artificial intelligence chip technology is still in the bud stage, and the architectural design of the photonic artificial intelligence chip still has a lot of defects, so that the advantages of the photonic artificial intelligence chip cannot be fully exerted.

Disclosure of Invention

The invention solves the technical problem of providing an improved photon neural network which can simultaneously carry out linear and nonlinear operations.

To solve the above technical problem, an embodiment of the present invention provides a photonic neural network, including: the optical modulation unit is used for modulating a signal to be processed to real amplitude of light, introducing a nonlinear corresponding relation between the real amplitude and the phase of the light during modulation, and recording the output of the optical modulation unit as an optical signal of a first array; a photon matrix calculation unit coupled to the light modulation unit to receive the optical signals of the first array, the photon matrix calculation unit performing matrix calculation on the optical signals of the first array to obtain optical signals of a second array, wherein matrix multiplication and nonlinear transformation are simultaneously performed when performing matrix calculation on the optical signals of the first array; a projection computation unit coupled to the photon matrix computation unit to receive the optical signals of the second array, the projection computation unit extracting real parts of complex amplitudes of the optical signals of the second array based on optical interference to obtain optical signals of a third array, wherein the real parts represent operation results of performing the matrix multiplication and nonlinear transformation on the optical signals of the first array; a light receiving unit coupled with the projection computation unit to receive the optical signals of the third array, the light receiving unit acquiring processed signals based on the optical signals of the third array.

Optionally, the transformation function of the nonlinear transformation is related to a phase variation amount caused when the light modulation unit performs real amplitude modulation and a phase shifter coefficient, where the phase shifter coefficient is a phase shift parameter of a phase shifter adopted by an equivalent diagonal matrix in the photon matrix calculation unit.

Optionally, the non-linear correspondence between the real amplitude and the phase of the light is due to a chirp effect of the light modulation unit.

Optionally, the light modulation unit includes: an optical interference unit, an upper arm or a lower arm of which is provided with a first phase shifter to modulate the signal to be processed to a real amplitude of light while adjusting a non-linear correspondence between the real amplitude and a phase of the light.

Optionally, a second phase shifter is disposed at an input end or an output end of the optical interference unit to adjust a nonlinear correspondence between the real amplitude and the phase of the light.

Optionally, the light modulation unit includes: the micro-ring resonator comprises a ring waveguide and a straight waveguide which are coupled, wherein the ring waveguide is provided with a first phase shifter to modulate the signal to be processed to the real amplitude of light and adjust the nonlinear corresponding relation between the real amplitude and the phase of the light.

Optionally, a second phase shifter is disposed at an input end or an output end of the straight waveguide to adjust a nonlinear correspondence between the real amplitude and the phase of the light.

Optionally, the light modulation unit includes: and the input end of the semiconductor optical amplifier receives the light and the signal to be processed, and the output end of the semiconductor optical amplifier outputs the optical signal of the first array.

Optionally, a second phase shifter is disposed at an input end or an output end of the semiconductor optical amplifier to adjust a nonlinear correspondence between the real amplitude and the phase of the light.

Optionally, the offset of the phase is determined according to a phase shift parameter of the second phase shifter, wherein the offset of the phase is an initial phase of a transform function of the nonlinear transform.

Optionally, the projection calculation unit includes: and the optical interference unit comprises a first input arm and a second input arm, wherein the first input arm receives the optical signals of the second array, the second input arm receives the optical signals of the reference array, and the optical signals of the second array and the optical signals of the reference array are output to obtain the optical signals of the third array after the optical interference unit generates optical interference.

Optionally, the light receiving unit includes: a photodetector for photoelectrically converting the optical signals of the third array; and the high-pass filtering unit is coupled with the optical detector and is used for performing high-pass filtering on the output of the optical detector to obtain the processed signal.

Optionally, the optical signals of the third array include an alternating current component related to a phase difference, and the high-pass filtering unit extracts the alternating current component as the processed signal based on high-pass filtering, where the phase difference is a phase difference between the optical signals of the second array and the optical signals of the reference array.

Optionally, the light receiving unit includes: a balance detection unit for acquiring the processed signal from the optical signals of the third array based on a balance detection method.

Optionally, the optical signals of the third array include an alternating current component related to a phase difference, and the balance detection unit extracts the alternating current component as the processed signal based on balance detection, where the phase difference is a phase difference between the optical signals of the second array and the optical signals of the reference array.

Optionally, the processed signal is proportional to the alternating current component.

Optionally, the photon matrix calculation unit is a unitary matrix, and the unitary matrix includes a plurality of optical interference units connected in series and parallel with each other.

Optionally, the optical modulation unit and the photon matrix calculation unit that are coupled to each other are referred to as a neural network unit, the photon neural network includes a plurality of cascaded neural network units, where an output of a preceding neural network unit is an input of a succeeding neural network unit, an input of a first neural network unit is the signal to be processed, and an output of a last neural network unit is the optical signal of the second array.

Optionally, an output end of the projection calculation unit is coupled to an input end of the photon matrix calculation unit, so as to re-input the optical signals of the third array into the photon matrix calculation unit to perform matrix calculation cyclically.

Optionally, the photonic neural network is used for image processing, image recognition, voice recognition, gene sequencing, quantum communication, or quantum computation.

Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:

an embodiment of the present invention provides a photonic neural network, including: the optical modulation unit is used for modulating a signal to be processed to real amplitude of light, introducing a nonlinear corresponding relation between the real amplitude and the phase of the light during modulation, and recording the output of the optical modulation unit as an optical signal of a first array; a photon matrix calculation unit coupled to the light modulation unit to receive the optical signals of the first array, the photon matrix calculation unit performing matrix calculation on the optical signals of the first array to obtain optical signals of a second array, wherein matrix multiplication and nonlinear transformation are simultaneously performed when performing matrix calculation on the optical signals of the first array; a projection computation unit coupled to the photon matrix computation unit to receive the optical signals of the second array, the projection computation unit extracting real parts of complex amplitudes of the optical signals of the second array based on optical interference to obtain optical signals of a third array, wherein the real parts represent operation results of performing the matrix multiplication and nonlinear transformation on the optical signals of the first array; a light receiving unit coupled with the projection computation unit to receive the optical signals of the third array, the light receiving unit acquiring processed signals based on the optical signals of the third array.

From the above, the present embodiment provides a photonic artificial intelligence chip capable of performing linear and nonlinear operations simultaneously, which has low power consumption, fast operation speed, high neural network integration level, and reconfigurable nonlinear operation.

Specifically, compared with the existing photonic neural network which needs to realize nonlinear operation based on an electronic domain, the nonlinear activation function of the neural network in the embodiment is realized in an optical domain, so that the energy consumption of a chip can be effectively reduced, the operation speed of a neural network algorithm is improved, and the possibility of further improving the computational power of a photonic artificial intelligence chip is realized.

Compared with the existing photonic neural network which needs a special processing module to independently perform the nonlinear activation function on the optical domain, the nonlinear activation function is introduced through the modulator unit in the embodiment, and the nonlinear activation function is specially realized without an additional device unit, so that the number of modules needed by the photonic neural network is reduced, and the integration level of the neural network is favorably improved.

Furthermore, the preparation process of the photonic neural network in the embodiment is completely consistent with the process required by the existing optical chip which simply executes linear operation, and reconfigurable nonlinear operation can be realized. The reconfigurable nonlinear operation means that an existing optical chip based on a photon integration process can be defined into any linear calculation matrix according to needs, and the scheme of the application can realize the nonlinear operation based on the existing optical chip process, so that the reconfigurable nonlinear operation can be realized by combining the optical chip and the optical chip.

Further, the non-linear correspondence between the real amplitude and the phase of the light is due to a chirp effect of the light modulation unit. Thus, the photonic neural network described in this implementation is based on the modulator unit loading the signal to be processed at the real amplitude of the light while introducing phase changes related to the real amplitude. The output optical signal of the modulator unit is the input optical signal of the photon matrix calculation unit, i.e. the optical signal of the first array. The phase of the optical signals of the first array changes simultaneously when the optical signals are subjected to real amplitude modulation, that is, the real amplitude modulation and the phase change are coupled, so that the result of performing nonlinear operation and then performing linear operation on the optical signals of the first array can be obtained through the matrix calculation of the photon matrix calculation unit. Therefore, the photonic neural network can simultaneously realize linear operation and nonlinear operation based on the chirp effect of the modulator unit and the cooperation of the photon matrix calculation unit.

Drawings

FIG. 1 is a schematic diagram of a photonic neural network in accordance with an embodiment of the present invention;

fig. 2 is a schematic diagram of a first basic structure of the optical modulation unit in fig. 1;

fig. 3 is a schematic diagram of a second basic structure of the optical modulation unit in fig. 1;

fig. 4 is a schematic diagram of a third basic structure of the light modulation unit in fig. 1;

FIG. 5 is a schematic diagram of a first basic structure of the projection calculation unit in FIG. 1;

FIG. 6 is a diagram of a second basic structure of the projection calculation unit of FIG. 1;

FIG. 7 is a diagram showing a first basic structure of the photon matrix calculating unit in FIG. 1;

FIG. 8 is a diagram showing a second basic structure of the photon matrix calculating unit in FIG. 1;

FIG. 9 is a schematic diagram of an exemplary application scenario in accordance with an embodiment of the present invention;

FIG. 10 is a schematic diagram of another photonic neural network in accordance with an embodiment of the present invention;

FIG. 11 is a schematic diagram of a multi-layer neural network, in accordance with embodiments of the present invention;

FIG. 12 is a schematic diagram of a recurrent neural network in accordance with an embodiment of the present invention;

FIGS. 13 and 14 are non-linear functions simulated when the optical modulation unit 11 in the photonic neural network 1 shown in FIG. 1 adopts the structure shown in FIG. 2;

fig. 15 and 16 are nonlinear functions obtained by simulation when the optical modulation unit 11 in the photonic neural network 1 shown in fig. 1 adopts the structure shown in fig. 3.

Detailed Description

As background art, in order to solve the exponential growth of data volume and the gradual termination of moore's law, innovative and subversive technologies are urgently needed to break through the problems faced by electronic chips in the AI field.

In the current most popular deep learning of artificial intelligence, the operation process mainly involves two parts: matrix multiplication and nonlinear activation functions. The nonlinear operation is the root of the strong expression capability of the artificial neural network, can accelerate the convergence rate of the neural network and improve the identification accuracy, and is an indispensable component in the neural network.

The photonic artificial intelligence chip completes a deep learning algorithm by using light, and is essentially characterized in that matrix multiplication and a nonlinear activation function are realized by using light. In which, matrix multiplication is completed by light, and the conventional common method is to use a Mach-Zehnder interferometer (MZI) array, a Micro-Ring Resonator (MRR) array, etc. to implement the matrix multiplication.

Implementing a nonlinear function in the optical domain presents two significant challenges: (1) the generation of optical nonlinear effects generally requires higher optical power; (2) in the photonic artificial intelligence chip architecture, the flexibility requirement on the nonlinear activation function is higher, and the control difficulty of the existing optical nonlinear effect is high, so that the flexibility requirement cannot be met.

In addition, the integration of nonlinear optical elements on a chip also has many problems in terms of process compatibility and device uniformity.

Therefore, the embodiment can realize the photonic artificial intelligence chip with high computational power, and the chip can realize the optical nonlinear activation function with low power consumption, high speed, easy realization and rich expression forms.

Next, embodiments of the present invention will be described in detail with reference to the drawings. Like parts are designated by like reference numerals throughout the several views. The embodiments are merely illustrative, and it is needless to say that partial substitutions or combinations of the structures shown in the different embodiments may be made. In the modification, descriptions of common matters with the first embodiment are omitted, and only different points will be described. In particular, the same operational effects produced by the same structures are not mentioned one by one for each embodiment.

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.

Fig. 1 is a schematic diagram of a photonic neural network according to an embodiment of the present invention.

The embodiment can be applied to application scenes such as image processing, image recognition, voice recognition, gene sequencing, quantum communication or quantum computing, and the like, and the photonic neural network 1 can realize recognition of images, videos, voices and the like with relatively high computing power.

Specifically, referring to fig. 1, the photonic neural network 1 according to this embodiment may include: the optical modulation unit 11 is configured to modulate a signal to be processed to a real amplitude of light, introduce a non-linear correspondence between the real amplitude and a phase of the light during modulation, and record an output of the optical modulation unit 11 as an optical signal of a first array.

The optical signals of the first array may comprise at least one optical signal, wherein each optical signal carries information of the signal to be processed.

In one implementation, the signal to be processed may be characterized as an electrical signal carrying corresponding information, and the optical modulation unit 11 is enabled to load the information in the electrical signal onto the optical signal by transmitting the electrical signal to the optical modulation unit 11 to form the optical signal of the first array.

Further, the electrical signal transmitted to the optical modulation unit 11 may be an analog signal.

In one implementation, the analog signal may be data to be processed that is directly obtained from outside the photonic neural network 1. For example, the data to be processed may be an image signal for image recognition characterized by an analog signal, or a voice signal for voice recognition. The optical modulation unit 11 encodes the analog signal into an optical signal for subsequent operation.

For example, the photonic neural network 1 may receive, for example, a voice signal from a speaker and process the received voice signal to recognize the content of the voice signal. Thus, speech recognition can be realized based on the photonic neural network 1.

For another example, the photonic neural network 1 may receive, for example, an image signal from an image capture device and process the received image signal to identify the content of the image signal. Thereby, image recognition can be realized based on the photonic neural network 1. Correspondingly, the signal to be processed may be the gray scale of a plurality of pixel points of the image.

In one variation, the analog signal may be converted from a digital signal, which is the data to be processed input externally by the photonic neural network 1.

For example, with continued reference to fig. 1, the photonic neural network 1 may include a first electrical processing unit 12 for performing digital-to-analog conversion on the data to be processed to generate a corresponding analog signal. Further, the generated analog signal is transmitted to the optical modulation unit 11 as the signal to be processed.

Further, the first electrical processing unit 12 may be integrated into the photonic neural network 1. Alternatively, the first electrical processing unit 12 and the photonic neural network 1 may be separately disposed, and both may be coupled for data transmission.

In one implementation, the transmission of optical signals between modules within the photonic neural network 1 according to the present embodiment may be carried on a waveguide basis. In particular, a waveguide refers to any structure capable of guiding an optical signal in any manner. Such as an optical fiber, a semiconductor waveguide fabricated in a substrate, a photonic crystal structure configured to guide an optical signal, or any other suitable structure.

For example, the optical signals of the first array may be transmitted through a first waveguide array that couples the optical modulation unit 11 and the photon matrix calculation unit 14.

Also for example, optical signals of a second array, which will be mentioned below, may be transmitted through a second waveguide array, which couples the photon matrix calculation unit 14 and the projection calculation unit 15.

For another example, optical signals of a third array, which will be mentioned below, may be transmitted through a third waveguide array that couples the projection calculation unit 15 and the light reception unit 16.

In one implementation, the photonic neural network 1 may further include a light source 13 for generating an array of optical signals, each optical signal in the array of optical signals including continuous light of the same or different wavelengths.

Further, the light modulation unit 11 may include an array of light modulators coupled to the light source to receive the array of optical signals generated by the light source 13. For example, the array of optical modulators includes a plurality of optical modulators in one-to-one correspondence with the optical signals in the array of optical signals to modulate the corresponding optical signals.

The optical modulation unit 11 applies a modulation signal to the optical signal array to obtain the optical signals of the first array, wherein the modulation signal is associated with the signal to be processed.

Further, the association of the modulation signal with the signal to be processed may refer to determining a specific value of the modulation signal according to specific information of the signal to be processed.

For example, the intensity of the optical signal array after passing through the optical modulation unit 11 can be adjusted by the optical modulator array, and the intensity adjustment amount of each optical signal is different according to the applied voltage. The intensity of the input light and the output light has a relationship, and information of the signal to be processed can be loaded on the light through the relationship.

Further, the optical modulation unit 11 also causes a phase change of the optical signal in a nonlinear relationship with the real amplitude while modulating the signal to be processed to the real amplitude of the optical signal.

The non-linear correspondence between the real amplitude and the phase of the light is due to the chirp effect of the light modulation unit 11.

The real amplitude refers to the mode of the complex amplitude of light, i.e. the square root of the intensity of the light.

In one implementation, referring to fig. 2, the light modulation unit 11 may include: the optical interference unit 111 is, for example, a 1 × 1 mach-zehnder interferometer as the optical interference unit 111, and the 1 × 1 mach-zehnder interferometer may include two 1 × 2 beam splitters (respectively denoted by a beam splitter 1 and a beam splitter 2 in the figure). The beam splitter 1 is disposed at the input end and coupled to the input waveguide, and is configured to receive one optical signal in the optical signal array transmitted through the input waveguide. The splitter 2 is disposed at the output end and coupled to the output waveguide, and is configured to transmit the modulated optical signal, i.e., one of the optical signals of the first array, to the outside through the output waveguide.

For example, the beam splitter may employ a Directional Coupler (DC), a multi-Mode Interferometer (MMI), or the like.

The upper arm of the 1 × 1 mach-zehnder interferometer is provided with a first phase shifter (denoted by phase shifter 1 in the figure) 112, and the first phase shifter 112 is configured to modulate the signal to be processed to a real amplitude of light and adjust a nonlinear correspondence between the real amplitude and the phase of the light.

Assume that the phase shift parameter of the first phase shifter 112 in FIG. 2 is β₁The first phase shifter 112 can load information on the mode of the optical wave and cause the phase of the optical wave to change to β₁/2。

Further, the output terminal of the 1 × 1 mach-zehnder interferometer may be provided with a second phase shifter (denoted by phase shifter 2 in the figure) 113 to adjust the nonlinear correspondence between the real amplitude and the phase of the light. Wherein, the output end is close to one end of the output waveguide.

Let the phase shift parameter of the second phase shifter 113 in FIG. 2 be β₂The phase of the light inputted from the input waveguide after passing through the optical interference unit 111 is changed to

In this embodiment, the phase shift parameter β of the second phase shifter 113₂Can be used as the offset value of the phase for defining the starting point of the phase, namely the starting phase of the transformation function of the nonlinear transformation, so that the user can obtain the desired nonlinear function. Phase shift parameter β of the first phase shifter 112₁Can be used to both adjust the intensity of the light and induce phase changes.

Accordingly, the transmission matrix M using the optical interference unit 111 shown in fig. 2 can be as shown in formula (1):

wherein, the transmission matrix M can be used to describe the variation of the complex amplitude of the optical signal.

The first phase shifter 112 and the second phase shifter 113 may be used to perform a phase shifting operation on an optical signal. For example, the first phase shifter 112 and the second phase shifter 113 may be implemented by thermo-optic, electro-optic, phase change, plasma dispersion, or the like.

In one modification, the first phase shifter 112 may be provided in the lower arm of the 1 × 1 mach-zehnder interferometer.

Alternatively, both the upper and lower arms of the 1 × 1 mach-zehnder interferometer may be provided with the first phase shifter 112.

In one variation, the second phase shifter 113 may be disposed at an input end of the 1 × 1 mach-zehnder interferometer, where the input end is an end near an input waveguide.

Alternatively, both the input and output terminals of the 1 × 1 mach-zehnder interferometer may be provided with the second phase shifter 113.

In one implementation, referring to fig. 3, the light modulation unit 11 may include: a micro-ring resonator 114, wherein the micro-ring resonator 114 may include a coupled ring waveguide (also referred to as a curved waveguide) 115 and a straight waveguide 116, and the ring waveguide 115 is provided with a first phase shifter (identified as phase shifter 1 in the figure) 112 to modulate the signal to be processed to a real amplitude of light while introducing a non-linear correspondence between the real amplitude and the phase of the light.

Specifically, the annular waveguide 115 is a closed waveguide.

Further, the straight waveguide 116 is provided with a second phase shifter (identified as phase shifter 2 in the figure) 113 to introduce a non-linear correspondence between the real amplitude and the phase of the light. For example, the second phase shifter 113 may be located between the ring waveguide 115 and the output waveguide, i.e., at the output end of the straight waveguide 116.

Output complex amplitude E using all-pass type microring resonator shown in fig. 3_passAnd input complex amplitude E_inputCan refer to formula (2):

where phi is the phase change of light around the micro-ring resonator (i.e., the ring waveguide 115), and may be based on the formula phi ═ n_effβL+β₁Is calculated to obtain, wherein n_effIs the effective index of refraction of the annular waveguide 115,

λ is the wavelength of light, β, as the propagation constant₁Is the phase shift parameter of the first phase shifter 112; a is²＝e^-αLWhere α is a loss coefficient of the micro-ring resonator 114, L ═ 2 π R is a perimeter of the micro-ring resonator 114, and R is a radius of the micro-ring resonator 114; r is the self-coupling coefficient.

Fig. 3 shows k as the coupling coefficient of the circular waveguide 115 and the straight waveguide 116 in the microring resonator 114.

Based on the structure shown in fig. 3, the phase of light input from the input waveguide is changed after passing through the micro-ring resonator 114

As shown in equation (3):

for a micro-ring resonator with a low Quality factor (Q value), the phase shift parameter β of the first phase shifter 112 can be used₁The phase change phi of the light around one revolution in the microring resonator (i.e., the ring waveguide 115) is adjusted to achieve intensity modulation while causing a phase change of the output light with respect to the input light by a specific change amount as described in the above equation (3).

In this embodiment, the phase shift parameter β of the second phase shifter 113₂Can be used as an offset value of the phase for defining the starting point of the phase, so that a user can obtain a desired nonlinear function. Phase shift parameter β of the first phase shifter 112₁Can be used to both adjust the intensity of the light and induce phase changes.

In one variation, the second phase shifter 113 may be located between the ring waveguide 115 and the input waveguide, i.e., at the input end of the straight waveguide 116.

Alternatively, the number of the second phase shifters 113 may be plural and located between the ring waveguide 115 and the input waveguide and between the ring waveguide 115 and the output waveguide, respectively.

In one implementation, referring to fig. 4, the light modulation unit 11 may include: a Semiconductor Optical Amplifier (SOA for short, referred to as SOA mark in the figure) 117, an input end of the Semiconductor Optical Amplifier 117 receives the light and the signal to be processed, and an output end of the Semiconductor Optical Amplifier 117 outputs the Optical signal of the first array. The input end of the semiconductor optical amplifier 117 is coupled to one end of the input waveguide, and the output end of the semiconductor optical amplifier 117 is coupled to one end of the output waveguide.

Further, the output end of the semiconductor optical amplifier 117 may be provided with a second phase shifter (denoted by phase shifter 2 in the figure) 113 to adjust the non-linear correspondence between the real amplitude and the phase of the light.

Specifically, the semiconductor optical amplifier 117 has a self-phase modulation effect. Injection of the optical signal into the semiconductor optical amplifier 117 causes a change in the concentration of carriers in the semiconductor optical amplifier 117, thereby affecting the optical gain and optical refractive index of the active region in the semiconductor optical amplifier 117.

Therefore, in the present embodiment, while the data to be processed is loaded to the input optical signal based on the semiconductor optical amplifier 117, a phase change related to the real amplitude of the optical signal can be brought at the same time.

For example, the data to be processed may be loaded by controlling different gains.

In this embodiment, the phase shift parameter β of the second phase shifter 113₂Can be used as an offset value of the phase for defining the starting point of the phase, so that a user can obtain a desired nonlinear function. Phase shift parameter beta of semiconductor optical amplifier 117₁Can be used to both adjust the intensity of the light and induce phase changes.

In a variation, the input end of the semiconductor optical amplifier 117 may be provided with the second phase shifter 113.

In one implementation, the light modulation unit 11 may include a plurality of optical interference units 111 as shown in fig. 2 connected in parallel.

Alternatively, the optical modulation unit 11 may include a plurality of micro-ring resonators 114 as shown in fig. 3 connected in parallel.

Still alternatively, the optical modulation unit 11 may include a plurality of semiconductor optical amplifiers 117 as shown in fig. 4 connected in parallel.

Alternatively, the optical modulation unit 11 may be a combination of the optical interference unit 111 shown in fig. 2, the microring resonator 114 shown in fig. 3, and the semiconductor optical amplifier 117 shown in fig. 4.

In one specific implementation, the modulator used by the light modulation unit 11 may be a modulator using electro-optical, acousto-optical, thermo-optical, or the like.

In one implementation, the optical modulation unit 11 and other modules of the photonic neural network 1 may be integrated together in a photonic artificial intelligence chip.

Alternatively, the optical modulation unit 11 and other modules of the photonic neural network 1 may be separately disposed. For example, the other modules of the photonic neural network 1 are integrated into a photonic artificial intelligence chip, and the light modulation unit 11 is externally coupled to the photonic artificial intelligence chip.

In a specific implementation, with continuing reference to fig. 1, the photonic neural network 1 of this embodiment may further include a photonic matrix calculation unit 14 coupled to the optical modulation unit 11 to receive the optical signals of the first array, where the photonic matrix calculation unit 14 performs a matrix calculation on the optical signals of the first array to obtain optical signals of a second array, and performs a matrix multiplication and a nonlinear transformation simultaneously when performing the matrix calculation on the optical signals of the first array.

Specifically, the optical signal modulated by the optical modulation unit 11 (i.e., the optical signal of the first array) enters the photon matrix calculation unit 14 through the first waveguide array as an input vector. Since the optical signals of the first array carry both linear and nonlinear relations, the photon matrix calculation unit 14 can simultaneously perform linear calculation and nonlinear calculation on the optical signals of the first array. That is, the input vector represented by the modulated optical signal is multiplied by the matrix S defined by the optical matrix calculation unit 14, and matrix multiplication and nonlinear transformation can be simultaneously achieved. The output of the photon matrix computation unit 14 (i.e. the optical signals of the second array) enters the second waveguide array as a result of the matrix computation for subsequent processing.

Further, the photon matrix calculation unit 14 may include a plurality of optical interference units connected in series and parallel with each other. For example, a combination of one or more of mach-zehnder interferometers, multimode interferometers, directional couplers, photonic crystals, and microring resonators may be used.

In one specific implementation, with continued reference to fig. 1, the photonic neural network 1 of this implementation may further include a projection calculation unit 15 coupled to the photonic matrix calculation unit 14 to receive the optical signals of the second array, where the projection calculation unit 15 extracts real parts of complex amplitudes of the optical signals of the second array based on optical interference to obtain optical signals of a third array, where the real parts represent operation results of performing the matrix multiplication and the nonlinear transformation on the optical signals of the first array.

Specifically, the projection calculation unit 15 may project the optical signal complex amplitude representing the calculation result of the photon matrix calculation unit 14 (i.e., the optical signals of the second array) to the complex plane real axis based on optical interference, thereby extracting the real part of the optical signal complex amplitude (i.e., the optical signals of the third array).

Further, the output representing the real part of the complex amplitude of the optical signal (i.e., the optical signal of the third array) enters the third waveguide array for subsequent processing.

In one implementation, the projection calculation unit 15 may include an optical interference unit, and the optical interference unit includes a first input arm 151 (shown in fig. 5 or 6) and a second input arm 152 (shown in fig. 5 or 6), where the first input arm 151 receives the optical signals of the second array, the second input arm 152 receives the optical signals of the reference array, and the optical signals of the second array and the optical signals of the reference array are output to obtain the optical signals of the third array after optical interference occurs in the optical interference unit.

Specifically, the optical signal of the reference array is the optical signal output by the light source 13 and not modulated and phase-adjusted by the optical modulation unit 11. The optical signal of the reference array is continuous light.

For example, referring to fig. 5, the optical interference unit may be a 2 × 2 Directional Coupler (DC). The 2 × 2 directional coupler includes two input arms, which are respectively referred to as a first input arm 151 and a second input arm 152, the first input arm 151 receives signal light (i.e., one of the optical signals of the second array), and the second input arm 152 receives reference light (i.e., one of the optical signals of the reference array). The optical signals of the second array and the optical signals of the reference array are transmitted to the outside through two outputs (marked by output 1 and output 2 in the figure) of the 2 × 2 directional coupler after optical interference occurs in the 2 × 2 directional coupler, and the output of the 2 × 2 directional coupler is the optical signals of the third array.

For another example, referring to fig. 6, the optical interference unit may be a 2 × 2 multi-Mode Interferometer (MMI). The 2 x 2 multimode interferometer comprises two input arms, denoted first input arm 151 and second input arm 152, respectively, the first input arm 151 receiving signal light (i.e. one of the optical signals of the second array) and the second input arm 152 receiving reference light (i.e. one of the optical signals of the reference array). The optical signals of the second array and the optical signals of the reference array are transmitted to the outside through two outputs (marked as output 1 and output 2 in the figure) of the 2 × 2 multimode interferometer after optical interference occurs in the 2 × 2 multimode interferometer, and the output of the 2 × 2 multimode interferometer is the optical signals of the third array.

Since the output of the optical interference unit is two-way, the optical signal is a pair of optical signals for each of the optical signals of the third array. The DC components of the pair of optical signals are the same, and the AC components are negative numbers of each other. Specific explanations regarding the direct current component and the alternating current component will be described in detail below.

Further, the number of the optical interference units may be multiple, and an optical interference array composed of multiple optical interference units receives the optical signals of the second array, where each optical interference unit is configured to process one optical signal of the optical signals of the second array.

For example, the optical interference array may include a plurality of 2 x 2 directional couplers as shown in fig. 5.

As another example, the optical interference array may include a plurality of 2 x 2 multimode interferometers as shown in FIG. 6.

As another example, the optical interference array may be a set of multi-mode interferometers, directional couplers, or other combinations of components that enable optical interference of two paths of light. Each component in the combination receives one of the optical signals of the second array, the outputs of the combinations together constituting the optical signals of the third array.

In one implementation, with continued reference to fig. 1, the photonic neural network 1 of this implementation may further include a light receiving unit 16 coupled to the projection computing unit 15 to receive the optical signals of the third array, where the light receiving unit 16 acquires the processed signals based on the optical signals of the third array.

Further, the photonic neural network 1 may further include a second electrical processing unit 17 coupled to the light receiving unit 16. The second electrical processing unit 17 may be an analog-to-digital converter for converting the processed signal characterized by an analog signal into a digital signal, thereby obtaining processed data that can be directly read and processed by a computer.

Further, the processed signal may also be amplified and the like.

In one implementation, the light receiving unit 16 may include: a photodetector for photoelectrically converting the optical signals of the third array; and the high-pass filtering unit is coupled with the optical detector and is used for performing high-pass filtering on the output of the optical detector to obtain the processed signal.

Specifically, the number of the photo-detectors may be multiple, and a photo-detector array composed of a plurality of photo-detectors may be coupled to the optical interference array of the projection calculation unit 15, where each photo-detector is configured to process one optical signal of the optical signals of the third array output by the corresponding optical interference unit.

For example, the photodetector may be coupled to either of the two outputs of the optical interference unit of FIG. 5 or FIG. 6. The optical detector is used for performing photoelectric conversion on the received optical signals of the third array to obtain electric signals, and the electric signals are the processed signals.

Further, the optical signals of the third array include an alternating current component related to a phase difference between the optical signals of the second array and the optical signals of the reference array, and the high-pass filtering unit may extract the alternating current component as the processed signal based on high-pass filtering. The alternating current component represents the real part of the complex amplitude of the light output by the photon matrix calculation unit 14.

In one variation, the light receiving unit may include: a balance detection unit to acquire the processed signal from the optical signal of the third array based on balance detection.

Specifically, the number of the balanced detection units may be multiple, and a light detector array composed of a plurality of balanced detection units may be coupled to the optical interference array of the projection calculation unit 15, where each balanced detection unit is configured to process one optical signal of the optical signals of the third array output by the corresponding optical interference unit.

The balanced detection unit may include a balanced photodetector.

For example, the balanced detection unit may include two detectors, which are respectively coupled to the two outputs of the optical interference unit shown in fig. 5 or fig. 6, and process the optical signals transmitted by the two outputs of the optical interference unit based on the balanced detection manner. The specific processing logic may be equivalent to subtracting one of the two outputs from the other.

Further, the optical signals of the third array include an alternating current component related to a phase difference between the optical signals of the second array and the optical signals of the reference array, and the balance detection unit extracts the alternating current component as the processed signal based on balance detection. The alternating current component represents the real part of the complex amplitude of the light output by the photon matrix calculation unit 14. Therefore, in the present modification, the real part of the complex amplitude of the light output by the photon matrix calculation unit 14 is directly obtained by the detector based on the balanced detection without additionally providing a high-pass filter unit.

Through the data processing process from the data to be processed to the processed data realized based on the photonic neural network 1, the operation of multiplying the input optical data by any real matrix M can be completed in the optical domain, and the nonlinear operation f can be synchronously realized. The real matrix M is not directly equal to the matrix S defined by the optical matrix calculation unit 14, but is determined by the real part of the matrix S: and M ═ Re (S) is called a pseudo real number photon calculation structure. Where Re () represents a real part.

Accordingly, the photonic neural network 1 of this embodiment may be a pseudo real network.

Next, the technical principle of the present embodiment will be described in detail by taking a pseudo real number network as an example.

The conventional optical neural network (also called as photonic neural network) uses Singular Value Decomposition (SVD) method to realize matrix multiplication, and its basic process is to use arbitrary matrix M_aDecomposed into two unitary matrices (denoted as unitary matrix U and unitary matrix V, respectively)^T) Multiplication M with a diagonal matrix Σ_a＝UΣV^TEach of these three matrices may be implemented by a mach-zehnder interferometer network, as shown in fig. 7. (superscript T stands for transpose)

Specifically, fig. 7 shows a 4 × 4 arbitrary matrix M composed of a plurality of optical interference units 141 cascaded in cascade_a(also referred to as a 4 × 4 convolution kernel unit). The optical interference unit 141 in fig. 7 is exemplarily illustrated by MZI. The MZI is provided with a phase shifter 142.

Further, In fig. 7, In1 to In4 are optical signals (i.e., input light) of the first array, and Out1 to Out4 are optical signals (i.e., output light) of the second array.

Further, the relationship between the input light and the output light may be as shown in equation (4):

wherein, I_in1The optical intensity, I, of the first array of optical signals received for In1_in2The optical intensity, I, of the optical signal of the second array received for In2_in3The optical intensity, I, of the optical signal of the third array received for In3_in4The optical intensity, φ, of the optical signal of the fourth array received for In4₁Phase of the output light for Out1, phi₂Phase of the output light for Out2, phi₃Phase of the output light for Out3, phi₄For the phase of the output light of Out4, I_out1Is the light intensity of the output light of Out1, I_out2Is the light intensity of the output light of Out2, I_out3Is the light intensity of the output light of Out3, I_out4For the intensity of the light output by Out4, exp () is an exponential function with e as the base, and j is an imaginary unit.

In fact, however, since the conventional optical detector can detect only the light intensity and cannot directly detect the phase, the matrix M formed by cascading three mach-zehnder interferometer networks as shown in fig. 7_aThe actual available matrix is limited to the modulus of the matrix.

Therefore, for a pseudo real photon neural network (pseudo real network for short), a unitary matrix and a coherent detection method can be used to realize an arbitrary matrix, thereby realizing matrix multiplication. For example, referring to fig. 8, the photon matrix calculation unit 14 according to this embodiment may only include a 4 × 4 unitary matrix as shown in fig. 8.

Specifically, the 4 × 4 unitary matrix may include 6 optical interference units 141, wherein one input arm and one interference arm of each optical interference unit 141 are provided with a phase shifter 142. The optical interference unit in fig. 8 is exemplarily illustrated by MZI.

In a pseudo real number network, by introducing the projection calculation unit 15, the representation of an arbitrary real matrix requires only one mach-zehnder interferometer network.

Taking an optical interference array composed of 2 × 2 directional couplers as shown in fig. 5 as an example of the projection calculation unit 15, one path of output of the photon matrix calculation unit 14 is output

As an input of a 2 × 2 directional coupler in the projection calculation unit 15 (corresponding to the first input arm 151), the reference light E with a certain intensity from the light source 13_refAs the other input of the 2 × 2 directional coupler (corresponding to the second input arm 152), the two outputs of the 2 × 2 directional coupler, i.e., the output of output 1, can be shown as formula (5), and the output of output 2 can be shown as formula (6):

wherein, I₁For outputting the light intensity of the optical signal output by 1, I₂For outputting the light intensity of the optical signal output 2, E_outFor calculating the complex amplitude, E, of the light output by the unit 14_refIs the complex amplitude of the reference light,

is the phase difference between the reference light and the output light of the photon matrix calculation unit 14, | a | is the modulus of the complex number a, a is E_refOr E_out。

Without loss of generality, the phase of the reference light is taken as 0 and the amplitude is taken as unit 1. Meanwhile, the reference light is not attenuated by the mach-zehnder interferometer network shown in fig. 7 or fig. 8, and is obviously stronger than the light calculated and output by the photon matrix calculating unit 14 (i.e. E)_ref>>E_out，“>>"is much larger than the symbol), the output result (i.e., the optical signal of the third array) passing through the projection calculation unit 15 is approximately one constant unit 1 plus or minus the real part of the complex amplitude of the light output by the photon matrix calculation unit 14

Twice as much.

Refer to formula (5) and formula (6), E_refAnd E_outCan be considered as a constant value, therefore | E in the formula_ref|²+|E_out|²Corresponding to a direct current component. Further, in the present invention,

will follow the phase difference

Is changed, corresponding to the alternating current component. In other words, the optical signals of the third array output by the projection calculation unit 15 include a constant value of the dc component and a phase difference

The alternating component of (a).

In one specific implementation, after a certain output (i.e. the aforementioned output 1 or output 2) of the optional 2 × 2 directional coupler is received by the optical detector in the projection calculation unit 15, the real part of the optical complex amplitude calculated and output by the photon matrix calculation unit 14 is obtained through high-pass filtering.

Specifically, by the cooperation of the photodetector and the high-pass filter, the dc component in the foregoing formula (5) or formula (6) can be eliminated and the ac component can be retained. Thus, the light receiving unit 16 according to the present embodiment can extract the real part of the complex amplitude of the light output by the photon matrix calculation unit 14.

In a variation, two outputs (i.e. output 1 and output 2) of the 2 × 2 directional coupler can be simultaneously connected to two different detectors, and the output of the light receiving unit 16 is proportional to the output of the light receiving unit in a balanced detection manner

Thus, the balance detection unit can directly eliminate the direct current component, and thus directly obtain an output 4 times the real part of the complex amplitude of light calculated and output by the photon matrix calculation unit 14.

The optical signal represented by the real part of the optical complex amplitude calculated and output by the photon matrix calculation unit 14 can be substantially regarded as the input optical signal (i.e. the optical signal of the first array) and the matrix M represented by the photon matrix calculation unit 14_aThe result of multiplying the real part of (c) is shown in equation (7):

for an M × n real number matrix M, it can be decomposed into M ═ U Σ V by SVD decomposition method^TWherein U and V^TAs unitary orthogonal matrices, i.e. UU^TI and VV^TI, Σ is a diagonal matrix, with values only on the main diagonal, and the other elements are all 0. Dimension of the upper three matricesDegree is respectively U ∈ R_m×m，Σ∈R_m×n，V∈R_n×n. In general, if the dimension of M is 3 × 3, the diagonal matrix Σ can be expressed based on equation (8):

wherein σ₁，σ₂，σ₃Are the singular values of the matrix M.

Dividing the diagonal matrix sigma by a coefficient alpha such that sigma₁′、σ₂′、σ₃' ≦ 1 because the diagonal matrix of the photonic MZI network can only attenuate light and cannot gain light, and the elements of the simulated diagonal matrix cannot be greater than 1. At this time, the expression form of the diagonal matrix Σ may be updated as shown in equation (9):

the coefficient alpha can be compensated for by gain matching at the location of the light detector. An imaginary part is added to the elements of the updated diagonal matrix Σ' such that each of its elements becomes a complex number modulo 1. In this case, based on the updated diagonal matrix Σ', the diagonal matrix Σ may be further updated to the diagonal matrix Σ ″ in the form shown in equation (10):

wherein the content of the first and second substances,

since the imaginary part may be positive or negative, the diagonal matrix Σ "may have 2^NN is the number of elements of the diagonal matrix Σ.

Further, formula (11) can be obtained:

wherein, Σ "^*Is a conjugate matrix of Σ ", and I is an identity matrix.

Since | det (Σ ") | 1 where det () is used to solve the determinant of the matrix, the diagonal matrix Σ" is a unitary matrix. Calculating M ═ U ∑ V^TBecause of U and V^TIs a unit orthogonal matrix and is a unitary matrix, sigma 'satisfies sigma'^*I and | det (Σ ") | 1 is also a unitary matrix, and the product of the unitary matrices is still a unitary matrix. Therefore, M' is also a unitary matrix.

Therefore, the real part of the matrix M' can be represented by a mach-zehnder interference network, as shown in equation (12).

Therefore, the real part of the unitary matrix can be used for representing a real matrix with a modulus smaller than 1 of any determinant, and the real matrix can be represented by multiplying the real matrix by a coefficient alpha. Therefore, an arbitrary real matrix can be represented by using only the mach-zehnder interferometer network shown in fig. 8.

In a typical application scenario, the photonic neural network 1 in this embodiment may be an optical neural network (pseudo real network for short) 2 based on pseudo real numbers as shown in fig. 9.

Specifically, the continuous light (shown as 1, 2.,. M.) output via the light source 13 shown in FIG. 1₁An optical signal array identification consisting of the beam optical signals) is input to the modulator array 21 via the waveguide array. The modulator array 21 may correspond to the optical modulation unit 11 in fig. 1, and is used for loading the signal to be processed on the optical amplitude and causing a non-linear variation relationship between the real amplitude and the phase of the light.

Further, the optical signal modulated by the modulator array 21 (shown as 1, 2.., M in the figure)₁Optical signal identification of a first array of beam optical signals) is input M via the waveguide array₂×M₁A convolution kernel unit 22. The M is₂×M₁The convolution kernel unit 22 may correspond to the photon matrix calculation unit 14 in fig. 1 for performing a matrix calculation on the optical signals of the first array. For example, the M₂×M₁The specific structure of the convolution kernel unit 22 may be as shown in fig. 7 or fig. 8.

Further, M₂×M₁The convolution kernel unit 22 computes the output optical signal (shown as 1, 2.., M in the figure)₂The optical signal identification of the second array of beam optical signals) is input to the projection unit array 23 via the waveguide array and interferes with the optical signals of the reference array within the projection unit array 23. The array of projection units 23 may correspond to the projection computation unit 15 in fig. 1 for extracting real parts in complex amplitudes of the optical signals of the second array based on optical interference. The array of projection units may also be referred to as a beam splitter array.

Further, the optical signals of the third array output by the projection unit array 23 are input into the detector array 24 through the waveguide array, and the obtained result is the output signal of the layer based on the pseudo real number optical neural network 2. Wherein the detector array 24 may include M₂Two paths of outputs of each projection unit in the projection unit array 23 are input into the corresponding balanced photodetectors. The specific structure of the projection unit may be as shown in fig. 5 or fig. 6.

Alternatively, detector array 24 may include M₂For each detector, two outputs of each projection unit in the projection unit array 23 can be selectively input to the detector corresponding to the detector array 24. The signal obtained by the detector is filtered by high pass, and then the real part of the optical complex amplitude calculated and output by the optical neural network 2 based on the pseudo real number of the layer can be obtained.

In the application scenario, the modulator array 21 performs intensity modulation and causes phase change of light, which is further correlated with M₂×M₁The convolution kernel units 22 are matched, and the structure shown in this embodiment can simultaneously implement linear and nonlinear operations of the neural network.

Continuous light to be input to the modulator array 21The complex amplitude of (D) is denoted as E_inFrom m to m₁A modulator m₁(0<m₁<M₁Integer) modulated, and then output as m-th optical signals of the first array₁The complex amplitude of the beam optical signal is

Wherein the real amplitude

And phase

There is a non-linear correspondence.

Taking a 3 × 3 real matrix M as an example, the real matrix M may be represented as M ═ U Σ V after singular value decomposition^TWherein U and V^TIs a real unitary matrix and sigma is a real diagonal matrix.

The real unitary matrix U is shown in equation (13):

real unitary matrix V^TAs shown in equation (14):

the real diagonal matrix Σ is shown in equation (15):

wherein, theta₁、θ₂And theta₃Phase parameters in the equivalent diagonal matrix Σ ', such as phase shift parameters of the phase shifter employed by the equivalent diagonal matrix Σ'; alpha is a proportionality coefficient.

An imaginary part is added to the element in the equivalent diagonal matrix Σ' so that each of its elements becomes a complex number modulo 1. In this case, the equivalent diagonal matrix Σ' is more equivalent to the diagonal matrix Σ ″ as shown in equation (16):

the complex amplitude of the optical signal of the first array output after modulation by the modulator array 21 is shown as formula (17):

wherein the real amplitude E_mAnd phase

There is a non-linear correspondence (m ═ 1,2, 3).

Accordingly, said M₂×M₁The complex amplitude of the optical signals of the second array output by the convolution kernel unit 22 is shown in equation (18):

the output after photoelectric conversion and differential operation by the balanced photodetector is shown in formula (19):

the output I_outIs characterized by being M₂×M₁The convolution kernel unit 22 outputs the real part of the complex amplitude of the optical signals of the second array. It can be seen that the processed signal finally obtained by the photonic neural network 1 is proportional to the alternating current component.

When the modulator in the modulator array 21 uses an ideal intensity modulator, only intensity modulation is performed on the input optical signal array without causing phase change of light, that is,

the output at this time is shown by equation (20):

when non-simple intensity modulators are used as the modulators in the modulator array 21, phase variations are introduced

Thus, M₂×M₁The convolution kernel unit 22 performs nonlinear operation in addition to the matrix multiplication.

Comparing the formula (19) and the formula (20), it can be found that the ratio of the two is not equal to 1, which indicates that the chirp effect of the light modulation unit 11 can affect the result of the matrix multiplication performed by the photon matrix calculation unit 14. That is, the output result of the light modulated by the light modulation unit 11 having the chirp effect after matrix calculation by the photon matrix calculation unit 14 has deviated from the strictly linear matrix multiplication effect, and the degree of deviation reflects the strength of the nonlinearity.

Accordingly, the expression of the nonlinear function of the nonlinear operation implemented based on the photonic neural network 1 according to the present embodiment can be defined as formula (21):

equation (21) is the result of dividing equation (19) by equation (20).

It can be seen that the transformation function f of the nonlinear transformation and the amount of change in phase change caused when the light modulation unit 11 performs real amplitude modulation

(simply referred to as phase variation) and phase shifter coefficient θ of equivalent diagonal matrix Σ_mAnd (4) correlating. Coefficient theta of the phase shifter_mIs the photon momentAnd the phase shift parameters of the phase shifter adopted by the equivalent diagonal matrix sigma' in the array computing unit.

From above, in conjunction with FIGS. 1 and 9, M₁The path of continuous light is modulated by the light modulation unit 11 to obtain M₁The path modulates the light (i.e., the optical signals of the first array). The first array employs M for optical signal input₂×M₁The photon matrix calculation unit 14 with the convolution kernel unit 22 structure outputs M₂Road lights (i.e., optical signals of the second array). Optical signals of the second array and M₂The path reference light interferes with the projection calculating unit 15 to output M₂An optical signal (i.e., an optical signal of the third array). The optical signal of the third array is M₂And the detector configured in a balanced detection mode receives and outputs the result of multiplying the real part of the photon matrix by the optical signal of the first array and nonlinear calculation. Finally, the second electrical processing unit 17 performs analog-to-digital conversion to complete data processing, and processed data are obtained.

M₂×M₁The specific connection mode of each optical interference unit 141 in the convolution kernel unit 22 can be adjusted according to actual needs, and is not limited herein.

When the photon matrix calculation unit 14 adopts a matrix M formed by cascading three Mach-Zehnder interferometer networks shown in FIG. 7_aIn this embodiment, since the projection calculation unit 15 extracts the real part of the optical complex amplitude calculated and output by the photon matrix calculation unit 14, the value of the real part of the optical complex amplitude detected by the optical detector in the subsequent light receiving unit 16 is obtained. Different from the prior art that only the intensity of light can be detected and only a linear operation result can be obtained, the real part of the complex amplitude of the light is finally detected and represented by the real part, and the real part represents the results of the linear operation and the nonlinear operation. Thus, the photonic neural network 1 according to the present embodiment can obtain the calculation results after two operations of linear and nonlinear operations.

In one implementation, with continued reference to fig. 1, the optical signals of the third array transmitted via the third waveguide array may be directly received and detected by the light receiving unit 16 to obtain linear and nonlinear operation results (i.e., the processed signals) of the photonic neural network 1.

In a variation, the optical signals of the third array transmitted via the third waveguide array may be transmitted back to the input of the photon matrix calculation unit 14 or to the input of another photon calculation structure to perform another round of operation.

For example, fig. 10 shows another photonic neural network 3 of the present embodiment, and only the differences between the photonic neural network 3 and the photonic neural network 1 shown in fig. 1 will be mainly described here.

The photonic neural network 3 shown in fig. 10 is different from the photonic neural network 1 shown in fig. 1 in that the optical signals of the third array output by the third waveguide array are received and detected by the light receiving unit 16 to obtain the real part of the optical complex amplitude output by the layer of photonic matrix calculation unit 14, and are also retransmitted to the input end of the photonic matrix calculation unit 14 for a new round of matrix calculation.

Accordingly, the optical signals of the third array obtained through another round of matrix calculation may be received by the light receiving unit 16, or may be returned to the input end of the photon matrix calculating unit 14 again to perform a new round of matrix calculation.

In a variation, as shown in fig. 10, the optical signals of the third array output by the third waveguide array are not transmitted back to the photon matrix calculation unit 14, but are transmitted to the new photon calculation structure 31 to start a new round of operation.

After one or more rounds of the above linear and nonlinear calculations, the finally output optical signal is received by the light receiving unit 16 anyway.

In one implementation, the multi-layer neural network 4 shown in fig. 11 can be constructed on the basis of the photonic neural network 1 shown in fig. 1.

Specifically, the light modulation unit 11 and the photon matrix calculation unit 14 coupled in fig. 1 are referred to as a neural network unit 41. Referring to fig. 11, the multi-layer neural network 4 may include a plurality of cascaded neural network units 41, wherein an output of a preceding neural network unit 41 is an input of a succeeding neural network unit 41, an input of a first neural network unit 41 is the signal to be processed, and an output of a last neural network unit 41 is the optical signal of the second array.

Further, the specific structure of the light modulation unit 11 in different neural network units 41 may be the same or different. For example, the basic structure of the optical modulation unit 11 in the first stage neural network unit 41 may adopt the micro-ring resonator 114 shown in fig. 3, and the basic structure of the optical modulation unit 11 in the second stage neural network unit 41 may adopt the semiconductor optical amplifier 117 shown in fig. 4.

Further, the specific structure of the photon matrix calculating unit 14 in different neural network units 41 may be the same or different. For example, the photon matrix calculation unit 14 in the intermediate stage neural network unit 41 may adopt a neural network structure composed of three mach-zehnder interferometer networks as shown in fig. 7, and the photon matrix calculation unit 14 in the last stage neural network unit 41 may adopt a pseudo-real number network composed of a single mach-zehnder interferometer network as shown in fig. 8.

As another example, the photon matrix computation unit 14 in the first level neural network unit 41 may be M₂×M₁The convolution kernel unit, and the photon matrix calculation unit 14 in the last stage neural network unit 41 may be M₃×M₂And a convolution kernel unit.

In this implementation, referring to FIG. 11, M₁The continuous light is modulated by the light modulation unit 11 in the first stage neural network unit 41, and the signal to be processed is loaded on the real amplitude of the light during modulation, and simultaneously a phase change related to the real amplitude is introduced due to the chirp effect of the modulator unit 11.

M modulated by light modulation unit 11₁The path modulation light is processed by the photon matrix calculation unit 14 in the first-stage neural network unit 41 and then output M₂And (4) light path. The photon matrix calculation unit 14 performs simultaneous calculation of linearity and nonlinearity in performing matrix calculation.

This M₂The path light sequentially enters the second level neural network unit 41 and the third level neural network unit 41 and … until the path light passes through the last stepThe first-level neural network unit 41 outputs M₃And (4) light path.

This M₃Road light projection calculation unit 15 and M₃The reference light interferes and outputs M₃For the optical signal, i.e. the optical signal of the third array.

Then, in the light receiving unit 16, M₃For optical signal by M₃And the detector configured by adopting a balanced detection mode receives and outputs the multilayer neural network 4. The output result of the multi-layer neural network 4 is the result obtained by multiplying the real matrix part of the neural network unit 41 by the input optical signal and then performing nonlinear calculation.

Finally, M₃The signals after the circuit processing are subjected to analog-to-digital conversion by the second electric processing unit 17 to complete data processing, and processed data are obtained.

In one implementation, the recurrent neural network 5 shown in fig. 12 can be constructed on the basis of the photonic neural network 1 shown in fig. 1.

Specifically, in addition to the coupling relationship shown in fig. 1, in the recurrent neural network 5 shown in fig. 12, the output end of the projection calculation unit 15 may be coupled to the input end of the photon matrix calculation unit 14, so as to input the optical signals of the third array into the photon matrix calculation unit 14 again to perform matrix calculation cyclically.

Further, since modulation by the light modulation unit 11 is not performed before the re-input to the photon matrix calculation unit 14, only linear operation is realized in matrix calculation cyclically. Alternatively, if the output of the photon matrix calculation unit 14 is coupled to an optical modulation unit without chirp effect (such as an ideal intensity modulator), then pure linear operation is performed when the output is transmitted to the photon matrix calculation unit 14 again.

In this embodiment, referring to fig. 12, after N paths of continuous light pass through the optical modulation unit 11, the signal to be processed is loaded on the real amplitude of the light, and a phase change related to the real amplitude is introduced due to the chirp effect.

The N paths of modulated light modulated by the light modulation unit 11 are output after passing through the photon matrix calculation unit 14, so that the simultaneous calculation of linearity and nonlinearity is realized.

The N paths of light output by the photon matrix calculation unit 14 interfere with the N paths of reference light in the projection calculation unit 15, and output N pairs of optical signals to enter the third waveguide array.

For the N pairs of optical signals of the third waveguide array, the light of a part of the ports is connected to the input end of the photon matrix calculation unit 14. The light of the other part of the ports is received by the light receiving unit 16 to output the calculation result of the recurrent neural network 5, and finally the data processing is completed by the analog-to-digital conversion of the second electrical processing unit 17.

Fig. 13 and 14 are nonlinear functions obtained by simulation when the optical modulation unit 11 in the photonic neural network 1 shown in fig. 1 adopts the structure shown in fig. 2. In the simulation of FIG. 13, the phase shift parameter β of the second phase shifter 113 provided in the Mach-Zehnder interferometer 111 of 1 × 1₂When the phase shift parameter β of the second phase shifter 113 provided in the mach-zehnder interferometer 111 in the simulation of fig. 14 is 0, the phase shift parameter β is₂Pi. In fig. 13 and 14, the abscissa is the result of normalizing the real amplitude of the optical signal of the first array by using the input light (i.e., the optical signal array) as a normalization parameter (referred to as normalized real amplitude), and the ordinate is the transmittance (which may also be referred to as transmittance).

FIG. 13 shows a curve of the phase shift parameter β of the second phase shifter 113₂When the phase parameter θ of the equivalent diagonal matrix is different, the nonlinear response of the real amplitude of the input light and the real amplitude of the output light is obtained.

FIG. 14 shows a curve of the phase shift parameter β of the second phase shifter 113₂And pi, the nonlinear response of the real amplitude of the input light and the real amplitude of the output light obtained when different phase parameters theta of the equivalent diagonal matrix are adopted.

Wherein, the solid line corresponds to θ being 0; the dotted line corresponds to θ being 0.2 pi; the dotted line corresponds to θ being 0.3 π; the solid line plus the circle mark corresponds to 0.7 pi; realizing that the inverted triangle mark corresponds to 0.8 pi; the solid plus square mark corresponds to θ ═ pi.

Due to the phase shift parameter β of the second phase shifter 113₂Different, and therefore the phase offset of the nonlinear function obtained in FIGS. 13 and 14 is notThe same is true. At this time, if the phase shift parameter β of the first phase shifter 112 for modulating the intensity is adjusted₁The mode of the light amplitude will change accordingly.

Fig. 15 and 16 are nonlinear functions obtained by simulation when the optical modulation unit 11 in the photonic neural network 1 shown in fig. 1 adopts the structure shown in fig. 3. Wherein, the phase shift parameter β of the second phase shifter 113 set by the micro-ring resonator 114 in the simulation of FIG. 15₂When the phase shift parameter β of the second phase shifter 113 provided in the microring resonator 114 is 0, fig. 16 is simulated₂Pi. In fig. 14 and 15, the abscissa represents normalized real amplitude, and the ordinate represents transmittance.

The parameters of the microring resonator 114 are as follows: radius R of the microring resonator 114 is 5um, and effective refractive index n of the annular waveguide 115_eff3.476, self-coupling coefficient r 0.7, quality factor Q379, resonance wavelength λ_res＝1538.053nm。

FIG. 15 shows a curve of the phase shift parameter β of the second phase shifter 113₂When the phase parameter θ of the equivalent diagonal matrix is different, the nonlinear response of the real amplitude of the input light and the real amplitude of the output light is obtained.

FIG. 15 shows a curve of the phase shift parameter β of the second phase shifter 113₂Pi, the nonlinear response of the real amplitude of the input light and the real amplitude of the output light obtained using the phase parameter θ of the different equivalent diagonal matrices.

Further, the non-linear correspondence between the real amplitude and the phase of the light is due to a chirp effect of the light modulation unit. Thus, the photonic neural network described in this implementation is based on the modulator unit loading the signal to be processed at the real amplitude of the light while introducing phase changes related to the real amplitude. The output optical signal of the modulator unit is the input optical signal of the photon matrix calculation unit, i.e. the optical signal of the first array. The phase of the optical signals of the first array changes simultaneously when the optical signals are subjected to real amplitude modulation, that is, the real amplitude modulation and the phase change are coupled, so that the result of performing nonlinear operation and then performing linear operation on the optical signals of the first array can be obtained through the matrix calculation of the photon matrix calculation unit.

Therefore, the photonic neural network can simultaneously realize linear operation and nonlinear operation based on the chirp effect of the modulator unit and the cooperation of the photon matrix calculation unit.

Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A photonic neural network, comprising:

the optical modulation unit is used for modulating a signal to be processed to real amplitude of light, introducing a nonlinear corresponding relation between the real amplitude and the phase of the light during modulation, and recording the output of the optical modulation unit as an optical signal of a first array;

a photon matrix calculation unit coupled to the light modulation unit to receive the optical signals of the first array, the photon matrix calculation unit performing matrix calculation on the optical signals of the first array to obtain optical signals of a second array, wherein matrix multiplication and nonlinear transformation are simultaneously performed when performing matrix calculation on the optical signals of the first array;

a projection computation unit coupled to the photon matrix computation unit to receive the optical signals of the second array, the projection computation unit extracting real parts of complex amplitudes of the optical signals of the second array based on optical interference to obtain optical signals of a third array, wherein the real parts represent operation results of performing the matrix multiplication and nonlinear transformation on the optical signals of the first array;

a light receiving unit coupled with the projection computation unit to receive the optical signals of the third array, the light receiving unit acquiring processed signals based on the optical signals of the third array.

2. The photonic neural network according to claim 1, wherein the transformation function of the nonlinear transformation is related to a phase change amount caused when the optical modulation unit performs real amplitude modulation and a phase shifter coefficient, wherein the phase shifter coefficient is a phase shift parameter of a phase shifter employed by an equivalent diagonal matrix in the photonic matrix calculation unit.

3. The photonic neural network of claim 1, wherein the non-linear correspondence between the real amplitude and the phase of the light is due to a chirp effect of the light modulating unit.

4. The photonic neural network of claim 1, wherein the optical modulating unit comprises: an optical interference unit, an upper arm or a lower arm of which is provided with a first phase shifter to modulate the signal to be processed to a real amplitude of light while adjusting a non-linear correspondence between the real amplitude and a phase of the light.

5. The photonic neural network of claim 4, wherein a second phase shifter is provided at an input or output of the optical interference unit to adjust the non-linear correspondence between the real amplitude and the phase of the light.

6. The photonic neural network of claim 1, wherein the optical modulating unit comprises: the micro-ring resonator comprises a ring waveguide and a straight waveguide which are coupled, wherein the ring waveguide is provided with a first phase shifter to modulate the signal to be processed to the real amplitude of light and adjust the nonlinear corresponding relation between the real amplitude and the phase of the light.

7. The photonic neural network of claim 6, wherein the input or output end of the straight waveguide is provided with a second phase shifter to adjust the non-linear correspondence between the real amplitude and the phase of the light.

8. The photonic neural network of claim 1, wherein the optical modulating unit comprises: and the input end of the semiconductor optical amplifier receives the light and the signal to be processed, and the output end of the semiconductor optical amplifier outputs the optical signal of the first array.

9. The photonic neural network of claim 8, wherein a second phase shifter is provided at an input or output of the semiconductor optical amplifier to adjust the non-linear correspondence between the real amplitude and the phase of the light.

10. The photonic neural network of claim 5, 7 or 9, wherein the offset of the phase is determined according to a dephasing parameter of the second phase shifter, wherein the offset of the phase is a starting phase of a transform function of the non-linear transformation.

11. The photonic neural network of claim 1, wherein the projection computation unit comprises:

and the optical interference unit comprises a first input arm and a second input arm, wherein the first input arm receives the optical signals of the second array, the second input arm receives the optical signals of the reference array, and the optical signals of the second array and the optical signals of the reference array are output to obtain the optical signals of the third array after the optical interference unit generates optical interference.

12. The photonic neural network of claim 1, wherein the light receiving unit comprises: a photodetector for photoelectrically converting the optical signals of the third array;

and the high-pass filtering unit is coupled with the optical detector and is used for performing high-pass filtering on the output of the optical detector to obtain the processed signal.

13. The photonic neural network of claim 12, wherein the optical signals of the third array comprise an alternating current component related to a phase difference, and the high-pass filtering unit extracts the alternating current component as the processed signal based on high-pass filtering, wherein the phase difference is a phase difference between the optical signals of the second array and the optical signals of the reference array.

14. The photonic neural network of claim 1, wherein the light receiving unit comprises: a balance detection unit for acquiring the processed signal from the optical signals of the third array based on a balance detection method.

15. The photonic neural network of claim 14, wherein the optical signals of the third array include an alternating current component related to a phase difference, and the balance detection unit extracts the alternating current component as the processed signal based on a balance detection method, wherein the phase difference is a phase difference between the optical signals of the second array and the optical signals of the reference array.

16. The photonic neural network of claim 15, wherein the processed signal is proportional to the alternating current component.

17. The photonic neural network of claim 1, wherein the photonic matrix calculating unit is a unitary matrix including a plurality of optical interference units connected in series and parallel with each other.

18. The photonic neural network according to claim 1, wherein the coupled optical modulation unit and the photonic matrix calculation unit are denoted as a neural network unit, and the photonic neural network comprises a plurality of cascaded neural network units, wherein an output of a preceding neural network unit is an input of a succeeding neural network unit, an input of a first neural network unit is the signal to be processed, and an output of a last neural network unit is the optical signal of the second array.

19. The photonic neural network of claim 1, wherein an output of the projection computation unit is coupled to an input of the photonic matrix computation unit to re-input the optical signals of the third array to the photonic matrix computation unit for performing the matrix computation cyclically.

20. The photonic neural network of claim 1, wherein the photonic neural network is used for image processing, image recognition, voice recognition, genetic sequencing, quantum communication, or quantum computation.