CN111680796B - On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip - Google Patents

On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip Download PDF

Info

Publication number
CN111680796B
CN111680796B CN202010605656.1A CN202010605656A CN111680796B CN 111680796 B CN111680796 B CN 111680796B CN 202010605656 A CN202010605656 A CN 202010605656A CN 111680796 B CN111680796 B CN 111680796B
Authority
CN
China
Prior art keywords
output
mth
neural network
optical
repeating unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010605656.1A
Other languages
Chinese (zh)
Other versions
CN111680796A (en
Inventor
罗章
赖明澈
翦杰
王子聪
孙岩
黎渊
张建民
徐金波
董德尊
熊泽宇
欧洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202010605656.1A priority Critical patent/CN111680796B/en
Publication of CN111680796A publication Critical patent/CN111680796A/en
Application granted granted Critical
Publication of CN111680796B publication Critical patent/CN111680796B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
    • G06N3/0675—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means using electro-optical, acousto-optical or opto-electronic means
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Optical Communication System (AREA)
  • Optical Modulation, Optical Deflection, Nonlinear Optics, Optical Demodulation, Optical Logic Elements (AREA)

Abstract

The invention discloses an on-chip photonic neural network device, a chip based on a cascade optical couplerThe application method comprises that the photonic neural network device comprises an all-optical photonic neural network, wherein the all-optical photonic neural network comprises an amplitude modulation unit, a multi-layer neuron and a detection unit, and the amplitude modulation unit comprises N 0 A circuit modulator, the multi-layer neuron including M repeating units, the repeating units including a plurality of amplitude-modulated phase modulators and a MIMO optical coupler, the plurality of amplitude-modulated phase modulators of any mth repeating unit being used to divide N into N m‑1 The optical signals input by the paths are subjected to amplitude modulation and phase modulation and then are obtained to N through the MIMO optical coupler m A path output, the detection unit comprises N M A path detector. The invention can realize the deep learning system of the all-optical photonic neural network with large bit width input in the on-chip integrated system based on the cascade optical coupler, has extremely low calculation cost and low power consumption in the forward reasoning process after training and learning are finished, and the on-chip system is easy to integrate and supports large bit width data input.

Description

On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip
Technical Field
The invention relates to a neural network deep learning technology, in particular to an on-chip photonic neural network device and a chip based on a cascade optical coupler and an application method thereof, which are used for realizing an all-optical photonic neural network deep learning system with large bit width input in an on-chip integrated system based on the cascade optical coupler.
Background
Artificial intelligence is a new technical science for researching and developing intelligent theory, method technology and application system for simulating extended and extended people. On one hand, the development of algorithms, calculation power and big data drives artificial intelligence to break through in all aspects of calculation intelligence, perception intelligence and cognition intelligence from three dimensions; on the other hand, artificial intelligence is being rapidly fused with various industries, and the traditional industry is assisted in upgrading and transforming, improving quality and enhancing efficiency, and inducing brand new industry surge in the global scope.
The dominant technology of artificial intelligence today is deep neural networks for large data processing. The deep neural network is a computational network model which is inspired by a brain signal processing process, and since the computational model is proposed, the computational network model is widely applied to the fields of computer vision, voice recognition, natural semantic understanding, man-machine game and the like, and the performance of machine learning in the related fields is greatly improved.
In recent decades, the neural network accelerator is greatly improved by the continuous improvement of a computing architecture, various technical routes such as a convolver-based structure, a multi-tree-shaped structure, a matrix multiplier-based structure and the like are continuously emerging, and deep learning is pushed to obtain one more achievement with milestone significance. However, the current electric domain computing system computing hardware still has a great gap from the brain in terms of power consumption and performance when processing the neural network. Taking an artificial intelligence program alpha Go based on a deep learning algorithm and based on a neural network accelerator TPU with a matrix multiplier structure, which is proposed by Google corporation, as an example, the power consumption of a computing system required by alpha Go is 20MW, and the volume is up to 30m 3 The method comprises the steps of carrying out a first treatment on the surface of the The human brain power consumption for running the same calculation task is only 20W, and the volume is only 0.0015m 3 . Under the current architecture of an electric domain computing system, the hardware performance of the neural network cannot be fully utilized. Although the electric domain computing system is continuously improved in computing architecture according to the characteristics of the neural network, the electric domain computing system is still centered on the data processing computation, and the processor reads the data computation from the memoryAfter processing, the data is written back to the memory. The neural network algorithm is centered on the data stream, and the data operation of the neural network algorithm is completed in the neural network following the data stream. The neural network algorithm is realized on the basis of hardware taking data processing as a center, so that massive data streams are necessarily required to be meshed and stored, and are moved to an operation processing component for operation processing when needed. On the one hand, this requires a massive storage, on the other hand, since the storage of data and the arithmetic processing of the data are separated from each other, which in turn leads to the movement of massive data. It is this architectural conflict that results in inefficiencies in the operation of the neural network algorithms by the electrical domain computing hardware.
While computer domain students are continuously improving the computing architecture to improve the computing efficiency of the neural network, optical domain students are also continuously proposing novel schemes to realize the neural network computing. Photon computing research has also made significant progress over the years, but limited by the physical nature of the photon's zero-stationary mass, has not yet made an exciting breakthrough in achieving optical caching. Without optical information storage, the traditional digital logic computer cannot be finally realized based on the photonic device, and the development of a photonic computing system is still in existence due to the lack of optical storage. But the architecture of the neural network algorithm which takes the data stream as the center can be well matched with a photon computing system. On the other hand, each neuron of the neural network algorithm can be used for input data without buffering, so that the defect that the photon computing network has no buffering function is not important.
In 2017, the photon computing team of the united states MIT first proposed an all-optical photonic neural network, which is composed of Mach-zehnder interferometers (Mach-Zehnder interferometer, MZI) with cascaded additional phase shift functions, and can realize arbitrary linear transformation and output of 4 input signals. The team uses the network to complete linear transformation and is assisted by a computer to complete nonlinear transformation, and the two are matched to realize a first-stage 4-path neuron layer. By multiplexing the all-optical photonic neural network, the 4-layer neural network is realized, and the recognition accuracy of 4 vowels exceeds 90%. The first reported all-optical photonic neural network capable of actually working is poor in realizability, on one hand, the all-optical photonic neural network is not used for realizing complete neural network operation in an optical domain, but more serious, the photonic network is based on an MZI device array, when the input bit width is higher and the number of network layers is larger, a large number of MZI devices are needed, the network scale is extremely huge, and the process preparation difficulty is great. This is because the implementation of the all-optical photonic neural network is a mechanical simulation of electronic logic computation, and does not make good use of photonic characteristics.
The U.S. UCLA team in 2019 proposed a deep diffraction neural network based on a multi-layer diffraction film structure. The design utilizes expansibility, locality and coherence of a diffraction light field to complete convolution linear transformation between two layers of neurons, utilizes the projection rate and phase shift of a diffraction film as learning parameters, fully utilizes the physical characteristics of the light field, has concise physical realization of the neurons and has good realizability. The team successfully realizes the identification of 28 pixels by 28 pixels pictures by using a convolutional neural network formed by 5 layers of diffraction films, and the identification success rate is up to 93%. Meanwhile, the scheme is used as a complete optical system, transmission imaging of an external light source is used as input, and the optical neural network system has almost no power consumption in the recognition process after learning. The scheme promotes the all-optical photonic neural network to take a great step towards practical application, and attracts wide attention in terms of excellent power consumption performance. However, since the implementation principle is natural two-dimensional convolution-like calculation, the method cannot be applied to neural network algorithms of non-two-dimensional convolution, and in practical application, the convolution neural network only accounts for less than 10% of all neural network applications. On the other hand, the scheme is based on the principle of space optics, and is difficult to integrate into a system-on-chip in a dimension-reducing way. The chip is a mainstream way for realizing miniaturization and integration of the photon computing system, and the all-optical photon neural network is taken as a special photon computing system, so that on-chip integration is an important development direction.
Disclosure of Invention
The invention aims to solve the technical problems: aiming at the problems in the prior art, the invention provides an on-chip photonic neural network device and a chip based on a cascade optical coupler and an application method thereof.
In order to solve the technical problems, the invention adopts the following technical scheme:
an on-chip photonic neural network device based on a cascade optical coupler comprises an all-optical photonic neural network, wherein the all-optical photonic neural network comprises an amplitude modulation unit, a multi-layer neuron and a detection unit, and the amplitude modulation unit comprises N 0 The multi-layer neuron comprises M repeating units which are sequentially layered and cascaded, the repeating units comprise a plurality of amplitude modulation modulators and a multi-input multi-output optical coupler, and the plurality of amplitude modulation modulators of any mth repeating unit are used for respectively amplifying N m-1 The optical signal inputted by the path is subjected to amplitude modulation and phase modulation and then is obtained by a multi-input multi-output optical coupler to obtain N m The detection unit comprises N with the Mth repeating unit M And the detectors are in one-to-one correspondence with the path output optical signals.
Optionally, the multi-input multi-output optical coupler is a star coupler or a multi-mode interference coupler.
Optionally, an input end of the amplitude modulation unit is connected with a coherent light source, and the coherent light source comprises M output light paths.
Optionally, the coherent light source comprises a laser and a single-ended input multi-output optical coupler, an output end of the laser is connected with an input of the single-ended input multi-output optical coupler, and an output end of the single-ended input multi-output optical coupler comprises N 0 And a plurality of output light paths.
In addition, the invention also provides an all-optical photon neural network chip based on the cascade optical coupler, which comprises a chip body, wherein the chip body comprises an amplitude modulation unit, a plurality of layers of neurons and a detection unit, and the amplitude modulation unit comprises N 0 The multi-layer neuron comprises M heavy-duty light emitting devices which are sequentially layered and cascadedA complex unit including a plurality of AM phase modulators and a multi-input multi-output optical coupler, the plurality of AM phase modulators of any mth repeating unit being used for respectively coupling N m-1 The optical signal inputted by the path is subjected to amplitude modulation and phase modulation and then is obtained by a multi-input multi-output optical coupler to obtain N m The detection unit comprises N with the Mth repeating unit M And the detectors are in one-to-one correspondence with the path output optical signals.
Optionally, the multi-input multi-output optical coupler is a star coupler or a multi-mode interference coupler.
In addition, the invention also provides an application method of the on-chip photonic neural network device based on the cascade optical coupler, and the step of classifying and reasoning through the photonic neural network device comprises the following steps:
1) Splitting the laser output into N by a single-end input multi-end output optical coupler 0 All coherent light sources are arranged, and N 0 The intensity of all coherent light sources is I 0 ；
2) Will N 0 Amplitude modulation is carried out on an amplitude modulator loaded into an amplitude modulation unit by an intensity modulation mode through a full coherent light source to form optical input excitation, and the complex amplitude of the ith excitation is obtained
Figure GDA0004203978610000031
Is->
Figure GDA0004203978610000032
Wherein->
Figure GDA0004203978610000033
Represents normalized i-th input data, with a maximum value of 1, N 0 The optical input stimulus of the path is marked as +.>
Figure GDA0004203978610000034
3)N 0 Optical excitation of path a (0) Amplitude-modulated phase modulator respectively entering first repeating unit in multilayer neuron for amplitude and phase modulation and outputting to first repeating unitMultiple-input multiple-output optical coupler, through which N is output after coherent superposition 1 The complex amplitude of the i-th output signal is
Figure GDA0004203978610000035
N 1 The optical output of the path is marked->
Figure GDA0004203978610000041
In any of the following m-th repeating units: n (N) m-1 Road light signal->
Figure GDA0004203978610000042
Multiple-input multiple-output optical coupler for outputting amplitude and phase modulated input amplitude modulator to mth repeating unit to output N m Way signal->
Figure GDA0004203978610000043
N of the multi-input multi-output optocoupler output of the final Mth repeating unit M Road light signal->
Figure GDA0004203978610000044
Directly captured by the photodetector in the detection unit to output N M Road light intensity->
Figure GDA0004203978610000045
Optionally, the functional expression for amplitude and phase modulation in step 3) is as follows:
Figure GDA0004203978610000046
in the above-mentioned method, the step of,
Figure GDA0004203978610000047
output of the kth way of the mth repeating unit,/->
Figure GDA0004203978610000048
Transformation matrix N being the mth repeating unit m ×N m-1 The k-th to i-th matrix element, j is an imaginary unit,/>
Figure GDA0004203978610000049
Output of the ith path which is the m-1 th repeating unit, +.>
Figure GDA00042039786100000410
An amplitude attenuation factor of the mth level i path,>
Figure GDA00042039786100000411
an additional phase for the mth stage i-th way.
Optionally, the N M Road light intensity
Figure GDA00042039786100000412
Output light intensity I of any ith path i The functional expression of (2) is as follows:
Figure GDA00042039786100000413
in the above-mentioned method, the step of,
Figure GDA00042039786100000414
is->
Figure GDA00042039786100000415
Conjugation amount of->
Figure GDA00042039786100000416
The output of the ith path of the Mth repeating unit.
Optionally, step 1) further includes a step of training the photonic neural network device:
s1) randomly selecting n samples from the training set as a sample group, wherein each sample in the sample group
Figure GDA00042039786100000417
With corresponding labels->
Figure GDA00042039786100000418
Wherein x is i All co-N representing sample x 0 An ith component in the data, where y i All co-N representing sample y M The ith component in the data, each component y of tag y i Only the component value of the position corresponding to the classification set of the sample is 1, and the rest components are 0;
s2) setting initial parameters for amplitude and phase modulation in the all-optical photon neural network, inputting each sample in the sample group into the all-optical photon neural network to output an inference result, comparing the inference result with a label y to obtain accuracy evaluation, and finishing training and exiting if the accuracy meets the requirement; otherwise, jumping to execute the next step;
s3) outputting light intensity to all-optical photon neural network
Figure GDA00042039786100000419
Press->
Figure GDA00042039786100000420
The normalized mode is marked as +.>
Figure GDA00042039786100000421
Then->
Figure GDA00042039786100000422
According to a preset back propagation bias function +.>
Figure GDA00042039786100000423
Starting back propagation from the mth repeating unit, and adjusting the amplitude attenuation factor in the mth repeating unit according to a preset gradient during back propagation
Figure GDA00042039786100000424
Additional phase->
Figure GDA00042039786100000425
Jump execution step S1);
the back propagation bias function
Figure GDA00042039786100000426
The functional expression of the initial value of (2) is as follows:
Figure GDA0004203978610000051
in the above-mentioned method, the step of,
Figure GDA0004203978610000052
s is the initial value of the back propagation bias function i Ith path of normalization result, y, representing output light intensity of all-optical photonic neural network i All co-N representing sample y M The ith component, s, in the data k The k-th path normalization result, y, of the output light intensity of the all-optical photonic neural network is shown k All co-N representing sample y M The kth component in the data, n represents the number of samples in the sample group, +.>
Figure GDA0004203978610000053
Conjugation of the output of the ith path representing the mth repeating unit;
the back propagation bias function
Figure GDA0004203978610000054
The recurrence relation of the back propagation is shown as follows:
Figure GDA0004203978610000055
in the above-mentioned method, the step of,
Figure GDA0004203978610000056
representing the back propagation bias function to the m-1 th repeating unit,/>
Figure GDA0004203978610000057
An amplitude attenuation factor of the mth-stage kth path,>
Figure GDA0004203978610000058
for the additional phase of the mth stage k-th way, j is the imaginary unit, ++>
Figure GDA0004203978610000059
Transformation matrix N being the mth repeating unit m ×N m-1 The conjugate of the matrix element of the i-th to k-th way,/th>
Figure GDA00042039786100000510
An amplitude attenuation factor of the mth level i path,>
Figure GDA00042039786100000511
representing a back propagation bias function propagated to the mth repeating unit;
amplitude attenuation factor in the mth repeating unit
Figure GDA00042039786100000512
The gradient function expression of (2) is as follows:
Figure GDA00042039786100000513
in the above-mentioned method, the step of,
Figure GDA00042039786100000514
an amplitude attenuation factor +.>
Figure GDA00042039786100000515
Re represents the real part operation, +.>
Figure GDA00042039786100000516
Conjugate of the output of the ith path representing the mth repeating unit, < >>
Figure GDA00042039786100000517
Representing a back propagation bias function propagated to the mth repeating unit;
additional phases in the mth repeat unit
Figure GDA00042039786100000518
The gradient function expression of (2) is as follows:
Figure GDA00042039786100000519
in the above-mentioned method, the step of,
Figure GDA00042039786100000520
represents the additional phase of the ith path of the mth stage +.>
Figure GDA00042039786100000521
Im represents the operation of taking the imaginary part,
Figure GDA00042039786100000522
conjugate of the output of the ith path representing the mth repeating unit, < >>
Figure GDA00042039786100000523
Representing the back propagation bias function propagated to the mth repeating unit.
Compared with the prior art, the on-chip photonic neural network device based on the cascade optical coupler has the following advantages: the cascade optical coupler comprises an all-optical photonic neural network, wherein the all-optical photonic neural network comprises an amplitude modulation unit 1, a multi-layer neuron 2 and a detection unit 3, and the amplitude modulation unit 1 comprises N 0 The multi-layer neuron 2 comprises M repeating units which are sequentially layered and cascaded, wherein each repeating unit comprises a plurality of amplitude modulation modulators and a multi-input multi-output optical coupler, and the plurality of amplitude modulation modulators of any mth repeating unit are used for respectively amplifying N m-1 After amplitude modulation and phase modulation, the optical signal input by the path is optically coupled through multi-input multi-output lightThe device obtains N m The detection unit 3 includes N with the Mth repeating unit M The invention can realize the deep learning system of the all-optical photonic neural network with large bit width input in the on-chip integrated system based on the cascade optical coupler, has extremely low calculation cost and low power consumption in the forward reasoning process after training and learning are finished, and is easy to integrate and support large bit width data input.
Drawings
Fig. 1 is a schematic structural diagram of a photonic neural network device according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a training principle of a photonic neural network device according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of an optical path principle of a single-ended input multi-ended output optical coupler according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of an optical path principle of a multi-input multi-output optical coupler according to an embodiment of the present invention.
Detailed Description
As shown in fig. 1, the present embodiment provides an on-chip photonic neural network device based on a cascade optical coupler, which includes an all-optical photonic neural network, the all-optical photonic neural network includes an amplitude modulation unit 1, a multi-layer neuron 2, and a detection unit 3, and the amplitude modulation unit 1 includes N 0 The multi-layer neuron 2 comprises M repeating units which are sequentially layered and cascaded, wherein each repeating unit comprises a plurality of amplitude modulation modulators and a multi-input multi-output optical coupler, and the plurality of amplitude modulation modulators of any M repeating unit are used for respectively amplifying N m-1 The optical signal inputted by the path is subjected to amplitude modulation and phase modulation and then is obtained by a multi-input multi-output optical coupler to obtain N m The optical signal output by the path, the detecting unit 3 includes N with the mth repeating unit M And the detectors are in one-to-one correspondence with the path output optical signals.
The photonic neural network chip is composed of cascaded on-chip optical couplers, typical Multi-port on-chip optical couplers including, but not limited to, star couplers (Star couplers) and multimode interference couplers (Multi-Mode Interference Coupler, MMI couplers). The low-linewidth high-power laser source is used for realizing the generation of a Multi-path coherent light source by beam splitting through a Single-end Input Multi-Output Coupler (figure 3), and is injected into an all-optical photonic neural network. The multi-bit wide data to be classified is modulated onto each optical signal using an electrical injection or thermal control modulator, the intensity of the optical signal being proportional to the normalized input data. In this embodiment, the multi-input multi-output optical coupler is a star coupler or a multi-mode interference coupler.
In this embodiment, the input end of the amplitude modulation unit 1 is connected with a coherent light source 4, and the coherent light source 4 includes M output light paths.
In this embodiment, the coherent light source 4 includes a laser 41 and a single-ended input multiple-output optical coupler 42, the output end of the laser 41 is connected with the input of the single-ended input multiple-output optical coupler 42, and the output end of the single-ended input multiple-output optical coupler 42 includes N 0 And a plurality of output light paths.
On the basis, the embodiment also provides an all-optical photon neural network chip based on the cascade optical coupler, which comprises a chip body, wherein the chip body comprises an amplitude modulation unit 1, a plurality of layers of neurons 2 and a detection unit 3, and the amplitude modulation unit 1 comprises N 0 The multi-layer neuron 2 comprises M repeating units which are sequentially layered and cascaded, wherein each repeating unit comprises a plurality of amplitude modulation modulators and a multi-input multi-output optical coupler, and the plurality of amplitude modulation modulators of any M repeating unit are used for respectively amplifying N m-1 The optical signal inputted by the path is subjected to amplitude modulation and phase modulation and then is obtained by a multi-input multi-output optical coupler to obtain N m The optical signal output by the path, the detecting unit 3 includes N with the mth repeating unit M And the detectors are in one-to-one correspondence with the path output optical signals.
As can be seen from the above, in this embodiment, by using the coherent coupling characteristic of the multiport on-chip optical coupler, linear transformation between output and input signals of two adjacent layers of neurons is implemented in an on-chip integrated optical system by using a single photonic device, and fine tuning of the linear transformation is implemented by modulating amplitude and phase of the input signals of the optical coupler, so as to complete the functions of the basic unit of the neurons;on the basis, cascading the multi-stage on-chip optical coupler to construct a multi-layer neuron and form an on-chip integrated all-optical photonic neural network; and a single-stage iterative reverse propagation gradient descent learning algorithm compatible with a complex propagation function and designed based on a standard reverse propagation gradient descent algorithm framework is matched, so that the efficient learning convergence of the photonic neural network is realized. Referring to fig. 2, the on-chip photonic neural network system based on the cascade optical coupler is composed of two parts of all-optical photonic neural network chip hardware based on the cascade optical coupler and learning feedback network software matched with the all-optical photonic neural network chip hardware based on the cascade optical coupler and based on a counter-propagation gradient descent algorithm. Referring to fig. 4, the modulated optical signal a (0) And then the amplitude and the phase of the signal are subjected to secondary modulation by a primary amplitude modulation modulator and then enter a Multi-Input Multi-Output optical Coupler (fig. 4) for coherent superposition. The amplitude modulation phase modulator modulates the amplitude and the phase of the optical signal at the same time, the modulation depth is determined after self-learning by a learning feedback network, and the modulation depth is used for changing the coherent coupling characteristic of the optical signal in the optical coupler to obtain different linear transformation outputs. The optical coupler is matched with an amplitude modulation phase modulator at the front end of the optical coupler to form a basic unit of the cascade optical coupler photonic neural network, so that a layer of neuron function is realized. Output a of optocoupler (1) The input of the next layer of neurons is input into the next-stage optical coupler after passing through the first-stage amplitude modulation phase modulator. Repeating the steps, and outputting an optical signal a by the last-stage optical coupler after passing through the multi-stage optical coupler (M) The light is directly captured by a group of light detectors, and the classification corresponding to the channel with the maximum light intensity is used as the classification output of the photonic neural network, so that the forward propagation reasoning process is completed.
In this embodiment, the step of performing classification inference by the photonic neural network device includes:
1) Splitting the laser output into N by a single-end input multi-end output optical coupler 0 All coherent light sources are arranged, and N 0 The intensity of all coherent light sources is I 0 ；
2) Will N 0 Amplitude modulation is carried out on an amplitude modulator loaded in an amplitude modulation unit 1 by an intensity modulation mode through a full coherent light source to form optical input excitation, and the ith excitation is carried outComplex amplitude
Figure GDA0004203978610000071
Is->
Figure GDA0004203978610000072
Wherein->
Figure GDA0004203978610000073
Represents normalized i-th input data, with a maximum value of 1, N 0 The optical input stimulus of the path is marked as +.>
Figure GDA0004203978610000074
3)N 0 Optical excitation of path a (0) Amplitude modulation and phase modulation of amplitude modulators respectively entering a first repeating unit in the multilayer neuron 2 and outputting the amplitude modulation and phase modulation to a multi-input multi-output optical coupler of the first repeating unit, and coherently adding and outputting N through the multi-input multi-output optical coupler 1 The complex amplitude of the i-th output signal is
Figure GDA0004203978610000075
N 1 The optical output of the path is marked->
Figure GDA0004203978610000076
In any of the following m-th repeating units: n (N) m-1 Road light signal->
Figure GDA0004203978610000077
Multiple-input multiple-output optical coupler for outputting amplitude and phase modulated input amplitude modulator to mth repeating unit to output N m Way signal->
Figure GDA0004203978610000081
N of the multi-input multi-output optocoupler output of the final Mth repeating unit M Road light signal->
Figure GDA0004203978610000082
The output N is captured directly by the photodetector in the detection unit 3 M Road light intensity->
Figure GDA0004203978610000083
In this embodiment, the functional expression for amplitude and phase modulation in step 3) is as follows:
Figure GDA0004203978610000084
in the above-mentioned method, the step of,
Figure GDA0004203978610000085
output of the kth way of the mth repeating unit,/->
Figure GDA0004203978610000086
Transformation matrix N being the mth repeating unit m ×N m-1 The k-th to i-th matrix element, j is an imaginary unit,/>
Figure GDA0004203978610000087
Output of the ith path which is the m-1 th repeating unit, +.>
Figure GDA0004203978610000088
An amplitude attenuation factor of the mth level i path,>
Figure GDA0004203978610000089
an additional phase for the mth stage i-th way.
In this embodiment, the N is M Road light intensity
Figure GDA00042039786100000810
Output light intensity I of any ith path i The functional expression of (2) is as follows:
Figure GDA00042039786100000811
in the above-mentioned method, the step of,
Figure GDA00042039786100000812
is->
Figure GDA00042039786100000813
Conjugation amount of->
Figure GDA00042039786100000814
The output of the ith path of the Mth repeating unit.
When the network training is carried out, the light intensity normalization result output by the photonic neural network chip is input into the learning feedback network, and the amplitude modulation and phase modulation depth of the front ends of all stages of optical couplers of the forward propagation network is adjusted through a gradient descent learning algorithm. In this embodiment, the step 1) further includes a step of training the photonic neural network device:
s1) randomly selecting n samples from the training set as a sample group, wherein each sample in the sample group
Figure GDA00042039786100000815
With corresponding labels->
Figure GDA00042039786100000816
Wherein x is i All co-N representing sample x 0 An ith component in the data, where y i All co-N representing sample y M The ith component in the data, each component y of tag y i Only the component value of the position corresponding to the classification set of the sample is 1, and the rest components are 0;
s2) setting initial parameters for amplitude and phase modulation in the all-optical photon neural network, inputting each sample in the sample group into the all-optical photon neural network to output an inference result, comparing the inference result with a label y to obtain accuracy evaluation, and finishing training and exiting if the accuracy meets the requirement; otherwise, jumping to execute the next step;
s3) outputting light intensity to all-optical photon neural network
Figure GDA00042039786100000817
Press->
Figure GDA00042039786100000818
The normalized mode is marked as +.>
Figure GDA00042039786100000819
Then->
Figure GDA00042039786100000820
According to a preset back propagation bias function +.>
Figure GDA00042039786100000821
Starting back propagation from the mth repeating unit, and adjusting the amplitude attenuation factor in the mth repeating unit according to a preset gradient during back propagation
Figure GDA00042039786100000822
Additional phase->
Figure GDA00042039786100000823
Jump execution step S1);
the back propagation bias function
Figure GDA00042039786100000824
The functional expression of the initial value of (2) is as follows:
Figure GDA0004203978610000091
in the above-mentioned method, the step of,
Figure GDA0004203978610000092
s is the initial value of the back propagation bias function i Ith path of normalization result, y, representing output light intensity of all-optical photonic neural network i All co-N representing sample y M The ith component, s, in the data k Representing all lightThe k-th path normalization result of the output light intensity of the photonic neural network, y k All co-N representing sample y M The kth component in the data, n represents the number of samples in the sample group, +.>
Figure GDA0004203978610000093
Conjugation of the output of the ith path representing the mth repeating unit;
the back propagation bias function
Figure GDA0004203978610000094
The recurrence relation of the back propagation is shown as follows:
Figure GDA0004203978610000095
in the above-mentioned method, the step of,
Figure GDA0004203978610000096
representing the back propagation bias function to the m-1 th repeating unit,/>
Figure GDA0004203978610000097
An amplitude attenuation factor of the mth-stage kth path,>
Figure GDA0004203978610000098
for the additional phase of the mth stage k-th way, j is the imaginary unit, ++>
Figure GDA0004203978610000099
Transformation matrix N being the mth repeating unit m ×N m-1 The conjugate of the matrix element of the i-th to k-th way,/th>
Figure GDA00042039786100000910
An amplitude attenuation factor of the mth level i path,>
Figure GDA00042039786100000911
representing a back propagation bias function propagated to the mth repeating unit;
amplitude attenuation factor in the mth repeating unit
Figure GDA00042039786100000912
The gradient function expression of (2) is as follows:
Figure GDA00042039786100000913
in the above-mentioned method, the step of,
Figure GDA00042039786100000914
an amplitude attenuation factor +.>
Figure GDA00042039786100000915
Re represents the real part operation, +.>
Figure GDA00042039786100000916
Conjugate of the output of the ith path representing the mth repeating unit, < >>
Figure GDA00042039786100000917
Representing a back propagation bias function propagated to the mth repeating unit;
additional phases in the mth repeat unit
Figure GDA00042039786100000918
The gradient function expression of (2) is as follows:
Figure GDA00042039786100000919
in the above-mentioned method, the step of,
Figure GDA00042039786100000920
represents the additional phase of the ith path of the mth stage +.>
Figure GDA00042039786100000921
Is represented by the imaginary part of the gradient ImIn the operation of the device,
Figure GDA00042039786100000922
conjugate of the output of the ith path representing the mth repeating unit, < >>
Figure GDA00042039786100000923
Representing the back propagation bias function propagated to the mth repeating unit.
In this embodiment, the cost function C is defined as:
Figure GDA00042039786100000924
i.e. the cost function C is defined as 1/4 of the average of the inferred output s of all samples x in the sample set and the square of the distance of the corresponding label y.
Defining a complex deviation function as:
Figure GDA0004203978610000101
there is a recurrence of the back propagation of the bias function and the initial value is as previously indicated. Finally, the obtained gradient is utilized to finish the adjustment of each control parameter according to a standard gradient descent algorithm, and the study of the next group of parameters is carried out until convergence.
The photonic neural network device in this embodiment has the following advantages: 1. after training and learning are completed, the calculation cost of the forward reasoning process is extremely low, and the power consumption is low; 2. the system on a chip is easy to integrate; 3. large bit width data entry is supported.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is directed to methods, apparatus (systems), and computer program products in accordance with embodiments of the present application, and to apparatus for performing functions specified in a flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the present invention may occur to one skilled in the art without departing from the principles of the present invention and are intended to be within the scope of the present invention.

Claims (4)

1. An application method of an on-chip photonic neural network device based on a cascade optical coupler is characterized in that the on-chip photonic neural network device comprises an all-optical photonic neural network, the all-optical photonic neural network comprises an amplitude modulation unit (1), a multi-layer neuron (2) and a detection unit (3), and the amplitude modulation unit (1) comprises N 0 The multi-layer neuron (2) comprises M repeating units which are sequentially layered and cascaded, and the repeating units comprise a plurality of amplitude-modulated modulators and one phase modulatorMultiple input multiple output optical couplers, multiple amplitude modulation modulators of any mth repeating unit are used for respectively using N m-1 The optical signal inputted by the path is subjected to amplitude modulation and phase modulation and then is obtained by a multi-input multi-output optical coupler to obtain N m The detection unit (3) comprises N with the Mth repeating unit M The path output optical signals are detectors corresponding to each other one by one, and the step of classification reasoning by the photonic neural network device comprises the following steps:
1) Splitting the laser output into N by a single-end input multi-end output optical coupler 0 All coherent light sources are arranged, and N 0 The intensity of all coherent light sources is I 0 ；
2) Will N 0 Amplitude modulation is carried out on the full coherent light source by an amplitude modulator loaded into an amplitude modulation unit (1) in an intensity modulation mode to form optical input excitation, and the complex amplitude of the ith excitation
Figure FDA0004203978600000011
Is->
Figure FDA0004203978600000012
Wherein->
Figure FDA0004203978600000013
Represents normalized i-th input data, with a maximum value of 1, N 0 The optical input stimulus of the path is marked as +.>
Figure FDA0004203978600000014
3)N 0 Optical excitation of path a (0) Amplitude modulation and phase modulation are carried out on amplitude modulation modulators respectively entering a first repeating unit in the multi-layer neuron (2), then the amplitude modulation modulators are output to a multi-input multi-output optical coupler of the first repeating unit, and the coherent superposition is carried out on the output N through the multi-input multi-output optical coupler 1 The complex amplitude of the i-th output signal is
Figure FDA0004203978600000015
N 1 The path optical output is recorded as
Figure FDA0004203978600000016
In any of the following m-th repeating units: n (N) m-1 Road light signal->
Figure FDA0004203978600000017
Multiple-input multiple-output optical coupler for outputting amplitude and phase modulated input amplitude modulator to mth repeating unit to output N m Way signal->
Figure FDA0004203978600000018
N of the multi-input multi-output optocoupler output of the final Mth repeating unit M Road light signal
Figure FDA0004203978600000019
Directly captured by the photodetector in the detection unit (3) to output N M Road light intensity->
Figure FDA00042039786000000110
2. The method for applying the cascade optical coupler-based on-chip photonic neural network device according to claim 1, wherein the functional expression of amplitude and phase modulation in the step 3) is as follows:
Figure FDA00042039786000000111
in the above-mentioned method, the step of,
Figure FDA00042039786000000112
output of the kth way of the mth repeating unit,/->
Figure FDA00042039786000000113
Transformation matrix N being the mth repeating unit m ×N m-1 The k-th to i-th matrix element, j is an imaginary unit,/>
Figure FDA00042039786000000114
The output of the ith path for the m-1 th repeating unit,
Figure FDA00042039786000000115
an amplitude attenuation factor of the mth level i path,>
Figure FDA00042039786000000116
an additional phase for the mth stage i-th way.
3. The method of using a cascade optocoupler-based on-chip photonic neural network device of claim 2, wherein the N M Road light intensity
Figure FDA0004203978600000021
Output light intensity I of any ith path i The functional expression of (2) is as follows:
Figure FDA0004203978600000022
in the above-mentioned method, the step of,
Figure FDA0004203978600000023
is->
Figure FDA0004203978600000024
Conjugation amount of->
Figure FDA0004203978600000025
The output of the ith path of the Mth repeating unit.
4. The method for applying the cascade optical coupler-based on-chip photonic neural network device according to claim 3, wherein the step 1) further comprises the step of training the photonic neural network device:
s1) randomly selecting n samples from the training set as a sample group, wherein each sample in the sample group
Figure FDA0004203978600000026
With corresponding labels->
Figure FDA0004203978600000027
Wherein x is i All co-N representing sample x 0 An ith component in the data, where y i All co-N representing sample y M The ith component in the data, each component y of tag y i Only the component value of the position corresponding to the classification set of the sample is 1, and the rest components are 0;
s2) setting initial parameters for amplitude and phase modulation in the all-optical photon neural network, inputting each sample in the sample group into the all-optical photon neural network to output an inference result, comparing the inference result with a label y to obtain accuracy evaluation, and finishing training and exiting if the accuracy meets the requirement; otherwise, jumping to execute the next step;
s3) outputting light intensity to all-optical photon neural network
Figure FDA0004203978600000028
Press->
Figure FDA0004203978600000029
The mode is normalized and then is marked as
Figure FDA00042039786000000210
Then->
Figure FDA00042039786000000211
According to a preset back propagation bias function +.>
Figure FDA00042039786000000212
Starting back propagation from the mth repeating unit, and adjusting the amplitude attenuation factor in the mth repeating unit according to a preset gradient during back propagation
Figure FDA00042039786000000213
Additional phase->
Figure FDA00042039786000000214
Jump execution step S1);
the back propagation bias function
Figure FDA00042039786000000215
The functional expression of the initial value of (2) is as follows:
Figure FDA00042039786000000216
in the above-mentioned method, the step of,
Figure FDA00042039786000000217
s is the initial value of the back propagation bias function i Ith path of normalization result, y, representing output light intensity of all-optical photonic neural network i All co-N representing sample y M The ith component, s, in the data k The k-th path normalization result, y, of the output light intensity of the all-optical photonic neural network is shown k All co-N representing sample y M The kth component in the data, n represents the number of samples in the sample group, +.>
Figure FDA00042039786000000218
Conjugation of the output of the ith path representing the mth repeating unit;
the back propagation bias function
Figure FDA00042039786000000219
The recurrence relation of the back propagation is shown as follows:
Figure FDA00042039786000000220
in the above-mentioned method, the step of,
Figure FDA00042039786000000221
representing the back propagation bias function to the m-1 th repeating unit,/>
Figure FDA00042039786000000222
An amplitude attenuation factor of the mth-stage kth path,>
Figure FDA0004203978600000031
for the additional phase of the mth stage k-th way, j is the imaginary unit, ++>
Figure FDA0004203978600000032
Transformation matrix N being the mth repeating unit m ×N m-1 The conjugate of the matrix element of the i-th to k-th way,/th>
Figure FDA0004203978600000033
Is the amplitude attenuation factor of the ith path of the mth level,
Figure FDA0004203978600000034
representing a back propagation bias function propagated to the mth repeating unit;
amplitude attenuation factor in the mth repeating unit
Figure FDA0004203978600000035
The gradient function expression of (2) is as follows:
Figure FDA0004203978600000036
in the above-mentioned method, the step of,
Figure FDA0004203978600000037
an amplitude attenuation factor +.>
Figure FDA0004203978600000038
Re represents the real part taking operation,
Figure FDA0004203978600000039
conjugate of the output of the ith path representing the mth repeating unit, < >>
Figure FDA00042039786000000310
Representing a back propagation bias function propagated to the mth repeating unit;
additional phases in the mth repeat unit
Figure FDA00042039786000000311
The gradient function expression of (2) is as follows:
Figure FDA00042039786000000312
in the above-mentioned method, the step of,
Figure FDA00042039786000000313
represents the additional phase of the ith path of the mth stage +.>
Figure FDA00042039786000000314
Is a gradient of Im representing an operation taking the imaginary part,/>
Figure FDA00042039786000000315
Conjugate of the output of the ith path representing the mth repeating unit, < >>
Figure FDA00042039786000000316
Representing the back propagation bias function propagated to the mth repeating unit.
CN202010605656.1A 2020-06-29 2020-06-29 On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip Active CN111680796B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010605656.1A CN111680796B (en) 2020-06-29 2020-06-29 On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010605656.1A CN111680796B (en) 2020-06-29 2020-06-29 On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip

Publications (2)

Publication Number Publication Date
CN111680796A CN111680796A (en) 2020-09-18
CN111680796B true CN111680796B (en) 2023-07-11

Family

ID=72456870

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010605656.1A Active CN111680796B (en) 2020-06-29 2020-06-29 On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip

Country Status (1)

Country Link
CN (1) CN111680796B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112988113B (en) * 2021-04-29 2021-09-14 中国科学院西安光学精密机械研究所 Photon matrix vector multiplier
CN113298246B (en) * 2021-05-27 2023-02-28 山东云海国创云计算装备产业创新中心有限公司 Data processing method, device and computer readable storage medium
CN117592533B (en) * 2023-10-30 2024-07-02 浙江大学 Method for realizing neural network nonlinear operation by using semiconductor laser

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10268232B2 (en) * 2016-06-02 2019-04-23 Massachusetts Institute Of Technology Apparatus and methods for optical neural network
TW202036134A (en) * 2018-11-02 2020-10-01 美商萊特美特股份有限公司 Matrix multiplication using optical processing
CN110247728B (en) * 2019-07-01 2021-06-25 光子算数(北京)科技有限责任公司 Photon neural network chip
CN110929864B (en) * 2019-12-05 2023-04-18 北京超放信息技术有限公司 Optical diffraction neural network on-line training method and system
CN111291881B (en) * 2020-03-05 2022-10-18 西南大学 Mode identification device and method based on cascade VCSELs photonic nerves

Also Published As

Publication number Publication date
CN111680796A (en) 2020-09-18

Similar Documents

Publication Publication Date Title
CN111680796B (en) On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip
Harris et al. Linear programmable nanophotonic processors
De Marinis et al. Photonic neural networks: A survey
Li et al. Class-specific differential detection in diffractive optical neural networks improves inference accuracy
Sui et al. A review of optical neural networks
Bagherian et al. On-chip optical convolutional neural networks
Qu et al. Inverse design of an integrated-nanophotonics optical neural network
AU2017273863B2 (en) Apparatus and methods for optical neural network
CN110929864B (en) Optical diffraction neural network on-line training method and system
Stark et al. Opportunities for integrated photonic neural networks
CN111478735B (en) General quantum walking simulation chip structure based on integrated optics
Gu et al. SqueezeLight: Towards scalable optical neural networks with multi-operand ring resonators
Huang et al. Programmable matrix operation with reconfigurable time-wavelength plane manipulation and dispersed time delay
US11600060B2 (en) Nonlinear all-optical deep-learning system and method with multistage space-frequency domain modulation
Ishihara et al. An optical neural network architecture based on highly parallelized WDM-multiplier-accumulator
Zang et al. Optoelectronic convolutional neural networks based on time-stretch method
Qiu et al. Decision-making and control with diffractive optical networks
Zhao et al. On-chip reconfigurable optical neural networks
Xu et al. Photoelectric hybrid convolution neural network with coherent nanophotonic circuits
CN115496195A (en) MZI array and FP-SA-based optical pulse neural network implementation device
Hattori et al. Optical-electronic implementation of artificial neural network for ultrafast and accurate inference processing
Cartlidge Photonic computing for sale
De Marinis et al. A photonic accelerator for feature map generation in convolutional neural networks
Jonuzi et al. Numerical Analysis of a Self-Calibrating Time-Spatial Interleaving Photonic Convolutional Accelerator
Zhang et al. Time-stretch optical neural network with time-division multiplexing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant