CN111680796B - On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip - Google Patents
On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip Download PDFInfo
- Publication number
- CN111680796B CN111680796B CN202010605656.1A CN202010605656A CN111680796B CN 111680796 B CN111680796 B CN 111680796B CN 202010605656 A CN202010605656 A CN 202010605656A CN 111680796 B CN111680796 B CN 111680796B
- Authority
- CN
- China
- Prior art keywords
- output
- mth
- neural network
- optical
- repeating unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
- G06N3/0675—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means using electro-optical, acousto-optical or opto-electronic means
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Optical Communication System (AREA)
- Optical Modulation, Optical Deflection, Nonlinear Optics, Optical Demodulation, Optical Logic Elements (AREA)
Abstract
The invention discloses an on-chip photonic neural network device, a chip based on a cascade optical couplerThe application method comprises that the photonic neural network device comprises an all-optical photonic neural network, wherein the all-optical photonic neural network comprises an amplitude modulation unit, a multi-layer neuron and a detection unit, and the amplitude modulation unit comprises N 0 A circuit modulator, the multi-layer neuron including M repeating units, the repeating units including a plurality of amplitude-modulated phase modulators and a MIMO optical coupler, the plurality of amplitude-modulated phase modulators of any mth repeating unit being used to divide N into N m‑1 The optical signals input by the paths are subjected to amplitude modulation and phase modulation and then are obtained to N through the MIMO optical coupler m A path output, the detection unit comprises N M A path detector. The invention can realize the deep learning system of the all-optical photonic neural network with large bit width input in the on-chip integrated system based on the cascade optical coupler, has extremely low calculation cost and low power consumption in the forward reasoning process after training and learning are finished, and the on-chip system is easy to integrate and supports large bit width data input.
Description
Technical Field
The invention relates to a neural network deep learning technology, in particular to an on-chip photonic neural network device and a chip based on a cascade optical coupler and an application method thereof, which are used for realizing an all-optical photonic neural network deep learning system with large bit width input in an on-chip integrated system based on the cascade optical coupler.
Background
Artificial intelligence is a new technical science for researching and developing intelligent theory, method technology and application system for simulating extended and extended people. On one hand, the development of algorithms, calculation power and big data drives artificial intelligence to break through in all aspects of calculation intelligence, perception intelligence and cognition intelligence from three dimensions; on the other hand, artificial intelligence is being rapidly fused with various industries, and the traditional industry is assisted in upgrading and transforming, improving quality and enhancing efficiency, and inducing brand new industry surge in the global scope.
The dominant technology of artificial intelligence today is deep neural networks for large data processing. The deep neural network is a computational network model which is inspired by a brain signal processing process, and since the computational model is proposed, the computational network model is widely applied to the fields of computer vision, voice recognition, natural semantic understanding, man-machine game and the like, and the performance of machine learning in the related fields is greatly improved.
In recent decades, the neural network accelerator is greatly improved by the continuous improvement of a computing architecture, various technical routes such as a convolver-based structure, a multi-tree-shaped structure, a matrix multiplier-based structure and the like are continuously emerging, and deep learning is pushed to obtain one more achievement with milestone significance. However, the current electric domain computing system computing hardware still has a great gap from the brain in terms of power consumption and performance when processing the neural network. Taking an artificial intelligence program alpha Go based on a deep learning algorithm and based on a neural network accelerator TPU with a matrix multiplier structure, which is proposed by Google corporation, as an example, the power consumption of a computing system required by alpha Go is 20MW, and the volume is up to 30m 3 The method comprises the steps of carrying out a first treatment on the surface of the The human brain power consumption for running the same calculation task is only 20W, and the volume is only 0.0015m 3 . Under the current architecture of an electric domain computing system, the hardware performance of the neural network cannot be fully utilized. Although the electric domain computing system is continuously improved in computing architecture according to the characteristics of the neural network, the electric domain computing system is still centered on the data processing computation, and the processor reads the data computation from the memoryAfter processing, the data is written back to the memory. The neural network algorithm is centered on the data stream, and the data operation of the neural network algorithm is completed in the neural network following the data stream. The neural network algorithm is realized on the basis of hardware taking data processing as a center, so that massive data streams are necessarily required to be meshed and stored, and are moved to an operation processing component for operation processing when needed. On the one hand, this requires a massive storage, on the other hand, since the storage of data and the arithmetic processing of the data are separated from each other, which in turn leads to the movement of massive data. It is this architectural conflict that results in inefficiencies in the operation of the neural network algorithms by the electrical domain computing hardware.
While computer domain students are continuously improving the computing architecture to improve the computing efficiency of the neural network, optical domain students are also continuously proposing novel schemes to realize the neural network computing. Photon computing research has also made significant progress over the years, but limited by the physical nature of the photon's zero-stationary mass, has not yet made an exciting breakthrough in achieving optical caching. Without optical information storage, the traditional digital logic computer cannot be finally realized based on the photonic device, and the development of a photonic computing system is still in existence due to the lack of optical storage. But the architecture of the neural network algorithm which takes the data stream as the center can be well matched with a photon computing system. On the other hand, each neuron of the neural network algorithm can be used for input data without buffering, so that the defect that the photon computing network has no buffering function is not important.
In 2017, the photon computing team of the united states MIT first proposed an all-optical photonic neural network, which is composed of Mach-zehnder interferometers (Mach-Zehnder interferometer, MZI) with cascaded additional phase shift functions, and can realize arbitrary linear transformation and output of 4 input signals. The team uses the network to complete linear transformation and is assisted by a computer to complete nonlinear transformation, and the two are matched to realize a first-stage 4-path neuron layer. By multiplexing the all-optical photonic neural network, the 4-layer neural network is realized, and the recognition accuracy of 4 vowels exceeds 90%. The first reported all-optical photonic neural network capable of actually working is poor in realizability, on one hand, the all-optical photonic neural network is not used for realizing complete neural network operation in an optical domain, but more serious, the photonic network is based on an MZI device array, when the input bit width is higher and the number of network layers is larger, a large number of MZI devices are needed, the network scale is extremely huge, and the process preparation difficulty is great. This is because the implementation of the all-optical photonic neural network is a mechanical simulation of electronic logic computation, and does not make good use of photonic characteristics.
The U.S. UCLA team in 2019 proposed a deep diffraction neural network based on a multi-layer diffraction film structure. The design utilizes expansibility, locality and coherence of a diffraction light field to complete convolution linear transformation between two layers of neurons, utilizes the projection rate and phase shift of a diffraction film as learning parameters, fully utilizes the physical characteristics of the light field, has concise physical realization of the neurons and has good realizability. The team successfully realizes the identification of 28 pixels by 28 pixels pictures by using a convolutional neural network formed by 5 layers of diffraction films, and the identification success rate is up to 93%. Meanwhile, the scheme is used as a complete optical system, transmission imaging of an external light source is used as input, and the optical neural network system has almost no power consumption in the recognition process after learning. The scheme promotes the all-optical photonic neural network to take a great step towards practical application, and attracts wide attention in terms of excellent power consumption performance. However, since the implementation principle is natural two-dimensional convolution-like calculation, the method cannot be applied to neural network algorithms of non-two-dimensional convolution, and in practical application, the convolution neural network only accounts for less than 10% of all neural network applications. On the other hand, the scheme is based on the principle of space optics, and is difficult to integrate into a system-on-chip in a dimension-reducing way. The chip is a mainstream way for realizing miniaturization and integration of the photon computing system, and the all-optical photon neural network is taken as a special photon computing system, so that on-chip integration is an important development direction.
Disclosure of Invention
The invention aims to solve the technical problems: aiming at the problems in the prior art, the invention provides an on-chip photonic neural network device and a chip based on a cascade optical coupler and an application method thereof.
In order to solve the technical problems, the invention adopts the following technical scheme:
an on-chip photonic neural network device based on a cascade optical coupler comprises an all-optical photonic neural network, wherein the all-optical photonic neural network comprises an amplitude modulation unit, a multi-layer neuron and a detection unit, and the amplitude modulation unit comprises N 0 The multi-layer neuron comprises M repeating units which are sequentially layered and cascaded, the repeating units comprise a plurality of amplitude modulation modulators and a multi-input multi-output optical coupler, and the plurality of amplitude modulation modulators of any mth repeating unit are used for respectively amplifying N m-1 The optical signal inputted by the path is subjected to amplitude modulation and phase modulation and then is obtained by a multi-input multi-output optical coupler to obtain N m The detection unit comprises N with the Mth repeating unit M And the detectors are in one-to-one correspondence with the path output optical signals.
Optionally, the multi-input multi-output optical coupler is a star coupler or a multi-mode interference coupler.
Optionally, an input end of the amplitude modulation unit is connected with a coherent light source, and the coherent light source comprises M output light paths.
Optionally, the coherent light source comprises a laser and a single-ended input multi-output optical coupler, an output end of the laser is connected with an input of the single-ended input multi-output optical coupler, and an output end of the single-ended input multi-output optical coupler comprises N 0 And a plurality of output light paths.
In addition, the invention also provides an all-optical photon neural network chip based on the cascade optical coupler, which comprises a chip body, wherein the chip body comprises an amplitude modulation unit, a plurality of layers of neurons and a detection unit, and the amplitude modulation unit comprises N 0 The multi-layer neuron comprises M heavy-duty light emitting devices which are sequentially layered and cascadedA complex unit including a plurality of AM phase modulators and a multi-input multi-output optical coupler, the plurality of AM phase modulators of any mth repeating unit being used for respectively coupling N m-1 The optical signal inputted by the path is subjected to amplitude modulation and phase modulation and then is obtained by a multi-input multi-output optical coupler to obtain N m The detection unit comprises N with the Mth repeating unit M And the detectors are in one-to-one correspondence with the path output optical signals.
Optionally, the multi-input multi-output optical coupler is a star coupler or a multi-mode interference coupler.
In addition, the invention also provides an application method of the on-chip photonic neural network device based on the cascade optical coupler, and the step of classifying and reasoning through the photonic neural network device comprises the following steps:
1) Splitting the laser output into N by a single-end input multi-end output optical coupler 0 All coherent light sources are arranged, and N 0 The intensity of all coherent light sources is I 0 ;
2) Will N 0 Amplitude modulation is carried out on an amplitude modulator loaded into an amplitude modulation unit by an intensity modulation mode through a full coherent light source to form optical input excitation, and the complex amplitude of the ith excitation is obtainedIs->Wherein->Represents normalized i-th input data, with a maximum value of 1, N 0 The optical input stimulus of the path is marked as +.>
3)N 0 Optical excitation of path a (0) Amplitude-modulated phase modulator respectively entering first repeating unit in multilayer neuron for amplitude and phase modulation and outputting to first repeating unitMultiple-input multiple-output optical coupler, through which N is output after coherent superposition 1 The complex amplitude of the i-th output signal isN 1 The optical output of the path is marked->In any of the following m-th repeating units: n (N) m-1 Road light signal->Multiple-input multiple-output optical coupler for outputting amplitude and phase modulated input amplitude modulator to mth repeating unit to output N m Way signal->N of the multi-input multi-output optocoupler output of the final Mth repeating unit M Road light signal->Directly captured by the photodetector in the detection unit to output N M Road light intensity->
Optionally, the functional expression for amplitude and phase modulation in step 3) is as follows:
in the above-mentioned method, the step of,output of the kth way of the mth repeating unit,/->Transformation matrix N being the mth repeating unit m ×N m-1 The k-th to i-th matrix element, j is an imaginary unit,/>Output of the ith path which is the m-1 th repeating unit, +.>An amplitude attenuation factor of the mth level i path,>an additional phase for the mth stage i-th way.
Optionally, the N M Road light intensityOutput light intensity I of any ith path i The functional expression of (2) is as follows:
in the above-mentioned method, the step of,is->Conjugation amount of->The output of the ith path of the Mth repeating unit.
Optionally, step 1) further includes a step of training the photonic neural network device:
s1) randomly selecting n samples from the training set as a sample group, wherein each sample in the sample groupWith corresponding labels->Wherein x is i All co-N representing sample x 0 An ith component in the data, where y i All co-N representing sample y M The ith component in the data, each component y of tag y i Only the component value of the position corresponding to the classification set of the sample is 1, and the rest components are 0;
s2) setting initial parameters for amplitude and phase modulation in the all-optical photon neural network, inputting each sample in the sample group into the all-optical photon neural network to output an inference result, comparing the inference result with a label y to obtain accuracy evaluation, and finishing training and exiting if the accuracy meets the requirement; otherwise, jumping to execute the next step;
s3) outputting light intensity to all-optical photon neural networkPress->The normalized mode is marked as +.>Then->According to a preset back propagation bias function +.>Starting back propagation from the mth repeating unit, and adjusting the amplitude attenuation factor in the mth repeating unit according to a preset gradient during back propagationAdditional phase->Jump execution step S1);
the back propagation bias functionThe functional expression of the initial value of (2) is as follows:
in the above-mentioned method, the step of,s is the initial value of the back propagation bias function i Ith path of normalization result, y, representing output light intensity of all-optical photonic neural network i All co-N representing sample y M The ith component, s, in the data k The k-th path normalization result, y, of the output light intensity of the all-optical photonic neural network is shown k All co-N representing sample y M The kth component in the data, n represents the number of samples in the sample group, +.>Conjugation of the output of the ith path representing the mth repeating unit;
the back propagation bias functionThe recurrence relation of the back propagation is shown as follows:
in the above-mentioned method, the step of,representing the back propagation bias function to the m-1 th repeating unit,/>An amplitude attenuation factor of the mth-stage kth path,>for the additional phase of the mth stage k-th way, j is the imaginary unit, ++>Transformation matrix N being the mth repeating unit m ×N m-1 The conjugate of the matrix element of the i-th to k-th way,/th>An amplitude attenuation factor of the mth level i path,>representing a back propagation bias function propagated to the mth repeating unit;
amplitude attenuation factor in the mth repeating unitThe gradient function expression of (2) is as follows:
in the above-mentioned method, the step of,an amplitude attenuation factor +.>Re represents the real part operation, +.>Conjugate of the output of the ith path representing the mth repeating unit, < >>Representing a back propagation bias function propagated to the mth repeating unit;
in the above-mentioned method, the step of,represents the additional phase of the ith path of the mth stage +.>Im represents the operation of taking the imaginary part,conjugate of the output of the ith path representing the mth repeating unit, < >>Representing the back propagation bias function propagated to the mth repeating unit.
Compared with the prior art, the on-chip photonic neural network device based on the cascade optical coupler has the following advantages: the cascade optical coupler comprises an all-optical photonic neural network, wherein the all-optical photonic neural network comprises an amplitude modulation unit 1, a multi-layer neuron 2 and a detection unit 3, and the amplitude modulation unit 1 comprises N 0 The multi-layer neuron 2 comprises M repeating units which are sequentially layered and cascaded, wherein each repeating unit comprises a plurality of amplitude modulation modulators and a multi-input multi-output optical coupler, and the plurality of amplitude modulation modulators of any mth repeating unit are used for respectively amplifying N m-1 After amplitude modulation and phase modulation, the optical signal input by the path is optically coupled through multi-input multi-output lightThe device obtains N m The detection unit 3 includes N with the Mth repeating unit M The invention can realize the deep learning system of the all-optical photonic neural network with large bit width input in the on-chip integrated system based on the cascade optical coupler, has extremely low calculation cost and low power consumption in the forward reasoning process after training and learning are finished, and is easy to integrate and support large bit width data input.
Drawings
Fig. 1 is a schematic structural diagram of a photonic neural network device according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a training principle of a photonic neural network device according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of an optical path principle of a single-ended input multi-ended output optical coupler according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of an optical path principle of a multi-input multi-output optical coupler according to an embodiment of the present invention.
Detailed Description
As shown in fig. 1, the present embodiment provides an on-chip photonic neural network device based on a cascade optical coupler, which includes an all-optical photonic neural network, the all-optical photonic neural network includes an amplitude modulation unit 1, a multi-layer neuron 2, and a detection unit 3, and the amplitude modulation unit 1 includes N 0 The multi-layer neuron 2 comprises M repeating units which are sequentially layered and cascaded, wherein each repeating unit comprises a plurality of amplitude modulation modulators and a multi-input multi-output optical coupler, and the plurality of amplitude modulation modulators of any M repeating unit are used for respectively amplifying N m-1 The optical signal inputted by the path is subjected to amplitude modulation and phase modulation and then is obtained by a multi-input multi-output optical coupler to obtain N m The optical signal output by the path, the detecting unit 3 includes N with the mth repeating unit M And the detectors are in one-to-one correspondence with the path output optical signals.
The photonic neural network chip is composed of cascaded on-chip optical couplers, typical Multi-port on-chip optical couplers including, but not limited to, star couplers (Star couplers) and multimode interference couplers (Multi-Mode Interference Coupler, MMI couplers). The low-linewidth high-power laser source is used for realizing the generation of a Multi-path coherent light source by beam splitting through a Single-end Input Multi-Output Coupler (figure 3), and is injected into an all-optical photonic neural network. The multi-bit wide data to be classified is modulated onto each optical signal using an electrical injection or thermal control modulator, the intensity of the optical signal being proportional to the normalized input data. In this embodiment, the multi-input multi-output optical coupler is a star coupler or a multi-mode interference coupler.
In this embodiment, the input end of the amplitude modulation unit 1 is connected with a coherent light source 4, and the coherent light source 4 includes M output light paths.
In this embodiment, the coherent light source 4 includes a laser 41 and a single-ended input multiple-output optical coupler 42, the output end of the laser 41 is connected with the input of the single-ended input multiple-output optical coupler 42, and the output end of the single-ended input multiple-output optical coupler 42 includes N 0 And a plurality of output light paths.
On the basis, the embodiment also provides an all-optical photon neural network chip based on the cascade optical coupler, which comprises a chip body, wherein the chip body comprises an amplitude modulation unit 1, a plurality of layers of neurons 2 and a detection unit 3, and the amplitude modulation unit 1 comprises N 0 The multi-layer neuron 2 comprises M repeating units which are sequentially layered and cascaded, wherein each repeating unit comprises a plurality of amplitude modulation modulators and a multi-input multi-output optical coupler, and the plurality of amplitude modulation modulators of any M repeating unit are used for respectively amplifying N m-1 The optical signal inputted by the path is subjected to amplitude modulation and phase modulation and then is obtained by a multi-input multi-output optical coupler to obtain N m The optical signal output by the path, the detecting unit 3 includes N with the mth repeating unit M And the detectors are in one-to-one correspondence with the path output optical signals.
As can be seen from the above, in this embodiment, by using the coherent coupling characteristic of the multiport on-chip optical coupler, linear transformation between output and input signals of two adjacent layers of neurons is implemented in an on-chip integrated optical system by using a single photonic device, and fine tuning of the linear transformation is implemented by modulating amplitude and phase of the input signals of the optical coupler, so as to complete the functions of the basic unit of the neurons;on the basis, cascading the multi-stage on-chip optical coupler to construct a multi-layer neuron and form an on-chip integrated all-optical photonic neural network; and a single-stage iterative reverse propagation gradient descent learning algorithm compatible with a complex propagation function and designed based on a standard reverse propagation gradient descent algorithm framework is matched, so that the efficient learning convergence of the photonic neural network is realized. Referring to fig. 2, the on-chip photonic neural network system based on the cascade optical coupler is composed of two parts of all-optical photonic neural network chip hardware based on the cascade optical coupler and learning feedback network software matched with the all-optical photonic neural network chip hardware based on the cascade optical coupler and based on a counter-propagation gradient descent algorithm. Referring to fig. 4, the modulated optical signal a (0) And then the amplitude and the phase of the signal are subjected to secondary modulation by a primary amplitude modulation modulator and then enter a Multi-Input Multi-Output optical Coupler (fig. 4) for coherent superposition. The amplitude modulation phase modulator modulates the amplitude and the phase of the optical signal at the same time, the modulation depth is determined after self-learning by a learning feedback network, and the modulation depth is used for changing the coherent coupling characteristic of the optical signal in the optical coupler to obtain different linear transformation outputs. The optical coupler is matched with an amplitude modulation phase modulator at the front end of the optical coupler to form a basic unit of the cascade optical coupler photonic neural network, so that a layer of neuron function is realized. Output a of optocoupler (1) The input of the next layer of neurons is input into the next-stage optical coupler after passing through the first-stage amplitude modulation phase modulator. Repeating the steps, and outputting an optical signal a by the last-stage optical coupler after passing through the multi-stage optical coupler (M) The light is directly captured by a group of light detectors, and the classification corresponding to the channel with the maximum light intensity is used as the classification output of the photonic neural network, so that the forward propagation reasoning process is completed.
In this embodiment, the step of performing classification inference by the photonic neural network device includes:
1) Splitting the laser output into N by a single-end input multi-end output optical coupler 0 All coherent light sources are arranged, and N 0 The intensity of all coherent light sources is I 0 ;
2) Will N 0 Amplitude modulation is carried out on an amplitude modulator loaded in an amplitude modulation unit 1 by an intensity modulation mode through a full coherent light source to form optical input excitation, and the ith excitation is carried outComplex amplitudeIs->Wherein->Represents normalized i-th input data, with a maximum value of 1, N 0 The optical input stimulus of the path is marked as +.>
3)N 0 Optical excitation of path a (0) Amplitude modulation and phase modulation of amplitude modulators respectively entering a first repeating unit in the multilayer neuron 2 and outputting the amplitude modulation and phase modulation to a multi-input multi-output optical coupler of the first repeating unit, and coherently adding and outputting N through the multi-input multi-output optical coupler 1 The complex amplitude of the i-th output signal isN 1 The optical output of the path is marked->In any of the following m-th repeating units: n (N) m-1 Road light signal->Multiple-input multiple-output optical coupler for outputting amplitude and phase modulated input amplitude modulator to mth repeating unit to output N m Way signal->N of the multi-input multi-output optocoupler output of the final Mth repeating unit M Road light signal->The output N is captured directly by the photodetector in the detection unit 3 M Road light intensity->
In this embodiment, the functional expression for amplitude and phase modulation in step 3) is as follows:
in the above-mentioned method, the step of,output of the kth way of the mth repeating unit,/->Transformation matrix N being the mth repeating unit m ×N m-1 The k-th to i-th matrix element, j is an imaginary unit,/>Output of the ith path which is the m-1 th repeating unit, +.>An amplitude attenuation factor of the mth level i path,>an additional phase for the mth stage i-th way.
In this embodiment, the N is M Road light intensityOutput light intensity I of any ith path i The functional expression of (2) is as follows:
in the above-mentioned method, the step of,is->Conjugation amount of->The output of the ith path of the Mth repeating unit.
When the network training is carried out, the light intensity normalization result output by the photonic neural network chip is input into the learning feedback network, and the amplitude modulation and phase modulation depth of the front ends of all stages of optical couplers of the forward propagation network is adjusted through a gradient descent learning algorithm. In this embodiment, the step 1) further includes a step of training the photonic neural network device:
s1) randomly selecting n samples from the training set as a sample group, wherein each sample in the sample groupWith corresponding labels->Wherein x is i All co-N representing sample x 0 An ith component in the data, where y i All co-N representing sample y M The ith component in the data, each component y of tag y i Only the component value of the position corresponding to the classification set of the sample is 1, and the rest components are 0;
s2) setting initial parameters for amplitude and phase modulation in the all-optical photon neural network, inputting each sample in the sample group into the all-optical photon neural network to output an inference result, comparing the inference result with a label y to obtain accuracy evaluation, and finishing training and exiting if the accuracy meets the requirement; otherwise, jumping to execute the next step;
s3) outputting light intensity to all-optical photon neural networkPress->The normalized mode is marked as +.>Then->According to a preset back propagation bias function +.>Starting back propagation from the mth repeating unit, and adjusting the amplitude attenuation factor in the mth repeating unit according to a preset gradient during back propagationAdditional phase->Jump execution step S1);
the back propagation bias functionThe functional expression of the initial value of (2) is as follows:
in the above-mentioned method, the step of,s is the initial value of the back propagation bias function i Ith path of normalization result, y, representing output light intensity of all-optical photonic neural network i All co-N representing sample y M The ith component, s, in the data k Representing all lightThe k-th path normalization result of the output light intensity of the photonic neural network, y k All co-N representing sample y M The kth component in the data, n represents the number of samples in the sample group, +.>Conjugation of the output of the ith path representing the mth repeating unit;
the back propagation bias functionThe recurrence relation of the back propagation is shown as follows:
in the above-mentioned method, the step of,representing the back propagation bias function to the m-1 th repeating unit,/>An amplitude attenuation factor of the mth-stage kth path,>for the additional phase of the mth stage k-th way, j is the imaginary unit, ++>Transformation matrix N being the mth repeating unit m ×N m-1 The conjugate of the matrix element of the i-th to k-th way,/th>An amplitude attenuation factor of the mth level i path,>representing a back propagation bias function propagated to the mth repeating unit;
amplitude attenuation factor in the mth repeating unitThe gradient function expression of (2) is as follows:
in the above-mentioned method, the step of,an amplitude attenuation factor +.>Re represents the real part operation, +.>Conjugate of the output of the ith path representing the mth repeating unit, < >>Representing a back propagation bias function propagated to the mth repeating unit;
in the above-mentioned method, the step of,represents the additional phase of the ith path of the mth stage +.>Is represented by the imaginary part of the gradient ImIn the operation of the device,conjugate of the output of the ith path representing the mth repeating unit, < >>Representing the back propagation bias function propagated to the mth repeating unit.
In this embodiment, the cost function C is defined as:
i.e. the cost function C is defined as 1/4 of the average of the inferred output s of all samples x in the sample set and the square of the distance of the corresponding label y.
Defining a complex deviation function as:
there is a recurrence of the back propagation of the bias function and the initial value is as previously indicated. Finally, the obtained gradient is utilized to finish the adjustment of each control parameter according to a standard gradient descent algorithm, and the study of the next group of parameters is carried out until convergence.
The photonic neural network device in this embodiment has the following advantages: 1. after training and learning are completed, the calculation cost of the forward reasoning process is extremely low, and the power consumption is low; 2. the system on a chip is easy to integrate; 3. large bit width data entry is supported.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is directed to methods, apparatus (systems), and computer program products in accordance with embodiments of the present application, and to apparatus for performing functions specified in a flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the present invention may occur to one skilled in the art without departing from the principles of the present invention and are intended to be within the scope of the present invention.
Claims (4)
1. An application method of an on-chip photonic neural network device based on a cascade optical coupler is characterized in that the on-chip photonic neural network device comprises an all-optical photonic neural network, the all-optical photonic neural network comprises an amplitude modulation unit (1), a multi-layer neuron (2) and a detection unit (3), and the amplitude modulation unit (1) comprises N 0 The multi-layer neuron (2) comprises M repeating units which are sequentially layered and cascaded, and the repeating units comprise a plurality of amplitude-modulated modulators and one phase modulatorMultiple input multiple output optical couplers, multiple amplitude modulation modulators of any mth repeating unit are used for respectively using N m-1 The optical signal inputted by the path is subjected to amplitude modulation and phase modulation and then is obtained by a multi-input multi-output optical coupler to obtain N m The detection unit (3) comprises N with the Mth repeating unit M The path output optical signals are detectors corresponding to each other one by one, and the step of classification reasoning by the photonic neural network device comprises the following steps:
1) Splitting the laser output into N by a single-end input multi-end output optical coupler 0 All coherent light sources are arranged, and N 0 The intensity of all coherent light sources is I 0 ;
2) Will N 0 Amplitude modulation is carried out on the full coherent light source by an amplitude modulator loaded into an amplitude modulation unit (1) in an intensity modulation mode to form optical input excitation, and the complex amplitude of the ith excitationIs->Wherein->Represents normalized i-th input data, with a maximum value of 1, N 0 The optical input stimulus of the path is marked as +.>
3)N 0 Optical excitation of path a (0) Amplitude modulation and phase modulation are carried out on amplitude modulation modulators respectively entering a first repeating unit in the multi-layer neuron (2), then the amplitude modulation modulators are output to a multi-input multi-output optical coupler of the first repeating unit, and the coherent superposition is carried out on the output N through the multi-input multi-output optical coupler 1 The complex amplitude of the i-th output signal isN 1 The path optical output is recorded asIn any of the following m-th repeating units: n (N) m-1 Road light signal->Multiple-input multiple-output optical coupler for outputting amplitude and phase modulated input amplitude modulator to mth repeating unit to output N m Way signal->N of the multi-input multi-output optocoupler output of the final Mth repeating unit M Road light signalDirectly captured by the photodetector in the detection unit (3) to output N M Road light intensity->
2. The method for applying the cascade optical coupler-based on-chip photonic neural network device according to claim 1, wherein the functional expression of amplitude and phase modulation in the step 3) is as follows:
in the above-mentioned method, the step of,output of the kth way of the mth repeating unit,/->Transformation matrix N being the mth repeating unit m ×N m-1 The k-th to i-th matrix element, j is an imaginary unit,/>The output of the ith path for the m-1 th repeating unit,an amplitude attenuation factor of the mth level i path,>an additional phase for the mth stage i-th way.
3. The method of using a cascade optocoupler-based on-chip photonic neural network device of claim 2, wherein the N M Road light intensityOutput light intensity I of any ith path i The functional expression of (2) is as follows:
4. The method for applying the cascade optical coupler-based on-chip photonic neural network device according to claim 3, wherein the step 1) further comprises the step of training the photonic neural network device:
s1) randomly selecting n samples from the training set as a sample group, wherein each sample in the sample groupWith corresponding labels->Wherein x is i All co-N representing sample x 0 An ith component in the data, where y i All co-N representing sample y M The ith component in the data, each component y of tag y i Only the component value of the position corresponding to the classification set of the sample is 1, and the rest components are 0;
s2) setting initial parameters for amplitude and phase modulation in the all-optical photon neural network, inputting each sample in the sample group into the all-optical photon neural network to output an inference result, comparing the inference result with a label y to obtain accuracy evaluation, and finishing training and exiting if the accuracy meets the requirement; otherwise, jumping to execute the next step;
s3) outputting light intensity to all-optical photon neural networkPress->The mode is normalized and then is marked asThen->According to a preset back propagation bias function +.>Starting back propagation from the mth repeating unit, and adjusting the amplitude attenuation factor in the mth repeating unit according to a preset gradient during back propagationAdditional phase->Jump execution step S1);
the back propagation bias functionThe functional expression of the initial value of (2) is as follows:
in the above-mentioned method, the step of,s is the initial value of the back propagation bias function i Ith path of normalization result, y, representing output light intensity of all-optical photonic neural network i All co-N representing sample y M The ith component, s, in the data k The k-th path normalization result, y, of the output light intensity of the all-optical photonic neural network is shown k All co-N representing sample y M The kth component in the data, n represents the number of samples in the sample group, +.>Conjugation of the output of the ith path representing the mth repeating unit;
the back propagation bias functionThe recurrence relation of the back propagation is shown as follows:
in the above-mentioned method, the step of,representing the back propagation bias function to the m-1 th repeating unit,/>An amplitude attenuation factor of the mth-stage kth path,>for the additional phase of the mth stage k-th way, j is the imaginary unit, ++>Transformation matrix N being the mth repeating unit m ×N m-1 The conjugate of the matrix element of the i-th to k-th way,/th>Is the amplitude attenuation factor of the ith path of the mth level,representing a back propagation bias function propagated to the mth repeating unit;
amplitude attenuation factor in the mth repeating unitThe gradient function expression of (2) is as follows:
in the above-mentioned method, the step of,an amplitude attenuation factor +.>Re represents the real part taking operation,conjugate of the output of the ith path representing the mth repeating unit, < >>Representing a back propagation bias function propagated to the mth repeating unit;
in the above-mentioned method, the step of,represents the additional phase of the ith path of the mth stage +.>Is a gradient of Im representing an operation taking the imaginary part,/>Conjugate of the output of the ith path representing the mth repeating unit, < >>Representing the back propagation bias function propagated to the mth repeating unit.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010605656.1A CN111680796B (en) | 2020-06-29 | 2020-06-29 | On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010605656.1A CN111680796B (en) | 2020-06-29 | 2020-06-29 | On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111680796A CN111680796A (en) | 2020-09-18 |
CN111680796B true CN111680796B (en) | 2023-07-11 |
Family
ID=72456870
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010605656.1A Active CN111680796B (en) | 2020-06-29 | 2020-06-29 | On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111680796B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112988113B (en) * | 2021-04-29 | 2021-09-14 | 中国科学院西安光学精密机械研究所 | Photon matrix vector multiplier |
CN113298246B (en) * | 2021-05-27 | 2023-02-28 | 山东云海国创云计算装备产业创新中心有限公司 | Data processing method, device and computer readable storage medium |
CN117592533B (en) * | 2023-10-30 | 2024-07-02 | 浙江大学 | Method for realizing neural network nonlinear operation by using semiconductor laser |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10268232B2 (en) * | 2016-06-02 | 2019-04-23 | Massachusetts Institute Of Technology | Apparatus and methods for optical neural network |
TW202036134A (en) * | 2018-11-02 | 2020-10-01 | 美商萊特美特股份有限公司 | Matrix multiplication using optical processing |
CN110247728B (en) * | 2019-07-01 | 2021-06-25 | 光子算数(北京)科技有限责任公司 | Photon neural network chip |
CN110929864B (en) * | 2019-12-05 | 2023-04-18 | 北京超放信息技术有限公司 | Optical diffraction neural network on-line training method and system |
CN111291881B (en) * | 2020-03-05 | 2022-10-18 | 西南大学 | Mode identification device and method based on cascade VCSELs photonic nerves |
-
2020
- 2020-06-29 CN CN202010605656.1A patent/CN111680796B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN111680796A (en) | 2020-09-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111680796B (en) | On-chip photonic neural network device and chip based on cascade optical coupler and application method of device and chip | |
Harris et al. | Linear programmable nanophotonic processors | |
De Marinis et al. | Photonic neural networks: A survey | |
Li et al. | Class-specific differential detection in diffractive optical neural networks improves inference accuracy | |
Sui et al. | A review of optical neural networks | |
Bagherian et al. | On-chip optical convolutional neural networks | |
Qu et al. | Inverse design of an integrated-nanophotonics optical neural network | |
AU2017273863B2 (en) | Apparatus and methods for optical neural network | |
CN110929864B (en) | Optical diffraction neural network on-line training method and system | |
Stark et al. | Opportunities for integrated photonic neural networks | |
CN111478735B (en) | General quantum walking simulation chip structure based on integrated optics | |
Gu et al. | SqueezeLight: Towards scalable optical neural networks with multi-operand ring resonators | |
Huang et al. | Programmable matrix operation with reconfigurable time-wavelength plane manipulation and dispersed time delay | |
US11600060B2 (en) | Nonlinear all-optical deep-learning system and method with multistage space-frequency domain modulation | |
Ishihara et al. | An optical neural network architecture based on highly parallelized WDM-multiplier-accumulator | |
Zang et al. | Optoelectronic convolutional neural networks based on time-stretch method | |
Qiu et al. | Decision-making and control with diffractive optical networks | |
Zhao et al. | On-chip reconfigurable optical neural networks | |
Xu et al. | Photoelectric hybrid convolution neural network with coherent nanophotonic circuits | |
CN115496195A (en) | MZI array and FP-SA-based optical pulse neural network implementation device | |
Hattori et al. | Optical-electronic implementation of artificial neural network for ultrafast and accurate inference processing | |
Cartlidge | Photonic computing for sale | |
De Marinis et al. | A photonic accelerator for feature map generation in convolutional neural networks | |
Jonuzi et al. | Numerical Analysis of a Self-Calibrating Time-Spatial Interleaving Photonic Convolutional Accelerator | |
Zhang et al. | Time-stretch optical neural network with time-division multiplexing |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |