CN116484931B - Photon matrix multiplication operation device and operation method for neural network - Google Patents
Photon matrix multiplication operation device and operation method for neural network Download PDFInfo
- Publication number
- CN116484931B CN116484931B CN202310703095.2A CN202310703095A CN116484931B CN 116484931 B CN116484931 B CN 116484931B CN 202310703095 A CN202310703095 A CN 202310703095A CN 116484931 B CN116484931 B CN 116484931B
- Authority
- CN
- China
- Prior art keywords
- signal
- optical
- matrix
- frequency
- frequency hopping
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 169
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 title claims abstract description 13
- 230000003287 optical effect Effects 0.000 claims abstract description 239
- 239000013307 optical fiber Substances 0.000 claims abstract description 35
- 230000000737 periodic effect Effects 0.000 claims description 74
- 125000004122 cyclic group Chemical group 0.000 claims description 22
- 238000012545 processing Methods 0.000 claims description 9
- 230000001360 synchronised effect Effects 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 230000001105 regulatory effect Effects 0.000 claims description 5
- 230000001276 controlling effect Effects 0.000 claims description 3
- 230000001537 neural effect Effects 0.000 claims 5
- 238000005516 engineering process Methods 0.000 abstract description 7
- 238000004364 calculation method Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 10
- 238000013507 mapping Methods 0.000 description 8
- 230000001133 acceleration Effects 0.000 description 4
- 239000004973 liquid crystal related substance Substances 0.000 description 4
- 238000010521 absorption reaction Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 239000000969 carrier Substances 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
- G06N3/0675—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means using electro-optical, acousto-optical or opto-electronic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06E—OPTICAL COMPUTING DEVICES; COMPUTING DEVICES USING OTHER RADIATIONS WITH SIMILAR PROPERTIES
- G06E3/00—Devices not provided for in group G06E1/00, e.g. for processing analogue or hybrid data
- G06E3/001—Analogue devices in which mathematical operations are carried out with the aid of optical or electro-optical elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06E—OPTICAL COMPUTING DEVICES; COMPUTING DEVICES USING OTHER RADIATIONS WITH SIMILAR PROPERTIES
- G06E3/00—Devices not provided for in group G06E1/00, e.g. for processing analogue or hybrid data
- G06E3/008—Matrix or vector computation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a photon matrix multiplication device facing a neural network, which takes a periodical frequency hopping optical signal as an optical carrier wave, loads a first matrix signal and a second matrix signal through serially connecting fewer electro-optical modulators, and multiplies matrix elements, so that the complexity of a traditional photon matrix calculation system can be simplified. The optical fiber delay unit is used for realizing time alignment of a plurality of frequency hopping components in the periodical frequency hopping optical signal and the periodical pulse optical carrier signal, and based on a time-frequency multiplexing technology, multiplication of two matrix elements can be realized in a single period, and the scheme is simple and efficient. The electro-optical modulator is used for loading matrix signals to be operated, the calculation speed can be greatly improved based on the large bandwidth advantage of the modulator, and the matrix can be dynamically adjusted. The invention also discloses a method for carrying out photon matrix multiplication operation by utilizing the photon matrix multiplication operation device.
Description
Technical Field
The invention belongs to the field of photon computing systems, and particularly relates to a photon matrix multiplication device and a photon matrix multiplication method for a neural network.
Background
With the rapid development of artificial intelligence technology, the global computing power increases sharply, and massive data needs to be processed in a faster and efficient manner, which puts higher demands on the computing power and energy efficiency of computing hardware.
At present, the electronic chip adopts a classical computer structure which separates a program space from a data space, so that data load between a storage unit and a computing unit is unstable, power consumption is high, and the traditional electronic computing encounters a bottleneck which is difficult to break through in terms of speed and energy efficiency.
The literature [ Shastri B J, tait a N, ferreira de Lima T, et al Photonics for artificial intelligence and neuromorphic computing Nature Photonics, 2021, 15 (2): 102-114 ] discloses that photonic technologies using photons as information carriers have characteristics of large bandwidth, low loss, parallelism, and the like, and has attracted researchers to apply photonic technologies in the computing field.
Currently, literature [ photoelectric intelligent computing research progress and challenge chinese laser, 2022, 49 (12): 1219001.] discloses neuromorphic computing hardware constructed from conventional photonic devices, such as mach-zehnder interferometer networks and micro-ring resonator arrays, the energy consumption of each multiply add is approximately at the fly-focus level, which is two orders of magnitude smaller than the most advanced complementary metal oxide semiconductor computing hardware, indicating that optical neural networks are far superior to electrical neural networks in computing energy efficiency while achieving ultra-high speed computation, and have significant advantages naturally in application scenarios such as high concurrency, high throughput, computationally intensive supercomputer platforms, data centers, and the like.
Chinese patent publication No. CN114358271a discloses a convolutional acceleration chip for time-wavelength interleaved photonic neural networks, which is suitable for all deep learning networks including convolutional operations. The invention integrates the modulator, the wavelength division delay weighting micro-ring array and the balance photoelectric detector which finish convolution acceleration operation through a photon integration technology. The method is characterized in that signals to be processed are respectively loaded onto a plurality of optical carriers based on a wavelength division multiplexing technology, convolution kernel coefficient weighting and time interleaving of different carrier signals are achieved through a micro-ring and a delay line, and summation operation after weighting is achieved through a balanced photoelectric detector. The invention can realize the construction of any convolution kernel matrix by utilizing the resonance characteristic of the integrated micro-circulator, and can complete convolution acceleration operation of any signal by combining time delay. The speed and energy efficiency ratio of convolution operation can be greatly improved by taking light as an information carrier. However, the wavelength division delay weighted micro-ring array disclosed in the above patent needs more optical modulators, namely micro-rings, so that the matrix vector photon multiplier proposed in the above patent has complex structure and poor operation stability. Therefore, it is needed to design a photonic matrix multiplication device with simple structure and good stability.
Disclosure of Invention
The invention provides a photon matrix multiplication device facing a neural network, which can carry out photon matrix multiplication by using fewer electro-optical modulators, has simple equipment and more accurate operation result.
The embodiment of the invention provides a photon matrix multiplication device facing a neural network, which comprises:
the light source input unit is used for carrying out pulse width regulation on the obtained continuous optical carrier signals to obtain periodic pulse optical carrier signals, and setting the time period of photon matrix multiplication operation;
the cyclic frequency shift loop is used for performing optical domain frequency shift on the periodic pulse optical carrier signal or the frequency hopping component obtained in the last cycle through the frequency shift signal to obtain an initial frequency hopping component, amplifying and delaying the initial frequency hopping component to obtain a frequency hopping component in the current cycle, obtaining a plurality of frequency hopping components with different frequencies and time through a plurality of cycles, and constructing a periodic frequency hopping optical signal based on the plurality of frequency hopping components and the periodic pulse optical carrier signal;
the electro-optical modulator is used for modulating the first matrix signal onto the periodical frequency hopping optical signal to obtain a first modulated optical signal, and modulating the second matrix signal onto the first modulated optical signal to obtain a second modulated optical signal, so that the code element of the first matrix signal and the corresponding code element of the second matrix signal are multiplied;
a synchronization control unit for aligning time sequences of the periodic frequency hopping optical signal, the first matrix signal and the second matrix signal with each other;
the optical fiber delay unit is used for demultiplexing the second modulated optical signal into a plurality of sub-modulated optical signals corresponding to the frequency of the periodic frequency hopping optical signal, and adding the plurality of sub-modulated optical signals after time alignment through wavelength division multiplexing so as to obtain a product optical signal by adding a plurality of multiplied code elements; and
and the result output unit is used for sequentially carrying out photoelectric and digital conversion on the product optical signal to obtain a digital signal, thereby obtaining a multiplication result.
Further, the electro-optic modulator comprises a first Mach-Zehnder modulator and a second Mach-Zehnder modulator;
the first Mach-Zehnder modulator is used for modulating code elements of the first matrix signal onto corresponding frequency hopping components or periodic pulse optical carrier signals in the periodic frequency hopping optical signals to obtain first modulated optical signals;
the second Mach-Zehnder modulator is used for modulating the code element of the second matrix signal to the corresponding frequency hopping component or the periodic pulse optical carrier signal in the periodic frequency hopping optical signal loaded with the code element of the first matrix signal to obtain a second modulated optical signal.
Further, the cyclic frequency shift loop comprises an optical coupler, a frequency shifter, a single-frequency signal source, an optical amplifier and an optical delay line;
the single-frequency signal source is used for generating a first frequency shift signal;
the optical coupler is used for respectively inputting the periodic pulse optical carrier signal or the frequency hopping component obtained in the last cycle to the electro-optical modulator and the frequency shifter;
the frequency shifter is used for performing optical domain frequency shifting on the periodic optical carrier signal or the frequency hopping component obtained in the last cycle based on the first frequency shifting signal to obtain initial frequency hopping components with different frequencies;
the optical amplifier is used for amplifying the initial frequency hopping component;
the optical delay line is used for delaying the amplified initial frequency hopping component to obtain a current frequency hopping component, and inputting the current frequency hopping component into the optical coupler for next circulation;
and (3) performing multiple circulation until the time sequence of the obtained periodic frequency hopping optical signal reaches a set time period, and stopping circulation to obtain multiple frequency hopping components.
Further, the synchronization control unit is further configured to output a frequency control signal to the single-frequency signal source, enable the single-frequency signal source to generate a second frequency shift signal based on the frequency control signal, and perform optical domain frequency shift on the periodic pulse optical carrier signal or the frequency hopping component obtained in the last cycle based on the second frequency shift signal to obtain an initial frequency hopping component, so that the frequency of the initial frequency hopping component corresponds to the frequency of the sub-modulated optical signal output by the wavelength division multiplexing.
Further, the synchronization control unit simultaneously transmits synchronization control information to the light source input unit, the cyclic frequency shift ring and the data source to be operated, and the symbol time sequence of the first matrix signal, the symbol time sequence of the second matrix signal and the time sequence of the periodic frequency hopping optical signal are mutually aligned based on the synchronization control information, so that the symbols of the first matrix signal and the second matrix signal can be loaded on the periodic frequency hopping optical signal at corresponding time;
the first matrix signal and the second matrix signal are output through a data source to be operated.
Further, the data source to be operated is configured to perform flattening processing on the obtained one-dimensional matrix or multidimensional matrix to obtain a first matrix signal and a second matrix signal, where the number of symbols of the first matrix signal and the second matrix signal is the same.
Further, the light source input unit comprises a laser and an optical switch, wherein the laser is used for outputting continuous optical carrier signals, and the optical switch is used for regulating the pulse width of the optical carrier signals and setting the time period of photon matrix multiplication operation based on the code element number of the first matrix signals and the second matrix signals.
Further, the optical fiber delay unit comprises a wavelength division demultiplexer, an optical fiber delay array and a wavelength division multiplexer;
the wavelength division demultiplexer is used for decomposing the second modulated optical signal into a plurality of sub-modulated optical signals, and the frequencies of the sub-modulated optical signals correspond to the frequencies of the periodic frequency hopping optical signals;
the optical fiber delay array is used for regulating and controlling the lengths of optical fibers corresponding to the plurality of sub-modulation optical signals so as to align the time of the plurality of sub-modulation optical signals;
the wavelength division multiplexer is used for integrating the plurality of time-aligned sub-modulated optical signals to obtain a product optical signal.
Further, the result output unit comprises a photoelectric detector and an acquisition processing module;
the photoelectric detector is used for carrying out photoelectric conversion on the product optical signal to obtain an electric output signal, and the acquisition processing module is used for carrying out digital conversion on the electric output signal to obtain a digital signal.
The embodiment of the invention also provides a photon matrix multiplication method facing the neural network, which is characterized by comprising the following steps: the photon matrix multiplication device facing the neural network is adopted to multiply the code element of the obtained first matrix signal with the second matrix signal to obtain a product optical signal, and the product optical signal is converted into a digital signal, so that matrix multiplication is realized.
Compared with the prior art, the invention has the beneficial effects that:
(1) According to the invention, the periodic frequency hopping optical signal is used as an optical carrier, and the first matrix signal and the second matrix signal which are obtained by converting matrix data of the neural network are loaded through the electro-optical modulator with fewer serial connections and are multiplied by matrix elements, so that the complexity of a traditional photon matrix computing system can be simplified.
(2) The invention realizes the time alignment of a plurality of frequency hopping components in the periodical frequency hopping optical signal and the periodical pulse optical carrier signal through the optical fiber extension unit, and can realize the multiplication of two matrix elements in a single period based on the time-frequency multiplexing technology, and the scheme is simple and efficient.
(3) The invention realizes the loading of matrix signals to be operated through the electro-optical modulator, can greatly improve the calculation speed based on the large bandwidth advantage of the modulator, and can dynamically adjust the matrix.
Drawings
FIG. 1 is a block diagram of a photonic matrix multiplication device for a neural network according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a photonic matrix multiplication device facing to a neural network according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a cyclic shift ring according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an optical fiber delay unit according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a photonic matrix multiplication device facing a neural network, where the electro-optical modulator is a mach-zehnder modulator according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a time-frequency mapping relationship of a periodic frequency-hopping optical signal according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a time-frequency mapping relationship of a first modulated optical signal according to an embodiment of the present invention;
FIG. 8 is a schematic diagram illustrating a time-frequency mapping relationship of a second modulated optical signal according to an embodiment of the present invention;
fig. 9 is a schematic diagram of time-frequency mapping relationship of a product optical signal according to an embodiment of the present invention.
Detailed Description
Aiming at the defects of the prior art, the specific embodiment of the invention provides a photon matrix multiplication device facing a neural network, and the specific thought is to sequentially load a first matrix signal and a second matrix signal based on periodic frequency hopping optical signals, realize delay alignment of different frequency component signals by combining an optical fiber delay array, and realize product operation of two matrices in a single frequency hopping period. In the scheme, the matrix can be flexibly expanded, and the signal processing is real-time and efficient.
The embodiment of the invention provides a photon matrix multiplication device facing a neural network, as shown in fig. 1, comprising:
the light source input unit is used for carrying out pulse width regulation and control on the continuous optical carrier signals to obtain periodic pulse optical carrier signals;
the cyclic frequency shift loop is used for performing optical domain frequency shift on the periodic pulse optical carrier signal or the frequency hopping component obtained in the last cycle through the frequency shift signal to obtain an initial frequency hopping component, sequentially amplifying and delaying the initial frequency hopping component to obtain a frequency hopping component in the current cycle, obtaining a plurality of frequency hopping components through multiple cycles, and constructing a periodic frequency hopping optical signal based on the plurality of frequency hopping components and the periodic pulse optical carrier signal;
a synchronization control unit for aligning timings of the periodic frequency hopping optical signal, the first matrix signal, and the second matrix signal with each other;
the electro-optical modulator is used for modulating the first matrix signal to the periodical frequency hopping optical signal to obtain a first modulated optical signal, and modulating the second matrix signal to the first modulated optical signal to obtain a second modulated optical signal;
the optical fiber delay unit is used for performing wavelength division multiplexing on the second modulated optical signals to obtain a plurality of sub-modulated optical signals, and performing wavelength division multiplexing on the plurality of sub-modulated optical signals after time alignment to obtain product optical signals; and
and the result output unit is used for converting the product optical signal into a digital signal so as to obtain a multiplication result.
In a specific embodiment, the embodiment provides a photon matrix multiplication device facing a neural network, as shown in fig. 2, which comprises a laser source, an optical switch, a cyclic frequency shift ring, a synchronous control unit, a data source to be operated, a first electro-optic modulator, a second electro-optic modulator, an optical fiber delay unit, a photoelectric detector and an acquisition processing module.
The laser source provided by the embodiment of the invention is used for outputting the optical carrier signal to the optical switch, and the pulse width fatin of the optical carrier signal is regulated and controlled based on the code element number of the first matrix signal and the second matrix signal through the optical switchtAnd simultaneously setting a time period T of photon matrix multiplication operation.
In a specific embodiment, the optical switch provided by the invention is a silicon-based optical switch, a micro-electro-mechanical optical switch and a rectangular pulse modulation electro-optical modulator.
The cyclic frequency shift ring provided by the embodiment of the invention comprises a 2×2 optical coupler, a frequency shifter, a single-frequency signal source, an optical amplifier and an optical delay line, as shown in fig. 3.
Wherein, the periodic pulse optical carrier signal or the frequency hopping component obtained in the last cycle is respectively input into the first electro-optical modulator and the frequency shifter through the 2X 2 optical coupler, and the frequency is generated by a single frequency signal source∆fInputting the first frequency shift signal into a frequency shifter, and frequency-shifting the periodic pulse optical carrier signal or the frequency-hopping component obtained in the last cycle by the first frequency shift signalThe method comprises the steps of obtaining initial frequency hopping components with different frequencies by realizing optical domain frequency shifting, amplifying the initial frequency hopping components through an optical amplifier, and delaying the amplified initial frequency hopping components through an optical delay line pairtObtaining a frequency hopping component of the current cycle, wherein the frequency and time of the frequency hopping component are different from those of a periodic pulse optical carrier signal before frequency shift or the frequency hopping component obtained in the last cycle, and the obtained frequency hopping component of the current cycle is subjected to the next cycle through a 2X 2 optical coupler. And (3) performing multiple circulation until the time sequence of the obtained periodic frequency hopping optical signal reaches a set time period, stopping circulation, and obtaining a plurality of frequency hopping components with different frequencies and times. Because the time and the frequency of the obtained frequency hopping components are different, each code element of the first matrix signal and the second matrix signal output by the data source to be operated can be loaded on the corresponding frequency hopping component through fewer electro-optical modulators, and the complexity of the device is reduced.
In one embodiment, the frequency shifter provided by the present invention is an acousto-optic frequency shifter, a dual parallel Mach-Zehnder modulator (DPMZM).
The synchronization control unit provided by the embodiment of the invention is used for simultaneously sending the synchronization control information to the optical switch, the cyclic frequency shift ring and the data source to be operated, and based on the synchronization control information, the symbol time sequence of the first matrix signal, the symbol time sequence of the second matrix signal and the time sequence of the periodic frequency hopping optical signal, namely the frequency hopping components with different frequencies in the periodic frequency hopping optical signal and the time sequence of the periodic pulse optical carrier signal are mutually aligned, so that the symbols of the first matrix signal and the second matrix signal can be loaded on the periodic frequency hopping optical signal with corresponding time.
In a specific embodiment, as shown in fig. 3, the synchronization control unit provided by the present invention is further configured to output a frequency control signal to the single-frequency signal source, enable the single-frequency signal source to generate a second frequency shift signal based on the frequency control signal, and perform optical domain frequency shift on the periodic pulse optical carrier signal or the frequency-hopping component obtained in the last cycle based on the second frequency shift signal to obtain an initial frequency-hopping component, where the frequency of the initial frequency-hopping component corresponds to the frequency of the sub-modulated optical signal output by the wavelength division multiplexing.
In a specific embodiment, the synchronization control unit provided by the specific embodiment of the invention simultaneously sends synchronization control information to the light source input unit, the cyclic frequency shift ring and the data source to be operated, and respectively controls the time of outputting the periodic pulse optical carrier signal by the optical switch and the time of outputting the periodic pulse optical carrier signal by the 2×2 optical coupler and the time of outputting the corresponding code element of the first matrix signal by the data source to be operated and the time of outputting the corresponding code element of the second matrix signal by the synchronous control information, so that the corresponding code element of the first matrix signal and the corresponding code element of the second matrix signal can be accurately and sequentially loaded by the periodic frequency hopping optical signal, thereby realizing accurate multiplication of the corresponding code element of the first matrix signal and the corresponding code element of the second matrix signal.
The data source to be operated provided by the embodiment of the invention is used for flattening the obtained one-dimensional matrix data or multidimensional matrix data of the neural network to be multiplied to obtain a first matrix signal and a second matrix signal respectively, wherein the first matrix signal and the second matrix signal are one-dimensional matrix electric signals, the number of code elements of the first matrix signal and the second matrix signal is the same, the code elements of the first matrix signal and the second matrix signal are N, the duration of a single code element of the first matrix signal, the duration of a single code element of the second matrix signal, the pulse width of each frequency hopping component and the pulse width of the periodical pulse optical carrier signal are equalEqual.
The first electro-optical modulator provided by the embodiment of the invention is used for modulating the code element of the first matrix signal onto a frequency hopping component periodic pulse optical carrier signal corresponding to time and frequency in the periodic frequency hopping optical signal to obtain a first modulated optical signal; the second electro-optical modulator is used for modulating the code elements of the second matrix signal to the frequency hopping component periodic pulse optical carrier signals corresponding to the time and the frequency in the periodic frequency hopping optical signals loaded with the code elements of the first matrix signal to obtain a second modulated optical signal.
In a particular embodiment, the first electro-optic modulator comprises an electro-absorption modulator or a Mach-Zehnder modulator and the second electro-optic modulator comprises an electro-absorption modulator or a Mach-Zehnder modulator.
The optical fiber delay unit provided by the embodiment of the invention, as shown in fig. 4, comprises a wavelength division demultiplexer, an optical fiber delay array and a wavelength division multiplexer;
the wavelength division demultiplexer is used for decomposing the second modulated optical signal into a plurality of sub-modulated optical signals, and the frequencies of the sub-modulated optical signals correspond to the frequencies of the periodic frequency hopping optical signals;
the optical fiber delay array is used for regulating and controlling the lengths of optical fibers corresponding to the plurality of sub-modulated optical signals, namely, the time alignment of the plurality of sub-modulated optical signals is realized through delay optical fibers with equal interval increase of delay;
the wavelength division multiplexer is used for integrating the plurality of time-aligned sub-modulated optical signals to obtain a product optical signal.
The photoelectric detector provided by the embodiment of the invention is used for carrying out photoelectric conversion on the product optical signal to obtain an electric output signal, and the acquisition processing module is used for carrying out digital conversion on the electric output signal to obtain a digital signal. The multiplication of the first matrix signal and the second matrix signal can be completed, thereby completing the multiplication of one-dimensional matrix data or multidimensional matrix data.
The invention provides a specific embodiment, which is used for describing a photon matrix multiplication device facing a neural network in detail.
As shown in fig. 5, the photonic two-dimensional convolution acceleration system based on time-wavelength interleaving of this specific embodiment includes: the system comprises 1 laser source, 1 optical switch, 1 cyclic frequency shift ring, 1 synchronous control unit, 2 Mach-Zehnder modulators (MZM 1, MZM 2), signal source to be convolved, optical fiber delay array, photoelectric detector, acquisition processing unit, optical fiber delay unit composed of 1 wavelength division multiplexer, optical fibers with different M sections of lengths, 1 wavelength division demultiplexer, etc.
The cyclic frequency shift loop provided in this embodiment includes 12×2 optical coupler, 1 single frequency signal, 1 double parallel mach-zehnder modulator (DPMZM), 1 optical amplifier, and 1 optical delay line.
First, the laser source generates a laser beam with a frequency ofA kind of electronic devicef 1 The optical carrier signal enters an optical switch, and the optical switch performs periodic switching with the period of T on the optical carrier signal to obtain a pulse width ofIs a periodic pulse optical carrier signal of which the time domains（t） c Expressed as:
s（t） c =A 0 exp（j2πf 1 t）（0≤t≤） (1)
wherein, the liquid crystal display device comprises a liquid crystal display device,A 0 is the amplitude of the periodically pulsed optical carrier signal,jis imaginary, t is time.
The embodiment sends the periodic pulse optical carrier signal into the cyclic frequency shift ring, the periodic pulse optical carrier signal enters the optical coupler through the first input port of the 2X 2 optical coupler and is divided into two paths, one path is output from the first output end of the 2X 2 optical coupler, the other path enters the cyclic frequency shift ring from the second output end, the periodic pulse optical carrier signal entering the cyclic frequency shift ring is sent into the double parallel Mach-Zehnder modulator, the single frequency signal source generates the second frequency shift signal to realize carrier single sideband modulation on the periodic pulse optical carrier signal, the optical domain frequency shift is realized to obtain the initial frequency hopping component, the frequency of the initial frequency hopping component corresponds to the frequency of the sub-modulated optical signal output by the de-wavelength division multiplexing, the initial frequency hopping component is sent into the optical amplifier to be amplified, the first frequency hopping component is obtained by the optical delay line delay, the currently circulated frequency hopping component is sent into the second input end of the 2X 2 optical coupler, and the cyclic frequency shift ring is delayed as follows. At this time, the time domain of the first frequency hopping components（t） C1 The method comprises the following steps:
s（t） C1 =A 1 exp（j2πf 2 t） （ ≤t≤2/>） (2)
wherein A is 1 For the first frequency-hopped signal amplitude,f 2 and (3) withf 1 The frequency difference is. The first frequency hopping component is divided into two paths as the periodic pulse optical carrier signal, one path is output from the first output end of the 2 x 2 optical coupler, and the other path enters the loop of the cyclic frequency shift loop from the second output end. The first frequency hopping component entering the ring is subjected to the same loop as the periodical pulse optical carrier signal, and the frequency hopping component after the subsequent cyclic frequency shifting ring is subjected to the same loop, so that a periodical pulse optical carrier signal containing M-1 frequency hopping components and one periodical pulse optical carrier signal is obtained, namely, the periodical frequency hopping optical signal with the period of the M frequency components being T, wherein M is a positive integer. A schematic diagram of a time-frequency mapping relationship of the periodic frequency hopping optical signal output by the cyclic frequency hopping loop is shown in FIG. 6, wherein the first time of the periodic frequency hopping optical signalmFrequency hopping signals（t） Cm The method comprises the following steps:
s（t） Cm =A m exp（j2πf m+1 t） （3）
wherein A is m Is the firstmThe amplitude of the frequency-hopped signal,f m+1 and (3) withf 1 The frequency difference ism . The amplitude of each frequency-hopping signal of the periodic frequency-hopping optical signal in one period can be represented as a= [ a ] by a matrix 0 ,A 1 ,A 2 ,…,A M-1 ] T M×1 In actual operation, the amplitude of each frequency hopping signal can be operated to be the same size through a beam shaper, and can also be compensated through an algorithm in matrix operation, so that influence of inconsistent amplitude on an operation result is avoided, and the embodiment uses the amplitude of each frequency hopping signalThe degree is equal, that is, the amplitude of each frequency hopping signal of the periodic frequency hopping optical signal is equal in one period, and the matrix can be expressed as a= [ a, …, a] T M×1 。
The method comprises the steps that a periodic frequency hopping optical signal is sent to a first Mach-Zehnder modulator (MZM 1), a first matrix signal to be output by a data source to be operated is modulated onto the periodic frequency hopping optical signal through the first Mach-Zehnder modulator to obtain a first modulated optical signal, and a second matrix signal to be output by the data source to be operated is modulated onto the first modulated optical signal through a second Mach-Zehnder modulator (MZM 2) to obtain a second modulated optical signal; meanwhile, the synchronous control unit outputs a synchronous control signal to realize synchronous alignment of the time sequence of the frequency hopping optical signal output by the cyclic frequency shifting ring and the time sequence of the first matrix signal and the second matrix signal output by the data source to be operated. It should be noted that the first matrix signal and the second matrix signal are one-dimensional matrices obtained by flattening the one-dimensional matrix or the multidimensional matrix, the number N of symbols corresponding to the first matrix signal and the second matrix signal is equal, and the duration of a single symbol of the first matrix signal, the duration of a single symbol of the second matrix signal and the pulse width of the periodic pulse optical carrier signal are equalEqual. The flattened first matrix signal matrix may be expressed as x= [ X ] 1 , x 2 , x 3 ,…, x N ] T N×1 The flattened matrix for the second matrix signal can be expressed as w= [ W ] 1 , w 2 , w 3 ,…, w N ] T N×1 . N is a positive integer, the number of code elements corresponding to the first matrix signal and the second matrix signal is smaller than or equal to the number M of the frequency components of the periodic frequency hopping optical signal, and N is equal to M in the embodiment. The corresponding time-frequency mapping relationship of the first modulated optical signal is shown in fig. 7, and the time-frequency mapping relationship of the second modulated optical signal is shown in fig. 8.
Then the second modulated optical signal is sent into an optical fiber delay array de-wavelength division multiplexer to be de-wavelength division multiplexed into M sub-modulated lightThe M sub-modulation optical signals are respectively time aligned by time delay optical fibers with equal-interval increased time delay, and the delayed M sub-modulation optical signals are combined into a product optical signal by a wavelength division multiplexer of an optical fiber delay array. The optical fiber length satisfaction relation of M sections of different lengths in the optical fiber delay array is as follows: with the mth optical fiber as a reference, the lengths of other optical fibers are sequentially increased by father l=c /n f WhereincFor the speed of the light in vacuum,n f is the refractive index of the optical fiber; the channel interval of the demultiplexer/wavelength division multiplexer in the optical fiber delay array is +.f. with the frequency interval of the adjacent frequency of the frequency hopping component of the periodical frequency hopping optical signal>Equal. The time-frequency mapping relation of the product optical signal output by the optical fiber delay array is shown in fig. 9. And the product optical signal output by the optical fiber delay array is sent to a photoelectric detector to complete photoelectric conversion to obtain an electric output signal, and multiplication operation of the first matrix and the second matrix can be completed after the electric output signal is acquired and reconstructed. Electric output signal strengthS ca The method comprises the following steps:
(4)
wherein, the liquid crystal display device comprises a liquid crystal display device,iis the index of the symbol.
The embodiment of the invention also provides a photon matrix multiplication method facing the neural network, which is characterized by comprising the following steps: the photon matrix multiplication device facing the neural network is adopted to multiply the code element of the obtained first matrix signal with the code element corresponding to the second matrix signal, the multiplied code elements are added to obtain a product optical signal, and the product optical signal is converted into a digital signal, so that matrix multiplication operation of one period T is realized.
Finally, it should be noted that the above list is only specific embodiments of the present invention. The invention is not limited to the above embodiments, but many variations are possible. All modifications directly derived or suggested to one skilled in the art from the present disclosure should be considered as being within the scope of the present invention.
Claims (8)
1. A neural network-oriented photon matrix multiplication device, comprising:
the light source input unit is used for carrying out pulse width regulation and control on the continuous optical carrier signals to obtain periodic pulse optical carrier signals;
the cyclic frequency shift loop is used for performing optical domain frequency shift on the periodic pulse optical carrier signal or the frequency hopping component obtained in the last cycle through the frequency shift signal to obtain an initial frequency hopping component, sequentially amplifying and delaying the initial frequency hopping component to obtain a frequency hopping component in the current cycle, obtaining a plurality of frequency hopping components through multiple cycles, and constructing a periodic frequency hopping optical signal based on the plurality of frequency hopping components and the periodic pulse optical carrier signal;
a synchronization control unit for aligning timings of the periodic frequency hopping optical signal, the first matrix signal, and the second matrix signal with each other;
the electro-optical modulator is used for modulating the first matrix signal to the periodical frequency hopping optical signal to obtain a first modulated optical signal, and modulating the second matrix signal to the first modulated optical signal to obtain a second modulated optical signal;
the optical fiber delay unit is used for performing wavelength division multiplexing on the second modulated optical signals to obtain a plurality of sub-modulated optical signals, and performing wavelength division multiplexing on the plurality of sub-modulated optical signals after time alignment to obtain product optical signals; and
the result output unit is used for converting the product optical signal into a digital signal so as to obtain a multiplication result;
the synchronous control unit simultaneously transmits synchronous control information to the light source input unit, the cyclic frequency shift ring and the data source to be operated, and the symbol time sequence of the first matrix signal, the symbol time sequence of the second matrix signal and the time sequence of the periodic frequency hopping optical signal are mutually aligned based on the synchronous control information, so that the symbols of the first matrix signal and the second matrix signal can be loaded on the periodic frequency hopping optical signal at corresponding time;
the first matrix signal and the second matrix signal are output through a data source to be operated;
the data source to be operated is used for flattening the obtained one-dimensional matrix or multidimensional matrix of the neural network to obtain a first matrix signal and a second matrix signal respectively, and the number of code elements of the first matrix signal and the second matrix signal is the same.
2. The neural network oriented photonic matrix multiplication device of claim 1, wherein the electro-optic modulator comprises a first mach-zehnder modulator and a second mach-zehnder modulator;
the first Mach-Zehnder modulator is used for modulating code elements of the first matrix signal onto corresponding frequency hopping components or periodic pulse optical carrier signals in the periodic frequency hopping optical signals to obtain first modulated optical signals;
the second Mach-Zehnder modulator is used for modulating the code element of the second matrix signal to the corresponding frequency hopping component or the periodic pulse optical carrier signal in the periodic frequency hopping optical signal loaded with the code element of the first matrix signal to obtain a second modulated optical signal.
3. The neural network-oriented photon matrix multiplication device of claim 1, wherein the cyclic frequency shift loop comprises an optical coupler, a frequency shifter, a single frequency signal source, an optical amplifier, and an optical delay line;
the single-frequency signal source is used for generating a first frequency shift signal;
the optical coupler is used for respectively inputting the periodic pulse optical carrier signal or the frequency hopping component obtained in the last cycle to the electro-optical modulator and the frequency shifter;
the frequency shifter is used for performing optical domain frequency shifting on the periodic optical carrier signal or the frequency hopping component obtained in the last cycle based on the first frequency shifting signal to obtain initial frequency hopping components with different frequencies;
the optical amplifier is used for amplifying the initial frequency hopping component;
the optical delay line is used for delaying the amplified initial frequency hopping component to obtain a current frequency hopping component, and inputting the current frequency hopping component into the optical coupler for next circulation;
and (3) performing multiple circulation until the time sequence of the obtained periodic frequency hopping optical signal reaches a set time period, stopping circulation, and obtaining a plurality of frequency hopping components with different frequencies and times.
4. The photonic matrix multiplication apparatus for a neural network according to claim 3, wherein the synchronization control unit is further configured to output a frequency control signal to the single-frequency signal source, cause the single-frequency signal source to generate a second frequency shift signal based on the frequency control signal, and perform optical domain frequency shift on the periodic pulsed optical carrier signal or the frequency-hopping component obtained in the last cycle based on the second frequency shift signal to obtain an initial frequency-hopping component, so that the frequency of the initial frequency-hopping component corresponds to the frequency of the sub-modulated optical signal output by the wavelength division multiplexing.
5. The neural network-oriented photon matrix multiplication device according to claim 1, wherein the light source input unit includes a laser for outputting a continuous optical carrier signal and an optical switch for adjusting a pulse width of the optical carrier signal and setting a time period of a photon matrix multiplication operation based on the number of symbols of the first matrix signal and the second matrix signal to obtain a periodic pulse optical carrier signal.
6. The neural network-oriented photon matrix multiplication device according to claim 1, wherein the optical fiber delay unit comprises a demultiplexer, an optical fiber delay array and a wavelength division multiplexer;
the wavelength division demultiplexer is used for decomposing the second modulated optical signal into a plurality of sub-modulated optical signals, and the frequencies of the sub-modulated optical signals correspond to the frequencies of the periodic frequency hopping optical signals;
the optical fiber delay array is used for regulating and controlling the lengths of optical fibers corresponding to the plurality of sub-modulation optical signals so as to align the time of the plurality of sub-modulation optical signals;
the wavelength division multiplexer is used for integrating the plurality of time-aligned sub-modulated optical signals to obtain a product optical signal.
7. The neural network-oriented photon matrix multiplication device according to claim 1, wherein the result output unit comprises a photodetector and an acquisition processing module;
the photoelectric detector is used for carrying out photoelectric conversion on the product optical signal to obtain an electric output signal, and the acquisition processing module is used for carrying out digital conversion on the electric output signal to obtain a digital signal.
8. The operation method of photon matrix multiplication facing to the neural network is characterized by comprising the following steps: the photon matrix multiplication device for the neural network according to any one of claims 1 to 7 is adopted to multiply the obtained code element of the first matrix signal with the second matrix signal to obtain a product optical signal, and the product optical signal is converted into a digital signal, so that matrix multiplication is realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310703095.2A CN116484931B (en) | 2023-06-14 | 2023-06-14 | Photon matrix multiplication operation device and operation method for neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310703095.2A CN116484931B (en) | 2023-06-14 | 2023-06-14 | Photon matrix multiplication operation device and operation method for neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116484931A CN116484931A (en) | 2023-07-25 |
| CN116484931B true CN116484931B (en) | 2023-09-01 |
Family
ID=87219865
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310703095.2A Active CN116484931B (en) | 2023-06-14 | 2023-06-14 | Photon matrix multiplication operation device and operation method for neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116484931B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2320613A1 (en) * | 1999-10-06 | 2001-04-06 | Alan F. Graves | Switch for optical signals |
| CN110009102A (en) * | 2019-04-12 | 2019-07-12 | 南京吉相传感成像技术研究院有限公司 | A kind of accelerated method of the depth residual error network based on photoelectricity computing array |
| CN110516802A (en) * | 2019-07-03 | 2019-11-29 | 上海交通大学 | Photon convolutional neural networks framework based on optical delay line caching |
| CN112232503A (en) * | 2020-06-09 | 2021-01-15 | 联合微电子中心有限责任公司 | Computing device, computing method, and computing system |
| CN112232504A (en) * | 2020-09-11 | 2021-01-15 | 联合微电子中心有限责任公司 | Photon neural network |
| CA3094544A1 (en) * | 2019-09-26 | 2021-03-26 | Xanadu Quantum Technologies Inc. | Management of power consumption in optical circuits for quantum computing |
| CN113705774A (en) * | 2021-07-29 | 2021-11-26 | 浪潮电子信息产业股份有限公司 | Optical circuit construction method, optical circuit, optical signal processing method and device |
| WO2022016894A1 (en) * | 2020-07-20 | 2022-01-27 | 联合微电子中心有限责任公司 | Photonic neural network |
| CN114358271A (en) * | 2022-03-18 | 2022-04-15 | 之江实验室 | Time-wavelength interweaving photon neural network convolution acceleration chip |
| CN115169542A (en) * | 2022-09-02 | 2022-10-11 | 之江实验室 | Two-dimensional photon convolution acceleration system and device for convolution neural network |
| DE102021121918A1 (en) * | 2021-08-24 | 2023-03-02 | Westfälische Wilhelms-Universität Münster, Körperschaft des öffentlichen Rechts | Optoelectronic computing unit and matrix processor |
-
2023
- 2023-06-14 CN CN202310703095.2A patent/CN116484931B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2320613A1 (en) * | 1999-10-06 | 2001-04-06 | Alan F. Graves | Switch for optical signals |
| CN110009102A (en) * | 2019-04-12 | 2019-07-12 | 南京吉相传感成像技术研究院有限公司 | A kind of accelerated method of the depth residual error network based on photoelectricity computing array |
| CN110516802A (en) * | 2019-07-03 | 2019-11-29 | 上海交通大学 | Photon convolutional neural networks framework based on optical delay line caching |
| CA3094544A1 (en) * | 2019-09-26 | 2021-03-26 | Xanadu Quantum Technologies Inc. | Management of power consumption in optical circuits for quantum computing |
| CN112232503A (en) * | 2020-06-09 | 2021-01-15 | 联合微电子中心有限责任公司 | Computing device, computing method, and computing system |
| WO2022016894A1 (en) * | 2020-07-20 | 2022-01-27 | 联合微电子中心有限责任公司 | Photonic neural network |
| CN112232504A (en) * | 2020-09-11 | 2021-01-15 | 联合微电子中心有限责任公司 | Photon neural network |
| CN113705774A (en) * | 2021-07-29 | 2021-11-26 | 浪潮电子信息产业股份有限公司 | Optical circuit construction method, optical circuit, optical signal processing method and device |
| DE102021121918A1 (en) * | 2021-08-24 | 2023-03-02 | Westfälische Wilhelms-Universität Münster, Körperschaft des öffentlichen Rechts | Optoelectronic computing unit and matrix processor |
| CN114358271A (en) * | 2022-03-18 | 2022-04-15 | 之江实验室 | Time-wavelength interweaving photon neural network convolution acceleration chip |
| CN115169542A (en) * | 2022-09-02 | 2022-10-11 | 之江实验室 | Two-dimensional photon convolution acceleration system and device for convolution neural network |
Non-Patent Citations (1)
| Title |
|---|
| 用于光学向量矩阵乘法器的光源阵列系统;卢洋洋;周平;朱巍巍;张磊;杨林;;红外与激光工程(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116484931A (en) | 2023-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022016894A1 (en) | Photonic neural network | |
| CN103678258B (en) | Method for improving data resolution ratio of silica-based optical matrix processor | |
| CN112232504A (en) | Photon neural network | |
| CN111860822A (en) | All-optical nonlinear activation function implementation method and device of optical neural network | |
| CN111882052A (en) | Photon convolution neural network system | |
| CN114819132B (en) | Photon two-dimensional convolution acceleration method and system based on time-wavelength interleaving | |
| CN115169542B (en) | Two-dimensional photon convolution acceleration system and device for convolution neural network | |
| CN115167610B (en) | Two-dimensional photon convolution acceleration chip and application system and method | |
| CN115222035B (en) | Photon neural network convolution acceleration chip | |
| CN114815959A (en) | Photon tensor calculation acceleration method and device based on wavelength division multiplexing | |
| CN117077750B (en) | Parallel photon convolution operation chip and system | |
| CN116432726B (en) | Photoelectric hybrid deep neural network operation device and operation method | |
| CN117097431B (en) | Time delay control equipment, optical time division multiplexing method, demultiplexing method and system | |
| CN114565091A (en) | Optical neural network device, chip and optical implementation method for neural network calculation | |
| CN116484931B (en) | Photon matrix multiplication operation device and operation method for neural network | |
| US20240086698A1 (en) | Two-dimensional photonic neural network convolutional acceleration chip based on series connection structure | |
| CN113315582A (en) | Microwave photon double-optical-frequency comb OFDM (orthogonal frequency division multiplexing) transmitting device and transmitting method | |
| CN115130666B (en) | Two-dimensional photon convolution acceleration method and system | |
| Mazur et al. | Optical frequency comb modulator for multichannel systems | |
| CN115001909B (en) | MIMO wireless communication rapid channel estimation device and method based on optical matrix calculation | |
| CN113630207A (en) | Photon neural network chip | |
| CN107346991B (en) | A kind of multichannel mpsk signal regeneration method based on Phase sensitive amplification | |
| CN116107037B (en) | Optical computing network structure of optical device on chip of dense waveguide array | |
| Mourgias-Alexandris et al. | Neuromorphic computing through photonic integrated circuits | |
| CN115001908B (en) | Wireless communication rapid channel estimation device and method based on optical matrix calculation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |