CN113657580A - Photon convolution neural network accelerator based on micro-ring resonator and nonvolatile phase change material - Google Patents
Photon convolution neural network accelerator based on micro-ring resonator and nonvolatile phase change material Download PDFInfo
- Publication number
- CN113657580A CN113657580A CN202110943559.8A CN202110943559A CN113657580A CN 113657580 A CN113657580 A CN 113657580A CN 202110943559 A CN202110943559 A CN 202110943559A CN 113657580 A CN113657580 A CN 113657580A
- Authority
- CN
- China
- Prior art keywords
- module
- convolution
- optical
- ring resonator
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 239000012782 phase change material Substances 0.000 title claims abstract description 27
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 20
- 230000003287 optical effect Effects 0.000 claims abstract description 74
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000004364 calculation method Methods 0.000 claims abstract description 19
- 239000011159 matrix material Substances 0.000 claims abstract description 15
- 238000006243 chemical reaction Methods 0.000 claims abstract description 13
- 230000005540 biological transmission Effects 0.000 claims abstract description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 15
- 230000004913 activation Effects 0.000 claims description 3
- 230000010363 phase shift Effects 0.000 claims description 3
- 238000013500 data storage Methods 0.000 abstract description 2
- 230000007547 defect Effects 0.000 abstract description 2
- 238000002425 crystallisation Methods 0.000 description 6
- 230000008025 crystallization Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000000034 method Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000002834 transmittance Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 238000005280 amorphization Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000001816 cooling Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 230000001808 coupling effect Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000031700 light absorption Effects 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 230000009022 nonlinear effect Effects 0.000 description 1
- 230000005693 optoelectronics Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
- G06N3/0675—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means using electro-optical, acousto-optical or opto-electronic means
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Optical Modulation, Optical Deflection, Nonlinear Optics, Optical Demodulation, Optical Logic Elements (AREA)
Abstract
The invention discloses a photon convolution neural network accelerator based on a micro-ring resonator and a nonvolatile phase-change material, which comprises a plurality of tile structures, wherein the tile structures are communicated through a router; the tile structure comprises a nonlinear module, an input/output module and an optical matrix vector multiplication module, wherein the nonlinear module is used for performing nonlinear operation, the input/output module is used for performing data transmission with the router, and the optical matrix vector multiplication module comprises a pulse processing module, a modulation module, a convolution module and a photoelectric conversion module and is used for completing convolution calculation. The invention introduces the light pulse to carry out multiply-add calculation, thereby realizing the multiplication of the calculation rate; GST is introduced to realize integrated processing of data storage and operation, so that on one hand, power loss is reduced, and on the other hand, the throughput of calculation is improved; and finally, an electric subtracter is introduced to obtain a negative weight value, so that the defect that the optical pulse cannot realize subtraction is overcome.
Description
Technical Field
The invention belongs to the field of photon signal processing, and particularly relates to a photonic convolution neural network accelerator architecture technology based on a micro-ring resonator and a nonvolatile phase change material.
Background
In convolutional neural networks, the convolution operation typically takes over 80% of the computation and processing time. In addition, with the introduction of the artificial intelligence era, the data volume shows an exponential growth trend. In order to meet the demand of the artificial intelligence era on the calculation of a super-large data set, the customized hardware for accelerating Matrix-Vector Multiplication (MVM) can be promoted to become a hot spot of the current research. Field-Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), and image Processing Units (cpus) have been developed to accelerate computations, but these electrical Processing based accelerators are limited in energy and rate by joule heating and electromagnetic crosstalk. Meanwhile, the interaction of data inside the electric accelerators requires charging and discharging of chip-level metal interconnection, thereby bringing about great power consumption cost.
Due to the continuous maturity of silicon photonics, photon convolution neural network accelerators have developed to some extent. The photon convolution neural network accelerator has the advantages that: (1) with extremely high modulation rates. The photon convolution neural network accelerator expands the calculation dimensionality from an electric domain to an optical domain, the currently known optical modulation rate can reach 10-40 GHz, the modulation rate of the optical overall architecture is improved by 1-2 orders of magnitude compared with that of a pure electrical calculation architecture, and the modulation rate is only influenced by the bandwidths of a photoelectric detector and an on-chip optical modulator; (2) performing large-scale parallel convolution operation by combining Wavelength Division Multiplexing (WDM) with multiple channels (optical splitters); (3) Multiply-add-Accumulate (MAC) operations can be implemented in photonic convolutional neural networks with very low energy. These unique advantages have led to further advances in power consumption and speed of photon convolutional neural network accelerators and have been extensively studied in recent years.
Although the existing photon convolution neural network is greatly improved in the aspects of speed and the like compared with an electric accelerator, the modulation of input and weight still needs to be realized by an external power supply. Therefore, the invention provides a photon convolution neural network accelerator structure based on phase change materials GST and ring resonators, wherein the GST is embedded into the top of a micro-ring, and different weight values are stored by utilizing the characteristic of non-volatility of the GST, so that the memory calculation of convolution is realized.
Disclosure of Invention
The invention aims to solve the problems of high power consumption, limited calculation rate and the like of the traditional electric accelerator. A photonic convolutional neural network accelerator architecture based on a micro-ring resonator and a nonvolatile phase change material Ge2Sb2Te5(GST) is provided. The dimensionality of convolution calculation is converted from an electric domain to an optical domain, a wavelength division multiplexing technology is introduced, parallel convolution processing is achieved, two ports of an upper path type ring resonator and a lower path type ring resonator are connected through a balanced photodiode, and positive and negative weight values are obtained. An important research is conducted on the communication between the optical Matrix-Vector Multiplication (MVM) architecture based on the ring resonator and the non-volatile phase change material GST and the devices in the chip. The optical parallel convolution processing on the chip is realized through the pulse processing module, the modulation module, the convolution module and the photoelectric conversion module, the speed of the chip is expected to be improved by 1-2 orders of magnitude compared with that of an electric accelerator, the power consumption is reduced, and the processing pressure caused by the increase of data explosion is effectively relieved.
In order to achieve the purpose, the technical scheme adopted by the invention is a photon convolution neural network accelerator based on a micro-ring resonator and a nonvolatile phase change material, and the photon convolution neural network accelerator comprises a plurality of tile structures, and all the tile structures are communicated through a router.
The tile structure comprises a nonlinear module, an input/output module and an optical matrix vector multiplication module, wherein the nonlinear module is used for row nonlinear operation, the input/output module is used for carrying out data transmission with a router, and the optical matrix vector multiplication module comprises a pulse processing module, a modulation module, a convolution module and a photoelectric conversion module; the pulse processing module comprises a multiplexer-based multiplexing module and an optical splitter-based optical splitting module, wherein the multiplexer-based multiplexing module is used for realizing multiplexing of different resonant wavelengths, and the optical splitter-based optical splitting module is used for splitting a single pulse into a plurality of optical pulses; the modulation module comprises a plurality of all-pass ring resonators and is used for modulating input optical pulses to change the pulse amplitude of the optical pulses; the convolution module comprises a plurality of upper and lower voice circuit type ring resonators embedded with nonvolatile phase change materials (GST) and used for storing weight values and realizing convolution calculation; the photoelectric conversion module is composed of a plurality of balanced photodiodes and a current adder, the balanced photodiodes are used for converting light pulses into currents and realizing current subtraction, the current adder is used for adding all current values to obtain a convolution characteristic, and the convolution characteristic is acted on the nonlinear module.
Further, the optical matrix vector multiplication module divides the N input optical pulses into M optical pulses after passing through the multiplexer and the optical splitter, and inputs the M optical pulses into M waveguides respectively; each modulation module consists of N multiplied by M all-pass ring resonators; each convolution module consists of N multiplied by M upper and lower voice path type ring resonators; each photoelectric conversion module comprises M balanced photodiodes, performs photoelectric conversion on the outputs of the M waveguides, and realizes subtraction operation.
Further, the all-pass ring resonator is composed of a straight waveguide and a first ring waveguide; used for modulating the input pulse in an electrical modulation mode. The upper and lower speech path type ring resonator consists of two straight waveguides and a second ring waveguide, wherein the second ring waveguide comprises a section of nonvolatile phase change material (GST), and each GST divides the second ring waveguide into 16 grades for storing different weight values; when the round-trip phase shift of the light wave in the second annular waveguide is equal to the integral multiple of 2 pi, the resonant cavity is in a resonance state, the input optical signal is completely output from the drop port, when the resonant cavity is detuned, the output power value of the drop port is reduced, and the output power passing through the port is increased.
Further, a power supply is applied to the all-pass ring resonator, and an input pulse is modulated in an electrical modulation mode.
Further, the balanced photodiode receives the light pulse from the falling port and the light pulse from the falling port, respectively, and subtracts the currents, and then flows into a current adder.
Further, in the nonlinear module, the shift addition operation is adopted to store the intermediate result in the output buffer, the result is sent to the activation unit, then the nonlinear function is applied to carry out nonlinear operation on the intermediate result, and the operation result is stored in the random access memory for the next layer of processing.
Further, the nonlinear function includes a sigmoid function and a maxpool function.
Further, the present invention includes a digital-to-analog converter and an analog-to-digital converter, wherein the digital-to-analog converter is configured to convert the digital signal to an analog signal to be used to modulate the input array; the analog-to-digital converter is used for converting the analog signal after convolution operation into a digital signal.
Further, the present invention also includes an optical pulse module for applying optical pulses to the convolution module to modify or erase internally stored weight values.
The invention has the following advantages and beneficial effects:
the invention provides a photonic neural network accelerator architecture based on a micro-ring resonator and a nonvolatile phase-change material. Firstly, modulating input and weight by using optical pulses, expanding the calculated dimensionality from an electrical domain to an optical domain, and simultaneously, effectively increasing the number of channels of an optical link by combining with a wavelength division multiplexing technology so as to realize the parallel processing of optical signals; secondly, a nonvolatile phase change material GST is introduced and embedded into the top of the micro-ring resonator, and a weight value is mapped into each GST by utilizing the characteristics of non-volatility and high contrast between an amorphous phase and a crystalline phase, so that the memory calculation of input and weight is realized. Due to the non-volatile characteristic of GST, the weight value is not easily influenced by the outside after being written into GST, so that power loss caused by continuous external power supply is avoided; finally, in view of the difficulty in achieving subtraction and storage of optical pulses, in the present architecture, balanced photodiodes are used for opto-electronic switching. The convolution kernel values output from the 'Through' and 'Drop' ports of the ring resonator are respectively input into a Photodiode (PD), and the current subtraction, namely 'T', is realized because the polarities of the two photodiodes are opposited-Tp”。These output values are added by an adder to produce a convolution signature value that is used in the nonlinear portion of the convolutional neural network. The architecture provided by the invention aims to realize the optical parallel memory computing of data so as to solve the challenge of the artificial intelligence era on the processing of a super-large data set.
Drawings
FIG. 1 is a photonic neural network accelerator chip structure of the present invention;
FIG. 2 is a schematic structural diagram of a microring resonator and a microring resonator with GST added on top;
FIG. 3 is a schematic diagram of an optical matrix vector multiplication module for generating a single convolution signature.
Detailed Description
The technical solutions in the embodiments of the present invention will be described in detail and clearly with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of a photonic neural network accelerator chip structure. A chip structure contains a plurality of tiles, each of which communicates with each other via a router. Each tile contains an input-output module, a non-linear module, and an optical matrix vector multiplication module for convolution operations. Wherein, the input value is taken out from the random access memory of the input-output module to form an input matrix which is transmitted to the convolution part of the optical matrix vector multiplication module; the optical pulse modulates the GST to generate different weight values, which is done off-chip. These weight values constitute a weight array that is also passed to the convolution section at the optical matrix vector multiplication module. The input array and the weight array will be convolved "in-memory calculations" within the optical matrix vector multiplication module. The resulting convolution value is sampled and held by a sample-and-hold unit and converted to a digital value by an analog-to-digital converter. The digital result is operated on by a sample and add unit and will be stored in an output buffer. The convolution value stored in the output buffer is finally transmitted to the nonlinear module, and an output result is generated through the nonlinear action of the activation, the pooling and the full connection layer.
Fig. 2 is a schematic structural diagram of a micro-ring resonator for building a photonic neural network. Fig. 2(a) shows an all-pass micro-ring resonator used for constructing a modulation array module. The ring resonator is composed of a straight waveguide and a ring waveguide. The ring resonator is used for modulating input optical pulses to enable the amplitude of the input optical pulses to change, and the modulation mode adopts electrical modulation. Fig. 2(b) shows an up-and-down channel type micro-ring resonator used for constructing the weight array. When the round-trip phase shift of the light wave in the annular waveguide is equal to integral multiple of 2 pi, the annular waveguide resonant cavity is in a resonance state, the input optical signal is completely output from a 'down' port, namely the transmissivity of the 'down' port is 1, and the transmissivity of the 'through' port is 0. When the ring waveguide resonator is detuned, the output power at the "down" port decreases and the output power through the "through" port increases. The invention embeds GST in the top of the micro-ring resonator, and controls the transmission of two ports by changing the crystallization of GST.
The GST crystallization degree influences the specific process of the output port:
GST is a new type of phase change material that has a high contrast between crystalline and amorphous states. When the GST is in a crystalline state, the GST has high absorptivity, and the light transmitted in the waveguide is completely absorbed by the GST, so that the projection rate of the waveguide is 0; when the GST is in an amorphous state, its absorption of light is almost 0, and light in the waveguide is transmitted through the GST, where the transmittance of the waveguide is 1. Between 0 and 1, a plurality of crystallization degree levels may be set according to a crystallization degree formula, which is as follows:
wherein p is the degree of crystallization, εaAnd εcPermittivity in the amorphous and crystalline states, epsiloneffFor GST crystallinity, differences in permittivity constants at different degrees of crystallinity will result in different degrees of crystallinity, and differences in degrees of crystallinity will result in differences in real and imaginary components of the GST effective refractive indexWill result in a phase different from the attenuation coefficient, the formula for the phase is as follows:
where theta, alpha represent attenuation factor and phase change factor, respectively, and neff,wgFor the refractive index of the waveguide, R represents the radius of the annular waveguide, LGSTIs the length of GST, neff,GSTDenotes the effective refractive index, k, of GSTeff,wgIs the imaginary part, k, of the effective refractive index of the waveguideeff,GSTRepresenting the imaginary part of the effective refractive index of GST, and λ is the input wavelength.
The output formulas of the two ports of the upper and lower path type ring resonator are as follows:
Tt、Tdrepresenting the transmissivity of the ring resonator "through" and "down" ports, t, respectively1、t2Representing the coupling coefficient.
From the above formula, it can be seen that the two-port transmission value and phase are related to the attenuation factor, so that changing the crystallization degree of GST will ultimately affect the two-port output.
In the invention, the convolution memory calculation is realized by embedding GST, and the power loss caused by continuous external power supply is avoided. GST, as a nonvolatile phase change material, has the characteristics of easy reading, writing, erasing, non-volatility, high contrast between crystalline phase and amorphous phase, and the like, can realize the memory calculation of input and weight, can modify or erase the weight value stored inside only by applying external light pulse, and can store the value stored inside for years or decades once the form of GST is fixed.
The specific implementation processes of GST read, write, and erase are as follows:
and (3) reading: a high-power input signal is injected into the input end, and the input signal can be absorbed into the GST by utilizing the evanescent wave coupling effect between the light in the waveguide and the GST, so that the temperature of the GST surface is increased. When the temperature is higher than the crystalline state threshold T1, the state of GST begins to change (i.e., amorphization process), affecting the transmittance of GST, thereby achieving writing of transmittance (weight) value.
And (3) storage operation: after the value of GST transmittance is written, the state structure of atoms in GST can be fixed by rapidly cooling GST to room temperature, and the weight can be stored. This state can be maintained at normal temperature for several decades and is therefore nonvolatile.
And (3) writing: after the weight value (assumed to be b) is written and stored, a small power signal of a is input at the input end, the energy of the signal is not enough to reach the crystalline threshold of GST, so that the state of GST is not changed, and the signal is transmitted to the output end through GST. The power c of the signal received by the output end is the product of the power a of the input signal and the weight b of GST, and the weight reading is realized.
And (3) erasing operation: the GST realizes the amorphous state to crystalline state through a high-power input pulse signal, thereby realizing the erasure of the weight data.
FIG. 3 is a schematic diagram of an optical matrix vector multiplication architecture for generating a single convolution signature. The architecture diagram comprises a pulse processing module, a modulation module, a convolution module and a photoelectric conversion module. The pulse processing module contains the multiplexing module that uses wavelength division multiplexer as the owner and uses the optical splitter as the optical splitting module of giving first place to, the modulation module contains a plurality of full logical type micro-ring syntonizers, the convolution module contains the last lower speech path type micro-ring syntonizers that GST has been inlayed at a plurality of tops, photoelectric conversion module contains a plurality of balanced photodiode and a current adder.
The pulse processing module consists of a wavelength division multiplexer and an optical splitter. Input optical pulses at N different resonant wavelengths are received at the input of fig. 3 and multiplexed by a wavelength division multiplexer into a single optical pulse that is transmitted along a waveguide to an optical splitter. The optical splitter splits the single pulse into M optical pulses, each of which contains the original N input pulses, except that the power is changed to 1/M of the original. The M split optical pulses are transmitted to the modulation module.
The modulation module consists of M multiplied by N all-pass micro-ring resonators. In FIG. 3, the modulation array (also called input array), a, is denoted by AijAn input value representing the ith row and jth column of the modulation array, the modulation of which requires a continuous applied power supply. The transmissivity of the port can be indirectly affected by adding a power supply to the all-pass ring resonator, thereby modulating the input value. The decomposed M optical pulses are transmitted to M waveguides and coupled with corresponding ring resonators according to different resonant frequencies.
The convolution module is composed of MxN uplink and downlink channel type micro-ring resonators with GST embedded at the tops, a weight array is represented by F, and FijRepresenting the weight value in row i and column j. In fig. 3, the optical pulses modulated by the modulation module reach the convolution module and are selectively coupled to the corresponding micro-rings according to wavelength. GST is embedded at the top of each micro-ring, and a weight value is stored in each GST. Once the weight value is written into GST, the value will be kept unchanged in the convolution process, and power loss caused by an external power supply is avoided. The inputs and weights are multiplied within the GST and the result of the operation will be output through the "down" and "through" ports.
The photoelectric conversion module mainly comprises a plurality of balanced photodiodes and a current adder. The light pulses from the 'down' and 'through' ports are input into two photodiodes, respectively, and the two photodiodes are in opposite directions, so that the current subtraction is realized, namely the 'T' is realizedd-Tp", positive and negative weight values are obtained. The subtracted current values are fed into a current adder where all current values are added to produceAnd generating convolution characteristics. This eigenvalue will be passed to the non-linear part of the neural network.
The above embodiments illustrate the operation of the photonic convolutional neural network based on micro-ring resonators and non-volatile phase change materials. In a convolutional neural network, 80% of the total processing time is spent on convolution operations, so it is important to improve the convolution operations. In the invention, the optical pulse is introduced to carry out multiply-add calculation, the original electric calculation is replaced, the operation range is expanded from an electric domain to an optical domain, and the multiplication rate is doubled; in addition, the GST is introduced as a nonvolatile phase change material, so that the integrated processing of data storage and operation is realized, on one hand, the power loss of changing and maintaining a weight value by an external power supply is reduced, on the other hand, the delay caused by frequent data switching is also reduced, and the throughput of calculation is improved; and finally, an electric subtracter is introduced to obtain a negative weight value, so that the defect that the optical pulse cannot realize subtraction is overcome, and the application range of the photon convolution neural network accelerator is further expanded. Meanwhile, the photon convolution neural network accelerator can be completely integrated into a chip, only external input optical pulses are needed, and the photon convolution neural network accelerator has high expandability. The invention is expected to be used in the fields of unmanned driving, aerospace, multi-position image processing, biomedicine and the like.
The above examples are to be construed as merely illustrative and not limitative of the remainder of the disclosure. After reading the description of the invention, the skilled person can make various changes or modifications to the invention, and these equivalent changes and modifications also fall into the scope of the invention defined by the claims.
Claims (9)
1. Photon convolution neural network accelerator based on micro-ring resonator and nonvolatile phase change material, its characterized in that: the system comprises a plurality of tile structures, wherein the tile structures are communicated with one another through a router;
the tile structure comprises a nonlinear module, an input/output module and an optical matrix vector multiplication module, wherein the nonlinear module is used for row nonlinear operation, the input/output module is used for carrying out data transmission with a router, and the optical matrix vector multiplication module comprises a pulse processing module, a modulation module, a convolution module and a photoelectric conversion module; the pulse processing module comprises a multiplexer-based multiplexing module and an optical splitter-based optical splitting module, wherein the multiplexer-based multiplexing module is used for realizing multiplexing of different resonant wavelengths, and the optical splitter-based optical splitting module is used for splitting a single pulse into a plurality of optical pulses; the modulation module comprises a plurality of all-pass ring resonators and is used for modulating input optical pulses; the convolution module comprises a plurality of upper and lower voice path type ring resonators embedded with nonvolatile phase change materials and used for storing weight values and realizing convolution calculation; the photoelectric conversion module is composed of a plurality of balanced photodiodes and a current adder, the balanced photodiodes are used for converting light pulses into currents and realizing current subtraction, the current adder is used for adding all current values to obtain a convolution characteristic, and the convolution characteristic is acted on the nonlinear module.
2. The photonic convolutional neural network accelerator based on a micro-ring resonator and a non-volatile phase change material of claim 1, wherein: the optical matrix vector multiplication module divides N input optical pulses into M optical pulses after passing through a multiplexer and a light splitter, and the M optical pulses are respectively input into M waveguides; each modulation module consists of N multiplied by M all-pass ring resonators; each convolution module consists of N multiplied by M upper and lower voice path type ring resonators; each photoelectric conversion module comprises M balanced photodiodes, performs photoelectric conversion on the outputs of the M waveguides, and realizes subtraction operation.
3. The photonic convolutional neural network accelerator based on the micro-ring resonator and the nonvolatile phase change material as claimed in claim 1 or 2, wherein: the all-pass ring resonator consists of a straight waveguide and a first ring waveguide; the up-down channel type ring resonator is composed of two straight waveguides and a second ring waveguide, wherein the second ring waveguide comprises a section of nonvolatile phase change material, when the round-trip phase shift of light waves in the second ring waveguide is equal to integral multiple of 2 pi, the resonant cavity is in a resonance state, input optical signals are completely output from the drop port, when the resonant cavity is detuned, the output power value of the drop port is reduced, and the output power passing through the port is increased.
4. The photonic convolutional neural network accelerator based on a micro-ring resonator and a non-volatile phase change material of claim 3, wherein: and applying a power supply to the all-pass ring resonator, and modulating input pulses in an electrical modulation mode.
5. The photonic convolutional neural network accelerator based on a micro-ring resonator and a non-volatile phase change material of claim 3, wherein: the balanced photodiode receives the light pulses from the down port and the through port respectively, performs current subtraction, and then flows into a current adder.
6. The photonic convolutional neural network accelerator based on the micro-ring resonator and the nonvolatile phase change material as claimed in claim 1, 2, 4 or 5, wherein: in the nonlinear module, the shift addition operation is adopted to store the intermediate result in an output buffer, the result is sent to an activation unit, then the nonlinear function is used to carry out nonlinear operation on the intermediate result, and the operation result is stored in a random access memory for the next layer of processing.
7. The photonic convolutional neural network accelerator based on a micro-ring resonator and a non-volatile phase change material of claim 6, wherein: the nonlinear function includes a sigmoid function and a maxpool function.
8. The photonic convolutional neural network accelerator based on a micro-ring resonator and a non-volatile phase change material of claim 6, wherein: the digital-to-analog converter is used for converting a digital signal into an analog signal, and the analog signal is used for modulating the input array; the analog-to-digital converter is used for converting the analog signal after convolution operation into a digital signal.
9. The photonic convolutional neural network accelerator based on a micro-ring resonator and a non-volatile phase change material of claim 6, wherein: an optical pulse module is also included for applying optical pulses to the convolution module to modify or erase internally stored weight values.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110943559.8A CN113657580B (en) | 2021-08-17 | 2021-08-17 | Photon convolution neural network accelerator based on micro-ring resonator and nonvolatile phase change material |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110943559.8A CN113657580B (en) | 2021-08-17 | 2021-08-17 | Photon convolution neural network accelerator based on micro-ring resonator and nonvolatile phase change material |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113657580A true CN113657580A (en) | 2021-11-16 |
| CN113657580B CN113657580B (en) | 2023-11-21 |
Family
ID=78480054
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110943559.8A Active CN113657580B (en) | 2021-08-17 | 2021-08-17 | Photon convolution neural network accelerator based on micro-ring resonator and nonvolatile phase change material |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113657580B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113933931A (en) * | 2021-08-27 | 2022-01-14 | 北京工业大学 | Annular cavity optical modulator based on vanadium dioxide nanowire |
| CN114488651A (en) * | 2022-01-04 | 2022-05-13 | 重庆邮电大学 | Photon digital-to-analog converter based on micro-ring array and GST (GST) and regulation and control method |
| CN114815074A (en) * | 2022-06-30 | 2022-07-29 | 中山大学 | Optical modulation micro-ring resonant cavity and preparation method thereof |
| CN114819132A (en) * | 2022-06-27 | 2022-07-29 | 之江实验室 | Photon two-dimensional convolution acceleration method and system based on time-wavelength interleaving |
| CN115085854A (en) * | 2022-06-13 | 2022-09-20 | 重庆邮电大学 | Optical matrix multiplier supporting storage and calculation integration and wavelength-mode hybrid multiplexing |
| CN115639698A (en) * | 2022-12-07 | 2023-01-24 | 之江实验室 | C-waveband silicon waveguide micro-ring modulator based on phase-change material antimony sulfide and modulation method |
| CN116009812A (en) * | 2023-02-02 | 2023-04-25 | 重庆邮电大学 | Photon matrix multiplier based on Mach-Zehnder interferometer and nonvolatile phase change material |
| CN116957031A (en) * | 2023-07-24 | 2023-10-27 | 浙江大学 | Photoelectric computer based on optical multi-neuron activation function module |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180374245A1 (en) * | 2017-06-26 | 2018-12-27 | Elekta, Inc. | Image quality in cone beam computed tomography images using deep convolutional neural networks |
| CN109460817A (en) * | 2018-09-11 | 2019-03-12 | 华中科技大学 | A kind of convolutional neural networks on piece learning system based on nonvolatile storage |
| CN110245324A (en) * | 2019-05-19 | 2019-09-17 | 南京惟心光电系统有限公司 | A kind of de-convolution operation accelerator and its method based on photoelectricity computing array |
| CN110276440A (en) * | 2019-05-19 | 2019-09-24 | 南京惟心光电系统有限公司 | A kind of convolution algorithm accelerator and its method based on photoelectricity computing array |
| CN110573066A (en) * | 2017-03-02 | 2019-12-13 | 光谱Md公司 | Machine learning systems and techniques for multi-spectral amputation site analysis |
| CN112232504A (en) * | 2020-09-11 | 2021-01-15 | 联合微电子中心有限责任公司 | Photon neural network |
| WO2021115262A1 (en) * | 2019-12-09 | 2021-06-17 | 南京惟心光电系统有限公司 | Pulse convolutional neural network algorithm, integrated circuit, computing apparatus, and storage medium |
-
2021
- 2021-08-17 CN CN202110943559.8A patent/CN113657580B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110573066A (en) * | 2017-03-02 | 2019-12-13 | 光谱Md公司 | Machine learning systems and techniques for multi-spectral amputation site analysis |
| US20180374245A1 (en) * | 2017-06-26 | 2018-12-27 | Elekta, Inc. | Image quality in cone beam computed tomography images using deep convolutional neural networks |
| CN109460817A (en) * | 2018-09-11 | 2019-03-12 | 华中科技大学 | A kind of convolutional neural networks on piece learning system based on nonvolatile storage |
| CN110245324A (en) * | 2019-05-19 | 2019-09-17 | 南京惟心光电系统有限公司 | A kind of de-convolution operation accelerator and its method based on photoelectricity computing array |
| CN110276440A (en) * | 2019-05-19 | 2019-09-24 | 南京惟心光电系统有限公司 | A kind of convolution algorithm accelerator and its method based on photoelectricity computing array |
| WO2021115262A1 (en) * | 2019-12-09 | 2021-06-17 | 南京惟心光电系统有限公司 | Pulse convolutional neural network algorithm, integrated circuit, computing apparatus, and storage medium |
| CN113033759A (en) * | 2019-12-09 | 2021-06-25 | 南京惟心光电系统有限公司 | Pulse convolution neural network algorithm, integrated circuit, arithmetic device, and storage medium |
| CN112232504A (en) * | 2020-09-11 | 2021-01-15 | 联合微电子中心有限责任公司 | Photon neural network |
Non-Patent Citations (2)
| Title |
|---|
| CLEMENT FARABET等: "hardware accelerated convolutional neural networks for synthetic vision systems", PROCEEDINGS OF 2010 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS, pages 257 - 260 * |
| 郭鹏星等: "相变材料辅助的光子卷积神经网络加速器", 光学学报, vol. 43, no. 4, pages 136 - 145 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113933931A (en) * | 2021-08-27 | 2022-01-14 | 北京工业大学 | Annular cavity optical modulator based on vanadium dioxide nanowire |
| CN114488651B (en) * | 2022-01-04 | 2022-12-27 | 重庆邮电大学 | Photon digital-to-analog converter based on micro-ring array and GST (GST) and regulation and control method |
| CN114488651A (en) * | 2022-01-04 | 2022-05-13 | 重庆邮电大学 | Photon digital-to-analog converter based on micro-ring array and GST (GST) and regulation and control method |
| CN115085854B (en) * | 2022-06-13 | 2023-08-29 | 重庆邮电大学 | Optical matrix multiplier supporting integrated memory and wavelength-mode mixed multiplexing |
| CN115085854A (en) * | 2022-06-13 | 2022-09-20 | 重庆邮电大学 | Optical matrix multiplier supporting storage and calculation integration and wavelength-mode hybrid multiplexing |
| CN114819132A (en) * | 2022-06-27 | 2022-07-29 | 之江实验室 | Photon two-dimensional convolution acceleration method and system based on time-wavelength interleaving |
| CN114815074B (en) * | 2022-06-30 | 2022-09-27 | 中山大学 | Optical modulation micro-ring resonant cavity and preparation method thereof |
| CN114815074A (en) * | 2022-06-30 | 2022-07-29 | 中山大学 | Optical modulation micro-ring resonant cavity and preparation method thereof |
| CN115639698A (en) * | 2022-12-07 | 2023-01-24 | 之江实验室 | C-waveband silicon waveguide micro-ring modulator based on phase-change material antimony sulfide and modulation method |
| CN116009812A (en) * | 2023-02-02 | 2023-04-25 | 重庆邮电大学 | Photon matrix multiplier based on Mach-Zehnder interferometer and nonvolatile phase change material |
| CN116009812B (en) * | 2023-02-02 | 2023-08-25 | 重庆邮电大学 | Photon matrix multiplier based on Mach-Zehnder interferometer and nonvolatile phase change material |
| CN116957031A (en) * | 2023-07-24 | 2023-10-27 | 浙江大学 | Photoelectric computer based on optical multi-neuron activation function module |
| CN116957031B (en) * | 2023-07-24 | 2024-05-24 | 浙江大学 | Photoelectric computer based on optical multi-neuron activation function module |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113657580B (en) | 2023-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113657580B (en) | Photon convolution neural network accelerator based on micro-ring resonator and nonvolatile phase change material | |
| Liu et al. | Holylight: A nanophotonic accelerator for deep learning in data centers | |
| CN106324865B (en) | One kind being based on the three-dimensionally integrated photoswitch of phase-change material | |
| CN114358271B (en) | Time-wavelength interweaving photon neural network convolution acceleration chip | |
| WO2023201970A1 (en) | Computing chip, system, and data processing method | |
| JP7115691B2 (en) | Training for Photonic Reservoir Computing Systems | |
| CN108665924B (en) | Array silicon-based programmable optical memory chip | |
| Peserico et al. | Integrated photonic tensor processing unit for a matrix multiply: a review | |
| Dabos et al. | Neuromorphic photonic technologies and architectures: scaling opportunities and performance frontiers | |
| Alkabani et al. | OE-CAM: a hybrid opto-electronic content addressable memory | |
| Kari et al. | Optical and electrical memories for analog optical computing | |
| CN114488651B (en) | Photon digital-to-analog converter based on micro-ring array and GST (GST) and regulation and control method | |
| Shafiee et al. | A survey on optical phase-change memory: The promise and challenges | |
| Zhu et al. | Dota: A dynamically-operated photonic tensor core for energy-efficient transformer accelerator | |
| CN115085854B (en) | Optical matrix multiplier supporting integrated memory and wavelength-mode mixed multiplexing | |
| CN116027481A (en) | Ultra-low power consumption photon chip and neural network architecture | |
| Yu et al. | All-chalcogenide programmable all-optical deep neural networks | |
| CN113839715B (en) | Optical neural network and chip | |
| CN116088245A (en) | Based on phase change material Sb 2 Se 3 Is a non-volatile reconfigurable compact all-optical logic gate | |
| CN113628653B (en) | All-optical Boolean logic device based on phase change material and binary logic implementation method thereof | |
| Popovskiy et al. | Features of the construction photonic tensor cores for neural networks | |
| Goswami | Optical computing: 2. Research trends | |
| Huang et al. | Neuromorphic photonics for optical communication systems | |
| CN113630207A (en) | Photon neural network chip | |
| Miyatake et al. | Photonic tensor core with low-insertion-loss, non-volatile Ge 2 Sb 2 Te 3 S 2 intensity modulators |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |