CN114358271A - Time-wavelength interweaving photon neural network convolution acceleration chip - Google Patents
Time-wavelength interweaving photon neural network convolution acceleration chip Download PDFInfo
- Publication number
- CN114358271A CN114358271A CN202210267027.1A CN202210267027A CN114358271A CN 114358271 A CN114358271 A CN 114358271A CN 202210267027 A CN202210267027 A CN 202210267027A CN 114358271 A CN114358271 A CN 114358271A
- Authority
- CN
- China
- Prior art keywords
- wavelength
- micro
- signal
- optical
- ring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Optical Modulation, Optical Deflection, Nonlinear Optics, Optical Demodulation, Optical Logic Elements (AREA)
- Optical Communication System (AREA)
- Optical Integrated Circuits (AREA)
Abstract
The invention discloses a time-wavelength interleaved photon neural network convolution acceleration chip which is suitable for all deep learning networks containing convolution operation. The modulator, the wavelength division delay weighting micro-ring array and the balance photoelectric detector which finish convolution acceleration operation are integrated through a photon integration technology. The method comprises the steps of loading signals to be processed on a plurality of optical carriers respectively based on a wavelength division multiplexing technology, realizing convolution kernel coefficient weighting and time interleaving of different carrier signals through a micro-ring and a delay line, and realizing summation operation after weighting through a balanced photoelectric detector. The invention can realize the construction of any convolution kernel matrix by utilizing the resonance characteristic of the integrated micro-circulator and can finish the convolution acceleration operation of any signal by combining the time delay. By using light as an information carrier, the speed and the energy efficiency ratio of convolution operation can be greatly improved.
Description
Technical Field
The invention relates to a deep learning-oriented photon neural network convolution acceleration chip, and belongs to the technical field of photon integration.
Background
Artificial intelligence is now widely used in the fields of machine vision, natural language processing, and automatic driving, and an artificial neural network, which is one of important models of artificial intelligence technology, is widely used due to its excellent generalization ability and stability. Artificial neural networks essentially create a similar pattern of interconnection of neural networks by mimicking the structure of the biological nervous system. Based on the mature development of electronic technology, nowadays, the mainstream neural network model training and testing mainly uses an electronic integrated chip as a carrier, for example, a CPU, a GPU, an FPGA, an application specific integrated circuit, and the like. Because the conventional computer structure which separates the program space from the data space is adopted by the electronic chip at present, the data load between the storage unit and the computing unit is unstable and the power consumption is higher, so that the efficiency of network model training is limited. Although the operational efficiency can be improved by improving the integration level of the electronic chip or by memory calculation, the technical directions are also faced with huge challenges due to the microscopic quantum characteristics and the macroscopic high frequency response characteristics of the electronic chip (see [1. chen-magnificent, zhang-ming, zhang-tian, etc.. photonic neural network development and challenges, china laser, 2020, 47(5): 0500004.]). The photon technology using photons as an information carrier has the characteristics of large bandwidth, low loss, parallelism and the like, and is widely applied to the fields of radar, communication, imaging and the like (see J. Capmann, D. Novak, "Microwave photonics combinations two works" Nature photonics, vol. 1, no. 6, pp. 319-330, 2007.]) The photon technology is combined with the traditional neural network, so that the advantages of the two technologies are expected to be fully exerted, and the technical development bottlenecks of high power consumption, long time delay and limited speed of the traditional electronic neural network are broken through (see [ Shen Y, Harris N C, Skirlo S, et al. "Deep learning with coherent nanophotonic circuits"Nature Photonics, vol. 11, no. 7, pp. 441-446, 2017.]). Firstly, the photon neural network adopts a simulation calculation framework, and the storage and the calculation are carried out simultaneously, so that the calculation speed is improved, and the calculation time delay can be reduced; secondly, based on the intrinsic characteristics of the optical transmission medium, the optical link has low loss characteristics, which indirectly reduces the systemSystem power consumption; finally, the effective working bandwidth of the photonic device is increased by several orders of magnitude compared with that of an electronic device, and the photonic device is more suitable for high-speed real-time operation of a neural network. At present, the photonic neural network model mainly includes three structures of a feedforward neural network, a cyclic neural network and an impulse neural network, such as a scheme (see [ Xu X, Tan M, Corcoran B, et al. "11 toas photonic connectivity access device for optical neural networks" Nature, vol. 589, no. 7840, pp. 45-51, 2021.]) The convolution operation and full-connection feedforward neural network for realizing the signal to be convolved based on the dispersion technology is provided, the operation speed is close to the latest chip based on the existing electronic technology, but the power consumption of the scheme is greatly reduced, and a reliable basis is provided for the practicability of the photonic neural network. However, the main network of the scheme is based on discrete optical modules, the size of each optical module is large, and the commercial instrument waveform shapers used in the scheme are difficult to integrate, so that the scheme is high in cost and difficult to apply to common commercial scenes in a large scale. In addition to discrete device based system-level neural networks, photonic integrated neural network chips are also being updated iteratively. For example (see (Shen Y, Harris N C, Skirlo S, et al. Deep learning with coherent nanophotonic circuits. Nature Photonics, 2017, 11(7): 441) a first photon computing chip in the world is developed based on a basic theoretical model of a trigonometric decomposition matrix operation algorithm, linear operation and a nonlinear activation function are realized by combining a photon chip with electric domain simulation, so that a fully-connected photon neural network is constructed, one layer of fully-connected neural network linear operation can be realized based on two operations of the chip, and meanwhile, nonlinear operation is realized based on the transmission characteristic of a computer town saturated absorber. Compared with the electronic neural network developed at present, the scheme still has wide promotion space in the aspects of system generalization, scale, practicability and the like.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the method overcomes the defects of the prior art, and utilizes a micro-ring resonator array containing delay waveguides to realize signal convolution kernel matrix coefficient weighting and time interleaving based on the photon integration technology. Except for the light source, all the photonic components of the whole acceleration chip are integrated on one chip, the system is compact and simple, small in size and low in cost, and the convolution kernel matrix can be flexibly expanded.
The invention specifically adopts the following technical scheme to solve the technical problems:
a time-wavelength interweaving photon neural network convolution acceleration chip is integrally integrated by a modulator, a wavelength division delay weighting micro-ring array and a balance photoelectric detector; all the photonic components are connected through optical waveguides; wherein:
the modulator is provided with 1 electric input end, 1 optical input end and 1 optical output end, the optical input end of the modulator is the optical input end of the whole chip and is used for receiving external multi-wavelength optical signals, the electric input end is used for receiving external signals to be convolved, and the optical output end of the modulator is connected with the optical input end of the wavelength division delay weighting micro-ring array;
the wavelength division delay weighting micro-ring array is provided with 1 optical input end, M electric control ends and 2 optical output ends, specifically, the wavelength division delay weighting micro-ring array comprises M pairs of micro-ring resonators, the M pairs of micro-ring resonators are connected in series through 1 through waveguide and 2 coupling waveguides, the through waveguide input end of the first pair of micro-ring resonators is the optical input end of the wavelength division delay weighting micro-ring array, and the 2 coupling waveguide output ends of the first pair of micro-ring resonators are 2 optical output ends of the wavelength division delay weighting micro-ring array; the balanced photodetector has 2 optical input ends and 1 electrical output end; two optical output ends of the wavelength division delay weighting micro-ring array are respectively connected with two optical input ends of the balance photoelectric detector; one electrical output end of the balanced photoelectric detector is the electrical output end of the whole chip;
the working process of the chip is as follows: firstly, modulating a multi-wavelength optical signal input to a modulator by a signal to be convolved through the intensity of the modulator, and respectively loading the signal to be convolved onto different carriers of the multi-wavelength optical signal to obtain a multi-wavelength intensity modulated optical signal containing O sub-intensity modulated optical signals; the multi-wavelength intensity modulation optical signal is sent into a wavelength division delay weighting micro-ring array, a control signal controls coupling coefficients of O pairs of adjacent micro-ring resonators in the wavelength division delay weighting micro-ring array, and O sub-intensity modulation optical signals are sequentially coupled into an upper coupling waveguide and a lower coupling waveguide according to different coupling coefficients to obtain a first coupling waveguide weighted intensity modulation optical signal and a second coupling waveguide weighted intensity modulation optical signal; and finally, respectively sending the first coupling waveguide weighted intensity modulation optical signal and the second coupling waveguide weighted intensity modulation optical signal to a balanced photoelectric detector to complete photoelectric conversion to obtain an electric output signal, wherein the electric output signal is a characteristic signal obtained after the convolution operation of the signal to be convolved is completed.
Preferably, M pairs of micro-ring resonators (each pair consisting of upper and lower 2 micro-ring resonators) in the wavelength division delay weighted micro-ring array have a length of L =between the ends of the through waveguidec∆t/n wThe delay waveguide of (1), whereincThe speed of the light in the vacuum is,n wis the effective refractive index of the waveguide delay linet=1/S M For a single symbol duration of the signal to be convolved,S M is the symbol rate of the signal to be convolved.
Preferably, the chip is integrated based on a iii-v material integration process or a silicon-based integration process, that is: the photonic components such as the modulator, the wave division delay weighting micro-ring array, the balanced photoelectric detector and the like and the optical waveguide are all prepared and integrated by three-five materials or silicon.
Furthermore, the number O of wavelengths of the multi-wavelength optical signal is equal to the number O of elements of the convolution kernel matrix, and is less than or equal to the logarithm M of the microring in the wavelength division delay weighted microring array.
Further, the control signal controls a coupling coefficient of O pairs of adjacent microring resonators in the wavelength division delay weighted microring array, specifically:
and determining the coupling coefficient of the micro-ring resonator according to the size of the convolution kernel matrix element and the initial signal intensity of each wavelength in the multi-wavelength optical signal, and changing the coupling coefficient of the micro-ring resonator through a thermo-optical effect or an electro-optical effect.
Furthermore, the radiuses of each pair of micro-ring resonators in the wavelength division delay weighting micro-ring array are the same, the micro-ring resonators correspond to a resonance wavelength respectively, and the free spectral range Δ λ corresponding to the micro-ring resonator with the largest radius isFSRShould be largeThe spectrum range O lambda occupied by the multi-wavelength optical signal diff ，∆λ diff Is the multi-wavelength optical signal wavelength spacing.
Further, the multi-wavelength optical signal is generated by a multi-wavelength laser, a mode-locked laser, a femtosecond laser, an optical frequency comb generator, or an optical soliton optical frequency comb generator.
Further, the signal intensities of the respective wavelengths in the multi-wavelength optical signal are equal.
Compared with the prior art, the technical scheme of the invention has the following beneficial effects:
1) the method realizes the arbitrary convolution kernel matrix coefficient weighting of the signal to be convolved based on the micro-ring resonator array, has simple and compact scheme and high adjusting speed, can realize real-time feedback training and extract the optimal convolution kernel matrix, and has the convolution operation speed limited only to the speed of a modulator.
2) Compared with the universal discrete photoelectric device, the monolithic photonic integration of all functional components does not need additional photoelectric functional devices, thereby simplifying the system, improving the stability of the system and expanding the scale of the chip in a large range.
3) The invention realizes the wavelength-time interleaving of the sub-intensity modulation signals with different wavelengths through the micro-ring resonator cascade integrated waveguide, has simple and efficient scheme, does not need dispersion calibration compensation, and can be integrated in a large scale.
Drawings
FIG. 1 is a schematic diagram of an exemplary convolution acceleration chip of a time-wavelength interleaved photonic neural network according to the present invention;
FIG. 2 is a schematic structural diagram of a convolution acceleration chip of a time-wavelength interleaved photonic neural network according to an embodiment of the present invention;
FIG. 3 is a schematic spectrum diagram of each working node of a convolution acceleration chip of a time-wavelength interleaved photonic neural network according to an embodiment of the present invention; the spectrum distribution diagram of the multi-wavelength optical signal is shown as A, the spectrum distribution diagram of the multi-wavelength intensity modulation optical signal is shown as B, the spectrum distribution diagram of the first coupling waveguide weighted intensity modulation optical signal is shown as C, the spectrum distribution diagram of the second coupling waveguide weighted intensity modulation optical signal is shown as D, the relation diagram of the first coupling waveguide weighted intensity modulation optical signal time sequence and the wavelength is shown as E, and the relation diagram of the second coupling waveguide weighted intensity modulation optical signal time sequence and the wavelength is shown as F.
Detailed Description
Aiming at the defects of the prior art, the idea of the invention is to realize the convolution kernel matrix coefficient weighting of the signal to be convolved and the time-wavelength interleaving of the multi-wavelength signal by utilizing the micro-ring resonator array of the cascade delay waveguide based on the photon integration technology. All the photon components of the photon neural network convolution acceleration chip are integrated on one chip, the scheme is compact and simple, the size is small, the cost is low, the convolution kernel matrix can be flexibly expanded, and the signal processing is real-time and efficient.
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
Fig. 1 shows an exemplary convolution acceleration chip structure of a time-wavelength interleaved photonic neural network according to the present invention, and as shown in fig. 1, the integrated photonic component has: a modulator, a wave division delay weighted micro-ring array and a Balanced Photodetector (BPD); all the photonic components are connected through optical waveguides; the modulator is provided with 1 electric input end, 1 optical input end and 1 optical output end, the optical input end of the modulator is the optical input end of the whole chip and is used for receiving external multi-wavelength optical signals, the electric input end is used for receiving external signals to be convolved, and the optical output end of the modulator is connected with the optical input end of the wavelength division delay weighting micro-ring array;
the wavelength division delay weighting micro-ring array is provided with 1 optical input end, M electric control ends and 2 optical output ends, specifically, the wavelength division delay weighting micro-ring array comprises M pairs of micro-ring resonators, the M pairs of micro-ring resonators are connected in series through 1 through waveguide and 2 coupling waveguides, the through waveguide input end of the first pair of micro-ring resonators is the optical input end of the wavelength division delay weighting micro-ring array, and the 2 coupling waveguide output ends of the first pair of micro-ring resonators are 2 optical output ends of the wavelength division delay weighting micro-ring array; the balanced photodetector has 2 optical input ends and 1 electrical output end; two optical output ends of the wavelength division delay weighting micro-ring array are respectively connected with two optical input ends of the balance photoelectric detector; one electrical output end of the balanced photoelectric detector is the electrical output end of the whole chip;
the working process of the chip is as follows: modulating the multi-wavelength optical signal input to the modulator through the intensity of the modulator, and loading the signal to be convolved to different carriers of the multi-wavelength optical signal respectively to obtain a multi-wavelength intensity modulated optical signal containing O sub-intensity modulated optical signals; the multi-wavelength intensity modulation optical signal is sent into a wavelength division delay weighting micro-ring array, a control signal controls coupling coefficients of O pairs of adjacent micro-ring resonators in the wavelength division delay weighting micro-ring array, and O sub-intensity modulation optical signals are sequentially coupled into an upper coupling waveguide and a lower coupling waveguide according to different coupling coefficients to obtain a first coupling waveguide weighted intensity modulation optical signal and a second coupling waveguide weighted intensity modulation optical signal; and finally, respectively sending the first coupling waveguide weighted intensity modulation optical signal and the second coupling waveguide weighted intensity modulation optical signal to a balanced photoelectric detector to complete photoelectric conversion to obtain an electric output signal, wherein the electric output signal is a characteristic signal obtained after the convolution operation of the signal to be convolved is completed.
The invention realizes arbitrary convolution kernel matrix coefficient weighting and wavelength-time interleaving based on an integrated micro-ring resonator array containing delay waveguides. The single integrated photon chip can realize the arbitrary convolution kernel convolution operation of the signal to be convolved, and the convolution kernel matrix coefficient can be updated rapidly and flexibly. The scheme is simple and compact, the power consumption is low, and the calculation real-time performance is high.
The multi-wavelength optical signal is generated by a multi-wavelength light source such as a multi-wavelength laser, a mode-locked laser, a femtosecond laser, an optical frequency comb generator, and an optical soliton optical frequency comb generator; in addition, when the signal intensity of each wavelength in the multi-wavelength light source is not equal, determining a micro-ring resonator coupling coefficient according to the size of the convolution kernel matrix element and the initial signal intensity of each wavelength in the multi-wavelength light signal, and when the signal intensity of each wavelength in the multi-wavelength light source is equal, determining the micro-ring resonator coupling coefficient according to the size of the convolution kernel matrix element; for ease of data processing, it is generally preferred that the intensity of each wavelength in the multi-wavelength light source be equal.
In practical operation, the number of wavelengths of the multi-wavelength signal is equal to the number of elements of the convolution kernel matrix, and is less than or equal to the number M of the micro-ring pairs in the wavelength-division delay weighted micro-ring array.
Furthermore, photonic components such as the modulator, the wavelength division delay weighting micro-ring array and the balanced photoelectric detector and the optical waveguide can be integrated by three-five family materials or silicon preparation, and the chip can be integrated based on mature processes such as three-five family material integration processes or silicon-based integration processes.
Wherein, the control signal controls the coupling coefficient of O pairs of adjacent micro-ring resonators in the wavelength division delay weighted micro-ring array, specifically:
and determining the coupling coefficient of the micro-ring resonator according to the size of the convolution kernel matrix element and the initial signal intensity of each wavelength in the multi-wavelength optical signal, and changing the coupling coefficient of the micro-ring resonator through a thermo-optical effect or an electro-optical effect.
The radiuses of each pair of micro-ring resonators in the wavelength division delay weighting micro-ring array are the same, the micro-ring resonators correspond to a resonance wavelength respectively, and the free spectral range lambda corresponding to the micro-ring resonator with the largest radiusFSRShould be larger than the spectrum range O λ occupied by the multi-wavelength signal diff ，∆λ diff Is the multi-wavelength optical signal wavelength spacing.
Fig. 2 shows a specific embodiment of a convolution operation application system based on a convolutional acceleration chip of a photonic neural network, which includes: the convolution acceleration chip for the photonic neural network, the multi-wavelength light source, the signal source to be convoluted, the convolution kernel matrix control signal and the analog-to-digital conversion and digital signal processor (ADC & DSP).
Firstly, the multi-wavelength light source outputs multi-wavelength light signals with equal wavelength intensity and enters the modulator through the optical input end of the photonic chip, and the wavelength intensities of the multi-wavelength light signals can be expressed as A = [ A, A, A, …, A ] by a matrix]T M×1M is the logarithm of the micro-ring in the wavelength division delay weighted micro-ring array, A is the wavelength intensity, and the spectral distribution is shown as A in FIG. 3. Modulating the multi-wavelength optical signal by a signal to be convolved output by a signal source to be convolved through a modulator, and respectively loading the signal to be convolved on different carriers of the multi-wavelength optical signal, wherein the signal to be convolved can be expressed as x (n) = [ ]x(1), x(2), x(3),…, x(N)]Wherein N represents a discretization time sequence number, N is the length of a signal to be convolved, the signal to be convolved is a one-dimensional signal obtained by matrix flattening processing of an actual signal, and the matrix flattening is specifically operated to convert a two-dimensional or multi-dimensional matrix into a one-dimensional matrix. Each intensity-modulated carrier corresponds to a signal to be convolved to obtain a multi-wavelength intensity-modulated optical signal for use in the multi-wavelength intensity-modulated optical signalThe matrix can be represented as:
the corresponding spectral distribution is shown as B in fig. 3. The multi-wavelength intensity modulation optical signal enters a wavelength division delay weighting micro-ring array of M pairs of micro-ring resonators connected in series, the wavelength division delay weighting micro-ring array consists of 1 through waveguide, 2 coupling waveguides and M pairs of micro-ring resonators, and a section of L = is arranged between the ends of the through waveguides of the micro-ring resonatorsc∆t/n wThe delay waveguide of (1), whereincThe speed of the light in the vacuum is,n wis the effective refractive index of the waveguide delay linet=1/S M The time difference between the single symbol duration of the signal to be convolved, i.e. x (n) and x (n-1),S M is the symbol rate of the signal to be convolved. The resonant characteristics of each pair of micro-rings in turn contribute to one wavelength. Firstly, a convolution kernel matrix control signal controls the resonance characteristic of a first pair of micro-ring resonators, so that corresponding wavelength sub-intensity modulation optical signals transmitted in a straight-through waveguide are respectively coupled into an upper coupling waveguide and a lower coupling waveguide according to different coupling coefficients, the coupling coefficients are set according to the sizes of the convolution kernel matrix coefficients, and the weighting of the convolution kernel matrix coefficients is realized. The multi-wavelength intensity modulation optical signal in the through waveguide enters the delay waveguide of the through waveguide after passing through the first micro-ring resonatortAnd (5) delaying. And the delayed multi-wavelength intensity modulation optical signals realize coefficient weighting on corresponding wavelength signals through a second pair of micro-ring resonators, and all wavelength signal weighting is completed in sequence after delay. And obtaining a first coupling waveguide weighted intensity modulation optical signal and a second coupling waveguide weighted intensity modulation optical signal at the output ends of the two coupling waveguides.
Setting the coefficient of the convolution kernel matrix after the flattening treatment as C w =[w 1,w 2,w 3,…,w M]TThe coupling coefficients of the modulated optical signals with different wavelength intensities on the upper micro-ring resonator and the lower micro-ring resonator of each pair of micro-ring resonators are respectively C wc1 =[w c1_1,w c1_2,w c1_3,…,w c1_M]TAnd C wc2 =[w c2_1,w c2_2,w c2_3,…,w c2_M]T，C w ，C wc1 ，C wc1 The following relationships are required:
C w=C wc1 - C wc2 or C w=C wc2 - C wc1 (2)
The first coupled waveguide weights the intensity modulated optical signal S Mod wc1_Can be expressed as:
the spectrum is shown as C in fig. 3, and the corresponding time series of signals versus wavelength is shown as E in fig. 3. Likewise, the second coupling waveguide weights the intensity modulated optical signal S Mod wc2_Can be expressed as:
the spectrum is shown as D in fig. 3, and the corresponding time series of signals versus wavelength is shown as F in fig. 3. After two paths of signals of the first coupling waveguide weighted intensity modulation optical signal and the second coupling waveguide weighted intensity modulation optical signal are sent to a balanced photoelectric detector to complete photoelectric conversion, the convolution multiply-add operation of the signal to be convolved can be completed, and a corresponding characteristic signal is obtained, wherein after the first coupling waveguide weighted intensity modulation optical signal is subjected to photoelectric conversion by one of the balanced photoelectric detectors, signals in an effective time sequence containing all wavelengths can be expressed as:
after the second coupling waveguide weighted intensity modulated optical signal is subjected to photoelectric conversion by the other balanced photoelectric detector, the signal in the effective time sequence can be represented as:
accordingly, the signal of the balanced photodetector in the active timing sequence can be expressed as:
wherein the content of the first and second substances,S ca (r) Is as followsrAs a result of the sub-multiply-add operation,w mare convolution kernel matrix coefficients. The active timing is shown as E, F in FIG. 3.
The analog-to-digital conversion and digital signal processor can acquire and process corresponding electric signals, and the corresponding characteristic signal matrix can be obtained by realizing signal reconstruction in a digital domain.
Finally, it should be noted that the above-mentioned list is only a specific embodiment of the present invention. The present invention is not limited to the above embodiments, and many variations are possible. All modifications which can be derived or suggested by a person skilled in the art from the disclosure of the present invention are to be considered within the scope of the invention.
Claims (8)
1. A time-wavelength interweaving photon neural network convolution acceleration chip is characterized in that the chip is integrated by a modulator, a wavelength division delay weighting micro-ring array and a balance photoelectric detector; wherein:
the modulator is provided with 1 electrical input end, 1 optical input end and 1 optical output end, wherein the optical input end is the optical input end of the whole chip and is used for receiving external multi-wavelength optical signals, and the optical output end is connected with the optical input end of the wavelength division delay weighting micro-ring array; the electrical input end is used for receiving an external signal to be convolved, and the signal to be convolved is subjected to intensity modulation on the multi-wavelength optical signal input to the modulator through the modulator to obtain a multi-wavelength intensity modulation optical signal containing O sub-intensity modulation optical signals;
the wave division delay weighting micro-ring array comprises M pairs of micro-ring resonators, wherein the M pairs of micro-ring resonators are connected in series through 1 through waveguide and 2 coupling waveguides, wherein the through waveguide input end of the first pair of micro-ring resonators is the optical input end of the wave division delay weighting micro-ring array, and the 2 coupling waveguide output ends of the first pair of micro-ring resonators are the 2 optical output ends of the wave division delay weighting micro-ring array; the wave division delay weighted micro-ring array controls the coupling coefficients of O pairs of adjacent micro-ring resonators according to the control signal, and sequentially couples O sub-intensity modulated optical signals into an upper coupled waveguide and a lower coupled waveguide according to different coupling coefficients to obtain a first coupled waveguide weighted intensity modulated optical signal and a second coupled waveguide weighted intensity modulated optical signal;
the balanced photoelectric detector is provided with 2 optical input ends which are respectively connected with 2 optical output ends of the wavelength division delay weighted micro-ring array and used for carrying out photoelectric conversion on the first coupling waveguide weighted intensity modulation optical signal and the second coupling waveguide weighted intensity modulation optical signal to obtain an electric output signal, wherein the electric output signal is a characteristic signal obtained after the convolution operation of the signal to be convolved is completed.
2. The time-wavelength interleaved photonic neural network convolution acceleration chip of claim 1, wherein M pairs of microring resonators in the wavelength division delay weighted microring array have a length Δ L =betweeneach other at the through waveguide endc∆t/n wThe delay waveguide of (1), whereincThe speed of the light in the vacuum is,n wis the effective refractive index of the waveguide delay linet=1/S M For a single symbol duration of the signal to be convolved,S M is the symbol rate of the signal to be convolved.
3. The time-wavelength interleaved photonic neural network convolution acceleration chip of claim 1 wherein the chip is integrated based on a iii-v material integration process or a silicon-based integration process.
4. The time-wavelength interleaved photonic neural network convolution acceleration chip of claim 1 wherein the number of wavelengths O of the multi-wavelength optical signal is equal to the number of elements of the convolution kernel matrix and is less than or equal to the number of microring logarithms M in the wavelength division delay weighted microring array.
5. The time-wavelength interleaved photonic neural network convolution acceleration chip of claim 1, wherein the control signal controls a coupling coefficient of O pairs of adjacent microring resonators in the wavelength division delay weighted microring array, specifically:
and determining the coupling coefficient of the micro-ring resonator according to the size of the convolution kernel matrix element and the initial signal intensity of each wavelength in the multi-wavelength optical signal, and changing the coupling coefficient of the micro-ring resonator through a thermo-optical effect or an electro-optical effect.
6. The time-wavelength interleaved photonic neural network convolution acceleration chip of claim 1, wherein each pair of micro-ring resonators in the wavelength division delay weighted micro-ring array have the same radius, respectively corresponding to a resonant wavelength, and the free spectral range Δ λ corresponding to the micro-ring resonator with the largest radiusFSRShould be larger than the spectrum range OΔ λ occupied by the multi-wavelength optical signal diff ，∆λ diff Is the multi-wavelength optical signal wavelength spacing.
7. The time-wavelength interleaved photonic neural network convolution acceleration chip of claim 1 wherein the multi-wavelength optical signal is generated by a multi-wavelength laser, a mode-locked laser, a femtosecond laser, an optical frequency comb generator, or an optical soliton optical frequency comb generator.
8. The time-wavelength interleaved photonic neural network convolution acceleration chip of claim 1 wherein the signal strength of each wavelength in the multi-wavelength optical signal is equal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210267027.1A CN114358271B (en) | 2022-03-18 | 2022-03-18 | Time-wavelength interweaving photon neural network convolution acceleration chip |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210267027.1A CN114358271B (en) | 2022-03-18 | 2022-03-18 | Time-wavelength interweaving photon neural network convolution acceleration chip |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114358271A true CN114358271A (en) | 2022-04-15 |
| CN114358271B CN114358271B (en) | 2022-07-12 |
Family
ID=81094877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210267027.1A Active CN114358271B (en) | 2022-03-18 | 2022-03-18 | Time-wavelength interweaving photon neural network convolution acceleration chip |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114358271B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114819132A (en) * | 2022-06-27 | 2022-07-29 | 之江实验室 | Photon two-dimensional convolution acceleration method and system based on time-wavelength interleaving |
| CN114815959A (en) * | 2022-06-27 | 2022-07-29 | 之江实验室 | Photon tensor calculation acceleration method and device based on wavelength division multiplexing |
| CN115146771A (en) * | 2022-09-02 | 2022-10-04 | 之江实验室 | Two-dimensional photon neural network convolution acceleration chip based on series structure |
| CN115169542A (en) * | 2022-09-02 | 2022-10-11 | 之江实验室 | Two-dimensional photon convolution acceleration system and device for convolution neural network |
| CN115167610A (en) * | 2022-09-08 | 2022-10-11 | 之江实验室 | Two-dimensional photon convolution acceleration chip and application system and method |
| CN115222035A (en) * | 2022-09-20 | 2022-10-21 | 之江实验室 | Photon neural network convolution acceleration chip |
| CN116029350A (en) * | 2023-03-29 | 2023-04-28 | 之江实验室 | Two-dimensional photon coherent convolution acceleration chip based on time interleaving and application system thereof |
| CN116432726A (en) * | 2023-06-14 | 2023-07-14 | 之江实验室 | Photoelectric hybrid deep neural network operation device and operation method |
| CN116484931A (en) * | 2023-06-14 | 2023-07-25 | 之江实验室 | Photon matrix multiplication operation device and operation method for neural network |
| CN117270100A (en) * | 2023-09-14 | 2023-12-22 | 之江实验室 | Monolithic photon integrated chip for realizing reserve tank circulation network |
| US11874501B1 (en) * | 2022-08-19 | 2024-01-16 | Huawei Technologies Canada Co., Ltd. | Systems and methods to maintain phase relations of an optical signal in a photonic dot product engine |
| CN117784313A (en) * | 2024-02-28 | 2024-03-29 | 之江实验室 | Two-dimensional photon convolution operation chip and system based on cyclic array waveguide grating |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180217328A1 (en) * | 2017-02-02 | 2018-08-02 | International Business Machines Corporation | Waveguide architecture for photonic neural component |
| CN109379138A (en) * | 2018-08-24 | 2019-02-22 | 中国电子科技集团公司第二十九研究所 | High Linear microwave photon link implementation method and structure based on photon neural network |
| CN109639359A (en) * | 2019-01-07 | 2019-04-16 | 上海交通大学 | Photon neural network convolutional layer chip based on micro-ring resonator |
| CN109981172A (en) * | 2019-03-01 | 2019-07-05 | 上海交通大学 | The multiply-add implementation method of full light matrix based on multi-wavelength modulation and dispersion time delay |
| CN110516802A (en) * | 2019-07-03 | 2019-11-29 | 上海交通大学 | Photon convolutional neural networks framework based on optical delay line caching |
| US20200019851A1 (en) * | 2018-07-10 | 2020-01-16 | The George Washington University | Optical convolutional neural network accelerator |
| CN111461317A (en) * | 2020-04-03 | 2020-07-28 | 上海交通大学 | Single-chip integrated photon convolution neural network computing system and preparation method thereof |
| CN111882052A (en) * | 2020-08-19 | 2020-11-03 | 北京大学 | Photon convolution neural network system |
| CN112308224A (en) * | 2020-10-09 | 2021-02-02 | 武汉光谷信息光电子创新中心有限公司 | Optical neural network device, chip and optical implementation method for neural network calculation |
| CN112327276A (en) * | 2020-11-09 | 2021-02-05 | 之江实验室 | Photon sampling chip for microwave photon radar and application system thereof |
| CN113630207A (en) * | 2020-05-07 | 2021-11-09 | 光子算数(苏州)智能科技有限公司 | Photon neural network chip |
| CN113890620A (en) * | 2020-07-01 | 2022-01-04 | 浙江大学 | Silicon substrate photonic neural network based on tunable filter and modulation method thereof |
| CN113961035A (en) * | 2021-10-15 | 2022-01-21 | 北京理工大学 | Deep learning-oriented high-efficiency reconfigurable all-optical neural network computing chip architecture |
-
2022
- 2022-03-18 CN CN202210267027.1A patent/CN114358271B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180217328A1 (en) * | 2017-02-02 | 2018-08-02 | International Business Machines Corporation | Waveguide architecture for photonic neural component |
| US20200019851A1 (en) * | 2018-07-10 | 2020-01-16 | The George Washington University | Optical convolutional neural network accelerator |
| CN109379138A (en) * | 2018-08-24 | 2019-02-22 | 中国电子科技集团公司第二十九研究所 | High Linear microwave photon link implementation method and structure based on photon neural network |
| CN109639359A (en) * | 2019-01-07 | 2019-04-16 | 上海交通大学 | Photon neural network convolutional layer chip based on micro-ring resonator |
| CN109981172A (en) * | 2019-03-01 | 2019-07-05 | 上海交通大学 | The multiply-add implementation method of full light matrix based on multi-wavelength modulation and dispersion time delay |
| CN110516802A (en) * | 2019-07-03 | 2019-11-29 | 上海交通大学 | Photon convolutional neural networks framework based on optical delay line caching |
| CN111461317A (en) * | 2020-04-03 | 2020-07-28 | 上海交通大学 | Single-chip integrated photon convolution neural network computing system and preparation method thereof |
| CN113630207A (en) * | 2020-05-07 | 2021-11-09 | 光子算数(苏州)智能科技有限公司 | Photon neural network chip |
| CN113890620A (en) * | 2020-07-01 | 2022-01-04 | 浙江大学 | Silicon substrate photonic neural network based on tunable filter and modulation method thereof |
| CN111882052A (en) * | 2020-08-19 | 2020-11-03 | 北京大学 | Photon convolution neural network system |
| CN112308224A (en) * | 2020-10-09 | 2021-02-02 | 武汉光谷信息光电子创新中心有限公司 | Optical neural network device, chip and optical implementation method for neural network calculation |
| CN112327276A (en) * | 2020-11-09 | 2021-02-05 | 之江实验室 | Photon sampling chip for microwave photon radar and application system thereof |
| CN113961035A (en) * | 2021-10-15 | 2022-01-21 | 北京理工大学 | Deep learning-oriented high-efficiency reconfigurable all-optical neural network computing chip architecture |
Non-Patent Citations (5)
| Title |
|---|
| QINGSHUI GUO 等: "Photonic Generation of Linear-Frequency-Modulated Waveforms With Improved Time-Bandwidth Product Based on Polarization Modulation", 《JOURNAL OF LIGHTWAVE TECHNOLOGY》, vol. 35, no. 10, 15 May 2017 (2017-05-15), pages 1821 - 1829, XP011646564, DOI: 10.1109/JLT.2017.2651902 * |
| XINGYUAN XU 等: "Photonic perceptron based on a Kerr microcomb for high-speed,scalable,optical neural networks", 《TECHRXIV》, 3 September 2020 (2020-09-03), pages 1 - 18 * |
| YUE JIANG 等: "PHOTONIC CONVOLUTION NEURAL NETWORK BASED ON INTERLEAVED TIME-WAVELENGTH MODULATION", 《ARXIV》, 22 February 2021 (2021-02-22), pages 1 - 14 * |
| 王健 等: "智能硅基多维复用与处理芯片", 《中兴通讯技术》, vol. 26, no. 2, 30 April 2020 (2020-04-30), pages 51 - 63 * |
| 白冰 等: "用于人工智能的硅基光电子芯片", 《中兴通讯技术》, vol. 27, no. 1, 28 February 2021 (2021-02-28), pages 77 - 82 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114815959A (en) * | 2022-06-27 | 2022-07-29 | 之江实验室 | Photon tensor calculation acceleration method and device based on wavelength division multiplexing |
| CN114819132A (en) * | 2022-06-27 | 2022-07-29 | 之江实验室 | Photon two-dimensional convolution acceleration method and system based on time-wavelength interleaving |
| US11874501B1 (en) * | 2022-08-19 | 2024-01-16 | Huawei Technologies Canada Co., Ltd. | Systems and methods to maintain phase relations of an optical signal in a photonic dot product engine |
| CN115146771A (en) * | 2022-09-02 | 2022-10-04 | 之江实验室 | Two-dimensional photon neural network convolution acceleration chip based on series structure |
| CN115169542A (en) * | 2022-09-02 | 2022-10-11 | 之江实验室 | Two-dimensional photon convolution acceleration system and device for convolution neural network |
| CN115146771B (en) * | 2022-09-02 | 2022-11-22 | 之江实验室 | Two-dimensional photon neural network convolution acceleration chip based on series structure |
| CN115169542B (en) * | 2022-09-02 | 2022-11-22 | 之江实验室 | Two-dimensional photon convolution acceleration system and device for convolution neural network |
| US11934943B1 (en) | 2022-09-02 | 2024-03-19 | Zhejiang Lab | Two-dimensional photonic neural network convolutional acceleration chip based on series connection structure |
| CN115167610A (en) * | 2022-09-08 | 2022-10-11 | 之江实验室 | Two-dimensional photon convolution acceleration chip and application system and method |
| CN115222035A (en) * | 2022-09-20 | 2022-10-21 | 之江实验室 | Photon neural network convolution acceleration chip |
| CN115222035B (en) * | 2022-09-20 | 2022-12-30 | 之江实验室 | Photon neural network convolution acceleration chip |
| CN116029350B (en) * | 2023-03-29 | 2023-07-07 | 之江实验室 | Two-dimensional photon coherent convolution acceleration chip based on time interleaving and application system thereof |
| CN116029350A (en) * | 2023-03-29 | 2023-04-28 | 之江实验室 | Two-dimensional photon coherent convolution acceleration chip based on time interleaving and application system thereof |
| CN116484931A (en) * | 2023-06-14 | 2023-07-25 | 之江实验室 | Photon matrix multiplication operation device and operation method for neural network |
| CN116432726B (en) * | 2023-06-14 | 2023-08-25 | 之江实验室 | Photoelectric hybrid deep neural network operation device and operation method |
| CN116484931B (en) * | 2023-06-14 | 2023-09-01 | 之江实验室 | Photon matrix multiplication operation device and operation method for neural network |
| CN116432726A (en) * | 2023-06-14 | 2023-07-14 | 之江实验室 | Photoelectric hybrid deep neural network operation device and operation method |
| CN117270100A (en) * | 2023-09-14 | 2023-12-22 | 之江实验室 | Monolithic photon integrated chip for realizing reserve tank circulation network |
| CN117784313A (en) * | 2024-02-28 | 2024-03-29 | 之江实验室 | Two-dimensional photon convolution operation chip and system based on cyclic array waveguide grating |
| CN117784313B (en) * | 2024-02-28 | 2024-06-07 | 之江实验室 | Two-dimensional photon convolution operation chip and system based on cyclic array waveguide grating |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114358271B (en) | 2022-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114358271B (en) | Time-wavelength interweaving photon neural network convolution acceleration chip | |
| CN109639359B (en) | Photonic neural network convolution layer chip based on micro-ring resonator | |
| CN114819132B (en) | Photon two-dimensional convolution acceleration method and system based on time-wavelength interleaving | |
| CN111882052B (en) | Photon convolution neural network system | |
| CN114815959B (en) | Photon tensor calculation acceleration method and device based on wavelength division multiplexing | |
| CN115167610B (en) | Two-dimensional photon convolution acceleration chip and application system and method | |
| CN115222035B (en) | Photon neural network convolution acceleration chip | |
| CN115169542B (en) | Two-dimensional photon convolution acceleration system and device for convolution neural network | |
| CN117077750B (en) | Parallel photon convolution operation chip and system | |
| CN117784313B (en) | Two-dimensional photon convolution operation chip and system based on cyclic array waveguide grating | |
| CN111561953A (en) | On-chip integrated optical matrix vector multiplier based on wavelength division multiplexing and balanced detection | |
| US11934943B1 (en) | Two-dimensional photonic neural network convolutional acceleration chip based on series connection structure | |
| CN116029350B (en) | Two-dimensional photon coherent convolution acceleration chip based on time interleaving and application system thereof | |
| CN115130666B (en) | Two-dimensional photon convolution acceleration method and system | |
| CN115374828A (en) | Tensor convolution kernel acceleration chip and method | |
| Xu et al. | High-energy-efficiency integrated photonic convolutional neural networks | |
| Xu et al. | Performance evaluation of an integrated photonic convolutional neural network based on delay buffering and wavelength division multiplexing | |
| CN117709423B (en) | Deep neural network photon acceleration chip and operation system thereof | |
| Karempudi et al. | A Low-Dissipation and Scalable GEMM Accelerator with Silicon Nitride Photonics | |
| Peserico et al. | PhotoFourier: silicon photonics joint transfer correlator for convolution neural network | |
| Shi et al. | Emulation and modelling of semiconductor optical amplifier-based all-optical photonic integrated deep neural network with arbitrary depth | |
| CN116739065A (en) | Photon tensor convolution calculation method and system for multichannel data processing | |
| Wang et al. | Non-volatile and ultra-fast photonic vector accelerator with optical phase change materials and integrated microcomb | |
| CN115688898A (en) | Photon convolution three-dimensional image processing system and method for automatic driving | |
| CN117436486A (en) | Optical convolution neural network based on thin film lithium niobate and silicon mixture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |