CN114520694A - Computing chip, system and data processing method - Google Patents
Computing chip, system and data processing method Download PDFInfo
- Publication number
- CN114520694A CN114520694A CN202210417937.3A CN202210417937A CN114520694A CN 114520694 A CN114520694 A CN 114520694A CN 202210417937 A CN202210417937 A CN 202210417937A CN 114520694 A CN114520694 A CN 114520694A
- Authority
- CN
- China
- Prior art keywords
- optical
- signal
- model
- target
- computing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 15
- 230000003287 optical effect Effects 0.000 claims abstract description 155
- 238000006243 chemical reaction Methods 0.000 claims abstract description 21
- 239000011159 matrix material Substances 0.000 claims description 91
- 238000004364 calculation method Methods 0.000 claims description 36
- 230000006870 function Effects 0.000 claims description 19
- 230000004913 activation Effects 0.000 claims description 14
- 238000005516 engineering process Methods 0.000 claims description 5
- 239000013307 optical fiber Substances 0.000 claims description 5
- 239000006096 absorbing agent Substances 0.000 claims description 4
- 230000004907 flux Effects 0.000 abstract description 4
- 230000000694 effects Effects 0.000 abstract description 2
- 238000000034 method Methods 0.000 description 23
- 239000012071 phase Substances 0.000 description 23
- 239000013598 vector Substances 0.000 description 15
- 230000008569 process Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 7
- 238000000354 decomposition reaction Methods 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 5
- 210000002569 neuron Anatomy 0.000 description 4
- 238000004377 microelectronic Methods 0.000 description 3
- 101000798940 Gallus gallus Target of Myb protein 1 Proteins 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 229910052710 silicon Inorganic materials 0.000 description 2
- 239000010703 silicon Substances 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000005374 Kerr effect Effects 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 230000008021 deposition Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000008384 inner phase Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000002207 thermal evaporation Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
- H04B10/50—Transmitters
- H04B10/516—Details of coding or modulation
- H04B10/54—Intensity modulation
- H04B10/541—Digital intensity or amplitude modulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/76—Architectures of general purpose stored program computers
- G06F15/78—Architectures of general purpose stored program computers comprising a single central processing unit
- G06F15/7807—System on chip, i.e. computer system on a single chip; System in package, i.e. computer system on one or more chips in a single package
- G06F15/7817—Specially adapted for signal processing, e.g. Harvard architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
- H04B10/50—Transmitters
- H04B10/501—Structural aspects
- H04B10/503—Laser transmitters
- H04B10/505—Laser transmitters using external modulation
- H04B10/5051—Laser transmitters using external modulation using a series, i.e. cascade, combination of modulators
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
- H04B10/50—Transmitters
- H04B10/501—Structural aspects
- H04B10/503—Laser transmitters
- H04B10/505—Laser transmitters using external modulation
- H04B10/5053—Laser transmitters using external modulation using a parallel, i.e. shunt, combination of modulators
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
- H04B10/50—Transmitters
- H04B10/516—Details of coding or modulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Computer Hardware Design (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Optics & Photonics (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Optical Modulation, Optical Deflection, Nonlinear Optics, Optical Demodulation, Optical Logic Elements (AREA)
Abstract
The application discloses a computing chip, a system and a data processing method in the technical field of computers. The electric domain controller in the computing chip can control the electro-optical modulator array to convert laser signals emitted by the signal emitter into target optical signals representing input data of the AI model, then the programmable optical structure carries out rapid computing on the target optical signals to obtain corresponding optical computing results, and then the photoelectric detector array is used for carrying out photoelectric conversion on the optical computing results to obtain electric signal representations corresponding to the optical computing results, wherein the electric signal representations are model processing results of the AI model aiming at the input data. The computing chip uses the programmable optical structure to accelerate the processing of the AI model, can improve the running speed of complex operation in the AI model, has the characteristics of low power consumption, high flux and low time delay, and improves the processing capability of hardware for the complex operation. The computing system and the data processing method also have the technical effects.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a computing chip, a computing system, and a data processing method.
Background
At present, the processing of specific operations such as convolution and the like can be accelerated by means of logic circuits such as an FPGA, a GPU and the like, but electronic chips such as the FPGA, the GPU and the like are influenced by Moore's law, and the computational power cannot be continuously increased, so that the computing power is limited, and the problems of crosstalk, high power consumption, high delay, heat deposition and the like are easy to occur.
Therefore, how to further improve the processing capability of the hardware for complex operations is a problem to be solved by those skilled in the art.
Disclosure of Invention
In view of the above, an object of the present application is to provide a computing chip, a computing system and a data processing method, so as to use an optical structure to process a complex operation, thereby improving the processing capability of hardware for the complex operation. The specific scheme is as follows:
in a first aspect, the present application provides a computing chip, comprising:
a signal transmitter for transmitting a laser signal;
the electro-optical modulator array is used for converting the laser signal into a target optical signal under the control of a domain controller in the computing chip; the target light signal is used for representing input data of a target AI model;
the programmable optical structure is used for realizing a model weight matrix of the target AI model, and is used for calculating the target optical signal and outputting an optical calculation result;
and the photoelectric detector array is used for performing photoelectric conversion on the light calculation result to obtain a model processing result of the target AI model aiming at the input data.
Optionally, the electric domain controller comprises:
the digital-to-analog conversion module is used for converting the first electric signal into a first analog signal;
and the logic control circuit is used for transmitting the first analog signal to the electro-optical modulator array so that the electro-optical modulator array converts the laser signal into the target optical signal according to the first analog signal.
Optionally, the electro-optical modulator array is specifically configured to:
and modulating the light intensity of the laser signal according to the first analog signal to obtain the target optical signal.
Optionally, the digital-to-analog conversion module is further configured to convert the second electrical signal into a second analog signal;
the logic control circuit is further configured to transmit the second analog signal to the programmable optical structure, so that the programmable optical structure adjusts a phase shifter in the programmable optical structure according to the second analog signal to implement the model weight matrix of the target AI model.
Optionally, the electric domain controller further comprises:
and the driving module is used for driving the logic control circuit to transmit the second analog signal to the programmable optical structure so that the programmable optical structure adjusts the phase shifter in the programmable optical structure according to the second analog signal.
Optionally, the electric domain controller further comprises:
a storage module for storing the model processing result, the first electrical signal, the second electrical signal, and/or an output result of a nonlinear activation function.
Optionally, the signal transmitter comprises:
a laser for generating the laser signal;
and the optical fiber array is connected with the laser and is used for transmitting the laser signals to the electro-optical modulator array by a preset number of input paths.
Optionally, the method further comprises:
saturable absorber structures, or bistable or MZI structures for realizing nonlinear activation functions.
Optionally, the programmable optical structure comprises: a cascaded MZI structure and a parallel optical attenuator structure.
In a second aspect, the present application provides a computing system comprising: a plurality of the computing chips of any of the above, each connected using optical interconnect technology.
In a third aspect, the present application provides a data processing method applied to any one of the above computing chips, including:
transmitting a laser signal by using a signal transmitter;
controlling an electro-optical modulator array by using a domain controller to convert the laser signal into a target optical signal; the target light signal is used for representing input data of a target AI model;
calculating the target optical signal by using a programmable optical structure, and outputting an optical calculation result; the programmable optical structure implements a model weight matrix with the target AI model;
and performing photoelectric conversion on the light calculation result by using a photoelectric detector array to obtain a model processing result of the target AI model aiming at the input data.
According to the above scheme, the present application provides a computing chip, including: a signal transmitter for transmitting a laser signal; the electro-optical modulator array is used for converting the laser signal into a target optical signal under the control of a domain controller in the computing chip; the target light signal is used for representing input data of a target AI model; the programmable optical structure is used for realizing a model weight matrix of the target AI model, and is used for calculating the target optical signal and outputting an optical calculation result; and the photoelectric detector array is used for performing photoelectric conversion on the light calculation result to obtain a model processing result of the target AI model aiming at the input data.
Therefore, the computing chip provided by the application uses the programmable optical structure to compute and process the target optical signal representing the input data of the AI model, and can quickly obtain the model processing result of the AI model aiming at the input data, so that the processing capacity of hardware aiming at complex operation is improved. The electric domain controller in the computing chip can control the electro-optical modulator array to convert the laser signals emitted by the signal emitter, so that common laser signals emitted by the signal emitter can be converted into target optical signals representing AI model input data, and then the programmable optical structure can rapidly calculate the target optical signals to obtain corresponding optical calculation results. In order to enable the display and application of the light calculation results, the light calculation results are photoelectrically converted using a photodetector array, so that an electrical signal representation corresponding to the light calculation results can be obtained. Because the optical signal has the advantages of super high speed, super large capacity, high parallelism, high anti-interference capability and the like, the programmable optical structure is used for accelerating the processing of the AI model, the running speed of complex operation in the AI model can be improved, and the computing chip has the characteristics of low power consumption, high flux and low time delay.
Correspondingly, the computing system and the data processing method provided by the application also have the technical effects.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic diagram of a computing chip structure disclosed in the present application;
FIG. 2 is a schematic diagram of a programmable optical architecture according to the present disclosure;
FIG. 3 is a schematic diagram of a single MZI structure disclosed herein;
FIG. 4 is a schematic diagram of a topology corresponding to the cascaded MZI structure of FIG. 2;
FIG. 5 is a schematic diagram of another computing chip structure disclosed in the present application;
FIG. 6 is a schematic diagram of a single optical attenuator as disclosed herein;
FIG. 7 is a schematic diagram of a weight matrix between two layers of the network disclosed in the present application;
fig. 8 is a flowchart of a data processing method disclosed in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
At present, the processing of specific operations such as convolution and the like can be accelerated by means of logic circuits such as an FPGA and the like, but electronic chips such as the FPGA, a GPU and the like are influenced by Moore's law, and the computational power cannot be continuously increased, so that the computing capability is limited, and the problems of crosstalk, high power consumption, high delay, thermal deposition and the like are easy to occur.
Therefore, the application provides a computing chip, a computing system and a data processing method, so that an optical structure is used for processing complex operation, and the processing capacity of hardware for the complex operation is improved.
Referring to fig. 1, an embodiment of the present application discloses a computing chip, including: the device comprises a signal emitter, an electro-optical modulator array, a programmable optical structure, a photoelectric detector array and a domain controller. Wherein the programmable optical structure implements a model weight matrix for the target AI model.
Wherein, the model weight matrix of the target AI model is obtained by software algorithm training. Namely: after an AI model is trained by using a software algorithm, a corresponding model weight matrix can be determined based on various parameters of the AI model, and then a corresponding programmable optical structure is built according to the model weight matrix, so that the programmable optical structure has the same function as the AI model realized by the algorithm. The programmable optical structure can be used for replacing a software algorithm AI model for calculation subsequently, so that the calculation speed of the model is improved, and the processing efficiency is accelerated. For example: if the AI model is an image classification model, the programmable optical structure used to implement the model weight matrix can also perform image classification, and ultimately output the image classification. Accordingly, the target light signal input to the programmable optical structure represents a certain image data to be classified.
In particular, the signal transmitter is used for transmitting a laser signal. The electro-optical modulator array is used for converting the laser signal into a target optical signal under the control of a domain controller in the computing chip; the target light signal is used to represent input data of the target AI model. The programmable optical structure is used for calculating the target optical signal and outputting an optical calculation result. The photoelectric detector array is used for performing photoelectric conversion on the light calculation result to obtain a model processing result of the target AI model aiming at the input data.
In one embodiment, a programmable optical structure comprises: a cascaded MZI structure and a parallel optical attenuator structure. The programmable optical structure may be as shown in FIG. 2, with FIG. 2 representing a 6-input, 6-output programmable optical structure, wherein
The representation of phase shifter. The first half of FIG. 2 is a cascaded MZI structure and the second half is a parallel optical attenuator structure.
One of the MZI structures is shown in FIG. 3. As shown in FIG. 3, an MZI structure comprises: two directional couplers:B 1andB 2an inner phase shifterR θAnd an external phase shifterR φ. The directional coupler is a 2-input 2-output 4-port device, and can couple the optical power of an input port to an output port according to a 50:50 splitting ratio. Internal phase shifter 2θ（0≤θLess than or equal to pi/2) is responsible for modulating the MZI output power. External phase shifterφ（0≤φLess than or equal to 2 pi) is responsible for compensating the relative phases of the two paths of light output by the MZI, so that both phase shifters have a programmable function. One MZI structure corresponds to one 2 × 2 dimensional unitary matrix.
From the comparison of fig. 2, the cascaded MZI structure in the first half of fig. 2 corresponds to a 6-dimensional unitary matrix, and the specific topology structure of the 6-dimensional unitary matrix can be referred to fig. 4. In FIG. 4, one is
Representing a MZI structure with an input end and an output end connected in an opposite way, 1-6 are 6 input signals, each
Of "xy"denotes the two input signals of the MZI structure there. For example: "65" indicates that the two input signals of the MZI structure are 6 and 5.
The computing chip in this embodiment uses the programmable optical structure to compute and process the target optical signal representing the input data of the AI model, and can quickly obtain the model processing result of the AI model for the input data, thereby improving the processing capability of hardware for complex operations.
Specifically, the electric domain controller in the computing chip can control the electro-optical modulator array to convert the laser signal emitted by the signal emitter, so that the common laser signal emitted by the signal emitter can be converted into a target optical signal representing AI model input data, and then the programmable optical structure performs rapid computation on the target optical signal to obtain a corresponding optical computation result. In order to display and apply the light calculation result, the light calculation result is subjected to photoelectric conversion by using a photoelectric detector array, so that an electric signal representation corresponding to the light calculation result can be obtained.
Therefore, the programmable optical structure is used for accelerating the processing of the AI model and improving the running speed of complex operation in the AI model, so that the computing chip has the characteristics of low power consumption, high flux and low time delay. The photon computing chip is a non-von neumann architecture, can perform operations at the light speed, and has higher computing power than an electronic AI chip.
Based on the above embodiment, the composition structure of the domain controller can refer to fig. 5. Referring to fig. 5, the domain controller includes: the digital-to-analog conversion module, the logic control circuit, the storage module SRAM (Static Random-Access Memory), the drive module.
Specifically, the digital-to-analog conversion module is used for converting the first electric signal into a first analog signal; the logic control circuit is used for transmitting a first analog signal to the electro-optical modulator array, so that the electro-optical modulator array converts the laser signal into a target optical signal according to the first analog signal. Wherein, the first electric signal is specifically: instructions capable of converting the laser signal to a target optical signal. Accordingly, the electro-optic modulator array is particularly useful for: and modulating the light intensity of the laser signal according to the first analog signal to obtain a target optical signal. It can be seen that the array of electro-optic modulators is capable of modulating the light intensity of the laser signal.
In one embodiment, the digital-to-analog conversion module is further configured to convert the second electrical signal into a second analog signal; the logic control circuit is further configured to transmit a second analog signal to the programmable optical structure such that the programmable optical structure adjusts the phase shifter in itself in accordance with the second analog signal to implement the model weight matrix for the target AI model. Wherein, the second electric signal is specifically: a model weight matrix for the target AI model.
Specifically, the driving module in the domain controller is configured to drive the logic control circuit to transmit the second analog signal to the programmable optical structure, so that the programmable optical structure adjusts the phase shifter in the programmable optical structure according to the second analog signal. It can be seen that the drive module is capable of controlling the phase shifters in the programmable optical structure. In general, the phase value of the phase shifter can be changed by modulating a voltage to change the refractive index of the material, and can also be changed by adjusting a voltage to interfere with the physical separation of the arms.
It should be noted that the model processing result of the target AI model output by the photodetector array for the input data may not be the final result. Since the programmable optical structure may only implement the weight matrix of a certain layer of the model, the model processing result of the target AI model for the input data cannot be calculated once, and then the calculation can be repeated using the programmable optical structure. It can be seen that the programmable optical structure can also implement only the weight matrix of a certain layer of the AI model. And the layers of the AI model are provided with corresponding weight matrixes, so that a plurality of model weight matrixes of the target AI model are possible, and if all the weight matrixes of the model are realized by using the programmable optical structure, the final output result of the model can be directly output by using the programmable optical structure. If only a weight matrix for a certain layer of the model is realized using the programmable optical structure, the result output using the programmable optical structure is a corresponding intermediate result. As described in the embodiments below with respect to the 6-input 6-output programmable optical structure, the processing result of the first output can be temporarily stored in a memory module of the domain controller for later use and re-calculation.
Accordingly, a storage module in the electric domain controller is used for storing the model processing result, the first electric signal, the second electric signal and/or the output result of the nonlinear activation function. The programmable optical structure calculates linear operation, so that the nonlinear activation function related to the target AI model can be calculated by using software, and the result calculated by the software is used by using a domain controller.
Based on the above embodiment, the constituent structure of the signal transmitter can refer to fig. 5. Referring to fig. 5, the signal transmitter includes: a laser and an array of optical fibers connected to the laser. In particular, a laser is used to generate a laser signal. The optical fiber array is used for transmitting laser signals to the electro-optical modulator array in a preset number of input paths. That is, a corresponding number of input paths are provided in the fiber array, and the laser signal generated by the laser can be divided into several optical signals and transmitted to the electro-optical modulator array. One input path corresponds to one electro-optic modulator, and thus multiple optical input paths correspond to one array of electro-optic modulators. Accordingly, each electro-optic modulator may adjust the intensity of its corresponding optical signal. Accordingly, the photodetector array includes a plurality of photodetectors, one for converting the optical signal of its corresponding path.
Referring to fig. 5, an embodiment of the present application discloses another design architecture of a computing chip. As shown in FIG. 5, the processor core of the computing chip is divided into an electrical domain and an optical domain. The electric domain is a CMOS microelectronic chip and comprises a logic control module, a storage module, a digital-to-analog conversion module, a driving module and the like. The digital-to-analog conversion module is used for performing D/A conversion or A/D conversion. The optical domain is a silicon optical chip of an integrated optical waveguide and an optical modulator, and mainly bears linear multiplication operation of matrixes and vectors, and the linear multiplication operation comprises an electro-optical modulator array, a programmable optical matrix (namely a programmable optical structure) and a photoelectric detector array. The electric domain part and the optical domain part are correspondingly connected through a bump array by adopting a flip-chip process on the package.
The programmable optical matrix is a cascaded optical modulation array (namely a cascaded MZI structure), and can perform linear multiplication operation of a two-dimensional weight matrix and a one-dimensional input vector. Specifically, the electronic chip converts the optimized model weight matrix (i.e., the second electrical signal) into a voltage signal (i.e., the second analog signal) through D/a, and drives the optical modulation array to perform intensity modulation on the laser signal in the waveguide using the voltage signal, that is: adjusting phase shifters in cascaded MZI structures to modulate phase intensity such that the optical modulation array implements a model weight matrixW. Wherein the model weight matrixWAnd training the model by using computer software. In the process of training the network model, the deviation between the network output and an actual value (label) can be calculated by using a cross entropy loss function, then the difference value is iteratively optimized by using an error back propagation algorithm, and each weight matrix of the network is obtained through training. And the trained weight matrix is loaded on an optical modulation array of the computing chip in an analog signal mode so as to carry out model inference application by using the optical modulation array.
The integrated silicon optical chip is connected with a peripheral unique laser light source through a coupling optical fiber array. When the computing chip works, the laser continuously outputs continuous laser signals for the chip.
The microelectronic chip converts preset input data (namely a first electric signal) into a voltage signal (namely a first analog signal) through the D/A conversion module, the voltage signal is used for driving the electro-optical modulator array to weaken the intensity of incident light, and the multiple paths of incident light signals are encoded into one-dimensional input column vectorsx(for representing model input data). This step is entered from the digital domainThe analog domain.
The optical signals from the electro-optical modulator array are input into the programmable optical matrix, and the one-dimensional vector result is calculated and output by the programmable optical matrix, and is received by the photoelectric detector array and converted into multiple current signals. The current signal is converted into a voltage signal through a trans-impedance amplifier, converted into a digital signal through A/D (analog/digital) and stored in a microelectronic chip. This step goes from the analog domain back to the digital domain.
According to the process, primary matrix and vector can be completed at the speed of lightxLinear multiplication of (2):a=W·x。arepresenting the output of the programmable optical matrix. In the above calculation procedure, the nonlinear activation function can be implemented by using materials and structures that can satisfy the conditions of the activation function, such as a saturable absorber, a bistable state, and a Kerr effect of MZI. Of course, in order to simplify the structural complexity of the photon computing chip, the nonlinear activation function can be realized in the electrical domain by using a software algorithm. In a specific embodiment, the nonlinear activation function may also be implemented using an optical structure, such as a saturable absorber structure, or a bistable or MZI structure.
Such as the computational chip shown in fig. 5, which can implement a matrix-vector linear multiplication operation. Specifically, the computing chip encodes data by modulating the amplitude or phase of the laser pulse, and the data is continuous real numbers. The computing chip can also use the traditional full-connection neural network architecture, and the network also comprises an input layer, a plurality of hidden layers and an output layer in principle. Generally, each layer of the network comprises a plurality of neuron nodes, and the neuron nodes of each layer are connected through a weight matrix to perform linear matrix multiplication. The neuron node values are de-linearized using a non-linear activation function before being input to the next layer. The calculating chip provided by the embodiment can use an optical structure to realize linear and nonlinear operation functions of neuron node calculation, weighting, activation and the like in the software sense.
It follows that programmable optical matrices are a key component of computing chips. The implementation principle of the programmable optical matrix is described below.
Mathematically, a two-dimensional real matrix of arbitrary dimensionsW(m,n) All can be decomposed into three matrixes by a singular value decomposition methodU、ΣAndV Tproduct of (i) i.e.W=U(m)ΣV T(n). Wherein the content of the first and second substances,U(m) Is composed ofm×mA dimensional unitary matrix is formed by a plurality of unitary matrices,Σfor diagonal non-negative realm×nThe diagonal matrix is then maintained,V T(n) Is composed ofn×nThe dimensional unitary matrix isV(n) The conjugate transpose of (c). Therefore, the model weight matrix can be decomposed into the product of two unitary matrices and a diagonal matrix, and then the two unitary matrices and the diagonal matrix are respectively realized by using an optical device, so that the programmable optical structure for realizing the model weight matrix can be obtained. If the model weight matrix is decomposed intoU、ΣAndV Tis then taken as the input optical signalV T(n)、Σ、U(m) And respectively obtaining corresponding model processing results by corresponding optical structures.
How to implement the weight matrix by adjusting the phase shifters in the MZI structure is described belowW. As can be seen from fig. 3, when the input optical signal propagates from left to right, a column vector is outputx out=U MZI·x inThen the transmission matrix of a single MZI structure can be expressed as:
for any onemThe dimensional unitary matrix can be realized by cascading a single MZI structure. In a mathematical sense, can be pairedmThe unitary matrix performs continuous dimensionality reduction transformation in two-dimensional subspace, i.e. firstlymDimension of the dimensional unitary matrix is reduced tom1 dimension, then repeating the process continuously, and finally reducing the dimension to two dimensions. Because the matrix operation corresponding to a single MZI structure is two-dimensional, the matrix operation is carried outmWhen factoring a dimensional unitary matrix, a two-dimensional matrix corresponding to a single MZI structure needs to be expanded into a two-dimensional matrix corresponding to a single MZI structuremDimension waiting matrixT qp 。
mDimension waiting matrixT qp As follows:
such asT qp As shown in the drawings, the above-described,pandqthe input port number of the optical matrix representing the two input ports into the MZI structure (see FIG. 4), 0 ≦p<q≤m. As shown in FIG. 4, the optical matrix input portp=1 andqconverging two optical signals of =6T 61In a corresponding certain MZI structure, the structure,T 61also shown is the spreading matrix of this MZI structure.
ByT qp As can be seen,T qp transformed from an identity matrix, but of the firstpLine ofpReplacement of column elements byu 11Of 1 atpLine ofqReplacement of column elements byu 12Of 1 atqLine ofpReplacement of column elements byu 21Of 1 atqLine ofqReplacement of column elements byu 22. The other diagonal elements are all 1, and the off-diagonal elements are all 0. That is, inT qp When the matrix participates in operation, only signals entering the ports of the corresponding MZI structures participate in change. The rest signals do not participate and correspond toT qp The value of (d) is a diagonal matrix. By controlling the value of the phase shifter in each MZI structure, a corresponding phase shifter is obtainedT qp 。
For any onemDimensional unitary matrix by right multiplicationT m m-(1), T m m-(2), … , T m2, T m1The method can be as followsmDimension of the dimensional unitary matrix is reduced tom-1, satisfying:
is provided withR(m)=T m m-(1) T m m-(2)…T m2 T m1Then satisfyU(m)R(m)R(m-1)…R(2)=D，DIs a diagonal matrix modulo 1. At this timemThe dimensional unitary matrix can be expressed as:U(m)=DR T (m)R T (m-1)…R T (2) Accordingly, the method can be realized by cascading MZI structuresmA dimensional unitary matrix.
Accordingly, the number of the first and second electrodes,nthe dimensional unitary matrix can be expressed as:V T(n)=DR T (n)R T (n-1)…R T (2) Accordingly, the method can be realized by cascading MZI structuresnA dimensional unitary matrix.
In addition, diagonal matrixΣOnly control of each diagonal element is required and can therefore be achieved with an optical attenuator based on the MZI structure.mA parallel optical attenuatormAnd programming a dimension diagonal matrix. The structure of a single optical attenuator is shown in fig. 6, and the input and output of the drop are blocked as shown in fig. 6. When the input light intensity isEWhile the output light intensity is attenuated toEcos2θ。
In accordance with the principles described above, a 6-input 6-output programmable optical structure as shown in FIG. 2 can be implemented. If the simplest 2-layer fully-connected neural network is constructed for realizing the classification and identification functions, the whole network comprises an input layer, a hidden layer and an output layer. The input layer is a feature vector extracted from the target to be classified, and contains 6 elements. The two weight matrices between the layers are shown in fig. 7, and fig. 7 only illustrates the two weight matrices, and the layers are not shown.
Since the eigenvectors of the input layer contain 6 elements, two 6-dimensional weight matrices are requiredWAnd two non-linear activation functionsfThe method is used for completing two times of matrix-vector linear multiplication operations and two times of nonlinear operations. Wherein each weight matrixWDecomposition into two 6-dimensional unitary matrixes and one 6-dimensional diagonal matrix through singular valuesΣ. Unitary and diagonal matrices by photonicsAnd (5) computing chip operation.
Current unitary matrix decomposition methods include the Reck trigonometric decomposition and the Clements rectangular decomposition. In this embodiment, taking triangle decomposition as an example, the 6-dimensional unitary matrix can be decomposed into:
specifically, the topology of the 6-dimensional unitary matrix is shown in fig. 4. Input laser signal fromT 65 TStart to propagateT 65 TIs thatT 65The transposition of (1) is equivalent to MZI reversal connection, and at the moment, an optical signal firstly passes through an external phase shifterR φ. By loading different phase values, sequential implementation can be realizedV 1 T(6)，U 1(6)，V 2 T(6) AndU 2(6). Nonlinear activation functionfThe operation is realized by software. Wherein the content of the first and second substances,fmay be a RelU, Sigmoid function, etc. For the classification identification network, the network is finally output through a normalized exponential function softmax.
In order to simplify the structure of the photon computing chip, only one computing structure corresponding to the 6-dimensional unitary matrix is arranged in the computing chip, and the structure is repeatedly used subsequently to finish the operation. Fig. 2 contains a computation structure corresponding to a 6-dimensional unitary matrix (i.e., the cascaded MZI structure shown in fig. 2) and a computation structure corresponding to a 6-dimensional diagonal matrix (i.e., the parallel optical attenuator structure shown in fig. 2). According to the structure shown in fig. 2, to complete a linear matrix-vector multiplication operation, a photon calculation chip needs to be processed twice.
The first calculation is as follows: editing phase shifter to realize unitary matrix structure of photon computing chipV 1 T(6) Implementation of diagonal matrix structureΣ 1. Second, the electro-optic modulator array encodes the input eigenvectorsxThe laser signal of (1), start the calculation. And then, collecting a first calculation result by using the photoelectric detector array, and temporarily storing the first calculation result in the memory.
And (3) calculating for the second time: editing phase shifter to make photon computing chip unitary matrix structure realNow thatU 1(6) The diagonal matrix structure implements a unit diagonal matrix. And secondly, loading the temporary storage data to the electro-optical modulator end to prepare for second calculation. Then, collecting the second calculation result by using a photoelectric detector, temporarily storing the second calculation result in an internal memory, and finishing a linear multiplication operation of a weight matrix and an input eigenvector at the momentW (1) x. Finally, completing the nonlinear operation at the electronic chip endz (1)=f(W (1) x) And the result is temporarily stored in the memory.
At this time, data enters the hidden layer from the input layer. The 2-layer fully-connected neural network needs to finish the operation again and carries out weight matrix-vector linear multiplication operationW (2)z(1)Finally, the channel with the maximum output power is identified by the photoelectric detector, namely, the object class is identified. If quantitative analysis is required, the softmax function can be calculated in software. At this time, data enters the output layer from the hidden layer.
It can be seen that the programmable optical structures in the photonic computing chip are suitable for operations between matrices and vectors. For the operation between matrixes frequently involved in the neural network, a certain matrix can be firstly split into a plurality of vectors in an electronic chip, then the matrix is sequentially operated with another matrix in a photon calculation chip to obtain a plurality of output vectors, and finally the output matrix is synthesized in the electronic chip. Of course, optical interconnection technology can also be used to operate multiple vectors in parallel in different photon computing chips. By using the photon computing chip provided by the embodiment, the running speed of matrix multiplication in the neural network can be increased, and the photon computing chip has the characteristics of low power consumption, high flux and low time delay.
In the following, a computing system provided by an embodiment of the present application is described, and a computing system described below and a computing chip described above may be referred to with each other.
The present embodiments provide a computing system, comprising: a plurality of the computing chips of any of the above embodiments, each computing chip connected using optical interconnect technology.
The embodiment provides a computing system, which can process complex operation by using an optical structure, thereby improving the processing capacity of hardware for the complex operation.
In the following, a data processing method provided by an embodiment of the present application is introduced, and a data processing method described below and a computing chip described above may be referred to each other.
Referring to fig. 8, the present embodiment provides a data processing method applied to the computing chip in any of the above embodiments, including:
and S801, transmitting a laser signal by using a signal transmitter.
S802, controlling an electro-optical modulator array by using a domain controller to convert the laser signal into a target optical signal; the target light signal is used to represent input data of the target AI model.
S803, calculating the target optical signal by using the programmable optical structure, and outputting an optical calculation result; the programmable optical structure implements a model weight matrix with a target AI model.
S804, photoelectric conversion is carried out on the light calculation result by using the photoelectric detector array, and a model processing result of the target AI model aiming at the input data is obtained.
In this embodiment, reference may be made to corresponding contents disclosed in the foregoing embodiments, which are not described herein again.
The data processing method provided by the computing chip of the embodiment can process complex operation by using an optical structure, so that the processing capacity of hardware for the complex operation is improved.
References in this application to "first," "second," "third," "fourth," etc., if any, are intended to distinguish between similar elements and not necessarily to describe a particular order or sequence. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, or apparatus.
It should be noted that the descriptions in this application referring to "first", "second", etc. are for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present application.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of readable storage medium known in the art.
The principle and the implementation of the present application are explained herein by applying specific examples, and the above description of the embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (11)
1. A computing chip, comprising:
a signal transmitter for transmitting a laser signal;
the electro-optical modulator array is used for converting the laser signal into a target optical signal under the control of a domain controller in the computing chip; the target light signal is used for representing input data of a target AI model;
the programmable optical structure is used for realizing a model weight matrix of the target AI model, and is used for calculating the target optical signal and outputting an optical calculation result;
and the photoelectric detector array is used for performing photoelectric conversion on the light calculation result to obtain a model processing result of the target AI model aiming at the input data.
2. The computing chip of claim 1, wherein the domain controller comprises:
the digital-to-analog conversion module is used for converting the first electric signal into a first analog signal;
and the logic control circuit is used for transmitting the first analog signal to the electro-optical modulator array so that the electro-optical modulator array converts the laser signal into the target optical signal according to the first analog signal.
3. The computing chip of claim 2, wherein the array of electro-optic modulators is specifically configured to:
and modulating the light intensity of the laser signal according to the first analog signal to obtain the target optical signal.
4. The computing chip of claim 2,
the digital-to-analog conversion module is also used for converting the second electric signal into a second analog signal;
the logic control circuit is further configured to transmit the second analog signal to the programmable optical structure, so that the programmable optical structure adjusts the phase shifter in itself according to the second analog signal.
5. The computing chip of claim 4, wherein the domain controller further comprises:
and the driving module is used for driving the logic control circuit to transmit the second analog signal to the programmable optical structure so that the programmable optical structure adjusts the phase shifter in the programmable optical structure according to the second analog signal.
6. The computing chip of claim 5, wherein the domain controller further comprises:
a storage module for storing the model processing result, the first electrical signal, the second electrical signal and/or the output result of the nonlinear activation function.
7. The computing chip of claim 1, wherein the signal transmitter comprises:
a laser for generating the laser signal;
and the optical fiber array is connected with the laser and is used for transmitting the laser signals to the electro-optical modulator array by a preset number of input paths.
8. The computing chip of claim 1, further comprising:
saturable absorber structures, or bistable or MZI structures for realizing nonlinear activation functions.
9. The computing chip of any of claims 1 to 8, wherein the programmable optical structure comprises: a cascaded MZI structure and a parallel optical attenuator structure.
10. A computing system, comprising: a plurality of computing chips as claimed in any one of claims 1 to 9, each connected by optical interconnect technology.
11. A data processing method applied to the computing chip according to any one of claims 1 to 9, comprising:
transmitting a laser signal by using a signal transmitter;
controlling an electro-optical modulator array by using a domain controller to convert the laser signal into a target optical signal; the target light signal is used for representing input data of a target AI model;
calculating the target optical signal by using a programmable optical structure, and outputting an optical calculation result; the programmable optical structure implements a model weight matrix with the target AI model;
and performing photoelectric conversion on the light calculation result by using a photoelectric detector array to obtain a model processing result of the target AI model aiming at the input data.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210417937.3A CN114520694A (en) | 2022-04-21 | 2022-04-21 | Computing chip, system and data processing method |
| PCT/CN2022/118103 WO2023201970A1 (en) | 2022-04-21 | 2022-09-09 | Computing chip, system, and data processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210417937.3A CN114520694A (en) | 2022-04-21 | 2022-04-21 | Computing chip, system and data processing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114520694A true CN114520694A (en) | 2022-05-20 |
Family
ID=81600299
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210417937.3A Pending CN114520694A (en) | 2022-04-21 | 2022-04-21 | Computing chip, system and data processing method |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114520694A (en) |
| WO (1) | WO2023201970A1 (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116777727A (en) * | 2023-06-21 | 2023-09-19 | 北京忆元科技有限公司 | Integrated memory chip, image processing method, electronic device and storage medium |
| WO2023201970A1 (en) * | 2022-04-21 | 2023-10-26 | 苏州浪潮智能科技有限公司 | Computing chip, system, and data processing method |
| WO2023226423A1 (en) * | 2022-05-24 | 2023-11-30 | 苏州元脑智能科技有限公司 | Auxiliary chip design method and apparatus, device and nonvolatile storage medium |
| WO2024012335A1 (en) * | 2022-07-15 | 2024-01-18 | 南京光智元科技有限公司 | Computing system and method for processing photonic computing result |
| WO2024021659A1 (en) * | 2022-07-27 | 2024-02-01 | 董毅博 | Three-dimensional photonic chip architecture based on vcsel array, application, and method for calculating structure of dnns |
| CN117891023A (en) * | 2024-03-15 | 2024-04-16 | 山东云海国创云计算装备产业创新中心有限公司 | Photonic chip, heterogeneous computing system, precision adjusting method and product |
| CN117891023B (en) * | 2024-03-15 | 2024-05-31 | 山东云海国创云计算装备产业创新中心有限公司 | Photonic chip, heterogeneous computing system, precision adjusting method and product |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110197277A (en) * | 2019-05-13 | 2019-09-03 | 浙江大学 | Realize the optical neural network method of number identification |
| CN110503196A (en) * | 2019-08-26 | 2019-11-26 | 光子算数(北京)科技有限责任公司 | A kind of photon neural network chip and data processing system |
| CN111898741A (en) * | 2020-08-04 | 2020-11-06 | 上海交通大学 | Lithium niobate-based on-chip cascaded MZI reconfigurable quantum network |
| CN112232504A (en) * | 2020-09-11 | 2021-01-15 | 联合微电子中心有限责任公司 | Photon neural network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114520694A (en) * | 2022-04-21 | 2022-05-20 | 苏州浪潮智能科技有限公司 | Computing chip, system and data processing method |
-
2022
- 2022-04-21 CN CN202210417937.3A patent/CN114520694A/en active Pending
- 2022-09-09 WO PCT/CN2022/118103 patent/WO2023201970A1/en unknown
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110197277A (en) * | 2019-05-13 | 2019-09-03 | 浙江大学 | Realize the optical neural network method of number identification |
| CN110503196A (en) * | 2019-08-26 | 2019-11-26 | 光子算数(北京)科技有限责任公司 | A kind of photon neural network chip and data processing system |
| CN111898741A (en) * | 2020-08-04 | 2020-11-06 | 上海交通大学 | Lithium niobate-based on-chip cascaded MZI reconfigurable quantum network |
| CN112232504A (en) * | 2020-09-11 | 2021-01-15 | 联合微电子中心有限责任公司 | Photon neural network |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023201970A1 (en) * | 2022-04-21 | 2023-10-26 | 苏州浪潮智能科技有限公司 | Computing chip, system, and data processing method |
| WO2023226423A1 (en) * | 2022-05-24 | 2023-11-30 | 苏州元脑智能科技有限公司 | Auxiliary chip design method and apparatus, device and nonvolatile storage medium |
| WO2024012335A1 (en) * | 2022-07-15 | 2024-01-18 | 南京光智元科技有限公司 | Computing system and method for processing photonic computing result |
| WO2024021659A1 (en) * | 2022-07-27 | 2024-02-01 | 董毅博 | Three-dimensional photonic chip architecture based on vcsel array, application, and method for calculating structure of dnns |
| CN116777727A (en) * | 2023-06-21 | 2023-09-19 | 北京忆元科技有限公司 | Integrated memory chip, image processing method, electronic device and storage medium |
| CN116777727B (en) * | 2023-06-21 | 2024-01-09 | 北京忆元科技有限公司 | Integrated memory chip, image processing method, electronic device and storage medium |
| CN117891023A (en) * | 2024-03-15 | 2024-04-16 | 山东云海国创云计算装备产业创新中心有限公司 | Photonic chip, heterogeneous computing system, precision adjusting method and product |
| CN117891023B (en) * | 2024-03-15 | 2024-05-31 | 山东云海国创云计算装备产业创新中心有限公司 | Photonic chip, heterogeneous computing system, precision adjusting method and product |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023201970A1 (en) | 2023-10-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114520694A (en) | Computing chip, system and data processing method | |
| Zhou et al. | Photonic matrix multiplication lights up photonic accelerator and beyond | |
| CN112232503B (en) | Computing device, computing method, and computing system | |
| Qu et al. | Inverse design of an integrated-nanophotonics optical neural network | |
| CN112232504A (en) | Photon neural network | |
| WO2021227767A1 (en) | All-optical diffractive neural network and system implemented on optical waveguide and/or optical chip | |
| Bai et al. | Towards silicon photonic neural networks for artificial intelligence | |
| JP7115691B2 (en) | Training for Photonic Reservoir Computing Systems | |
| JP6784644B2 (en) | Optical signal processing circuit | |
| EP4193306A1 (en) | Coherent photonic computing architectures | |
| CN111753977B (en) | Optical neural network convolution layer chip, convolution calculation method and electronic equipment | |
| CN112506265A (en) | Light calculation device and calculation method | |
| WO2023109065A1 (en) | Method and system for implementing recurrent neural network, and electronic device and storage medium | |
| Youngblood | Coherent photonic crossbar arrays for large-scale matrix-matrix multiplication | |
| CN113627605A (en) | Optical interference unit configuration method and device of photonic neural network and storage medium | |
| CN112101540A (en) | Optical neural network chip and calculation method thereof | |
| WO2023005084A1 (en) | Optical circuit building method, optical circuit, and optical signal processing method and apparatus | |
| WO2021170107A1 (en) | Optical computing device and system, and computing method | |
| TW202220401A (en) | Systems and methods for utilizing photonic degrees of freedom in a photonic processor | |
| JP2019148715A (en) | Optical signal processing device | |
| US20230351167A1 (en) | Frequency multiplexed photonic neural networks | |
| CN112232487B (en) | Optical neural network chip and calculation method thereof | |
| CN116888554A (en) | Optical vector multiplier | |
| JP6829700B2 (en) | Optical signal processor | |
| Chen et al. | Optical logic operations using a meshed Mach-Zehnder interference array |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220520 |
|
| RJ01 | Rejection of invention patent application after publication |