CN114519403B - Optical diagram neural classification network and method based on-chip diffraction neural network - Google Patents
Optical diagram neural classification network and method based on-chip diffraction neural network Download PDFInfo
- Publication number
- CN114519403B CN114519403B CN202210411556.4A CN202210411556A CN114519403B CN 114519403 B CN114519403 B CN 114519403B CN 202210411556 A CN202210411556 A CN 202210411556A CN 114519403 B CN114519403 B CN 114519403B
- Authority
- CN
- China
- Prior art keywords
- graph
- optical
- neural network
- structures
- diffraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Neurology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Optical Integrated Circuits (AREA)
Abstract
The application relates to the technical field of optical neural network computing, in particular to an optical diagram neural classification network and method based on an on-chip diffraction neural network, wherein the network comprises: the optical diagram feature extraction module is used for encoding the input node attribute information of the diagram structure onto an input optical signal of an input waveguide, extracting an output optical signal of an output waveguide after being transmitted through the on-chip diffraction neural network, and obtaining diagram feature information by utilizing the output optical signal; the optical graph characteristic aggregation module is used for aggregating graph characteristic information of a plurality of graph structures; and the classification module is used for classifying the multiple graph structures according to the aggregated graph feature information of the multiple graph structures to obtain classification results of the multiple graph structures. Therefore, the problem that the conventional optical neural network can only process regular data structures in the forms of vectors, matrixes and the like but can not process non-Euclidean space data structures such as graph structures and the like is solved, and graph structure data such as a social network, a paper mutual-introduction network and the like can be processed by utilizing the on-chip integrated optical neural network.
Description
Technical Field
The application relates to the technical field of optical neural network computing, in particular to an optical diagram neural classification network and method based on an on-chip diffraction neural network.
Background
Deep learning techniques have made tremendous progress in a wide range of Artificial Intelligence (AI) applications, including computer vision, speech recognition, natural language processing, autonomous driving, biomedical science, and so on. The core of the method is to learn complex features from big data by using a multilayer neural network under the driving of continuous development of comprehensive electronic computing platforms such as a Central Processing Unit (CPU), a Graphic Processing Unit (GPU), a Tensorial Processor (TPU), a Field Programmable Gate Array (FPGA), and the like. However, as the demand for artificial intelligence development increases, electronic computing performance is approaching its physical limits, and running large-scale deep neural models can bring about enormous energy consumption. Photon calculation takes photons as a calculation medium, and utilizes the advantages of high parallelism, low power consumption, high signal processing speed and the like to construct a photon neural network, which is a research hotspot in the field of photon calculation in recent years.
In recent years, many optical neural network structures for performing intelligent reasoning tasks, such as diffraction depth neural networks, optical interference neural networks, photon pulse neural networks, have been proposed, and have made great progress in tasks such as speech recognition, image classification, and the like. However, the existing optical neural networks can only process regular data structures in the form of vectors, matrixes and the like, but cannot process data structures in non-euclidean space such as graph structures and the like. However, the data analyzed in various scientific fields surpass this euclidean space category. As a typical representation, graph data structures encode rich relationships (edges) between nodes in complex systems, ubiquitous in the real world, from social networks to chemical molecules. The Neural network of electronic Graph (GNN) has been developed as a broad and novel method that can learn local node features and Graph topology features well to perform representation learning on Graph structure data. In these models, based on the GNN of message delivery, messages are generated by extracting node features through a trainable transformation matrix, and then messages are delivered to each neighboring node to generate topological features of the graph, which has been significantly successful in molecular property prediction, drug discovery, bone-based human behavior recognition, spatio-temporal prediction, etc. due to its flexibility. However, how to efficiently utilize optical computations to aid graph-based machine learning remains to be explored.
Disclosure of Invention
The application provides an optical diagram neural classification network and method based on an on-chip diffraction neural network, which can realize high-speed, low-power-consumption and large-scale diagram structure data processing, so that the all-optical neural network can better complete various types of machine learning tasks.
An embodiment of a first aspect of the present application provides an optical map neural classification network based on an on-chip diffraction neural network, including: the optical diagram feature extraction module is used for encoding the input node attribute information of the diagram structure onto an input optical signal of an input waveguide, extracting an output optical signal of an output waveguide after being transmitted through the on-chip diffraction neural network, and obtaining diagram feature information by using the output optical signal; the optical graph characteristic aggregation module is used for aggregating graph characteristic information of a plurality of graph structures; and the classification module is used for classifying the multiple graph structures according to the aggregated graph feature information of the multiple graph structures to obtain the classification results of the multiple graph structures.
Optionally, in an embodiment of the present application, the optical map feature extraction module includes: an optical node property input unit, configured to encode input node property information of the graph structure onto the light intensity or phase of the input optical signal through a modulator; the on-chip diffraction calculating unit comprises an on-chip diffraction neural network which is arranged in an integrated mode and is used for extracting the light intensity or the phase of the output light signal and obtaining the graph characteristic information based on the light intensity or the phase of the output light signal; and the optical node characteristic aggregation unit is used for splicing the graph characteristic information of a plurality of adjacent input nodes in the graph structure through the waveguide and the coupler.
Optionally, in one embodiment of the present application, the modulator comprises an electro-optic modulator, an acousto-optic modulator, or a thermo-optic modulator.
Optionally, in an embodiment of the present application, the on-chip diffraction neural network is formed by connecting multiple layers of diffraction lines through optical diffraction, the shape, size, and period of the diffraction lines are set, and an amplitude modulation coefficient and a phase modulation coefficient of light by the on-chip diffraction neural network are determined.
Optionally, in an embodiment of the present application, the classification module includes: and the optical neural network unit is used for inputting the graph characteristic information of the graph structures into the optical neural network through a waveguide, and classifying the graph structures by using the optical neural network to obtain the classification results of the graph structures.
Optionally, in an embodiment of the present application, the classification module includes: a photodetector for photoelectrically converting the aggregated pattern feature information of the plurality of pattern structures; and the electronic neural network unit is used for classifying the multiple graph structures according to the graph feature information of the multiple graph structures after photoelectric conversion to obtain the classification results of the multiple graph structures.
Optionally, in an embodiment of the present application, the method further includes: the processing module is used for simulating the electromagnetic field of the optical diagram neural classification network structure, obtaining the structural parameters of the optical diagram neural network, establishing a forward propagation numerical model according to the structural parameters, training the parameters of each modulation layer of the diffraction calculation unit by using an error back propagation algorithm, and establishing the optical diagram neural network structure according to the training result.
The embodiment of the second aspect of the application provides an optical diagram neural network classification method based on an on-chip diffraction neural network, which comprises the following steps: encoding input node attribute information of a graph structure to an input optical signal of an input waveguide, extracting an output optical signal of an output waveguide after being transmitted through an on-chip diffraction neural network, and obtaining graph characteristic information by using the output optical signal; and aggregating the graph feature information of the multiple graph structures, and classifying the multiple graph structures according to the aggregated graph feature information of the multiple graph structures to obtain classification results of the multiple graph structures.
Optionally, in an embodiment of the present application, the encoding input node attribute information of the graph structure onto an input optical signal of an input waveguide, extracting an output optical signal of an output waveguide after propagating through an on-chip diffraction neural network, and obtaining graph feature information by using the output optical signal includes: encoding the input node attribute information of the graph structure to the light intensity or phase of the input optical signal, extracting the light intensity or phase of the output optical signal, obtaining the graph characteristic information based on the light intensity or phase of the output optical signal, and splicing the graph characteristic information of a plurality of adjacent input nodes in the graph structure.
Optionally, in an embodiment of the present application, the classifying the multiple graph structures according to the aggregated graph feature information of the multiple graph structures to obtain a classification result of the multiple graph structures includes: inputting the graph feature information of the graph structures into an optical neural network, and classifying the graph structures by using the optical neural network to obtain classification results of the graph structures; and/or performing photoelectric conversion on the aggregated graph feature information of the multiple graph structures, and classifying the multiple graph structures according to the graph feature information of the multiple graph structures after the photoelectric conversion to obtain classification results of the multiple graph structures.
An embodiment of a third aspect of the present application provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to perform the method for classifying an optical pattern neural network based on an on-chip diffraction neural network as described in the above embodiments.
A fourth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, where the program is executed by a processor to execute the method for classifying an optical pattern neural network based on an on-chip diffraction neural network according to the foregoing embodiments.
The embodiment of the application has the following beneficial effects:
different from the conventional optical neural network which can only process vector or matrix forms, the method utilizes the on-chip integrated diffraction neural network to perform feature extraction on node attributes coded on light, generates messages of each node, realizes the transmission and aggregation of the messages by utilizing waveguide coupling, physically constructs topological connection of nodes of a graph structure, and provides a system and a method for processing non-Euclidean space data structures such as the graph structure and the like by utilizing the on-chip integrated optical neural network.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic structural diagram of an optical diagram neural classification network based on an on-chip diffraction neural network according to an embodiment of the present application;
FIG. 2 is a schematic structural diagram of an optical neural classification network based on an on-chip diffraction neural network according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an optical structure of an optical neural classification network based on an on-chip diffraction neural network according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a graph feature extraction module for a node with a high attribute dimension according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a diffraction line structure of an on-chip diffraction calculation unit provided according to an embodiment of the present application;
fig. 6 is a schematic diagram of FDTD (Finite-Difference Time-Domain) electromagnetic field simulation in which the amplitude and phase modulation coefficient of each pixel (slot) on the diffraction line vary with the slot width according to an embodiment of the present application;
FIG. 7 is a graph of amplitude and phase modulation factor for each pixel (slot) on a diffraction line as a function of slot width provided in accordance with an embodiment of the present application;
fig. 8 is a schematic diagram of an overall structure of an on-chip diffraction computing unit and a schematic diagram of FDTD electromagnetic field simulation provided according to an embodiment of the present application;
FIG. 9 is a schematic diagram of an optical field distribution of an input waveguide provided in accordance with an embodiment of the present application;
FIG. 10 is a graph of coupling coefficient versus incident light angle for an output waveguide provided in accordance with an embodiment of the present application;
FIG. 11 is a graph comparing errors of numerical modeling and FDTD physics simulation provided in accordance with an embodiment of the present application;
FIG. 12 is a graph comparing the classification result of the random block model for realizing synthesis with the performance of the conventional Multilayer Perceptron (MLP) and the electronic GNN (PPRGo), according to the embodiment of the present application;
FIG. 13 is a graph node classification comparing a graph node with a conventional multi-layer perceptron (MLP), electronic GNN (PPRGo) performance for a reference data set implementing Cora-ML, Citeser, Amazon Photoo, etc. according to an embodiment of the present application;
FIG. 14 is a flow chart of a method for optical map neural network classification based on an on-chip diffractive neural network according to an embodiment of the present application;
fig. 15 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
The optical diagram neural classification network and method based on the on-chip diffraction neural network according to the embodiment of the application are described below with reference to the attached drawings. Aiming at the problem that the existing optical neural network mentioned in the center of the background art can only process regular data structures in the forms of vectors, matrixes and the like but can not process data structures in non-Euclidean spaces such as graph structures and the like, the application provides an optical graph neural classification network based on an on-chip diffraction neural network, the diffraction computing unit (diffraction neural network) integrated on the chip is used for carrying out feature extraction on node attributes coded to light to generate messages of each node, waveguide coupling is used for realizing the transmission and aggregation of the messages, the topological connection of the nodes of the graph structures is physically constructed, and a system and a method for processing the data structures in the non-Euclidean spaces such as the graph structures and the like by using the optical neural network integrated on the chip are provided. Therefore, high-speed and low-power-consumption large-scale graph structure data processing can be realized, the all-optical neural network can better complete various types of machine learning tasks, excellent performance is achieved in the node and graph classification tasks on the reference data set, and a new direction is opened for designing an integrated photonic circuit for efficiently processing large-scale graph structure data by utilizing deep learning.
Specifically, fig. 1 is a schematic structural diagram of an optical diagram neural classification network based on an on-chip diffraction neural network according to an embodiment of the present application.
As shown in fig. 1, the optical diagram neural classification network 10 based on the on-chip diffraction neural network includes: an optical map feature extraction module 100, an optical map feature aggregation module 200, and a classification module 300.
Specifically, the optical diagram feature extraction module 100 is configured to encode the input node attribute information of the graph structure onto an input optical signal of an input waveguide, extract an output optical signal of an output waveguide that is propagated through the on-chip diffraction neural network, and obtain the diagram feature information by using the output optical signal. And an optical diagram feature aggregating module 200, configured to aggregate diagram feature information of a plurality of diagram structures. The classification module 300 is configured to classify the multiple graph structures according to the aggregated graph feature information of the multiple graph structures, so as to obtain classification results of the multiple graph structures. The network 10 enables processing of graph structure data for social networks, paper mutual-reference networks, etc. using on-chip integrated optical neural networks.
Further, the optical graph feature extraction module is composed of optical node attribute input, optical node feature extraction, optical node feature aggregation and the like. And the optical diagram feature aggregation module is used for splicing and aggregating the output features of the optical diagram feature extraction modules. And the classification module consists of an electronic neural network or an optical neural network including a diffraction neural network. The specific compositions are illustrated by the following examples.
In an embodiment of the present application, the optical map feature extraction module 100 includes: an optical node property input unit 101 is configured to encode input node property information of the graph onto the light intensity or phase of the input optical signal via the modulator. The on-chip diffraction calculating unit 102 includes an integrally arranged on-chip diffraction neural network, and is configured to extract the light intensity or phase of the output optical signal, and obtain the pattern feature information based on the light intensity or phase of the output optical signal. And the optical node characteristic aggregation unit 103 is used for splicing the graph characteristic information of a plurality of adjacent input nodes in the graph structure through the waveguide and the coupler.
It is understood that the optical map feature extraction module 100 is composed of optical node attribute input, optical node feature extraction (message passing), optical node feature aggregation, and the like, as shown in fig. 2, 3, and 4. The node attribute input is realized by encoding the attribute of an input node to the light intensity or phase in an input waveguide by an electro-optic, acousto-optic or thermo-optic modulator such as a Mach-Zehnder interferometer. Optical node feature extraction (generating a message to be communicated) is done by the on-chip diffraction computation unit, the extracted features being reflected in the light intensity or phase in the output waveguide. The features (transmitted messages) extracted by adjacent nodes are connected by the waveguide, and the complex addition of the features is realized by the coupler, so that the feature aggregation of the optical nodes is realized.
In the embodiment of the application, the on-chip diffraction neural network is formed by connecting multiple layers of diffraction lines through optical diffraction, the shape, the size and the period of the diffraction lines are set, and the amplitude modulation coefficient and the phase modulation coefficient of the on-chip diffraction neural network on light are determined.
Specifically, the feature extraction is completed by a diffraction calculation unit composed of diffraction lines of multi-layer modulation light field propagation etched On an insulating Silicon wafer (SOI), an On-chip diffraction neural network of the On-chip diffraction calculation unit is composed of a plurality of diffraction lines etched On the SOI at certain intervals, each pixel On the diffraction lines is a groove etched and filled with Silicon dioxide and other substances, the shape, the size and the period of the groove determine the amplitude modulation coefficient and the phase modulation coefficient of the pixel On light, each pixel is an optical neuron, the multi-layer diffraction lines are connected through optical diffraction to form the On-chip diffraction neural network, and tasks such as intelligent calculation, feature learning and the like are achieved.
Optionally, in an embodiment of the present application, the classification module 300 includes: and the optical neural network unit is used for inputting the graph characteristic information of the multiple graph structures into the optical neural network through the waveguide, and classifying the multiple graph structures by using the optical neural network to obtain the classification results of the multiple graph structures.
Optionally, in an embodiment of the present application, the classification module 300 includes: the photoelectric detector is used for carrying out photoelectric conversion on the aggregated graph characteristic information of the multiple graph structures; and the electronic neural network unit is used for classifying the multiple graph structures according to the graph feature information of the multiple graph structures after photoelectric conversion to obtain the classification results of the multiple graph structures.
Specifically, the optical diagram feature aggregation unit splices the output features of different optical diagram feature extraction modules 100 together by a waveguide, so as to realize optical diagram feature aggregation for subsequent classification. And finally, photoelectric conversion of the aggregated graph characteristics is realized by a photoelectric detector, and classification (DGNN-E) is realized by a simple electronic neural network, or the spliced graph characteristics are input into an optical neural network such as a diffraction neural network by a waveguide to be classified (DGNN-O). The error calibration of the optical diagram neural network physical system can be carried out through the two different classification units, and the result output by the optical diagram neural network physical system is finely adjusted through a simple electronic network. Specifically, an optical diagram feature extraction result is obtained according to a photoelectric detector of an actually processed physical system, and a simple electronic network is trained to fine-tune the result. The optical part completes large-scale efficient operation, and the electronic part ensures the performance of the final whole system through fine adjustment.
Optionally, in an embodiment of the present application, the method further includes: the processing module is used for simulating the structural electromagnetic field of the optical diagram neural classification network, obtaining the structural parameters of the optical diagram neural network, establishing a forward propagation numerical model according to the structural parameters, training the parameters of each modulation layer of the diffraction calculation unit by using an error back propagation algorithm, and establishing the optical diagram neural network structure according to the training result.
In order to realize the optical diagram neural classification network, the embodiment of the application carries out accurate numerical modeling and training on the optical diagram neural network, firstly obtains accurate waveguide input and output coupling coefficients and diffraction line modulation coefficients according to time domain finite difference method electromagnetic field simulation, establishes an accurate forward propagation numerical model, trains parameters of each modulation layer of a diffraction calculation unit through an error back propagation algorithm, then processes an optical diagram neural network structure according to a training result, executes all-optical diagram structure data characteristic learning and reasoning, and finally can calibrate errors of a physical system through an output electronic calculation layer, so that the performance is further improved.
Specifically, accurate FDTD simulation is carried out on micro-nano optical structure electromagnetic fields such as waveguides and diffraction lines, accurate waveguide input coupling coefficients, output coupling coefficients for incident light at different angles and range of amplitude and phase modulation coefficients which can be realized by the diffraction lines are obtained. And obtaining the light field distribution of the output surface according to the light field distribution of the input waveguide, the amplitude of diffraction lines of an angular spectrum diffraction propagation method and a phase modulation coefficient, decomposing the light field distribution into components of each angle, obtaining an output result by utilizing the relation between the coupling coefficient of the output waveguide and the angle of incident light, and establishing an accurate forward propagation model. And establishing a proper training set, a proper testing set and a proper loss function according to task requirements, training the amplitude and phase modulation coefficient of each neuron in the diffraction neural network by combining error back propagation and a random gradient descent algorithm, determining a network structure after the training is finished, and constructing a physical system by processing and integrating structures such as a laser light source input, a modulator, a waveguide, a diffraction line, a photoelectric detector and the like through a silicon optical process.
Fig. 5, 6 and 7 show the diffraction line structure of the on-chip diffraction calculating unit corresponding to fig. 2 and 3, and the simulation diagram and graph of the FDTD electromagnetic field of each pixel (groove) on the diffraction line with the variation of the amplitude and phase modulation coefficient of the groove width respectively. Fig. 8, 9, 10 and 11 show the overall structure schematic diagram and FDTD electromagnetic field simulation schematic diagram of the on-chip diffraction calculation unit corresponding to fig. 2, 3, 5, 6 and 7, as well as the optical field distribution schematic diagram of the input waveguide, the graph of the coupling coefficient of the output waveguide as a function of the incident light angle, and the error comparison diagram of numerical modeling and FDTD physical simulation. And obtaining the light field distribution of the output surface according to the light field distribution of the input waveguide, the amplitude of diffraction lines of an angular spectrum diffraction propagation method and a phase modulation coefficient, decomposing the light field distribution into components of all angles, obtaining an output result by utilizing the relationship between the coupling coefficient of the output waveguide and the angle of incident light, and establishing a forward propagation model. Fig. 11 shows the minimal error between the numerical simulation model and the FDTD physical simulation, verifying the forward propagation model accuracy. Establishing a proper training set, a proper testing set and a proper loss function according to task requirements, training the amplitude and phase modulation coefficient of each neuron in the diffraction neural network by combining an error back propagation and random gradient descent algorithm, determining a network structure after the training is finished, and constructing a physical system by processing and integrating structures such as a laser light source input, a modulator, a waveguide, a diffraction line, a photoelectric detector and the like through a silicon optical process.
The embodiment of the application provides an optical diagram neural classification network based on an on-chip diffraction neural network, which utilizes a diffraction computing unit (diffraction neural network) integrated on a chip to perform feature extraction on node attributes coded on light, generates a message of each node, and utilizes waveguide coupling to realize the transmission and aggregation of the messages. Therefore, an all-optical graph neural network for graph structure data processing is established, and the performance of the proposed optical graph neural network, which is comparable to that of an electronic GNN, is demonstrated on the basis of the benchmark data sets such as Cora-ML, Citeser, Amazon Photo and the like and the synthetic random block model, and the great potential of the optical graph neural network in executing the non-Euclidean space data structure machine learning task is verified, as shown in FIGS. 12 and 13.
Next, a method for classifying an optical pattern neural network based on an on-chip diffraction neural network according to an embodiment of the present application will be described with reference to the drawings.
FIG. 14 is a flowchart of a method for classifying an optical diagram neural network based on an on-chip diffraction neural network according to an embodiment of the present application.
As shown in fig. 14, the method for classifying the optical diagram neural network based on the on-chip diffraction neural network includes the following steps:
step S101, encoding the input node attribute information of the graph structure to the input optical signal of the input waveguide, extracting the output optical signal of the output waveguide after being transmitted by the on-chip diffraction neural network, and obtaining graph characteristic information by using the output optical signal.
And step S102, aggregating the graph feature information of the graph structures, and classifying the graph structures according to the aggregated graph feature information of the graph structures to obtain classification results of the graph structures.
Optionally, in an embodiment of the present application, encoding the input node attribute information of the graph structure onto an input optical signal of an input waveguide, extracting an output optical signal of an output waveguide after propagating through an on-chip diffraction neural network, and obtaining graph feature information by using the output optical signal includes: the method comprises the steps of encoding input node attribute information of the graph structure to light intensity or phase of an input optical signal, extracting the light intensity or phase of the output optical signal, obtaining graph characteristic information based on the light intensity or phase of the output optical signal, and splicing the graph characteristic information of a plurality of adjacent input nodes in the graph structure.
Optionally, in an embodiment of the present application, classifying the multiple graph structures according to the aggregated graph feature information of the multiple graph structures to obtain classification results of the multiple graph structures, including: inputting graph characteristic information of a plurality of graph structures into an optical neural network, and classifying the graph structures by using the optical neural network to obtain classification results of the graph structures; and/or performing photoelectric conversion on the aggregated graph feature information of the multiple graph structures, and classifying the multiple graph structures according to the graph feature information of the multiple graph structures after photoelectric conversion to obtain classification results of the multiple graph structures.
It should be noted that the foregoing explanation of the embodiment of the on-chip diffraction-based neural network-based optical diagram neural network classification method is also applicable to the on-chip diffraction-based neural network classification method of the embodiment, and details are not repeated here.
According to the optical graph neural network classification method based on the on-chip diffraction neural network, the on-chip integrated diffraction neural network is used for carrying out feature extraction on node attributes coded on light, information of each node is generated, and waveguide coupling is used for achieving transmission and aggregation of the information. The method realizes high-speed and low-power consumption large-scale graph structure data processing, and enables the all-optical neural network to better complete various types of machine learning tasks. The method achieves excellent performance in the tasks of node and graph classification on a reference data set, and opens up a new direction for designing an integrated photonic circuit which efficiently processes large-scale graph structure data by deep learning.
Fig. 15 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device may include:
a memory 1501, a processor 1502, and computer programs stored on the memory 1501 and executable on the processor 1502.
The processor 1502 executes a program to implement the method for classifying an optical pattern neural network based on an on-chip diffraction neural network provided in the above-described embodiments.
Further, the electronic device further includes:
a communication interface 1503 for communication between the memory 1501 and the processor 1502.
A memory 1501 for storing computer programs operable on the processor 1502.
The memory 1501 may include a high-speed RAM memory, and may further include a non-volatile memory (non-volatile memory), such as at least one disk memory.
If the memory 1501, the processor 1502 and the communication interface 1503 are implemented independently, the communication interface 1503, the memory 1501 and the processor 1502 may be connected to each other via a bus and perform communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 15, but this is not intended to represent only one bus or type of bus.
Optionally, in a specific implementation, if the memory 1501, the processor 1502 and the communication interface 1503 are integrated into one chip, the memory 1501, the processor 1502 and the communication interface 1503 may complete communication with each other through an internal interface.
The processor 1502 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement embodiments of the present Application.
The present embodiment also provides a computer-readable storage medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method for classifying an optical map neural network based on an on-chip diffraction neural network as described above.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Moreover, various embodiments or examples and features of various embodiments or examples described in this specification can be combined and combined by one skilled in the art without being mutually inconsistent.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "N" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more N executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried out in the method of implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and the program, when executed, includes one or a combination of the steps of the method embodiments.
Claims (10)
1. An optical pattern neural classification network based on an on-chip diffraction neural network, comprising:
the optical diagram feature extraction module is used for encoding the input node attribute information of the diagram structure onto an input optical signal of an input waveguide, extracting an output optical signal of an output waveguide after being transmitted through the on-chip diffraction neural network, and obtaining diagram feature information by using the output optical signal;
the optical graph characteristic aggregation module is used for aggregating graph characteristic information of a plurality of graph structures;
the classification module is used for classifying the graph structures according to the aggregated graph feature information of the graph structures to obtain classification results of the graph structures;
the optical map feature extraction module comprises:
an optical node property input unit, configured to encode input node property information of the graph structure onto the light intensity or phase of the input optical signal through a modulator;
the on-chip diffraction calculating unit comprises an on-chip diffraction neural network which is arranged in an integrated mode and is used for extracting the light intensity or the phase of the output light signal and obtaining the graph characteristic information based on the light intensity or the phase of the output light signal;
and the optical node characteristic aggregation unit is used for splicing the graph characteristic information of a plurality of adjacent input nodes in the graph structure through the waveguide and the coupler.
2. The network of claim 1, wherein the modulator comprises an electro-optic modulator, an acousto-optic modulator, or a thermo-optic modulator.
3. The network of claim 1, wherein the on-chip diffraction neural network is composed of multiple layers of diffraction lines connected by optical diffraction, the shapes, sizes and periods of the diffraction lines are set, and the amplitude modulation coefficient and the phase modulation coefficient of light by the on-chip diffraction neural network are determined.
4. The network of claim 1, wherein the classification module comprises:
and the optical neural network unit is used for inputting the graph characteristic information of the graph structures into the optical neural network through a waveguide, and classifying the graph structures by using the optical neural network to obtain the classification results of the graph structures.
5. The network according to claim 1 or 4, characterized in that said classification module comprises:
a photodetector for photoelectrically converting the aggregated graph feature information of the plurality of graph structures;
and the electronic neural network unit is used for classifying the multiple graph structures according to the graph feature information of the multiple graph structures after photoelectric conversion to obtain the classification results of the multiple graph structures.
6. The network of claim 5, further comprising:
the processing module is used for simulating the electromagnetic field of the optical diagram neural classification network structure, obtaining the structural parameters of the optical diagram neural network, establishing a forward propagation numerical model according to the structural parameters, training the parameters of each modulation layer of the diffraction calculation unit by using an error back propagation algorithm, and establishing the optical diagram neural network structure according to the training result.
7. An optical diagram neural network classification method based on an on-chip diffraction neural network is characterized by comprising the following steps:
encoding input node attribute information of a graph structure to an input optical signal of an input waveguide, extracting an output optical signal of an output waveguide after being transmitted through an on-chip diffraction neural network, and obtaining graph characteristic information by using the output optical signal;
aggregating the graph feature information of the multiple graph structures, and classifying the multiple graph structures according to the aggregated graph feature information of the multiple graph structures to obtain classification results of the multiple graph structures;
the encoding of the input node attribute information of the graph structure onto the input optical signal of the input waveguide, extracting the output optical signal of the output waveguide after being propagated through the on-chip diffraction neural network, and obtaining graph feature information by using the output optical signal includes:
encoding the input node attribute information of the graph structure to the light intensity or phase of the input optical signal, extracting the light intensity or phase of the output optical signal, obtaining the graph characteristic information based on the light intensity or phase of the output optical signal, and splicing the graph characteristic information of a plurality of adjacent input nodes in the graph structure.
8. The method according to claim 7, wherein the classifying the graph structures according to the aggregated graph feature information of the graph structures to obtain the classification result of the graph structures comprises:
inputting the graph feature information of the graph structures into an optical neural network, and classifying the graph structures by using the optical neural network to obtain classification results of the graph structures; and/or
And performing photoelectric conversion on the aggregated graph feature information of the multiple graph structures, and classifying the multiple graph structures according to the graph feature information of the multiple graph structures after the photoelectric conversion to obtain classification results of the multiple graph structures.
9. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the method for classifying a neural network of an on-chip diffraction-based optical map according to any one of claims 7 to 8.
10. A computer-readable storage medium, on which a computer program is stored, the program being executable by a processor for implementing the method for classifying a neural network of an optical map based on an on-chip diffraction neural network according to any one of claims 7 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210411556.4A CN114519403B (en) | 2022-04-19 | 2022-04-19 | Optical diagram neural classification network and method based on-chip diffraction neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210411556.4A CN114519403B (en) | 2022-04-19 | 2022-04-19 | Optical diagram neural classification network and method based on-chip diffraction neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114519403A CN114519403A (en) | 2022-05-20 |
| CN114519403B true CN114519403B (en) | 2022-09-02 |
Family
ID=81600387
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210411556.4A Active CN114519403B (en) | 2022-04-19 | 2022-04-19 | Optical diagram neural classification network and method based on-chip diffraction neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114519403B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111582468A (en) * | 2020-04-02 | 2020-08-25 | 清华大学 | Photoelectric hybrid intelligent data generation and calculation system and method |
| CN113780258A (en) * | 2021-11-12 | 2021-12-10 | 清华大学 | Intelligent depth classification method and device for photoelectric calculation light field |
| CN114002931A (en) * | 2021-10-08 | 2022-02-01 | 清华大学深圳国际研究生院 | Large-view-field holographic projection method and system based on deep learning accelerated calculation |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019200289A1 (en) * | 2018-04-13 | 2019-10-17 | The Regents Of The University Of California | Devices and methods employing optical-based machine learning using diffractive deep neural networks |
| CN113795773A (en) * | 2019-03-08 | 2021-12-14 | 欧司朗股份有限公司 | Component for a LIDAR sensor system, LIDAR sensor device, method for a LIDAR sensor system and method for a LIDAR sensor device |
-
2022
- 2022-04-19 CN CN202210411556.4A patent/CN114519403B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111582468A (en) * | 2020-04-02 | 2020-08-25 | 清华大学 | Photoelectric hybrid intelligent data generation and calculation system and method |
| CN114002931A (en) * | 2021-10-08 | 2022-02-01 | 清华大学深圳国际研究生院 | Large-view-field holographic projection method and system based on deep learning accelerated calculation |
| CN113780258A (en) * | 2021-11-12 | 2021-12-10 | 清华大学 | Intelligent depth classification method and device for photoelectric calculation light field |
Non-Patent Citations (2)
| Title |
|---|
| Solving Computer Vision Tasks with Diffractive Neural Networks;Yan Tao 等;《OPTOELECTRONIC IMAGING AND MULTIMEDIA TECHNOLOGY VI》;20200514;1-9 * |
| 计算光场成像;方璐 等;《光学学报》;20200131;第40卷(第1期);1-22 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114519403A (en) | 2022-05-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI735886B (en) | Computing system | |
| De Marinis et al. | Photonic neural networks: A survey | |
| US9477136B2 (en) | Reservoir computing using passive optical systems | |
| WO2022134391A1 (en) | Fusion neuron model, neural network structure and training and inference methods therefor, storage medium, and device | |
| TWI741533B (en) | Computing system, computing apparatus, and operating method of computing system | |
| CN109784486B (en) | Optical neural network processor and training method thereof | |
| CN109639359A (en) | Photon neural network convolutional layer chip based on micro-ring resonator | |
| CN109376855B (en) | Optical neuron structure and neural network processing system comprising same | |
| CN111753977B (en) | Optical neural network convolution layer chip, convolution calculation method and electronic equipment | |
| CN111582435B (en) | Diffraction depth neural network system based on residual error network | |
| CN103620624A (en) | Method and apparatus for local competitive learning rule that leads to sparse connectivity | |
| da Silva Ferreira et al. | Towards an integrated evolutionary strategy and artificial neural network computational tool for designing photonic coupler devices | |
| CN109784485A (en) | A kind of optical neural network processor and its calculation method | |
| CN113496281A (en) | Photoelectric computing system | |
| TW202029067A (en) | computing system, COMPUTING APPARATUS, AND OPERATING METHOD OF computing system | |
| CN113298246B (en) | Data processing method, device and computer readable storage medium | |
| CN111582468B (en) | Photoelectric hybrid intelligent data generation and calculation system and method | |
| CN115545173A (en) | Optical modulation neuron for signal processing and all-optical diffraction neural network method | |
| CN110892422A (en) | Training of photon reservoir computing systems | |
| CN110309916A (en) | The full optical depth learning system of multistage null tone domain Modulation and Nonlinear and method | |
| CN113627605A (en) | Optical interference unit configuration method and device of photonic neural network and storage medium | |
| CN114519403B (en) | Optical diagram neural classification network and method based on-chip diffraction neural network | |
| CN113705774A (en) | Optical circuit construction method, optical circuit, optical signal processing method and device | |
| Cem et al. | Data-efficient modeling of optical matrix multipliers using transfer learning | |
| CN113592084B (en) | Photon neural network on chip based on reverse optimization superstructure convolution kernel |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |