CN113627605A - Optical interference unit configuration method and device of photonic neural network and storage medium - Google Patents

Optical interference unit configuration method and device of photonic neural network and storage medium Download PDF

Info

Publication number
CN113627605A
CN113627605A CN202110722256.3A CN202110722256A CN113627605A CN 113627605 A CN113627605 A CN 113627605A CN 202110722256 A CN202110722256 A CN 202110722256A CN 113627605 A CN113627605 A CN 113627605A
Authority
CN
China
Prior art keywords
optical
matrix
input
neural network
interference unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110722256.3A
Other languages
Chinese (zh)
Inventor
吴睿振
王凛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Electronic Information Industry Co Ltd
Original Assignee
Inspur Electronic Information Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Electronic Information Industry Co Ltd filed Critical Inspur Electronic Information Industry Co Ltd
Priority to CN202110722256.3A priority Critical patent/CN113627605A/en
Publication of CN113627605A publication Critical patent/CN113627605A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
    • G06N3/0675—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means using electro-optical, acousto-optical or opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
    • G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
    • G06F7/544—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices for evaluating functions by calculation
    • G06F7/5443—Sum of products

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computational Mathematics (AREA)
  • Computing Systems (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Neurology (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Optical Modulation, Optical Deflection, Nonlinear Optics, Optical Demodulation, Optical Logic Elements (AREA)

Abstract

The application discloses a method and a device for configuring an optical interference unit of a photonic neural network and a computer readable storage medium. Decomposing a weight real matrix constructed by each set of weight values of a convolution kernel of the photon neural network in convolution operation to obtain a unitary matrix for calculating an interference light path link topological structure; determining a minimum multiply-add unit of an interference light path link topological structure based on the light path structure and the light conduction characteristics of an optical interference unit of the photonic neural network; based on the optical signal input relation and the unitary matrix correspondingly represented by the minimum multiplication and addition unit, carrying out multiple times of elimination operation processing with the elimination times smaller than the order number of the unitary matrix; and determining all configuration modes of the optical interference unit according to the elimination element operation result, thereby effectively improving the configuration efficiency of the optical interference unit of the photonic neural network.

Description

Optical interference unit configuration method and device of photonic neural network and storage medium
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for configuring an optical interference unit of a photonic neural network, and a computer-readable storage medium.
Background
With the development of Technology, society has now entered the era of cloud + AI (Artificial Intelligence) +5G (5th Generation Mobile Communication Technology), and a dedicated chip supporting a large amount of computation is required to meet the computation requirement of cloud + AI + 5G. The chip is one of the greatest inventions of human beings, and is also the foundation and the core of the modern electronic information industry. The technology is small enough for mobile phones, computers and digital cameras and large enough for 5G, Internet of things and cloud computing, and is a continuous breakthrough based on chip technology. The development of the semiconductor lithography process level is a fundamental stone of an electronic computer taking a chip as a core, the manufacturing process of the semiconductor lithography is almost the physical limit of the moore's law at present, with the smaller and smaller manufacturing process, the transistor unit in the chip is close to the molecular scale, and the bottleneck effect of the semiconductor manufacturing process is more and more obvious. With the high-speed development of globalization and science and technology, the amount of data to be processed is increased rapidly, corresponding data processing models and algorithms are also increased continuously, and the requirements on computing power and power consumption are increased continuously. However, the problems of transmission bottleneck, power consumption increase, computing power bottleneck and the like of the existing von neumann and harvard architecture electronic computers are increasingly difficult to meet the requirements of computing power and power consumption in big data era, for example, the artificial intelligence computing requirement is extremely not matched with the traditional chip computing power growth curve, so that the problem of increasing the computing speed and reducing the computing power consumption is a critical problem at present. The photon computing method is one of the potential ways to solve the problems of moore's law predicament and von neumann architecture, i.e. the current computational power and power consumption. Photons have the characteristics of light velocity propagation, electromagnetic interference resistance, arbitrary superposition and the like, and compared with electric calculation, the electric calculation has many advantages, such as: the optical signal is transmitted at the speed of light, so that the speed is greatly improved; the light has natural parallel processing capability and mature wavelength division multiplexing technology, has extremely high operation speed and is very suitable for parallel operation, thereby greatly improving the data processing capability, capacity and bandwidth; the optical computing power consumption is expected to be as low as 10-18J/bit, and under the same power consumption, a photonic device is hundreds of times faster than an electronic device, so that an integrated photonic chip with deep learning capability, high computing power and low power consumption is widely applied, for example, in distance measurement, speed measurement and high-resolution imaging laser radars of long-distance and high-speed moving targets, and in novel computing microscopic associated imaging equipment for realizing high-resolution nondestructive detection of internal structures of biological medicines, nano devices and the like.
In recent years, with the gradual failure of moore's law and the increasing requirements of the big data era on the power consumption and speed of the computing system, the characteristics of high speed and low power consumption of the optical computing technology are more and more emphasized. In addition, due to the parallelism operation characteristic of the optical computing technology and the development of algorithms and hardware architectures such as an optical neural network and the like, the most potential solution is provided for the demands of artificial intelligence technologies such as image recognition, voice recognition, virtual reality and the like on computing power. The light calculation can be divided into an analog light calculation and a digital light calculation. The most typical example of the analog light calculation is fourier operation, and fourier transform related calculation, such as convolution calculation, needs to be applied in the field of image processing and the like. The calculation of the fourier transform with a conventional computer is very computationally expensive, and the passage of light through the lens is itself a fourier transform process, which requires almost no time at all. The digital optical calculation is to form a classic logic gate by combining light and an optical device, construct a calculation system similar to the traditional digital electronic calculation principle, and realize calculation through complex logic gate combination operation.
ONN (optical neural network) includes an input layer, an output layer, and hidden layers in between. The intermediate hidden layers include Optical Interference Units (OIUs) and optical non-linear units (ONUs), which function as matrix multiplication and activation functions, respectively. Due to the fact that the mode of achieving optical operation through the MZI (Mach-Zehnder interferometer) has the characteristic of being easy to control, photon operation based on the MZI is the most common industrial solution in the aspect of photonic neural networks nowadays. Accordingly, the OIU may be implemented by a programmable nanophotonics device based on an MZI array. The convolution operation of the main operation in the photonic neural network, and the corresponding implementation manner of the convolution in the ONN based on the MZI, are the problems which must be solved in the photonic neural network. MZI is used as a basic device to be linked in different linking modes, and the MZI is configured for different linking modes, so that convolution multiplication and addition operation of weight values required by convolution operation in multiple groups of different ANNs (Artificial Neural networks) can be realized, and a final result is obtained. In the process of determining different link modes of the photonic neural network and corresponding MZI configuration modes, all MZI configuration modes can be obtained only by calculating a unitary matrix of any m × m at least m times, and the larger the scale of the unitary matrix is, the longer the time consumption is, and the lower the efficiency is.
In view of this, how to improve the configuration efficiency of the optical interference unit of the photonic neural network is a technical problem to be solved by those skilled in the art.
Disclosure of Invention
The application provides a method and a device for configuring an optical interference unit of a photonic neural network and a computer readable storage medium, which improve the configuration efficiency of the optical interference unit of the photonic neural network.
In order to solve the above technical problems, embodiments of the present invention provide the following technical solutions:
an embodiment of the present invention provides a method for configuring an optical interference unit of a photonic neural network, including:
decomposing a weight real matrix constructed by each set of weight values of a convolution kernel of the photon neural network in convolution operation to obtain a unitary matrix for calculating an interference light path link topological structure;
determining a minimum multiply-add unit of the interference light path link topological structure based on the light path structure and the light conduction characteristics of an optical interference unit of the photonic neural network;
performing multiple elimination operations based on the optical signal input relation correspondingly represented by the minimum multiplication and addition unit and the unitary matrix; the number of the element eliminating operations is smaller than the order number of the unitary matrix;
and determining all configuration modes of the optical interference unit according to the elimination element operation result.
Optionally, the minimum multiply-add unit for determining the interference optical path link topology structure based on the optical path structure and the light transmission characteristics of the optical interference unit of the photonic neural network includes:
the optical interference unit comprises a first input end, a second input end, a first phase shifter, a first optical splitter, a second phase shifter and a second optical splitter; the optical signals input through the first input end and the second input end are projected to the first optical splitter through the first phase shifter, enter the second phase shifter after being processed by the first optical splitter, enter the second optical splitter through the second phase shifter for processing, and are output;
determining an optical conversion relationship between the input optical signal and the output optical signal according to the optical structure of the optical interference unit and the optical conduction characteristic as follows:
Figure BDA0003136890240000041
in the formula (I), the compound is shown in the specification,
Figure BDA0003136890240000043
is an input optical signal at said first input terminal,
Figure BDA0003136890240000044
is the input optical signal at the second input terminal,
Figure BDA0003136890240000045
is the final output optical signal after the input optical signal of the first input end is processed by the optical interference unit,
Figure BDA0003136890240000046
the final output optical signal is the input optical signal of the second input end after being processed by the optical interference unit; theta is the phase modulation angle of the second phase shifter, phi is the phase modulation angle of the first phase shifter, e represents an index, and i represents an imaginary number;
based on the optical conversion relationship, setting the phase modulation angle of the first phase shifter to 0, and obtaining the minimum multiplication and addition unit R as:
Figure BDA0003136890240000042
optionally, decomposing the weight real matrix constructed by each set of weight values of the convolution kernel of the photon neural network in the convolution operation to obtain the unitary matrix for calculating the interferometric optical path link topology includes:
acquiring all weight values for performing convolution operation on a current convolution kernel;
expanding the weight value of the current convolution kernel needing to be operated into a matrix structure; each row of the matrix structure corresponds to a group of weight values of convolution operation, and each column corresponds to one to-be-input optical signal data;
and carrying out singular value decomposition on the matrix structure to obtain the unitary matrix.
Optionally, the performing multiple elimination operations based on the optical signal input relationship correspondingly represented by the minimum multiply-add unit and the unitary matrix includes:
constructing a diagonal matrix for representing the corresponding input relation of the optical interference unit according to the fact that the minimum multiplication and addition unit has unitary matrix characteristics;
performing matrix operation-based expansion on the diagonal matrix based on diagonal elements of the diagonal matrix to obtain a configuration relation matrix;
and carrying out elimination processing on the configuration relation matrix according to a preset elimination method, and calculating the input relation contained in the configuration relation matrix by combining the unitary matrix.
Optionally, the determining all configuration modes of the optical interference unit according to the result of the null operation includes:
and determining the configuration mode of the optical interference unit according to the input relation calculated in each elimination element operation processing process.
Another aspect of the embodiments of the present invention provides an optical interference unit configuration apparatus for a photonic neural network, including:
the topological structure representation module is used for decomposing a weight real matrix constructed by each set of weight values of a convolution kernel of the photon neural network in convolution operation to obtain a unitary matrix for calculating an interference light path link topological structure;
the basic unit determining module of the topological structure is used for determining the minimum multiplication and addition unit of the interference light path link topological structure based on the light path structure and the light conduction characteristic of the optical interference unit of the photonic neural network;
the elimination processing is used for carrying out multiple elimination operations based on the optical signal input relation correspondingly represented by the minimum multiplication and addition unit and the unitary matrix; the number of the element eliminating operations is smaller than the order number of the unitary matrix;
and the configuration mode determining module is used for determining all configuration modes of the optical interference unit according to the elimination element operation result.
Optionally, the topology base unit determining module is further configured to:
the optical interference unit comprises a first input end, a second input end, a first phase shifter, a first optical splitter, a second phase shifter and a second optical splitter; the optical signals input through the first input end and the second input end are projected to the first optical splitter through the first phase shifter, enter the second phase shifter after being processed by the first optical splitter, enter the second optical splitter through the second phase shifter for processing, and are output; determining an optical conversion relationship between the input optical signal and the output optical signal according to the optical structure of the optical interference unit and the optical conduction characteristic as follows:
Figure BDA0003136890240000051
in the formula (I), the compound is shown in the specification,
Figure BDA0003136890240000052
is an input optical signal at said first input terminal,
Figure BDA0003136890240000053
is the input optical signal at the second input terminal,
Figure BDA0003136890240000055
is the final output optical signal after the input optical signal of the first input end is processed by the optical interference unit,
Figure BDA0003136890240000054
the final output optical signal is the input optical signal of the second input end after being processed by the optical interference unit; theta is the phase modulation angle of the second phase shifter, phi is the phase modulation angle of the first phase shifter, e represents an index, and i represents an imaginary number;
based on the optical conversion relationship, setting the phase modulation angle of the first phase shifter to 0, and obtaining the minimum multiplication and addition unit R as:
Figure BDA0003136890240000061
optionally, the topology representation module is further configured to:
acquiring all weight values for performing convolution operation on a current convolution kernel;
expanding the weight value of the current convolution kernel needing to be operated into a matrix structure; each row of the matrix structure corresponds to a group of weight values of convolution operation, and each column corresponds to one to-be-input optical signal data;
and carrying out singular value decomposition on the matrix structure to obtain the unitary matrix.
The embodiment of the present invention further provides an optical interference unit configuration apparatus of a photonic neural network, including a processor, configured to implement the steps of the optical interference unit configuration method of the photonic neural network according to any one of the preceding claims when executing a computer program stored in a memory.
Finally, an embodiment of the present invention provides a computer-readable storage medium, on which an optical interference unit configuration program of a photonic neural network is stored, and when being executed by a processor, the optical interference unit configuration program of the photonic neural network implements the steps of the optical interference unit configuration method of the photonic neural network according to any one of the preceding items.
The technical scheme provided by the application has the advantages that different sets of weighted values of the same convolution kernel can be suitable for any unitary matrix through a decomposed matrix mapping calculation mode. And for any m × m unitary matrix, only m-1 times of calculation is needed to obtain the configuration modes of all optical interference units, so that the required m groups of weighted values are calculated, and compared with the prior art, the configuration efficiency of the optical interference units of the photonic neural network can be effectively improved.
In addition, the embodiment of the invention also provides a corresponding implementation device and a computer readable storage medium for the optical interference unit configuration method of the photonic neural network, so that the method has higher practicability, and the device and the computer readable storage medium have corresponding advantages.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the related art, the drawings required to be used in the description of the embodiments or the related art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow chart illustrating a method for configuring an optical interference unit of a photonic neural network according to an embodiment of the present invention;
fig. 2 is a schematic diagram of an optical path structure of an exemplary embodiment of the present invention;
FIG. 3 is an optical schematic of a matrix configuration of an illustrative example provided by an embodiment of the invention;
FIG. 4 is an optical schematic of another matrix configuration of an illustrative example provided by an embodiment of the invention;
FIG. 5 is a block diagram of an embodiment of an apparatus for configuring an optical interference unit of a photonic neural network according to the present invention;
fig. 6 is a structural diagram of another embodiment of an optical interference unit configuration apparatus of a photonic neural network according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may include other steps or elements not expressly listed.
Having described the technical solutions of the embodiments of the present invention, various non-limiting embodiments of the present application are described in detail below.
Referring to fig. 1, fig. 1 is a schematic flow chart of a method for configuring an optical interference unit of a photonic neural network according to an embodiment of the present invention, where the embodiment of the present invention includes the following steps:
s101: and decomposing the weight real matrix constructed by each set of weight values of the convolution kernel of the photon neural network in convolution operation to obtain the unitary matrix for calculating the interference light path link topological structure. It is understood that each hidden layer in the middle of the photonic neural network may include optical interference units to function as a matrix multiplication. The optical interference unit may be implemented by, for example, a programmable nanophotonic device based on a Mach-Zehnder interferometer (MZI), where as shown in fig. 2, a structure of the MZI generally includes two input ends, two output ends, two optical splitters, a first phase shifter and a second phase shifter, and an input optical signal enters the second phase shifter after being split by the first optical splitter, and is output by the second optical splitter after being processed by the second phase shifter. Where a first phase shifter is located between the input and the first splitter and a second phase shifter is located between the two splitters, both phase shifters being used to control the output splitting ratio, such as by changing the waveguide refractive index.
Generally speaking, a convolution kernel should have a set of weight values, but in an actual artificial neural network, the same set of convolution kernel generally has different weight calculation requirements, so that the convolution weight values for the same set of data calculation include multiple sets. Then, an interference light path linking method that can perform operation on a plurality of convolution weight values in parallel for the same set of input and obtain different output results needs to be implemented. The weight real matrix of the present application is a matrix formed by expanding all sets of weight values of the current convolution kernel in the convolution operation, and any matrix decomposition method can be used for the matrix decomposition. An optional implementation manner of the unitary matrix may be that, for the current convolution kernel, all weight values for performing convolution operation are obtained; expanding the weight value of the current convolution kernel needing to be operated into a matrix structure; each row of the matrix structure corresponds to a group of weight values of convolution operation, and each column corresponds to one to-be-input optical signal data; and carrying out singular value decomposition on the matrix structure to obtain the unitary matrix.
S102: and determining a minimum multiply-add unit of the interference optical path link topological structure based on the optical path structure and the light conduction characteristics of the optical interference unit of the photonic neural network. In the optical interference unit of the photonic neural network of the embodiment, the interference optical path link relationship is based on a minimum multiply-add unit, the light transmission characteristic means that in the transmission process of light in the photonic neural network, a real part and an imaginary part are required to be involved in transmission by combining with a mathematical expression form of an optical signal, each operation is that the imaginary part is required to participate as energy loss, and in the photoelectric conversion output, only the real part of the light can be identified.
S103: and performing multiple elimination operations based on the optical signal input relation correspondingly represented by the minimum multiplication and addition unit and the unitary matrix.
Since the interference optical path linking relationship is based on the minimum multiplication and addition unit, elements in the minimum multiplication and addition unit are used for representing the input relationship, and the unitary matrix is obtained based on weight value decomposition, all configuration parameters of the optical interference unit or the configuration mode of the optical interference unit can be obtained when the interference optical path corresponding to any unitary matrix such as MZI is realized based on the elimination processing of the minimum multiplication and addition unit and the unitary matrix. The number of the null operations in this embodiment is smaller than the order of the unitary matrix, so as to improve the configuration efficiency of the optical interference unit.
S104: and determining all configuration modes of the optical interference unit according to the elimination element operation result.
In the technical scheme provided by the embodiment of the invention, different sets of weight values of the same convolution kernel can be suitable for any unitary matrix through a matrix mapping calculation mode after decomposition. And for any m × m unitary matrix, only m-1 times of calculation is needed to obtain the configuration modes of all optical interference units, so that the required m groups of weighted values are calculated, and compared with the prior art, the configuration efficiency of the optical interference units of the photonic neural network can be effectively improved.
It should be noted that, in the present application, there is no strict sequential execution order among the steps, and as long as the logical order is met, the steps may be executed simultaneously or according to a certain preset order, and fig. 1 is only an exemplary manner, and does not represent that only the execution order is the order.
In the above embodiments, the calculation method of the minimum multiply-add unit of the photonic neural network is not limited, and in this embodiment, a specific implementation is given for the optical interference unit structure shown in fig. 2, where the optical interference unit includes a first input end, a second input end, a first phase shifter, a first optical splitter, a second phase shifter, and a second optical splitter; the optical signal input through the first input end and the second input end is projected to the first optical splitter through the first phase shifter, enters the second phase shifter after being processed by the first optical splitter, enters the second optical splitter through the second phase shifter for processing and then is output, and the method may include the following steps:
determining an optical conversion relationship between the input optical signal and the output optical signal according to the optical structure and the optical transmission characteristics of the optical interference unit as follows:
Figure BDA0003136890240000101
in the formula (I), the compound is shown in the specification,
Figure BDA0003136890240000103
is an input optical signal at a first input terminal,
Figure BDA0003136890240000104
is the input optical signal at the second input terminal,
Figure BDA0003136890240000105
is the final output optical signal after the input optical signal of the first input end is processed by the optical interference unit,
Figure BDA0003136890240000106
the final output optical signal is obtained after the input optical signal at the second input end is processed by the optical interference unit; theta is the phase modulation angle of the second phase shifter, phi is the phase modulation angle of the first phase shifter, e represents an index, and i represents an imaginary number;
based on the optical conversion relation, the phase modulation angle of the first phase shifter is set to be 0, and the minimum multiplication and addition unit R is obtained as follows:
Figure BDA0003136890240000102
in this embodiment, the optical conversion calculation relationship refers to the processing operation of the optical signal involved in the process of inputting the optical signal to the output end through the input end and outputting the optical signal.
In the foregoing embodiment, how to perform the elimination processing is not limited, and the present application further provides an elimination processing method, that is, a process of performing multiple elimination operations based on the unitary matrix and the optical signal input relationship correspondingly represented by the minimum multiply-add unit may include the following steps:
the whole element eliminating thought is as follows: constructing a diagonal matrix for representing the corresponding input relation of the optical interference unit according to the characteristic that the minimum multiplication and addition unit has a unitary matrix; performing matrix operation-based expansion on the diagonal matrix based on diagonal elements of the diagonal matrix to obtain a configuration relation matrix; and performing elimination processing on the configuration relation matrix according to a preset elimination method, and calculating the input relation contained in the configuration relation matrix by combining a unitary matrix. According to the practical application scene, the method can be executed as follows:
constructing a first diagonal matrix for representing a first input relation corresponding to the optical interference unit according to the characteristic that the minimum multiplication and addition unit has a unitary matrix;
performing matrix operation-based expansion on the first diagonal matrix based on diagonal elements of the first diagonal matrix to obtain a first configuration relation matrix;
carrying out element elimination operation on the first configuration relation matrix according to a preset element elimination method, and calculating a first input relation contained in the first configuration relation matrix by combining a unitary matrix;
judging whether the diagonal element number of the first configuration relation matrix is 1 or not;
if not, executing the subsequent steps, and if yes, determining the configuration mode of the optical interference unit according to the input relation obtained by each calculation.
Performing multiply-add operation on the angle configuration relationship matrix and the unitary matrix to obtain a first sub-unitary matrix;
constructing a second diagonal matrix for representing a second input relation corresponding to the optical interference unit according to the fact that the minimum multiplication and addition unit has the unitary matrix characteristic;
performing matrix operation-based expansion on the second diagonal matrix based on diagonal elements of the second diagonal matrix to obtain a second configuration relationship matrix;
performing element elimination operation on the second configuration relation matrix according to a preset element elimination method, and calculating a second input relation contained in the second configuration relation matrix by combining the first sub unitary matrix;
judging whether the number of diagonal elements of the second configuration relation matrix is 1 or not;
if not, executing the subsequent steps, and if yes, determining the configuration mode of the optical interference unit according to the input relation obtained by each calculation.
Performing multiply-add operation on the second configuration relation matrix and the unitary matrix to obtain a second sub-unitary matrix;
constructing a third diagonal matrix for representing a third input relation corresponding to the optical interference unit according to the fact that the minimum multiplication and addition unit has the unitary matrix characteristic;
performing matrix operation-based expansion on the third diagonal matrix based on diagonal elements of the third diagonal matrix to obtain a third configuration relation matrix;
and performing element elimination operation on the third configuration relation matrix according to a preset element elimination method, and calculating a third input relation contained in the third configuration relation matrix by combining the second sub unitary matrix.
Accordingly, the implementation process of S104 in the above embodiment is: and determining the configuration mode of the optical interference unit according to the input relation calculated in each elimination element operation processing process.
In order to make the technical solutions of the present application more obvious to those skilled in the art, the present application describes the whole process based on the optical interference unit structure shown in fig. 2 by way of specific schematic examples, which may include the following:
it can be understood that the optical interference unit implements convolution operation through an optical component, taking a convolution neural network for image processing as an example, where 2 × 2 convolution operation is as follows:
Figure BDA0003136890240000121
the forward propagation process of the artificial neural network strongly depends on multiply-add operation, and most of the operation in the inference process is essentially linear operation between the trained weight and the characteristic value. The use of optical chips to compute matrix multiplication is very different from electrical chips in terms of implementation principles. In digital integrated circuits, data is typically encoded as binary strings in the switching states of transistors. The numbers represented by binary strings are discrete, e.g., integers or floating point values; in photonics, data is encoded by modulating the amplitude or phase of a laser pulse, resulting in a continuous real value, changing the intensity or phase of the optical field changes the real number represented. The circuit can use conducting wire to guide electron, and the photonics can use silicon-based optical waveguide structure to transmit laser. On a mathematical model, a programmable phase shifter, a mach-zehnder interferometer, or other structures can implement matrix multiplication of any dimension in the optical domain by using a characteristic Decomposition method such as SVD (Singular Value Decomposition). In linear algebra, singular value decomposition is an important matrix decomposition mode, is one of algorithms commonly used in machine learning, and is widely applied to feature extraction, data simplification and recommendation systems. The real number matrix of any dimensionality can be decomposed into the product of three matrixes through a singular value decomposition method. Assuming that M is a matrix of M × n, U is a matrix of M × M, called unitary matrix, Σ is a diagonal matrix of M × n, values on the diagonal are non-negative real numbers, V is a matrix of n × n, also called unitary matrix, the complex conjugate matrix of V is represented by V, and the singular value decomposition of matrix M can be represented by formula (2):
M＝U∑V*。 (2)
based on the above theoretical basis, for the structure shown in fig. 2, L1 and L2 are the optical inputs of the MZI, and the output optical signals are L1 'and L2'. The MZI can couple the optical power of one dual port to the optical power of another dual port in a certain proportion, the splitting ratio is 50: 50. 2 theta and 2 phi are phase shifters, which have programmable functions, and generally, the programmable phase shifters can be implemented by several ways, such as plating a metal film on a section of waveguide material, and controlling a metal film heater to change the temperature of the waveguide by applying an external voltage to change the refractive index, thereby implementing phase shift; phase shift can also be introduced by altering the waveguide refractive index using the plasma dispersion effect, i.e., changing the concentration of electrons and holes and the electro-optic effect. If A represents amplitude, ω represents frequency, t represents time, θ1、θ2For the initial phase, L1 and L2 are expressed as
Figure BDA0003136890240000131
In the formula, A1、A2、θ1、θ2The amplitude and initial phase of the optical signal L1 of the first optical path channel and the optical signal L2 of the second optical path channel, respectively. Since only the real part of light can be identified during the photoelectric conversion process and the imaginary part represents the energy loss during transmission, equation (3) above can be converted based on the euler equation:
Figure BDA0003136890240000132
the expression (4) Re represents the real part. After L1 and L2 enter the MZI, it is known that the energy contained in the light is transferred to the optical signals corresponding to the two output ports through the coupler, and thus the amplitudes thereof are the original ones
Figure BDA0003136890240000133
The relationship of L1 and L2 to L1 'and L2' in fig. 2 can therefore be expressed by equation (5):
Figure BDA0003136890240000134
Figure BDA0003136890240000135
is an input optical signal at said first input terminal,
Figure BDA0003136890240000138
is the input optical signal at the second input terminal,
Figure BDA0003136890240000136
is the final output optical signal after the input optical signal of the first input end is processed by the optical interference unit,
Figure BDA0003136890240000137
the final output optical signal is the input optical signal of the second input end after being processed by the optical interference unit; theta is asThe phase modulation angle of the second phase shifter, phi, is the phase modulation angle of the first phase shifter, e represents an index, and i represents an imaginary number.
The sequence of the to-be-multiplied numbers for converting the matrix of the right 3x3 in equation (1) into the left convolution kernel operation can be expressed as:
Figure BDA0003136890240000141
combining equation (1), the weight value of the convolution kernel in equation (7) that needs to be mapped to 2 φ and 2 θ:
Figure BDA0003136890240000142
here, the corresponding relation of one convolution kernel corresponding to data of weight values in convolution operation is realized. In an actual ANN, a same set of convolution kernels generally has different weight operation requirements, and besides the weight value of the convolution kernel in formula (7), convolution operations corresponding to other weight values of a same set of data (shown in formula (6)) may be required, and all convolution weight values required to be operated for the same set of data are shown in formula (8):
Figure BDA0003136890240000143
then, it is necessary to implement an MZI linking method that performs multiple convolution weight values in parallel, can operate on the same set of input, and obtains different output results.
The method for solving the problem in real time comprises the following steps of firstly expanding the weight value needing to be calculated into a matrix structure, wherein the expansion is as shown in a formula (8):
Figure BDA0003136890240000144
each row in the formula (9) corresponds to a set of weight values of convolution operation, and each column corresponds to an input data to be multiplied. Considering the operation structure implemented by MZI, if the input data is L1-L4, the convolution operation structure to be implemented is:
Figure BDA0003136890240000151
the T matrix is a matrix relation that needs to be realized by different MZI link topologies, and L is input data.
It can be known that any matrix of m × n can be decomposed into two unitary matrices and one diagonal matrix through SVD, as shown in formula (2). Then, for the weight value in equation (10) as an example, performing singular value decomposition can obtain:
Figure BDA0003136890240000152
u and W in equation (10) are unitary matrices, and V is a diagonal matrix, respectively, UVW can be decomposed, and an attempt is made to construct a topological implementation using MZI. Considering the characteristics of MZI from equation (5), the phase modulators are respectively set, and for the convenience of derivation, we first set the phase modulator 2 Φ to 0, and then only the external phase modulator 2 θ needs to be adjusted.
The minimum multiply-add unit of the MZI is then improved as:
Figure BDA0003136890240000153
observing the characteristics of equation (11), it can be known that the V matrix as a diagonal matrix can be easily implemented by the MZI link relationship in fig. 2, and it is more complicated to solve the unitary matrices U and W. Taking the unitary matrix U as an example to explain the method of the present invention, the minimum multiplication and addition unit of equation (12) is assumed to perform gaussian elimination as follows:
Figure BDA0003136890240000161
the unitary matrix U is considered to be inherently invertible in its properties and to exhibit symmetric properties of the hilbert space. And decomposing the unitary matrix to obtain the diagonal matrix in consideration of the required operation result. Therefore, it can be seen that the U matrix performs the matrix operation of multiply-add, which certainly saves the diagonal elements and eliminates other elements. Further, as shown in equation (12), the minimum unit for performing the multiply-add operation is a unitary matrix represented by MZI, and therefore the multiplied matrix of the multiply-add operation must satisfy the characteristics thereof. Based on the above characteristics, it is necessary to perform an elimination operation on the U matrix, and the U matrix is used as a unitary matrix and must have a characteristic that all sub-matrices thereof satisfy the unitary matrix.
The arithmetic minimum multiply-add unit (MZI) expressed by equation (12) exhibits a unitary matrix characteristic (the inverse matrix is its transposed matrix). This property is true when cos and sin in (12) are reversed, so that a fixed unitary matrix property must be determined and then the result determined based on the expansion operation.
The multiplied matrix R at this time takes into account the characteristic of equation (12), i.e., the relationship determined in the present embodiment is the positional relationship of cos and sin given by equation (12). At this time cos is on the diagonal of the matrix, so all matrix operations based on (12) must retain cos on the diagonal, and correspondingly, for performing the primitive operation, the operation of adding or subtracting 0 must be sin related to the other two elements except the diagonal. Based on the above relationship, if the diagonal elements are saved in equation (14), the rest must be cos. And other operations based on cos. Thus, the cos value of the diagonal is stored in (14) as a base reference, and then the other operations are expanded based on the relation of the primitive operations in (12). When the outermost element elimination element is made, the element of the first row and the first column is necessarily saved, and other elements of the first row and the first column are eliminated. Based on the characteristics of (12) and the relationship of the elimination elements, a formula (14) of the first row and the first column of elimination elements is constructed, and the method comprises the following steps:
1: firstly, storing diagonal elements as follows:
Figure BDA0003136890240000171
wherein, theta12,θ13,θ14Respectively, the input relationship corresponding to MZI is shown, specifically, the embodiment is exemplified by 4 input signals, so θ12Is a relation between the first input signal and the second input signal, theta13Is the relationship between the first input signal and the third input signal, θ14Is the relation between the first input signal and the fourth input signal.
2: the method based on FIG. 3 expands the diagonal elements based on matrix operation to obtain a first configuration relationship matrix R1Namely, the following formula (15):
Figure BDA0003136890240000172
3: the following formula (16) is obtained by performing the operation of eliminating the elements in the first row and the first column, and θ can be calculated based on the formula12,θ13,θ14：
Figure BDA0003136890240000173
4: the iteration completes all other operations:
equation (17) can be obtained by performing a multiply-add operation on the unitary matrix and the first configuration relationship matrix:
Figure BDA0003136890240000181
then, the newly obtained sub-matrix is set as U', that is, the first unitary matrix:
Figure BDA0003136890240000182
using the first unitary matrix as the unitary matrix, stepping according to the above 1-3 stepsPerforming row calculation to obtain a second configuration relation matrix until theta is calculated12、θ13、θ14、θ24、θ23、θ34。θ24、θ23The calculation process of (a) may be:
firstly, storing diagonal elements as follows:
Figure BDA0003136890240000183
wherein, theta24Is the relationship between the second input signal and the fourth input signal, θ23Is the relationship between the second input signal and the third input signal.
The diagonal elements are expanded based on the matrix operation based on the method of FIG. 4 to obtain a second configuration relationship matrix R2Namely, the following formula (20):
Figure BDA0003136890240000184
the elimination operations of the first row and the first column can respectively calculate theta24、θ23。
θ34The calculation process can be performed according to the above method, and will not be described herein. After calculating to obtain theta12、θ13、θ14、θ24、θ23、θ34。θ24、θ23Then based on theta12、θ13、θ14、θ24、θ23、θ34。θ24、θ23The subscript, the sequence calculation order and the iteration relation order of the MZI link relation are corresponding to the MZI link relation, namely all the configuration modes of the MZI.
As can be seen from the above, this embodiment provides a fast operation mode for achieving ONN convolution based on MZI, and sets a processing sequence of round elimination by analyzing an operation relationship based on a unitary matrix and MZI, and a method for setting an operation matrix based on the processing sequence. By the embodiment, the angle configuration relationship of the MZI can be rapidly obtained when the MZI corresponding to any unitary matrix is realized. In addition, in consideration of the overall calculation efficiency and the calculation environment to be mounted, the present embodiment is suitable for operation within five squares.
The embodiment of the invention also provides a corresponding device for the optical interference unit configuration method of the photonic neural network, so that the method has higher practicability. Wherein the means can be described separately from the functional module point of view and the hardware point of view. The following describes an optical interference unit configuration apparatus of a photonic neural network according to an embodiment of the present invention, and the optical interference unit configuration apparatus of the photonic neural network described below and the optical interference unit configuration method of the photonic neural network described above may be referred to correspondingly.
Based on the angle of the functional module, referring to fig. 5, fig. 5 is a structural diagram of an optical interference unit configuration apparatus of a photonic neural network according to an embodiment of the present invention, in an embodiment, the apparatus may include:
the topology structure representation module 501 is configured to decompose a weight real matrix constructed by each set of weight values of a convolution kernel of the photonic neural network in convolution operation to obtain a unitary matrix for calculating the interferometric optical link topology structure.
A topology base unit determining module 502, configured to determine a minimum multiply-add unit of an interference optical path link topology based on an optical path structure and light conduction characteristics of an optical interference unit of the photonic neural network.
A elimination processing module 503, configured to perform multiple elimination operations based on the unitary matrix and the optical signal input relationship correspondingly represented by the minimum multiply-add unit; the number of the operation of the elimination element is smaller than the order of the unitary matrix.
And an arrangement mode determining module 504, configured to determine all arrangement modes of the optical interference unit according to the result of the null operation.
Optionally, in some implementations of this embodiment, the topology base unit determining module 502 is further configured to:
the optical interference unit comprises a first input end, a second input end, a first phase shifter, a first optical splitter, a second phase shifter and a second optical splitter; the optical signals input through the first input end and the second input end are projected to the first optical splitter through the first phase shifter, enter the second phase shifter after being processed by the first optical splitter, enter the second optical splitter through the second phase shifter for processing and then are output; determining an optical conversion relationship between the input optical signal and the output optical signal according to the optical structure and the optical transmission characteristics of the optical interference unit as follows:
Figure BDA0003136890240000201
in the formula (I), the compound is shown in the specification,
Figure BDA0003136890240000203
is an input optical signal at a first input terminal,
Figure BDA0003136890240000204
is the input optical signal at the second input terminal,
Figure BDA0003136890240000205
is the final output optical signal after the input optical signal of the first input end is processed by the optical interference unit,
Figure BDA0003136890240000206
the final output optical signal is obtained after the input optical signal at the second input end is processed by the optical interference unit; theta is the phase modulation angle of the second phase shifter, phi is the phase modulation angle of the first phase shifter, e represents an index, and i represents an imaginary number;
based on the optical conversion relation, the phase modulation angle of the first phase shifter is set to be 0, and the minimum multiplication and addition unit R is obtained as follows:
Figure BDA0003136890240000202
as an optional implementation manner of this embodiment, the topology representation module 501 is further configured to: acquiring all weight values for performing convolution operation on a current convolution kernel; expanding the weight value of the current convolution kernel needing to be operated into a matrix structure; each row of the matrix structure corresponds to a group of weight values of convolution operation, and each column corresponds to one to-be-input optical signal data; and carrying out singular value decomposition on the matrix structure to obtain the unitary matrix.
Optionally, in other embodiments of this embodiment, the elimination processing module 503 may be further configured to: constructing a diagonal matrix for representing the corresponding input relation of the optical interference unit according to the characteristic that the minimum multiplication and addition unit has a unitary matrix; performing matrix operation-based expansion on the diagonal matrix based on diagonal elements of the diagonal matrix to obtain a configuration relation matrix; and performing elimination processing on the configuration relation matrix according to a preset elimination method, and calculating the input relation contained in the configuration relation matrix by combining a unitary matrix.
As an optional implementation manner of this embodiment, the configuration manner determining module 504 may be configured to determine a configuration manner of the optical interference unit according to the input relationship calculated in each elimination operation process.
The functions of the functional modules of the optical interference unit configuration device of the photonic neural network according to the embodiments of the present invention may be specifically implemented according to the method in the above method embodiments, and the specific implementation process may refer to the description related to the above method embodiments, and will not be described herein again.
From the above, the embodiment of the invention can effectively improve the configuration efficiency of the optical interference unit of the photonic neural network.
The above mentioned optical interference unit configuration device of the photonic neural network is described from the perspective of a functional module, and further, the present application also provides an optical interference unit configuration device of the photonic neural network, which is described from the perspective of hardware. Fig. 6 is a structural diagram of an optical interference unit configuration device of another photonic neural network according to an embodiment of the present application. As shown in fig. 6, the apparatus comprises a memory 60 for storing a computer program; a processor 61 for implementing the steps of the method for configuring an optical interference unit of a photonic neural network as mentioned in any of the above embodiments when executing a computer program.
The processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 61 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 61 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 61 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 61 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.
Memory 60 may include one or more computer-readable storage media, which may be non-transitory. Memory 60 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 60 is at least used for storing a computer program 601, wherein the computer program is loaded and executed by the processor 61, and then the relevant steps of the optical interference unit configuration method of the photonic neural network disclosed in any one of the foregoing embodiments can be implemented. In addition, the resources stored by the memory 60 may also include an operating system 602, data 603, and the like, and the storage may be transient storage or permanent storage. Operating system 602 may include Windows, Unix, Linux, etc., among others. The data 603 may include, but is not limited to, data corresponding to the result of the configuration of the optical interference unit of the photonic neural network, and the like.
In some embodiments, the optical interference unit configuration device of the photonic neural network may further include a display 62, an input/output interface 63, a communication interface 64 or network interface, a power supply 65, and a communication bus 66. The display 62 and the input/output interface 63, such as a Keyboard (Keyboard), belong to a user interface, and the optional user interface may also include a standard wired interface, a wireless interface, and the like. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, as appropriate, is used for displaying information processed in the optical interference unit configuration means of the photonic neural network and for displaying a visualized user interface. The communication interface 64 may optionally include a wired interface and/or a wireless interface, such as a WI-FI interface, a bluetooth interface, etc., typically used to establish a communication link between the optical interference unit configuration apparatus of the photonic neural network and other electronic devices. The communication bus 66 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 6, but this is not intended to represent only one bus or type of bus.
It will be understood by those skilled in the art that the structure shown in fig. 6 does not constitute a limitation of the optical interference unit configuration means of the photonic neural network, and may include more or less components than those shown, for example, and may further include a sensor 67 for performing various functions.
The functions of the functional modules of the optical interference unit configuration device of the photonic neural network according to the embodiments of the present invention may be specifically implemented according to the method in the above method embodiments, and the specific implementation process may refer to the description related to the above method embodiments, and will not be described herein again.
From the above, the embodiment of the invention can effectively improve the configuration efficiency of the optical interference unit of the photonic neural network.
It is understood that, if the optical interference unit configuration method of the photonic neural network in the above embodiment is implemented in the form of a software functional unit and sold or used as a separate product, it may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application may be substantially or partially implemented in the form of a software product, which is stored in a storage medium and executes all or part of the steps of the methods of the embodiments of the present application, or all or part of the technical solutions. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), an electrically erasable programmable ROM, a register, a hard disk, a removable magnetic disk, a CD-ROM, a magnetic or optical disk, and other various media capable of storing program codes.
Based on this, the embodiment of the present invention further provides a computer readable storage medium, which stores an optical interference unit configuration program of a photonic neural network, where the optical interference unit configuration program of the photonic neural network is executed by a processor, and the steps of the optical interference unit configuration method of the photonic neural network are as described in any one of the above embodiments.
The functions of the functional modules of the computer-readable storage medium according to the embodiment of the present invention may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the related description of the foregoing method embodiment, which is not described herein again.
From the above, the embodiment of the invention can effectively improve the configuration efficiency of the optical interference unit of the photonic neural network.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The above detailed description describes a method, an apparatus, and a computer readable storage medium for configuring an optical interference unit of a photonic neural network provided in the present application. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present application.

Claims (10)

1. A method for configuring an optical interference unit of a photonic neural network, comprising:
decomposing a weight real matrix constructed by each set of weight values of a convolution kernel of the photon neural network in convolution operation to obtain a unitary matrix for calculating an interference light path link topological structure;
determining a minimum multiply-add unit of the interference light path link topological structure based on the light path structure and the light conduction characteristics of an optical interference unit of the photonic neural network;
performing multiple elimination operations based on the optical signal input relation correspondingly represented by the minimum multiplication and addition unit and the unitary matrix; the number of the element eliminating operations is smaller than the order number of the unitary matrix;
and determining all configuration modes of the optical interference unit according to the elimination element operation result.
2. The method for configuring optical interference unit of photonic neural network according to claim 1, wherein said determining the minimum multiply-add unit of the interference optical path link topology based on the optical path structure and optical transmission characteristics of the optical interference unit of photonic neural network comprises:
the optical interference unit comprises a first input end, a second input end, a first phase shifter, a first optical splitter, a second phase shifter and a second optical splitter; the optical signals input through the first input end and the second input end are projected to the first optical splitter through the first phase shifter, enter the second phase shifter after being processed by the first optical splitter, enter the second optical splitter through the second phase shifter for processing, and are output;
determining an optical conversion relationship between the input optical signal and the output optical signal according to the optical structure of the optical interference unit and the optical conduction characteristic as follows:
Figure FDA0003136890230000011
in the formula (I), the compound is shown in the specification,
Figure FDA0003136890230000012
is an input optical signal at said first input terminal,
Figure FDA0003136890230000013
is the input optical signal at the second input terminal,
Figure FDA0003136890230000014
is the final output optical signal after the input optical signal of the first input end is processed by the optical interference unit,
Figure FDA0003136890230000015
the final output optical signal is the input optical signal of the second input end after being processed by the optical interference unit; theta is the phase modulation angle of the second phase shifter, phi is the phase modulation angle of the first phase shifter, e represents an index, i represents a virtualCounting;
based on the optical conversion relationship, setting the phase modulation angle of the first phase shifter to 0, and obtaining the minimum multiplication and addition unit R as:
Figure FDA0003136890230000021
3. the method for configuring optical interference units of a photonic neural network according to claim 2, wherein decomposing a real weight matrix constructed by each set of weight values of a convolution kernel of the photonic neural network in a convolution operation to obtain a unitary matrix for calculating an interference optical path link topology comprises:
acquiring all weight values for performing convolution operation on a current convolution kernel;
expanding the weight value of the current convolution kernel needing to be operated into a matrix structure; each row of the matrix structure corresponds to a group of weight values of convolution operation, and each column corresponds to one to-be-input optical signal data;
and carrying out singular value decomposition on the matrix structure to obtain the unitary matrix.
4. The method of any one of claims 1 to 3, wherein the performing a plurality of apogee operations based on the unitary matrix and the optical signal input relationship correspondingly represented by the minimum multiply-add unit comprises:
constructing a diagonal matrix for representing the corresponding input relation of the optical interference unit according to the fact that the minimum multiplication and addition unit has unitary matrix characteristics;
performing matrix operation-based expansion on the diagonal matrix based on diagonal elements of the diagonal matrix to obtain a configuration relation matrix;
and carrying out elimination processing on the configuration relation matrix according to a preset elimination method, and calculating the input relation contained in the configuration relation matrix by combining the unitary matrix.
5. The method of claim 4, wherein the determining all configurations of the optical interference unit according to the result of the elimination operation comprises:
and determining the configuration mode of the optical interference unit according to the input relation calculated in each elimination element operation processing process.
6. An optical interference unit configuration device of a photonic neural network, comprising:
the topological structure representation module is used for decomposing a weight real matrix constructed by each set of weight values of a convolution kernel of the photon neural network in convolution operation to obtain a unitary matrix for calculating an interference light path link topological structure;
the basic unit determining module of the topological structure is used for determining the minimum multiplication and addition unit of the interference light path link topological structure based on the light path structure and the light conduction characteristic of the optical interference unit of the photonic neural network;
the elimination processing is used for carrying out multiple elimination operations based on the optical signal input relation correspondingly represented by the minimum multiplication and addition unit and the unitary matrix; the number of the element eliminating operations is smaller than the order number of the unitary matrix;
and the configuration mode determining module is used for determining all configuration modes of the optical interference unit according to the elimination element operation result.
7. The apparatus of claim 6, wherein the topology base unit determining module is further configured to:
the optical interference unit comprises a first input end, a second input end, a first phase shifter, a first optical splitter, a second phase shifter and a second optical splitter; the optical signals input through the first input end and the second input end are projected to the first optical splitter through the first phase shifter, enter the second phase shifter after being processed by the first optical splitter, enter the second optical splitter through the second phase shifter for processing, and are output; determining an optical conversion relationship between the input optical signal and the output optical signal according to the optical structure of the optical interference unit and the optical conduction characteristic as follows:
Figure FDA0003136890230000031
in the formula (I), the compound is shown in the specification,
Figure FDA0003136890230000032
is an input optical signal at said first input terminal,
Figure FDA0003136890230000033
is the input optical signal at the second input terminal,
Figure FDA0003136890230000034
is the final output optical signal after the input optical signal of the first input end is processed by the optical interference unit,
Figure FDA0003136890230000035
the final output optical signal is the input optical signal of the second input end after being processed by the optical interference unit; theta is the phase modulation angle of the second phase shifter, phi is the phase modulation angle of the first phase shifter, e represents an index, and i represents an imaginary number;
based on the optical conversion relationship, setting the phase modulation angle of the first phase shifter to 0, and obtaining the minimum multiplication and addition unit R as:
Figure FDA0003136890230000036
8. the apparatus of claim 7, wherein the topology representation module is further configured to:
acquiring all weight values for performing convolution operation on a current convolution kernel;
expanding the weight value of the current convolution kernel needing to be operated into a matrix structure; each row of the matrix structure corresponds to a group of weight values of convolution operation, and each column corresponds to one to-be-input optical signal data;
and carrying out singular value decomposition on the matrix structure to obtain the unitary matrix.
9. An optical interference unit configuration apparatus of a photonic neural network, comprising a processor for implementing the steps of the optical interference unit configuration method of the photonic neural network according to any one of claims 1 to 5 when executing a computer program stored in a memory.
10. A computer-readable storage medium, on which an optical interference unit configuration program of a photonic neural network is stored, which when executed by a processor implements the steps of the optical interference unit configuration method of the photonic neural network according to any one of claims 1 to 5.
CN202110722256.3A 2021-06-28 2021-06-28 Optical interference unit configuration method and device of photonic neural network and storage medium Pending CN113627605A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110722256.3A CN113627605A (en) 2021-06-28 2021-06-28 Optical interference unit configuration method and device of photonic neural network and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110722256.3A CN113627605A (en) 2021-06-28 2021-06-28 Optical interference unit configuration method and device of photonic neural network and storage medium

Publications (1)

Publication Number Publication Date
CN113627605A true CN113627605A (en) 2021-11-09

Family

ID=78378496

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110722256.3A Pending CN113627605A (en) 2021-06-28 2021-06-28 Optical interference unit configuration method and device of photonic neural network and storage medium

Country Status (1)

Country Link
CN (1) CN113627605A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113935474A (en) * 2021-12-15 2022-01-14 苏州浪潮智能科技有限公司 Method and system for realizing recurrent neural network, electronic equipment and storage medium
CN114037070A (en) * 2022-01-07 2022-02-11 苏州浪潮智能科技有限公司 Optical signal processing method, photonic neural network chip and design method thereof
WO2023005084A1 (en) * 2021-07-29 2023-02-02 浪潮电子信息产业股份有限公司 Optical circuit building method, optical circuit, and optical signal processing method and apparatus
WO2023179374A1 (en) * 2022-03-24 2023-09-28 苏州浪潮智能科技有限公司 Method and apparatus for training optical neural network, and device and medium

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023005084A1 (en) * 2021-07-29 2023-02-02 浪潮电子信息产业股份有限公司 Optical circuit building method, optical circuit, and optical signal processing method and apparatus
CN113935474A (en) * 2021-12-15 2022-01-14 苏州浪潮智能科技有限公司 Method and system for realizing recurrent neural network, electronic equipment and storage medium
CN113935474B (en) * 2021-12-15 2022-03-08 苏州浪潮智能科技有限公司 Method and system for realizing recurrent neural network, electronic equipment and storage medium
CN114037070A (en) * 2022-01-07 2022-02-11 苏州浪潮智能科技有限公司 Optical signal processing method, photonic neural network chip and design method thereof
WO2023179374A1 (en) * 2022-03-24 2023-09-28 苏州浪潮智能科技有限公司 Method and apparatus for training optical neural network, and device and medium

Similar Documents

Publication Publication Date Title
CN113627605A (en) Optical interference unit configuration method and device of photonic neural network and storage medium
CN113298246B (en) Data processing method, device and computer readable storage medium
US9704094B2 (en) Mapping of algorithms to neurosynaptic hardware
Zhou et al. Chip-scale optical matrix computation for PageRank algorithm
US10831691B1 (en) Method for implementing processing elements in a chip card
Wang et al. PR-ELM: Parallel regularized extreme learning machine based on cluster
CN113408720B (en) Optical neural network, data processing method and device based on optical neural network, and storage medium
KR20210125405A (en) Question analysis method, apparatus, knowledge graph question-answering system and electronic equipment
Giamougiannis et al. Neuromorphic silicon photonics with 50 GHz tiled matrix multiplication for deep-learning applications
US20240184847A1 (en) Optical circuit building method, optical circuit, and optical signal processing method and apparatus
Li et al. Quantum circuit design for several morphological image processing methods
Huang et al. Easily scalable photonic tensor core based on tunable units with single internal phase shifters
Kitayama Guiding principle of reservoir computing based on “small-world” network
US11209856B2 (en) Path-number-balanced universal photonic network
CN113325650B (en) Optical circuit, optical signal processing method, optical signal processing device and readable storage medium
Li et al. High-efficiency reinforcement learning with hybrid architecture photonic integrated circuit
CN113592084B (en) Photon neural network on chip based on reverse optimization superstructure convolution kernel
US20220121901A1 (en) Gated unit for a gated recurrent neural network
Hirose et al. Complex-valued neural networks for wave-based realization of reservoir computing
CN113392966A (en) Method, equipment and storage medium for realizing neural network average pooling
Shiflett et al. Bitwise neural network acceleration using silicon photonics
Kishore et al. Implementation of braun and baugh-wooley multipliers using qca
TW202013265A (en) Methods for computing convolutions using programmable nanophotonics
CN114021729B (en) Quantum circuit operation method and system, electronic device and medium
CN114548381B (en) Activation function construction method, device, equipment and medium of all-optical neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination