WO2021170107A1 - 一种光计算装置、系统以及计算方法 - Google Patents

一种光计算装置、系统以及计算方法 Download PDF

Info

Publication number
WO2021170107A1
WO2021170107A1 PCT/CN2021/078233 CN2021078233W WO2021170107A1 WO 2021170107 A1 WO2021170107 A1 WO 2021170107A1 CN 2021078233 W CN2021078233 W CN 2021078233W WO 2021170107 A1 WO2021170107 A1 WO 2021170107A1
Authority
WO
WIPO (PCT)
Prior art keywords
group
optical signals
optical
weight
module
Prior art date
Application number
PCT/CN2021/078233
Other languages
English (en)
French (fr)
Inventor
张佳旭
须江
王治飞
陈宣奇
陈世希
董晓文
嵇绍波
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP21760588.0A priority Critical patent/EP4099232A4/en
Publication of WO2021170107A1 publication Critical patent/WO2021170107A1/zh
Priority to US17/896,143 priority patent/US20220414442A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06E—OPTICAL COMPUTING DEVICES; COMPUTING DEVICES USING OTHER RADIATIONS WITH SIMILAR PROPERTIES
    • G06E3/00—Devices not provided for in group G06E1/00, e.g. for processing analogue or hybrid data
    • G06E3/001—Analogue devices in which mathematical operations are carried out with the aid of optical or electro-optical elements
    • G06E3/005—Analogue devices in which mathematical operations are carried out with the aid of optical or electro-optical elements using electro-optical or opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06E—OPTICAL COMPUTING DEVICES; COMPUTING DEVICES USING OTHER RADIATIONS WITH SIMILAR PROPERTIES
    • G06E3/00—Devices not provided for in group G06E1/00, e.g. for processing analogue or hybrid data
    • G06E3/008—Matrix or vector computation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
    • G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
    • G06F7/544—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices for evaluating functions by calculation
    • G06F7/5443—Sum of products
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]

Definitions

  • This application relates to the field of information technology, and in particular to an optical computing device, system and computing method.
  • Neural network models such as convolutional layers and fully connected layers, usually require a large amount of data operations, and in the process of data operations, a large amount of process data needs to be stored.
  • the optical computing system can be used to realize the data calculation process of the neural network model and obtain the final calculation result.
  • Optical computing systems can be divided into two types: space optical computing systems and on-chip optical computing systems according to their implementation forms.
  • the spatial light computing system is mainly based on the 4F optical system, which relies on the Fourier optical transform to implement convolution operations and simulate neural network models such as convolutional neural networks (CNN).
  • CNN convolutional neural networks
  • the on-chip optical computing system is based on silicon-based photonic integration technology, and mainly uses optical devices such as Mach-Zehnder interferometer (MZI) to construct optical multipliers and adders to achieve convolution operations.
  • MZI Mach-Zehnder interferometer
  • this type of optical computing system Including a limited number of MZIs, multiple multiplication and addition iterations are required to achieve multiplication and addition operations, and the on-chip optical computing system uses heating electrodes to adjust the phase of the optical signal, which makes the on-chip optical computing system prone to thermal crosstalk and reduces the accuracy of the calculation. Makes the efficiency of the entire on-chip optical computing system poor.
  • the present application provides an optical computing device, system, and computing method to provide an optical computing system with higher efficiency.
  • an embodiment of the present application provides an optical computing device, which includes a linear computing module, a first delay module, and a coupler.
  • the linear operation module may receive the first set of optical signals used to indicate the first set of data, and may also receive the first electrical signal indicating the first part of the weight value in the first weight.
  • the linear operation module modulates the first set of optical signals according to the first electrical signal to output a second set of optical signals, where the second set of optical signals is used to indicate the calculation result of the first set of data and the first part of the weight value, and the first weight Is the weight of the first neural network layer.
  • the linear operation module may subsequently receive a third group of optical signals for indicating the first group of data, and a second electrical signal for indicating a second part of the weight value in the first weight.
  • the linear operation module modulates the third group of optical signals according to the second electrical signal to output a fourth group of optical signals, where the fourth group of optical signals is used to indicate the calculation result of the first group of data and the second part of the weight value.
  • the first delay module can adjust the delay of the second group of optical signals; after that, the coupler can combine the delayed second group of optical signals and the fourth group of optical signals into a fifth group of optical signals, where the fifth group
  • the light signal indicates the first calculation result of the multiplication and addition of the first set of data, the first part weight and the second part weight.
  • the linear computing unit in the optical computing device can output a set of optical signals that can indicate the calculation result of the multiplication and addition operation by sequentially receiving multiple sets of optical signals and electrical signals, in cooperation with the coupler and the first delay module.
  • different multiplication and addition operations can be realized by adjusting the input multiple sets of optical properties and electrical signals.
  • the optical computing device may also include a filter module.
  • the optical computing device may also use the filter module to implement pooling operations. Take the optical computing device to perform the pooling operation on the sixth group of optical signals as an example for description.
  • the linear operation module can receive a sixth group of optical signals and a third electrical signal indicating the second weight, where the sixth group of optical signals is used to indicate the second calculation result after the first group of data and the first weight are multiplied and added. , Where the second calculation result may include the first calculation result, and the second weight is the weight of the second neural network layer.
  • the linear operation module modulates the sixth group of optical signals according to the third electrical signal to output the seventh group of optical signals.
  • the filter module can receive the seventh group of optical signals, filter the invalid signals in the seventh group of optical signals, and output the eighth group.
  • the optical signal, the eighth group of optical signals is used to indicate the calculation result of the pooling operation on the second calculation result.
  • the optical computing device can not only implement multiplication and addition operations but also implement pooling operations, which improves the versatility of the optical computing device.
  • the optical computing device may also implement convolution operation by means of the filter module. Take the optical computing device performing the convolution operation on the ninth group of optical signals as an example for description.
  • the linear operation module may receive the ninth group of optical signals and the fourth electrical signal indicating the third weight, and the ninth group of optical signals are used to indicate the second group of data.
  • the linear operation module can modulate the ninth group of optical signals according to the fourth electrical signal to output the tenth group of optical signals.
  • the tenth group of optical signals is used to indicate the third calculation after the second group of data and the third weight are multiplied and added.
  • the third weight is the weight of the third neural network layer;
  • the filter module receives the tenth group of optical signals, filters out the invalid signals in the tenth group of optical signals, and outputs the eleventh group of optical signals.
  • the eleventh group of optical signals is used to indicate the calculation result of the convolution operation on the third calculation result .
  • the optical computing device can not only implement multiplication and addition operations, but also convolution operations and pooling operations, and further improve the versatility of the optical computing device, so that the optical computing device can implement more neural network models. Operation.
  • the optical computing device may also include a nonlinear operation module to implement the activation function configured in the neural network model.
  • the nonlinear operation module can perform nonlinear operations on the signals output by the coupler, such as the sixth group of optical signals or the fifth group of optical signals, the eighth group of optical signals or the eleventh group of optical signals, and the nonlinear operation is an activation function operation.
  • the optical computing device in addition to multiplying and adding operations, convolution operations, and pooling operations, the optical computing device can also implement activation function operations on the calculation results of multiplying and adding operations, convolution operations, and pooling operations, ensuring optical computing The versatility of the device.
  • the optical computing device may also include a dual-mode beam splitter.
  • the dual-mode beam splitter can divide the received set of optical signals indicating a set of data into two identical sets indicating the same set of data.
  • Light signal Exemplarily, the dual-mode beam splitter may receive the twelfth group of optical signals for indicating the first group of data, and divide the twelfth group of optical signals into the first group of optical signals and the third group of optical signals.
  • the dual-mode beam splitter set by the optical computing device does not need to input a group of optical signals indicating the same data to the optical computing device multiple times, only one group of optical signals needs to be input, and multiple groups of instructions can be split.
  • the optical signal of the same data can effectively improve the calculation efficiency of the optical computing device and improve the performance of the optical computing device.
  • the optical computing device further includes a second delay module, and the dual-mode beam splitter can transmit the delayed second group of optical signals to the second delay module; the second delay module can be adjusted The time delay of the second group of optical signals, so that the second group of optical signals and the fourth group of optical signals are simultaneously transmitted to the coupler.
  • the coupler can receive multiple sets of optical signals at the same time, and can better combine a set of optical signals.
  • the linear operation module includes N photoelectric modulators.
  • any one of the N photoelectric modulators can receive one of the optical signals in the first group of optical signals and one of the first electrical signals for indicating the weight of the first part, and the received electrical signals are The received optical signal is modulated; similarly, any photoelectric modulator can also receive an optical signal in the third group of optical signals and an electrical signal in the second electrical signal for indicating the weight of the second part, according to the received The electrical signal modulates the received optical signal.
  • the linear computing module can better implement multiplication and addition operations with the help of N photoelectric modulators, and ensure the efficient computing of the optical computing device.
  • the linear operation module also includes N phase modulators and N-1 phase lock modules;
  • a phase modulator can be connected to an optoelectronic modulator, and the phase modulator can adjust the phase of the first group of optical signals or the second group of optical signals modulated by the modulator; a phase lock module can be connected to two of the N phase modulators A phase modulator is used to lock the phase of the first group of optical signals or the second group of optical signals modulated by the two phase modulators.
  • the phase of the optical signal can be adjusted by the phase modulator, and the symbol of the data indicated by the optical signal can be adjusted.
  • the phase lock module can ensure that the phase modulator can effectively adjust the phase of the optical signal.
  • the linear operation module further includes an amplifier, which can amplify the power of the fifth group of optical signals, the sixth group of optical signals, the eighth group of optical signals, or the eleventh group of optical signals.
  • the loss of the optical signal in the optical computing device can be reduced by the amplifier.
  • the present application provides a calculation method.
  • the method is executed by an optical computing device, and the method includes:
  • the linear operation module receives a first group of optical signals and a first electrical signal, where the first group of optical signals is used to indicate the first group of data, and the first electrical signal is used to indicate the first part of the weight value of the first weight;
  • the electrical signal modulates the first set of optical signals to output a second set of optical signals, where the second set of optical signals is used to indicate the calculation result of the first set of data and the first part of the weight value, and the first weight is the first neural network layer the weight of;
  • the linear operation module receives a third group of optical signals and a second electrical signal, where the third group of optical signals is used to indicate the first group of data, and the second electrical signal is used to indicate the second part of the weight value of the first weight, and according to The second electrical signal modulates the third group of optical signals to output a fourth group of optical signals, where the fourth group of optical signals is used to indicate the calculation result of the first group of data and the second part of the weight value.
  • the first delay module adjusts the delay of the second group of optical signals.
  • the coupler combines the delayed second group of optical signals and the fourth group of optical signals into a fifth group of optical signals, where the fifth group of optical signals instructs the first group of data to multiply and add the first group of weights and the second group of weights The first calculation result after the operation.
  • the linear operation module receives the sixth group of optical signals, and the sixth group of optical signals is used to indicate the second calculation result after the first set of data and the first weight are multiplied and added, and to indicate the second calculation result.
  • the linear operation module modulates the sixth group of optical signals according to the third electrical signal to output the seventh group of optical signals.
  • the filtering module receives the seventh group of optical signals, filters the invalid signals in the seventh group of optical signals, and outputs the eighth group of optical signals.
  • the eighth group of optical signals is used to indicate the calculation result of the pooling operation on the second calculation result.
  • the linear operation module receives the ninth group of optical signals, and the ninth group of optical signals is used to indicate the second group of data.
  • the linear operation module receives the fourth electrical signal for indicating the third weight, and modulates the ninth group of optical signals according to the fourth electrical signal to output the tenth group of optical signals, and the tenth group of optical signals is used to indicate the second group of data
  • the third calculation result after multiplying and adding with the third weight, and the third weight is the weight of the third neural network layer;
  • the filter module receives the tenth group of optical signals, filters out the invalid signals in the tenth group of optical signals, and outputs the eleventh group of optical signals.
  • the eleventh group of optical signals is used to indicate the calculation result of the convolution operation on the third calculation result .
  • the present application provides an optical computing system.
  • the optical computing device may include a processor and the optical computing device as described in the first aspect or any one of the first aspect.
  • the processing The device is used to transmit the data to be calculated to the optical computing device, such as the first group of data or the second group of data, and can also transmit the first weight or the second weight.
  • this application provides a computer-readable storage medium.
  • the computer-readable storage medium When the computer-readable storage medium is executed by a computing device, the computing device executes the foregoing second aspect or any possible design of the second aspect. method.
  • the storage medium stores the program.
  • the storage medium includes, but is not limited to, volatile memory, such as random access memory, non-volatile memory, such as flash memory, hard disk drive (HDD), and solid state drive (SSD).
  • this application provides a computing device program product.
  • the computing device program product includes computer instructions. When executed by a computing device, the computing device executes the foregoing second aspect or any possible design of the second aspect.
  • the computer program product may be a software installation package. In the case where the method provided in the foregoing second aspect or any possible design of the second aspect needs to be used, the computer program product may be downloaded and the computer program may be executed on a computing device product.
  • FIG. 1 is a schematic diagram of the structure of an optical computing device provided by this application.
  • FIG. 2 is a schematic diagram of a transmission path of an optical signal provided by this application.
  • Figure 3a is a schematic diagram of a multiplication and addition operation provided by this application.
  • FIG. 3b is a schematic diagram of a first part weight value and a second part weight value provided by this application;
  • Figure 4a is a schematic structural diagram of an optical computing device provided by this application.
  • 4b is a schematic structural diagram of an optical computing device provided by this application.
  • FIG. 5 is a schematic structural diagram of an optical computing device provided by this application.
  • Fig. 6a is a schematic structural diagram of an optical computing device provided by this application.
  • FIG. 6b is a schematic structural diagram of an optical computing device provided by this application.
  • FIG. 7 is a schematic structural diagram of an optical computing device provided by this application.
  • FIG. 8a is a schematic diagram of a transmission path of an optical signal provided by this application.
  • FIG. 8b is a schematic diagram of a transmission path of an optical signal provided by this application.
  • FIG. 9a is a schematic structural diagram of a linear operation module provided by this application.
  • FIG. 9b is a schematic structural diagram of a linear operation module provided by this application.
  • FIG. 10 is a schematic structural diagram of a first delay module provided by this application.
  • FIG. 11 is a schematic structural diagram of a filtering module provided by this application.
  • FIG. 12 is a schematic structural diagram of a dual-mode beam splitter provided by this application.
  • FIG. 13 is a schematic diagram of a multiplication and addition operation provided by this application.
  • FIG. 14 is a schematic diagram of a convolution operation provided by this application.
  • FIG. 15a is a schematic diagram of an input serial optical signal and an output optical signal in the first linear operation process provided by this application;
  • 15b is a schematic diagram of an input serial optical signal and an output optical signal in a second linear operation process provided by this application;
  • FIG. 15c is a schematic diagram of an optical signal filtered by a filter module provided by this application.
  • 15d is a schematic diagram of an optical signal output by an optical computing device provided by this application.
  • FIG. 16 is a schematic diagram of a pooling operation provided by this application.
  • FIG. 17 is a schematic diagram of an input serial optical signal and an output optical signal in a linear operation process provided by this application;
  • Figure 18 is a calculation method provided by this application.
  • Figure 19 is an optical computing system provided by this application.
  • an embodiment of the present application provides an optical computing device.
  • the optical computing device includes a linear computing module 100, a first delay module 200, and a coupler 300.
  • the embodiments of the present application do not limit the specific form of the optical computing device.
  • the optical computing device may be an optical computing chip.
  • the linear operation module 100 can modulate the optical signal input to the linear operation module 100 according to the received electrical signal.
  • the first delay module 200 can adjust the delay of the optical signal output by the linear operation module 100.
  • the coupler 300 can combine the linear operation module 100 to output multiple groups of optical signals successively, and output a group for indicating a group of data and The light signal of the calculation result of the weighted multiplication and addition operation.
  • the linear operation module 100 may also receive the first electrical signal used to indicate the weight value of the first part of the first weight.
  • the linear operation module 100 may The first electrical signal modulates the first group of optical signals, modulates the light intensity of the first group of optical signals, and outputs the second group of optical signals.
  • the output second group of optical signals can indicate the difference between the first group of data and the first part of the weight value. Calculation results.
  • the first weight is the weight set on the first neural network layer.
  • the linear calculation module 100 can perform a limited amount of calculations for an optical signal in a linear calculation process, that is to say, the linear calculation module 100 can only output the multiplication indicating the first set of data and part of the weight (such as the first part of the weight value).
  • the optical signal of the addition operation If the linear operation module 100 needs to perform all calculations, the linear operation module 100 also needs to continue to receive the next set of optical signals for linear operation, so that the coupler 300 can subsequently combine the linear operation module 100 to output multiple sets of different optical signals.
  • a set of optical signals indicating the calculation result of the multiplication and addition operation between the first set of data and the first weight is output.
  • the optical signal used for indicating the first group of data subsequently received by the linear operation module 100 is the third group of optical signals as an example.
  • the linear operation module 100 may also receive a second electrical signal used to indicate the second part of the weight value in the first weight.
  • the three sets of optical signals are modulated to modulate the light intensity of the third set of optical signals, and the fourth set of optical signals are output.
  • the output fourth set of optical signals can indicate the calculation results of the first set of data and the second part of the weight value.
  • the linear operation module 100 successively outputs the second group of optical signals and the fourth group of optical signals, and there is a time delay between the two optical signals, and they cannot be directly combined.
  • the first delay module 200 can adjust the delay of the second group of optical signals; after that, after the coupler 300 receives the fourth group of optical signals and the second group of optical signals with the delay adjusted, the second group of optical signals can be adjusted.
  • the optical signal and the delayed fourth group of optical signals are combined into a fifth group of optical signals, where the fifth group of optical signals is used to indicate the calculation result of the first group of data and the first part weight and the second part weight after the multiplication and addition operation .
  • the following describes the transmission paths of the second group of optical signals and the fourth group of optical signals in the optical computing device.
  • Figure 2 uses a black solid line to mark the transmission path 1 of the second group of optical signals output from the linear arithmetic module 100.
  • the second group of optical signals are output from the linear arithmetic module 100, It is transmitted to the first delay module 200, and then output from the first delay module 200 to the coupler 300.
  • Figure 2 uses the black dashed line to mark the transmission path 2 of the fourth group of optical signals after being output from the linear arithmetic module 100. It can be seen from Fig. 2 that the fourth group of optical signals can be directly transmitted to after being output from the linear arithmetic module 100 Coupler 300.
  • the second set of optical signals can indicate the calculation result of the first set of data and the first part of the weight value after the multiplication and addition operation
  • the fourth set of optical signals can indicate the first set of data
  • the coupler 300 After multiplying and adding the result of the second part of the weight value, after adjusting the delay of the second group of optical signals through the first delay module 200, the coupler 300 combines the delayed second group of optical signals with the fourth group of optical signals.
  • the optical signal is coupled into the fifth group of optical signals, and the multiplication and addition operation between the first group of data and the first part weight and the second part weight is also completed, and the calculation process of the fully connected layer is realized.
  • the left side of Figure 3a is a multiplication and addition operation that the fully connected layer needs to implement. For more intuitive, it can be converted to the right side of Figure 3a.
  • the linear operation is a 4x4 linear operation, which requires four input data (X c1 , X c2 , X c3 , X c4 ) and four output data (X n1 , X n2 , X n3 , X n4 ), and it needs to be applied
  • the weight of is (W 11 ⁇ W 44 ).
  • the linear operation module 100 can only perform two different modulations on one optical signal at a time (that is, the linear operation module 100 includes only two photoelectric modulators), it can be seen from the multiplication and addition operation shown in Figure 3a that for any input Data requires four different adjustments. Taking X c1 as an example, four different adjustments need to be performed on X c1 .
  • the linear operation module 100 needs to receive two sets of optical signals indicating X c1 , X c2 , X c3 , and X c4 , such as the first set of optical signals and the third set of optical signals, It is also necessary to successively receive the first electrical signal indicating the weight value of the first part and the second electrical signal indicating the weight value of the second part.
  • the solid line box is the weight value of the first part
  • the dashed line box is the weight value of the second part. It can cooperate with the first delay module 200 and the coupler 300, and output indications X n1 , X n2 , X n3 , X The fifth group of optical signals of n4.
  • the linear operation module 100 receives two sets of optical signals indicating the first set of data as an example. There is a third part of the weight value, the linear operation module 100 can continue to receive a group of optical signals indicating the first group of data, and the linear operation module 100 modulates the group of optical signals according to the electrical signal indicating the third part of the weight value, and outputs the modulation
  • the first delay module 200 can delay the fifth group of optical signals
  • the coupler 300 can combine the fifth group of optical signals and the modulated group of optical signals into a group of optical signals. .
  • the optical computing device can also use the aforementioned similar process, and the linear operation module 100 continues to receive a group of optical signals indicating the first group of data, and the group of optical signals For modulation, the first delay module 200 delays a group of optical signals that have been combined before, and the coupler 300 combines a group of optical signals delayed by the first delay module 200 with the one modulated by the linear operation module 100 this time. Groups of optical signals are combined, and the above process is performed cyclically until the multiplication and addition operation between the first group of data and the first weight is completed. In the embodiment of the present application, the sixth group of optical signals is used to indicate the difference between the first group of data and the first weight. The calculation result of the multiplication and addition operation.
  • the optical computing device can also perform operations on other neural network models, such as the convolution operation of the convolutional layer and the pooling operation of the pooling layer. Because of the need to remove some unnecessary operations during the convolution operation or pooling operation
  • the data involved in the calculation may further include a filtering module 400 in the optical computing device.
  • a filter module 400 can be provided on the transmission path of the second group of optical signals.
  • the filter module 400 can filter some optical signals that do not need to perform calculations from a group of optical signals, that is, invalid signals.
  • the signal indicates invalid data.
  • the remaining optical signals in the group of optical signals are transmitted to the first delay module 200 along the transmission path of the second group of optical signals.
  • the transmission process of the optical signals in the optical computing device is realized when the optical computing device is pooled. Be explained.
  • the coupler 300 outputs the sixth group of optical signals (the process of generating the sixth group of optical signals can be referred to the foregoing content, and will not be repeated here), the sixth group of optical signals passes through the first delay module 200 and the filter module 300 Transfer to the linear operation module 100.
  • the linear operation module 100 modulates the sixth group of optical signals according to the received third electrical signal indicating the second weight, and outputs the seventh group of optical signals, where the second weight is the second neural network layer (such as the pooling layer)
  • the seventh group of optical signals is transmitted to the filter module 400 through the first delay module 200 (the first delay module 200 may not adjust the delay of the second group of optical signals), and the sixth group of optical signals is transmitted to the filter module 400 ,
  • the filtering module 400 filters invalid signals from the seventh group of optical signals.
  • the invalid signals refer to invalid data that needs to be deleted in the pooling operation, and output the remaining optical signals in the seventh group of optical signals.
  • the eighth group of optical signals is used Indicate the remaining optical signals in the seventh group of optical signals, and the eighth group of optical signals can indicate the final result of the pooling operation on the calculation result of the multiplication and addition operation between the first group of data and the first weight.
  • the following describes the convolution operation of the optical computing device as an example. It is distinguished from the foregoing groups of optical signals and individual electrical signals.
  • the group of optical signals received by the linear operation module 100 is the ninth group of optical signals, and the received electrical signals are
  • the fourth optical signal indicating the third weight is taken as an example for description.
  • the third weight is the weight set on the third neural network layer (such as the convolutional layer).
  • the linear operation module 100 modulates the ninth group of optical signals according to the received fourth electrical signal, modulates the light intensity of the ninth group of optical signals, and outputs the tenth group of optical signals.
  • the output tenth group of optical signals can indicate the second group of optical signals. Group data and the calculation result of the third weight.
  • the linear operation module 100 receives a group of optical signals and outputs the tenth group of optical signals as an example for description. In actual operation, it is coupled with the linear operation module 100 and the first delay module 200.
  • the way of outputting the sixth group of optical signals is similar to the way of outputting the sixth group of optical signals.
  • the linear operation module 100 can also receive multiple groups of optical signals successively, and cooperate with the first delay module 200 and the coupler 300 to execute the linear operation multiple times in a loop to obtain the tenth group of optical signals. Group light signal.
  • the tenth group of optical signals is transmitted to the filtering module 400 through the first delay module 200 (the first delay module 200 may not adjust the delay of the tenth group of optical signals), and the tenth group of optical signals is transmitted to the filtering module 400, and the The module 400 filters invalid signals from the tenth group of optical signals.
  • the invalid signals refer to the invalid data that needs to be deleted in the pooling operation, and output the remaining optical signals in the tenth group of optical signals.
  • the eleventh group of optical signals is used to indicate For the remaining optical signals in the tenth group of optical signals, the eleventh group of optical signals can indicate the final result of the convolution operation on the second group of data.
  • first neural network layer, the second neural network layer, and the third neural network layer in the foregoing content can be any neural network layer in any neural network model, the first neural network layer and the second neural network layer It can also be two adjacent neural network layers in a neural network model.
  • the optical computing device may also include a second delay module 500, As shown in FIG. 4b, the second delay module 500 can be placed on the transmission path of the second group of optical signals to further adjust the delay of the second group of optical signals.
  • the control system may also include a dual-mode beam splitter 600, which can receive the first group of optical signals and the third group of optical signals, and The first group of optical signals and the third group of optical signals are transmitted to the linear operation module 100, and the dual-mode beam splitter 600 can also receive the second group of optical signals, and transmit the second group of optical signals to the second delay module 500 or the first Delay module 200.
  • a dual-mode beam splitter 600 which can receive the first group of optical signals and the third group of optical signals, and The first group of optical signals and the third group of optical signals are transmitted to the linear operation module 100, and the dual-mode beam splitter 600 can also receive the second group of optical signals, and transmit the second group of optical signals to the second delay module 500 or the first Delay module 200.
  • the dual-mode beam splitter 600 can also split a group of optical signals into The two groups of optical signals are the same as the group of optical signals before splitting.
  • the dual-mode beam splitter 600 may first receive the twelfth group of optical signals, and split the twelfth group of optical signals into the first group of optical signals and the third group of optical signals.
  • the optical computing device may also be provided with a nonlinear operation module 700, and the nonlinear operation module 700 may perform a nonlinear operation on the second group of optical signals. Realize the function of the activation function configured by the neural network model.
  • the embodiment of the present application does not limit the location of the non-linear calculation module 700.
  • the non-linear calculation module 700 can be connected to the coupler 300 to receive the fifth group of optical signals and the sixth group of optical signals from the coupler 300.
  • the optical signal, the eighth group of optical signals, and the eleventh group of optical signals can also be connected to the first delay module 200, so that the fifth group of optical signals or the sixth group of optical signals, the first delay module 200 can be received through the first delay module 200.
  • the eight groups of optical signals and the eleventh group of optical signals are shown in Figure 6a.
  • the optical computing device may further include an optical amplifier 1000, which may be combined with the coupler 300 (FIG. 6b) or the nonlinear operation unit 700 to amplify the fifth The power of the group of optical signals, the sixth group of optical signals, the eighth group of optical signals, and the eleventh group of optical signals.
  • the amplifier 1000 may be a semiconductor optical amplifier (SOA) or an erbium doped fiber application amplifier (EDFA).
  • the optical computing device may include multiple optical switches for changing the transmission direction of the optical signal in the optical computing device.
  • the optical computing device may also include multiple couplers to combine optical signals output by different modules or devices.
  • an optical computing device provided by an embodiment of this application includes a dual-mode beam splitter 600, a linear operation module 100, a second delay module 500, a first delay module 200, and a nonlinear operation module 700. And the filtering module 400.
  • the optical computing device also includes three couplers, namely a first coupler 810, a second coupler 820, and a third coupler 830, and the optical computing device also includes two optical switches, namely the first optical switch 910 And the second optical switch 920, the optical computing device further includes an amplifier 1000, which is connected to the third coupler 830 or the first optical switch 910. It should be noted that the embodiment of the present application defines the location of the amplifier 1000.
  • the amplifier 1000 can be set on the path of the second coupler 820 -> the third coupler 830 -> the first optical switch 910 -> the first coupler 810. Any position of.
  • the functions of the dual-mode beam splitter 600, the linear operation module 100, the second delay module 500, the first delay module 200, the nonlinear operation module 700, and the filter module 400 can be referred to the foregoing description, which will be repeated here.
  • the coupler is mainly used to combine the optical signals output by different modules or devices.
  • the first coupler 810 can couple the optical signal inputted into the optical computing device (such as the third group of optical signals) and the optical signal output by the filter module 400 (the second group of optical signals, the eighth group of optical signals, or the eleventh group of optical signals). Signal).
  • the second coupler 820 is used to combine the optical signal output by the linear operation module 100 (the fourth group of optical signals) and the optical signal output by the second delay module 500 (the second group of optical signals after adjusting the delay),
  • the third coupler 830 may combine the optical signal output by the second coupler 820 and the optical signal output by the nonlinear operation module 700.
  • the optical switch can change the transmission direction of the optical signal.
  • the first optical switch 910 can transmit the signal output by the first delay module 200 to the filter module 400 or the nonlinear operation module 700.
  • the second optical switch 920 can transmit the optical signal output by the nonlinear operation module 700 to the third coupler 830 or output the optical signal output by the nonlinear operation module 700.
  • the first group of optical signals is marked with a black solid line in FIG.
  • the seventh group of optical signals enters the optical computing device through the first coupler 810, and then passes through the dual-mode beam splitter 600.
  • the dual-mode beam splitter 600 divides the twelfth group of optical signals into The first group of optical signals and the third group of optical signals transmit the first group of optical signals to the linear operation module 100.
  • the linear operation module 100 modulates the first group of optical signals according to the received first electrical signal, and outputs the second group of optical signals.
  • the second group of optical signals are transmitted to the third coupler 830 through the second coupler 820, and the third coupler 830 transmits the second group of optical signals to the first delay module 200, and the first delay module 200 adjusts the second The delay of the group of optical signals, the second group of optical signals after the delay is adjusted is transmitted to the filter module 400 under the action of the first optical switch 910, and the filter module 400 does not need to perform any removal operation, and directly adjusts the delay The second group of optical signals is transmitted to the first coupler 810.
  • the third group of optical signals is transmitted in the optical computing device without passing through the linear operation module 100, and the transmission path is the second delay module 500->the second coupler 820->the third coupler 830->the first delay module 200 ->First optical switch 910->Filter module 400 First coupler 810->Dual-mode beam splitter 600, when the last optical signal in the first group of optical signals is input to linear operation module 100, the dual-module beam splitter 600 transmits the third group of optical signals to the linear operation module 100.
  • the black dashed line marks the transmission path of the fourth group of optical signals after the third group of optical signals are transformed into the fourth group of optical signals by the linear operation module 100. It can be seen from Fig. 8a that the dual-mode beam splitter 600 has The three groups of optical signals are transmitted to the linear operation module 100, the third group of optical signals are input to the linear operation module 100, and the linear operation module 100 modulates the third group of optical signals according to the received second electrical signal, and outputs the fourth group of optical signals.
  • the transmission path of the second group of optical signals after passing through the first coupler 810 is marked with a gray solid line.
  • the second group of optical signals passes through the dual-mode beam splitter 600.
  • the signal is transmitted to the second delay module 500, and the second delay module 500 further adjusts the delay of the second group of optical signals so that the second group of optical signals can communicate with the fourth group of optical signals after being output from the second delay module 500
  • the second coupler 820 is reached.
  • the second coupler 820 couples the second group of optical signals and the fourth group of optical signals, outputs a fifth group of optical signals, and transmits the fifth group of optical signals to the third coupler 830.
  • the third coupler 830 can output the fifth set of optical signals through the first delay module 200 (The first delay module 200 does not perform operations), the first optical switch 910, the nonlinear operation module 700 (the nonlinear operation module 700 does not perform operations), the second optical switch 920, output from the optical computing device, the fifth group of optical The signal can indicate the final calculation result.
  • the third coupler 830 may continue to transmit the fifth group of optical signals to the first delay module 200.
  • the transmission process of the fifth group of optical signals is similar to the transmission process of the second group of optical signals after being output from the third coupler 830.
  • the fifth group of optical signals and other groups of optical signals modulated by the linear operation module 100 are combined to output the sixth group of optical signals.
  • the sixth group of optical signals output by the third coupler 830 can pass through the first delay module 200 (first delay module 200 does not perform operations), the first optical switch 910, the nonlinear operation module 700 (the nonlinear operation module 700 does not perform operations), the second optical switch 920, output from the optical computing device, the fifth group of optical signals can indicate the final calculation result.
  • the transmission process of the optical signal shown in FIG. 8a only involves the linear operation process.
  • the optical signal coupled by the third coupler 830 after the linear operation needs to continue in the optical computing device. Transmit for further processing.
  • the third coupler 830 can output the fifth group of optical signals through the first delay module 200 (the first delay module 200 does not perform operations), the first optical switch 910, the filter module 400 (the filter module 400 does not perform operations), and the first delay module 200 (the first delay module 200 does not perform operations).
  • a coupler 810 and a dual-mode beam splitter 600 finally arrive at the linear operation module 100.
  • the linear operation module 100 modulates the sixth group of electrical signals according to the received third electrical signal indicating the second weight to output the seventh group of optical signals
  • the seventh group of optical signals passes through the second coupler 820, the third coupler 830, the first delay module 200 (the first delay module 200 does not perform operations), and the first optical switch 910, and reaches the filter module 400, the filter module 400 Filter the invalid signals in the seventh group of optical signals, and output the eighth group of optical signals.
  • the eighth group of optical signals passes through the second coupler 820, the third coupler 830, the first delay module 200, the first optical switch 910, and the The linear operation module 700 (the non-linear operation module 700 does not perform operations) and the second optical switch 920 output from the optical computing device.
  • the linear arithmetic module 100 receives the ninth group of optical signals, and the linear arithmetic module 100 modulates the ninth group of optical signals according to the received fourth electrical signal, and outputs the tenth group of optical signals.
  • the tenth group of optical signals indicating the second group of data and the second weight multiplication and addition operation result is modulated only once.
  • the generation of the tenth group of optical signals can also be similar to the fifth group of optical signals or the sixth group of optical signals
  • the generation process that is, the linear operation module 100, the first delay module 200, the second delay module 500, the second coupler 820, and the third coupler 830 can cooperate to generate the tenth group of optical signals), the tenth The group of optical signals is transmitted to the third coupler 830 through the second coupler 820, and the tenth group of optical signals is transmitted through the third coupler 830, the first delay module 200 (the first delay module 200 does not perform operations), and the first optical signal
  • the switch 910 reaches the filter module 400.
  • the filter module 400 filters the invalid signals in the tenth group of optical signals, and outputs the eleventh group of optical signals.
  • the eleventh group of optical signals passes through the second coupler 820, the second coupler 820, and the first coupler.
  • the triple coupler 830, the first delay module 200, the first optical switch 910, the nonlinear operation module 700 (the nonlinear operation module 700 does not perform operations), and the second optical switch 920 are output from the optical computing device.
  • the fifth group of optical signal transmission paths are marked with a black solid line. If the subsequent optical computing device does not need to continue to receive the optical signal indicating the first group of data, that is, the linear operation required by the optical computing device has been completed. , The third coupler 830 transmits the fifth group of optical signals to the first delay module 200, the first delay module 200 may not process the fifth group of optical signals, and transmits the fifth group of optical signals to the first optical switch 910. Through the first optical switch 910, the fifth group of optical signals can be transmitted to the non-linear calculation module 700, and the non-linear calculation module 700 can perform non-linear operations on the fifth group of optical signals.
  • the fifth group of optical signals are transmitted to the second optical switch 920, through the second optical switch 920, the fifth group of optical signals after the non-linear operation is output from the optical computing device, and the fifth group of optical signals after the non-linear operation is performed Can indicate the final calculation result.
  • FIG 8b another transmission path of the fifth group of optical signals is marked with a black dashed line. If the subsequent optical computing device needs to continue to process the fifth group of optical signals, the third coupler 830 can transfer the fifth group of optical signals Continue to transmit to the first delay module 200.
  • the transmission process of the fifth group of optical signals is similar to the transmission process of the second group of optical signals after being output from the third coupler 830. For details, please refer to the foregoing content and will not be repeated here.
  • the third coupler 830 can receive the combined optical signal from the second coupler 820. Here, the linear operation module 100 executes all the linear operations.
  • the optical signal received by the third coupler 830 from the second coupler 820 is the sixth group of optical signals.
  • the transmission path of the sixth group of optical signals is marked with a gray dashed line.
  • the second coupler 820 transmits the received sixth group of optical signals to the first delay module 200.
  • the first delay module 200 may not correspond to the sixth group.
  • the optical signal is processed, and the sixth group of optical signals is transmitted to the first optical switch 910. Through the first optical switch 910, the sixth group of optical signals can be transmitted to the non-linear calculation module 700, and the non-linear calculation module 700 can control the sixth group of optical signals.
  • the group of optical signals performs non-linear operation, and then the sixth group of optical signals after performing the non-linear operation are transmitted to the second optical switch 920, and output from the optical computing device through the second optical switch 920.
  • the non-linear operation is performed
  • the sixth group of optical signals can indicate the final calculation result.
  • the amplifier 1000 is not shown in FIGS. 8a to 8b, and the amplifier 1000 can amplify the optical signal output by the third coupler 830.
  • the linear operation module 100 can compare the received optical signals (the first group of optical signals, the third group of optical signals, and the sixth group of optical signals) according to the received electrical signals (the first electrical signal, the second electrical signal, or the third electrical signal). Signal) modulates, changes the light intensity of the optical signal, and realizes linear operation.
  • the linear operation module 100 may divide the optical signal into N optical signals with different delays, and then modulate the N optical signals with different delays.
  • the linear operation module 100 includes multiple beam splitters, multiple delay waveguides, multiple couplers, and N photoelectric modulators. Optionally, it may also include a delay control module.
  • the delay control module receives the optical signal input to the linear operation module, and calibrates the delay of the optical signal.
  • the composition of the delay control module, the first delay module 200 and the second delay module 500 are all to adjust the delay of the optical signal, the composition of the delay control module, the first delay module 200 and the second delay module
  • the structure of 500 is also similar.
  • the structure of the delay control module please refer to the description of the first delay module 200 and the second delay module 500 below, which will not be repeated here.
  • the beam splitter can divide one optical signal into two identical optical signals.
  • the beam splitter can be a Y-shaped beam splitter or a multimode interference coupler. Any method that can divide one optical signal into two identical optical signals is applicable to the embodiments of the present application.
  • the delay waveguide can transmit optical signals, and can also adjust the time delay of the transmitted optical signals.
  • Multiple beam splitters and multiple delay waveguides cooperate to divide the optical signal output by the delay control module into N optical signals with different delays.
  • the photoelectric modulator is used to modulate the optical signal according to the received electrical signal.
  • the photoelectric modulator can be an MZI or a tunable microring resonator.
  • Each photoelectric modulator is used to modulate one of the N different optical signals output by the multiple beam splitters and the multiple delay waveguides.
  • the multiple couplers cooperate to combine the optical signals output by the N photoelectric modulators into one optical signal.
  • the coupler can be a Y beam splitter, and the function of the coupler is realized by the reverse Y beam splitter.
  • the linear operation module 100 includes a delay control module 1100 and three beam splitters (using a beam splitter 1101, a beam splitter, respectively). 1102, beam splitter 1103), three time-delay waveguides (respectively use time-delay waveguide 1104, time-delay waveguide 1105, and time-delay waveguide 1106), four photoelectric modulators (respectively use photoelectric modulator 1107, photoelectric modulation The photoelectric modulator 1108, the photoelectric modulator 1109, and the photoelectric modulator 1110 are distinguished), and three couplers (the coupler 1111, the coupler 1112, and the coupler 1113 are distinguished, respectively).
  • the first group of optical signals are input to the delay control module 1100, and the delay control module 1000 adjusts the delay of the first group of optical signals.
  • the first group of optical signals after adjusting the delay are transmitted to the beam splitter 1101, and are divided into two identical first group optical signals.
  • the two first group optical signals are the first group optical signals. 1 and the second group of optical signals 2.
  • the first optical signal 1 does not need to pass through the delay waveguide, is transmitted to the beam splitter 1102, and is divided into two identical first-group optical signals.
  • the two first-group optical signals are respectively the first group Optical signal 3 and the second group of optical signals 4.
  • the first group of optical signals 3 are transmitted to the photoelectric modulator 1107.
  • the second group of optical signals 4 pass through the delay waveguide 1105, and the delay waveguide 1105 adjusts the delay of the second group of optical signals 4, and the first group of optical signals 4 with the adjusted delay is transmitted to the photoelectric modulator 1108.
  • the first optical signal 2 passes through the delay waveguide 1104 and is transmitted to the beam splitter 1103, where it is divided into two identical first group optical signals.
  • the two first group optical signals are the first group optical signals.
  • the first group of optical signals 5 are transmitted to the photoelectric modulator 1109.
  • the second group of optical signals 6 pass through the delay waveguide 1106, and the delay waveguide 1106 adjusts the delay of the second group of optical signals 6, and the first group of optical signals 4 with the adjusted delay is transmitted to the photoelectric modulator 1110.
  • the first electrical signal received by the linear operation module 100 can be loaded on the four photoelectric modulators, so that the four photoelectric modulators can respectively modulate the received first group of optical signals based on the first electrical signal.
  • the first electrical signal may include multiple sub-signals, each photoelectric modulator is loaded with a sub-signal, and each photoelectric modulator modulates the received first group of optical signals according to the loaded electrical signal.
  • the linear operation module 100 may also include N phase modulators, one phase modulator is connected to one photoelectric modulator, and the phase modulator can modulate the phase of the optical signal output by the photoelectric modulator.
  • the linear operation module 100 may also include at least N-1 phase-locked modules, and the phase-locked modules are used to stabilize the phase of the optical signal.
  • a linear operation module provided by an embodiment of the present application is different from Fig. 9a.
  • the linear operation module 100 includes four phase modulators, namely a phase modulator 1114 and a phase modulator 1115.
  • the phase modulator 1116 and the phase modulator 1117 also include three phase-locked units, which are a phase-locked unit 1118, a phase-locked unit 1119, and a phase-locked unit 1120, respectively.
  • the phase modulator 1114 is connected to the photoelectric modulator 1107, and adjusts the phase of the optical signal output by the photoelectric modulator 1107.
  • the phase modulator 1115 is connected to the photoelectric modulator 1108, and adjusts the phase of the optical signal output by the photoelectric modulator 1108.
  • the phase modulator 1116 is connected to the photoelectric modulator 1109, and adjusts the phase of the optical signal output by the photoelectric modulator 1109.
  • the phase modulator 1117 is connected to the photoelectric modulator 1110, and adjusts the phase of the optical signal output by the photoelectric modulator 1107.
  • the phase lock unit 1118 is connected with the phase modulator 1114, and the phase of the optical signal output by the phase modulator 1114 is locked according to the optical signal output by the coupler 1111, and the phase lock unit 1119 is connected with the phase modulator 1116 according to the optical signal output by the coupler 1112 The phase of the optical signal output by the phase modulator 1116 is locked, and the phase locking unit 1120 is connected to the coupler 1112 to lock the phase of the optical signal output by the coupler 1112 according to the optical signal output by the coupler 1113.
  • the functions of the first delay module 200 and the second delay module 500 are both to adjust the delay of the optical signal.
  • the embodiment of the present application does not limit the functions of the first delay module 200 and the second delay module 500.
  • any device capable of adjusting the delay of the optical signal is applicable to the embodiments of the present application.
  • the first delay module 200 may adopt a cascade structure, and the first delay module 200 may include a plurality of light Switches (in Figure 10 are distinguished by optical switch 1, optical switch 2... optical switch n), two optical signal transmission paths are arranged between the two optical switches, one is a long-delay transmission path, and the transmission path is Equipped with an adjustable delay device (such as the delay waveguide 1), which can add a longer delay to the optical signal, and the other is a short delay transmission path, which can be equipped with an adjustable delay device (Such as delay waveguide 2), this device can add a shorter time delay to the optical signal.
  • an adjustable delay device such as the delay waveguide 1
  • the delay waveguide 2 which can add a longer delay to the optical signal
  • the other is a short delay transmission path, which can be equipped with an adjustable delay device (Such as delay waveguide 2), this device can add a shorter time delay to the optical signal.
  • each optical switch can be controlled by an electrical signal for selecting the transmission path of the optical signal.
  • the filter module 400 can remove part of the optical signals in the serial optical signal, and can also adjust the time delay of the optical signal transmitted before the removed optical signal to ensure that there is no gap in the output of a group of optical signals by the filter module 400.
  • FIG. 11 it is a schematic structural diagram of a filtering module provided by an embodiment of this application.
  • the filter module 400 includes two optical switches (to distinguish between the optical switch 501 and the optical switch 502 for convenience of description) and a third delay module 503.
  • the seventh group of optical signals After the seventh group of optical signals are input to the filter module 400, they first enter the optical switch 501, and are transmitted to the third delay module 503 under the action of the optical switch 501.
  • the delay module 503 can directly transmit the seventh group of optical signals without processing.
  • the optical switch 502 outputs the invalid signals in the seventh group of optical signals to the outside of the optical computing device, and transmits the remaining optical signals in the seventh group of optical signals (the eighth group of optical signals) to the optical switch 501.
  • the switch 501 transmits the eighth group of optical signals to the third delay module 503. Since the signals included in the seventh group of optical signals are serial optical signals, when a part of the optical signals is removed, there will be a gap in the eighth group of optical signals.
  • the third delay module 503 can Adjust the delay of the optical signal whose transmission sequence is before the invalid signal in the eighth group of optical signals, instead of adjusting the delay of the optical signal whose transmission sequence is before the invalid signal, to ensure the transmission sequence of the optical signal and the transmission sequence before the invalid signal There is no vacancy in the optical signal after the invalid signal.
  • Dual-mode beam splitter 600 optical switches (first optical switch 910 and second optical switch 920).
  • the dual-mode beam splitter 600 and the optical switch are essentially beam splitters with a splitting ratio adjustment function.
  • the dual-mode beam splitter 600 includes a beam splitter 601 and two phase shifters (for the convenience of description , Distinguished by a phase shifter 602 and a phase shifter 603) and a coupler 604, respectively.
  • the beam splitter 601 can divide the optical signal into two identical optical signals, and the phase shifter 602 and the phase shifter 603 can compensate the phase of the optical signal to realize the adjustment of the light splitting ratio.
  • Couplers such as coupler 300, first coupler 810 and second coupler 820, and coupler 604.
  • the coupler in the embodiment of the present application may be a Y-type coupler, which is used to couple two optical signals into one optical signal.
  • Non-linear computing module 700
  • the nonlinear operation module 700 may include devices with optical nonlinear effects, such as a graphene layer.
  • the manner in which the optical computing device implements the calculation process of the fully connected layer, the convolutional layer, and the pooling layer will be described.
  • the optical computing device can be used to realize the multiplication and addition operation.
  • the optical computing device can receive serial optical signals multiple times (such as the first group of optical signals or the third group of optical signals).
  • serial optical signals multiple times (such as the first group of optical signals or the third group of optical signals).
  • Xc[1] ⁇ Xc[n] are used to represent the serial optical signals.
  • the optical signal of the row is composed of n sub-signals (such as Xc[1], Xc[n]).
  • the linear computing module 100 can The electrical signal (such as the first electrical signal and the second electrical signal) modulates the sub-signal, the modulated sub-signal continues to be transmitted in the optical computing device, and the modulated sub-signal passes through the first delay module 200 (adjusts the modulated sub-signal) , So that the adjusted sub-signal and the modulated sub-signal in the next serial optical signal Xc[1] ⁇ Xc[n] input to the optical computing device can be aligned in time delay),
  • the filtering module 400 (the filtering module 400 does not perform any processing), the dual-mode beam splitter 600, the second delay module 500, the second coupler 820, and the third coupler 830.
  • the optical computing device can continue to receive the serial optical signals Xc[1] to Xc[n] to perform the above operations until all linear operations are completed.
  • the linear operation module 100 receives Xc[1] ⁇ Xc[8] in turn, and the four modulators in the linear operation module 100 can adjust the intensity of Xc[1] ⁇ Xc[8] ,
  • w ij can indicate the degree of modulation of the sub-signal light intensity by the adjuster in the linear operation module 100 (that is, w ij can indicate the weight)
  • i indicates the number of the sub-signal adjusted by the modulator
  • j Indicates the number of the photoelectric modulator.
  • Xc[1] Take Xc[1] as an example. After Xc[1] is input to the linear operation module 100 in time period 1, it is split into four sub-signals Xc[1] with different delays, which are Xc with time delays in time period 1. [1], Xc[1] with time delay in time period 2, Xc[1] with time delay in time period 3, and Xc[1] with time delay in time period 4.
  • the first photoelectric modulator modulates the sub-signal of Xc[1] with a time delay of 1 as w 11 Xc[1].
  • the second photoelectric modulator modulates the sub-signal of Xc[1] with a time delay of 2 as w 12 Xc[1].
  • the third photoelectric modulator modulates the sub-signal of Xc[1] with a time delay of 3 as w 13 Xc[1].
  • the fourth photoelectric modulator modulates the sub-signal of Xc[1] with a time delay of 4 as w 14 Xc[1].
  • the modulated sub-signal continues to be transmitted in the optical computing device, and the transmission path is the second coupler 820->the third coupler 830->the first delay module 200->the filtering module 400->the dual-mode beam splitter 600- > The second delay module 500, until the last sub-signal in the serial optical signal passes through the linear operation module 100.
  • the first delay module 200 adjusts the delay of all sub-signals after modulation, and the added delay is 2*4 ⁇ ( ⁇ is the time length of the single signal pulse).
  • the optical computing device continues to receive the serial Optical signal Xc[1] ⁇ Xc[8], the subsequent received serial optical signal Xc[1] ⁇ Xc[8] in the optical computing device and the operation process of the previous serial optical signal Xc[1] ⁇ Xc[8] is similar.
  • the optical signal After the last sub-signal in the serial optical signal passes through the linear operation module 100, it passes through the second coupler 820 and the third coupler 830, the optical signal is combined into 8 serial optical signals, which can indicate the final calculation result .
  • the calculation process of the convolutional layer is similar to that of the fully connected layer. The difference is that when performing the calculation of the convolutional layer, the filtering module 400 can remove some optical signals that do not need to participate in the calculation from the optical signal modulated by the linear calculation module 100. It is noted that the light signal rejected by the filtering module 400 is related to the calculation to be completed.
  • the convolution operation of a 3x3 matrix and a 2x2 convolution kernel is shown in Figure 14, where Xc 1 ⁇ Xc 9 is the serial optical signal to be input, and a, b, c, and d are the weights indicated by the convolution kernel. Xn[1] ⁇ Xn[4] are the optical signals to be output, indicating the final calculation result.
  • the linear operation module 100 sequentially receives Xc 1 ⁇ Xc 9 , and the two modulators in the linear operation module 100 can adjust the intensity of Xc[1] ⁇ Xc[9], where d and c indicate linearity respectively.
  • a and b respectively indicate the degree of modulation of the sub-signal light intensity (that is, the weight indicated by the convolution kernel) by the adjuster during the second linear operation of the linear operation module 100.
  • Xn 1 to Xn 9 are optical signals output by the linear operation module 100 during a linear operation.
  • Figure 15a is the input serial optical signal and the output optical signal during the first linear operation of the linear operation module 100.
  • Figure 15b shows the input serial optical signal and the output optical signal during the second linear operation of the linear operation module 100.
  • Figure 15c shows the optical signal removed from the optical signal output by the linear operation module 100 by the filter module 400 (the optical signal in the linear frame is the removed signal Xn 1 to Xn 4 , Xn 7 ) and the remaining optical signal.
  • Figure 15d shows the final optical signal output by the optical computing device.
  • the operation process of the pooling layer is similar to that of the convolutional layer.
  • the difference is that the function of the convolution kernel in the pooling layer is to retain the maximum value (maximum pooling) or average value (average pooling) of specific data.
  • optical computing device to realize the average pooling operation process from 4x4 matrix to 2x2 matrix as an example to describe the operation process performed by the optical computing device:
  • FIG. 17 shows the input serial optical signal and the output optical signal during the linear operation performed by the linear operation module 100.
  • the linear operation module 100 sequentially receives Xc 1 to Xc 16 , where 0.25 indicates the degree of modulation of the sub-signal light intensity by the adjuster during the linear operation performed by the linear operation module 100, and the blank indicates the degree of light intensity of the sub-signal
  • the emphasis system is 0.
  • Xn 1 to Xn 16 are optical signals output by the linear arithmetic module 100. It can be seen from Figure 17 that the effective signals are Xn6, Xn8, Xn14, Xn16. Xn1 ⁇ Xn5, Xn7, Xn9 ⁇ Xn13, Xn15 are all invalid signals.
  • the filtering module 400 in the optical computing device can filter invalid signals.
  • the linear operation module 100 receives a first group of optical signals and a first electrical signal.
  • the linear operation module 100 modulates the first group of optical signals according to the first electrical signal to output the second group of optical signals.
  • the linear operation module 100 can receive the third group of optical signals and the second electrical signal, and the linear operation module 100 modulates the third group of optical signals according to the second electrical signal to output the fourth group of optical signals, where the fourth group of optical signals is used To indicate the calculation result of the first set of data and the second part of the weight value.
  • the first delay module 200 can adjust the delay of the second group of optical signals; after that, the coupler 300 can combine the delayed second group of optical signals and the fourth group of optical signals into a fifth group of optical signals, where the first The five groups of light signals indicate the first calculation result of the multiplication and addition of the first group of data, the first part of the weight and the second part of the weight.
  • the optical computing device may further include a filter module, and the optical computing device performs a pooling operation on the sixth group of optical signals as an example for description.
  • the linear operation module 100 can receive the sixth group of optical signals and the third electrical signal.
  • the linear operation module 100 modulates the sixth group of optical signals according to the third electrical signal to output the seventh group of optical signals.
  • the filtering module can receive the seventh group of optical signals, filter the invalid signals in the seventh group of optical signals, and output the eighth group of optical signals. Group light signal.
  • the optical computing device may also implement a convolution operation by means of the filter module. Take the optical computing device performing the convolution operation on the ninth group of optical signals as an example for description.
  • the linear operation module 100 may receive the ninth group of optical signals and the fourth electrical signal, and modulate the ninth group of optical signals according to the fourth electrical signal to output the tenth group of optical signals;
  • the filter module receives the tenth group of optical signals, filters out the invalid signals in the tenth group of optical signals, and outputs the eleventh group of optical signals.
  • the eleventh group of optical signals is used to indicate the calculation result of the convolution operation on the third calculation result .
  • the embodiment of the present application also provides an optical computing system.
  • the optical computing system includes a processor and an optical computing device as mentioned in the foregoing.
  • the optical computing system 10 includes an optical computing device 20 and a processor 30.
  • the processor 30 and the optical computing device 20 may be connected through a standard host interface or a network interface (network interface).
  • the host interface may include a Peripheral Component Interconnect Express (PCIE) interface.
  • the processor 30 may send data to be calculated to the optical computing device 20, where the data includes a first set of data and may also include a first weight.
  • the data can also be a second set of data and a second weight.
  • PCIE Peripheral Component Interconnect Express
  • the optical computing device 20 After the optical computing device 20 receives the data, it can determine the corresponding one or more groups of optical signals (such as the first group of optical signals, the third group of optical signals, the ninth group of optical signals, etc.) and electrical signals according to the data. Signal (the first electrical signal, the second electrical signal, the fourth electrical signal, etc.) to complete the corresponding calculation process.
  • groups of optical signals such as the first group of optical signals, the third group of optical signals, the ninth group of optical signals, etc.
  • Signal the first electrical signal, the second electrical signal, the fourth electrical signal, etc.
  • These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
  • the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Neurology (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Nonlinear Science (AREA)
  • Optics & Photonics (AREA)
  • Optical Modulation, Optical Deflection, Nonlinear Optics, Optical Demodulation, Optical Logic Elements (AREA)
  • Optical Communication System (AREA)

Abstract

一种光计算装置、系统以及计算方法，该光计算装置包括线性运算模块、第一延时模块以及耦合器。线性运算模块可以接收第一组光信号以及第一电信号，并根据第一电信号对第一组光信号进行调制以输出第二组光信号。线性运算模块后续还可以接收第三组光信号以及第二电信号，并根据第二电信号对第三组光信号进行调制以输出第四组光信号。第一延时模块可以调整第二组光信号的时延，耦合器将延时后的第二组光信号与第四组光信号合并为第五组光信号，第五组光信号指示第一组数据与第一部分权重以及第二部分权重进行乘加运算后的第一计算结果。针对该光计算系统，只需调整输入的光信号以及电信号就可以实现不同的乘加运算，保证了光计算系统的效率。

Description

一种光计算装置、系统以及计算方法 技术领域
本申请涉及信息技术领域，尤其涉及一种光计算装置、系统以及计算方法。
背景技术
神经网络模型，例如卷积层和全连接层，通常需要进行大量的数据运算，并且在数据运算过程中，还需要存储大量的过程数据。为了提升神经网络模型的运算速度，可以借助光计算系统实现神经网络模型的数据运算过程，获得最终的运算结果。
光计算系统按照其实现形式可以分为空间光计算系统和片上光计算系统两类。其中，空间光计算系统以4F光学系统为主，依托傅里叶光学变换实现卷积运算，模拟卷积神经网络(convolutional neural network，CNN)等神经网络模型，但是这种光计算系统在速度和体积上均不占优势。而片上光计算系统是以硅基光子集成技术为基础，主要利用马赫-曾德调制器(mach zehnder interferometer，MZI)等光学器件构建光学乘加器，实现卷积运算，但这类光计算系统包括有限数量的MZI，需要进行多次乘加迭代才能实现乘加运算，且片上光计算系统利用加热电极调整光信号的相位，使得片上光计算系统容易存在热串扰，降低了运算的准确性，使得整个片上光计算系统的效率较差。
发明内容
本申请提供一种光计算装置、系统以及计算方法，用以提供一种效率较高的光计算系统。
第一方面，本申请实施例提供了一种光计算装置，该光计算装置包括线性运算模块、第一延时模块以及耦合器。
线性运算模块可以接收用于指示第一组数据的第一组光信号，还可以接收指示第一权重中的第一部分权重值的第一电信号。线性运算模块根据第一电信号对第一组光信号进行调制以输出第二组光信号，其中，第二组光信号用于指示第一组数据与第一部分权重值的计算结果，第一权重为第一神经网络层的权重。
线性运算模块后续还可以接收用于指示第一组数据的第三组光信号，以及指示第一权重中的第二部分权重值的第二电信号。线性运算模块根据第二电信号对第三组光信号进行调制以输出第四组光信号，其中，第四组光信号用于指示第一组数据与第二部分权重值的计算结果。
第一延时模块可以调整第二组光信号的时延；之后，耦合器可以将延时后的第二组光信号与第四组光信号合并为第五组光信号，其中，第五组光信号指示第一组数据与第一部分权重以及第二部分权重进行乘加运算后的第一计算结果。
通过上述光计算装置，该光计算装置中线性运算单元可以通过先后接收多组光信号和电信号与耦合器、第一延时模块配合输出可以指示乘加运算的计算结果的一组光信号。在该光计算装置中，只需调整输入多组光性以及电信号就可以实现不同的乘加运算，且在光计算装置中不需要借助加热电极的方式实现光信号相位的调整，保证计算的准确性，进而保证光计算装置的效率。
在一种可能的设计中，光计算装置中还可以包括过滤模块，光计算装置除了乘加运算， 还可以借助过滤模块实现池化操作。以光计算装置对第六组光信号进行池化操作为例进行说明。线性运算模块可以接收第六组光信号，以及指示第二权重的第三电信号，其中，第六组光信号用于指示第一组数据与第一权重进行乘加运算后的第二计算结果，其中，第二计算结果可以包含第一计算结果，第二权重为第二神经网络层的权重。
线性运算模块根据第三电信号对第六组光信号进行调制以输出第七组光信号，过滤模块可以接收第七组光信号，并过滤第七组光信号中的无效信号后输出第八组光信号，第八组光信号用于指示对第二计算结果进行池化操作的计算结果。
通过上述光计算装置，光计算装置不仅可以实现乘加运算还可以实现池化操作，提高了光计算装置的通用性。
在一种可能的设计中，光计算装置包括过滤模块的情况下，光计算装置还可以借助过滤模块实现卷积运算。以光计算装置对第九组光信号进行卷积运算为例进行说明。
线性运算模块可以接收第九组光信号以及指示第三权重的第四电信号，第九组光信号用于指示第二组数据。
线性运算模块可以根据第四电信号对第九组光信号进行调制以输出第十组光信号，第十组光信号用于指示第二组数据与第三权重进行乘加运算后的第三计算结果，第三权重为第三神经网络层的权重；
过滤模块接收第十组光信号，并过滤第十组光信号中的无效信号后输出第十一组光信号，第十一组光信号用于指示对第三计算结果进行卷积操作的计算结果。
通过上述光计算装置，光计算装置不仅可以实现乘加运算，还可以实现卷积运算和池化操作，进一步，提高了光计算装置的通用性，使得光计算装置能实现较多神经网络模型的运算。
在一种可能的设计中，光计算装置中还可以包括非线性运算模块，以实现神经网络模型中配置的激活函数。非线性运算模块可以对耦合器输出的信号，如第六组光信号或第五组光信号、第八组光信号或第十一组光信号进行非线性操作，非线性操作为激活函数操作。
通过上述光计算装置，光计算装置除了实现乘加运算、卷积运算和池化操作，还可以在乘加运算、卷积运算和池化操作的计算结果上实现激活函数操作，确保了光计算装置的通用性。
在一种可能的设计中，光计算装置中还可以包括双模式分束器，双模式分束器能够将接收到的指示一组数据的一组光信号分为指示同一组数据的两组相同的光信号。示例性的，双模式分束器可以接收用于指示第一组数据的第十二组光信号，将第十二组光信号分为第一组光信号以及第三组光信号。
通过上述光计算装置，光计算装置设置的双模式分束器，不需要多次向光计算装置输入指示相同数据的一组光信号，仅需输入一组光信号，可以分束出多组指示同一数据的光信号，能够有效提升光计算装置的计算效率，提高了光计算装置的性能。
在一种可能的设计中，光计算装置还包括第二延时模块，双模式分束器可以将延时后的第二组光信号传输至第二延时模块；第二延时模块可以调整第二组光信号的时延，以使得第二组光信号与从第四组光信号同时传输至耦合器。
通过上述光计算装置，耦合器能够同时接收到多组光信号，能够较好的合并出一组光信号。
在一种可能的设计中，线性运算模块包括N个光电调制器。
N个光电调制器中的任一光电调制器可以接收第一组光信号中的一个光信号以及用于指 示第一部分权重的第一电信号中的一个电信号，根据接收到的电信号对接收到的光信号进行调制；类似的，任一光电调制器还可以接收第三组光信号中的一个光信号以及用于指示第二部分权重的第二电信号中的一个电信号，根据接收到的电信号对接收到的光信号进行调制。
通过上述光计算装置，线性运算模块借助N个光电调制器可以较好的实现乘加运算，保证光计算装置的高效运算。
在一种可能的设计中，线性运算模块还包括N个相位调制器以及N-1个锁相模块；
一个相位调制器可以连接一个光电调制器，相位调制器可以调整经过调制器调制后的第一组光信号或第二组光信号的相位；一个锁相模块可以连接N个相位调制器中的两个相位调制器，用于锁定经过两个相位调制器调制后的第一组光信号或第二组光信号的相位。
通过光计算装置，通过相位调制器可以调整光信号的相位，调整了光信号所指示的数据的符号，通过锁相模块能够保证相位调制器可以有效的调整光信号的相位。
在一种可能的设计中，线性运算模块还包括放大器，放大器可以放大第五组光信号、第六组光信号、第八组光信号或第十一组光信号的功率。
通过上述光计算装置，通过放大器可以减少光信号在光计算装置中的损耗。
第二方面，本申请提供了一种计算方法，有益效果可以参见第一方面的相关描述，此处不再赘述。该方法由光计算装置执行，该方法包括：
线性运算模块接收第一组光信号以及第一电信号，其中，第一组光信号用于指示第一组数据，第一电信号用于指示第一权重中的第一部分权重值；根据第一电信号对第一组光信号进行调制以输出第二组光信号，其中，第二组光信号用于指示第一组数据与第一部分权重值的计算结果，第一权重为第一神经网络层的权重；
线性运算模块接收第三组光信号以及第二电信号，其中，第三组光信号用于指示第一组数据，第二电信号用于指示第一权重中的第二部分权重值，并根据第二电信号对第三组光信号进行调制以输出第四组光信号，其中，第四组光信号用于指示第一组数据与第二部分权重值的计算结果。
第一延时模块调整第二组光信号的时延。
耦合器将延时后的第二组光信号与第四组光信号合并为第五组光信号，其中，第五组光信号指示第一组数据与第一部分权重以及第二部分权重进行乘加运算后的第一计算结果。
在一种可能的设计中，线性运算模块接收第六组光信号，第六组光信号用于指示第一组数据与第一权重进行乘加运算后的第二计算结果以及用于指示第二权重的第三电信号，其中，第二计算结果包含第一计算结果；其中，第二权重为第二神经网络层的权重。
线性运算模块根据第三电信号对第六组光信号进行调制以输出第七组光信号。
过滤模块接收第七组光信号，并过滤第七组光信号中的无效信号后输出第八组光信号，第八组光信号用于指示对第二计算结果进行池化操作的计算结果。
在一种可能的设计中，线性运算模块接收第九组光信号，第九组光信号用于指示第二组数据。
线性运算模块接收用于指示第三权重的第四电信号，并根据第四电信号对第九组光信号进行调制以输出第十组光信号，第十组光信号用于指示第二组数据与第三权重进行乘加运算后的第三计算结果，第三权重为第三神经网络层的权重；
过滤模块接收第十组光信号，并过滤第十组光信号中的无效信号后输出第十一组光信号，第十一组光信号用于指示对第三计算结果进行卷积操作的计算结果。
第三方面，本申请提供了一种光计算系统，所述光计算设备可以包括处理器以及如第一 方面或第一方面的任意一种可能的涉及中所述的光计算装置，所述处理器用于向光计算装置传输待计算的数据，如第一组数据或第二组数据，也可以传输第一权重或第二权重。
第四方面，本申请提供了一种计算机可读存储介质，所述计算机可读存储介质被计算设备执行时，所述计算设备执行前述第二方面或第二方面的任意可能的设计中提供的方法。该存储介质中存储了程序。该存储介质包括但不限于易失性存储器，例如随机访问存储器，非易失性存储器，例如快闪存储器、硬盘(hard disk drive，HDD)、固态硬盘(solid state drive，SSD)。
第五方面，本申请提供了一种计算设备程序产品，所述计算设备程序产品包括计算机指令，在被计算设备执行时，所述计算设备执行前述第二方面或第二方面的任意可能的设计中提供的方法。该计算机程序产品可以为一个软件安装包，在需要使用前述第二方面或第二方面的任意可能的设计中提供的方法的情况下，可以下载该计算机程序产品并在计算设备上执行该计算机程序产品。
附图说明
图1为本申请提供的一种光计算装置的结构示意图；
图2为本申请提供的一种光信号的传输路径示意图；
图3a为本申请提供的一种乘加运算示意图；
图3b为本申请提供的一种第一部分权重值和第二部分权重值的示意图；
图4a为本申请提供的一种光计算装置的结构示意图；
图4b为本申请提供的一种光计算装置的结构示意图；
图5为本申请提供的一种光计算装置的结构示意图；
图6a为本申请提供的一种光计算装置的结构示意图；
图6b为本申请提供的一种光计算装置的结构示意图；
图7为本申请提供的一种光计算装置的结构示意图；
图8a为本申请提供的一种光信号的传输路径示意图；
图8b为本申请提供的一种光信号的传输路径示意图；
图9a为本申请提供的一种线性运算模块的结构示意图；
图9b为本申请提供的一种线性运算模块的结构示意图；
图10为本申请提供的一种第一延时模块的结构示意图；
图11为本申请提供的一种过滤模块的结构示意图；
图12为本申请提供的一种双模式分束器的结构示意图；
图13为本申请提供的一种乘加运算示意图；
图14为本申请提供的一种卷积运算示意图；
图15a为本申请提供的一种第一次线性运算过程中的输入的串行光信号以及输出的光信号示意图；
图15b为本申请提供的一种第二次线性运算过程中的输入的串行光信号以及输出的光信号示意图；
图15c为本申请提供的一种过滤模块过滤的光信号示意图；
图15d为本申请提供的一种光计算装置输出的光信号示意图；
图16为本申请提供的一种池化操作示意图；
图17为本申请提供的一种线性运算过程中的输入的串行光信号以及输出的光信号示意图；
图18为本申请提供的一种计算方法；
图19为本申请提供的一种光计算系统。
具体实施方式
如图1所示，本申请实施例提供了一种光计算装置，该光计算装置包括线性运算模块100、第一延时模块200以及耦合器300。本申请实施例并不限定光计算装置的具体形态，例如光计算装置可以为一种光计算芯片。
在本申请实施例中线性运算模块100能够根据接收到的电信号对输入至线性运算模块100的光信号进行调制，第一延时模块200可以调整线性运算模块100输出的光信号的时延，在第一延时模块200调整了线性运算模块100输出的光信号的时延后，耦合器300可以将线性运算模块100先后输出多组光信号进行合并，输出一组用于指示一组数据与权重的乘加运算的计算结果的光信号。
下面以线性运算模块100先后接收两组光信号，分别为第一组光信号和第三组光信号为例，对光计算装置中的运算过程进行说明：
首先，线性运算模块100在接收到指示第一组数据的第一组光信号的过程中，还可以接收用于指示第一权重中第一部分权重值的第一电信号，线性运算模块100可以根据第一电信号对第一组光信号进行调制，调制第一组光信号的光强，并输出第二组光信号，输出的第二组光信号可以指示第一组数据与第一部分权重值的计算结果。第一权重为第一神经网络层上设置的权重。
由于线性运算模块100对一个光信号在一次线性运算过程中所能实现的运算量有限，也就是说线性运算模块100只能输出指示第一组数据和部分权重(如第一部分权重值)的乘加运算的光信号。若线性运算模块100需要执行完成所有运算量，线性运算模块100还需要继续接收下一组光信号进行线性运算，以便耦合器300后续可以将线性运算模块100先后输出多组不同的光信号合并，输出指示第一组数据与第一权重之间乘加运算的计算结果一组光信号。
为了与第一组光信号区分，以线性运算模块100后续接收的用于指示第一组数据的光信号为第三组光信号为例。
线性运算模块100在接收到第三组光信号的过程中，还可以接收用于指示第一权重中的第二部分权重值的第二电信号，线性运算模块100可以根据第二电信号对第三组光信号进行调制，调制第三组光信号的光强，输出第四组光信号，输出的第四组光信号可以指示第一组数据与第二部分权重值的计算结果。
由上可知，线性运算模块100先后输出了第二组光信号和第四组光信号，这两个光信号之间存在时延，并不能直接合并。
为此，第一延时模块200可以调整第二组光信号的时延；之后，耦合器300接收到第四组光信号以及调整了时延的第二组光信号后，可以将第二组光信号与延迟后的第四组光信号合并为第五组光信号，其中，第五组光信号用于指示第一组数据与第一部分权重、第二部分权重进行乘加运算后的计算结果。
下面对第二组光信号以及第四组光信号在光计算装置中的传输路径进行说明。
参见图2，图2用黑色的实线标注了第二组光信号从线性运算模块100输出后的传输路 径1，从图2可以看出，第二组光信号从线性运算模块100输出后，传输至第一延时模块200，之后从第一延时模块200输出至耦合器300。
图2用黑色的虚线标注了第四组光信号的从线性运算模块100输出后的传输路径2，从图2中可以看出第四组光信号从线性运算模块100输出后，可以直接传输至耦合器300。
在如图1～2所示的光计算装置中，第二组光信号可以指示第一组数据与第一部分权重值进行乘加运算后的计算结果，第四组光信号可以指示第一组数据与第二部分权重值进行乘加运算后的结果，通过第一延时模块200调整第二组光信号的时延后，耦合器300通过将延时后的第二组光信号和第四组光信号耦合为第五组光信号，也完成第一组数据与第一部分权重、以及第二部分权重之间的乘加运算，实现了全连接层的运算过程。
下面对光计算装置所执行的全连接层的运算进行说明，参见图3a，图3a左侧为全连接层所需实现的一种乘加运算，为了更加直观，可转换为图3a右侧所示的阵列的形式。该线性运算为4x4的线性运算，需要有四个输入数据(X c1、X c2、X c3、X c4)，四个输出数据(X n1、X n2、X n3、X n4)，所需施加的权重为(W 11～W 44)。若线性运算模块100每次只能对一个光信号进行两种不同的调制(也即线性运算模块100中只包括两个光电调制器)，从如图3a的乘加运算可知，对任一输入数据，需要进行四种不同的调整，以X c1为例，对X c1需要执行四种不同的调整，已获得W 11X c1、W 12X c1、W 13X c1、W 14X c1，为完成如图3a所示的乘加运算，线性运算模块100需要先后接收两组指示X c1、X c2、X c3、X c4的光信号，如作为第一组光信号和第三组光信号，还需要先后接收指示第一部分权重值的第一电信号，以及指示第二部分权重值的第二电信号。参见图3b，为实线框中为第一部分权重值，虚线框为第二部分权重值，才能与第一延时模块200、耦合器300配合，输出指示X n1、X n2、X n3、X n4的第五组光信号。
需要说明的是，上述光计算装置中仅是线性运算模块100接收两组指示第一组数据的光信号为例进行说明，若第一权重中除第一部分权重值和第二部分权重值外还有第三部分权重值，线性运算模块100还可以继续接收指示第一组数据的一组光信号，线性运算模块100根据指示第三部分权重值的电信号对该组光信号进行调制，输出调制后的一组光信号，第一延时模块200可以对第五组光信号进行延时，耦合器300可以将第五组光信号和该组调制后的一组光信号合并了一组光信号。类似的，如第一权重中还包括第四部分权重值，光计算装置还可以采用前述类似的过程，线性运算模块100继续接收指示第一组数据的一组光信号，并对该组光信号进行调制，第一延时模块200对之前已合并一组光信号进行延时，耦合器300将第一延时模块200延时后的一组光信号与线性运算模块100此次调制后的一组光信号进行合并，循环执行上述过程，直至完成第一组数据与第一权重之间的乘加运算，在本申请实施例中用第六组光信号指示第一组数据与第一权重之间乘加运算的计算结果。
光计算装置还可实现其他神经网络模型的运算，例如卷积层的卷积运算以及池化层的池化操作等，由于在进行卷积运算或池化操作的过程中，需要剔除一些不必要参与运算的数据，也即无效数据，在光计算装置还可以包括过滤模块400。如图4a所示，在第二组光信号的传输路径上可以设置过滤模块400，该过滤模块400可以从一组光信号中过滤一些不需要执行运算的光信号，也即无效信号，该无效信号指示无效数据。在过滤了一些不需要执行运算的光信号后，将该组光信号中剩余的光信号沿第二组光信号的传输路径传输至第一延时模块200。
下面以光计算装置中线性运算模块100接收的两组光信号分别为第一组光信号或第三光信号为例，对光计算装置实现池化操作时，光计算装置中光信号的传输过程进行说明。
首先，当耦合器300输出第六组光信号(第六组光信号的产生过程可参见前述内容，此处不再赘述)后，第六组光信号经过第一延时模块200、过滤模块300传输至线性运算模块 100。
线性运算模块100根据接收到的指示第二权重的第三电信号对第六组光信号进行调制，输出第七组光信号，其中，第二权重为第二神经网络层(如池化层)上设置的权重。
之后，第七组光信号通过第一延时模块200(第一延时模块200可以不调整第二组光信号的时延)传输至过滤模块400，将第六组光信号传输至过滤模块400，过滤模块400从第七组光信号中过滤无效信号，无效信号指池化操作中需删除的无效数据，输出第七组光信号中剩余的光信号，为方便说明，用第八组光信号指示第七组光信号中剩余的光信号，第八组光信号即可指示对第一组数据与第一权重之间乘加运算的计算结果进行池化操作的最终结果。
下面对光计算装置实现卷积操作为例进行说明，区分于前述各组光信号以及各个电信号，以线性运算模块100接收的一组光信号为第九组光信号，接收的电信号为指示第三权重的第四光信号为例进行说明。其中，第三权重为第三神经网络层(如卷积层)上设置的权重。
线性运算模块100根据接收到的第四电信号对第九组光信号进行调制，调制第九组光信号的光强，并输出第十组光信号，输出的第十组光信号可以指示第二组数据与第三权重的计算结果。
这里需要说明的是，这里仅是以线性运算模块100接收一组光信号，输出第十组光信号为例进行说明，在实际运算中，与线性运算模块100与第一延时模块200、耦合器300配合，输出第六组光信号的方式类似，线性运算模块100也可以先后接收多组光信号，与第一延时模块200、耦合器300配合，循环多次执行线性运算，获取第十组光信号。
第十组光信号通过第一延时模块200(第一延时模块200可以不调整第十组光信号的时延)传输至过滤模块400，将第十组光信号传输至过滤模块400，过滤模块400从第十组光信号中过滤无效信号，无效信号指池化操作中需删除的无效数据，输出第十组光信号中剩余的光信号，为方便说明，用第十一组光信号指示第十组光信号中剩余的光信号，第十一组光信号即可指示对第二组数据进行卷积操作的最终结果。
需要说明的是，前述内容中第一神经网络层、第二神经网络层以及第三神经网络层可以是任一个神经网络模型中任一神经网络层，第一神经网络层与第二神经网络层也可以是一个神经网络模型中是相邻的两个神经网络层。
通常，光信号在不同的传输路径上的传输时间不同，为了能够保证第二组光信号和第四组光信号同时传输至耦合器300，光计算装置中还可以包括第二延时模块500，如图4b所示，该第二延时模块500可以放置在第二组光信号的传输路径上，进一步调整第二组光信号的时延。
作为一种可能的实施方式，如图5所示，该控制系统中还可以包括双模式分束器600，该双模式分束器600能够接收第一组光信号和第三组光信号，将第一组光信号和第三组光信号传输至线性运算模块100，双模式分束器600还可以接收第二组光信号，将第二组光信号传输至第二延时模块500或第一延时模块200。
需要说明的是，双模式分束器600除了能够将一组光信号传输到线性运算模块100、第二延时模块500或第一延时模块200外，还可以将一组光信号分束为两组与分束前的一组光信号相同的光信号。例如，双模式分束器600可以先接收第十二组光信号，将第十二组光信号分束为第一组光信号和第三组光信号。
作为一种可能的实施方式，如图6a所示，该光计算装置中还可以设置非线性运算模块700，该非线性运算模块700可以对第二组光信号执行非线性操作，非线性操作以实现神经网络模型所配置的激活函数的功能。
需要说明的是，本申请实施例并不限定非线性运算模块700设置的位置，例如，非线性运算模块700可以与耦合器300连接，以便从耦合器300接收第五组光信号、第六组光信号、第八组光信号以及第十一组光信号，也可以与第一延时模块200连接，这样可以通过第一延时模块200接收第五组光信号或第六组光信号、第八组光信号以及第十一组光信号，如图6a所示。
作为一种可能的实施方式，如图6b所示，该光计算装置中还可以包括光放大器1000，该光放大器可以与耦合器300(如图6b)或非线性运算单元700，对放大第五组光信号、第六组光信号、第八组光信号以及第十一组光信号的功率。放大器1000可以是半导体光放大器(semiconductor optical amplifier，SOA)或者掺铒光纤放大器(erbium doped Fiber application amplifier，EDFA)。
在实际运算的过程中，并非所有的运算过程都需要非线性运算模块700参与。为了提高光计算装置的通用性，以适用各种运算场景，光计算装置中可以包括多个光开关，用于改变光信号在光计算装置中的传输方向。可选的，光计算装置中还可以包括多个耦合器，用以合并不同模块或器件输出的光信号。
参见图7，为本申请实施例提供的一种光计算装置，其中包括双模式分束器600、线性运算模块100、第二延时模块500、第一延时模块200、非线性运算模块700以及过滤模块400。该光计算装置中还包括三个耦合器，分别为第一耦合器810、第二耦合器820以及第三耦合器830，光计算装置中还包括两个光开关，分别为第一光开关910和第二光开关920，光计算装置中还包括放大器1000，与第三耦合器830或第一光开关910连接。需要说明的是本申请实施例并限定放大器1000的设置位置，放大器1000可以设置在第二耦合器820->第三耦合器830->第一光开关910->第一耦合器810该路径上的任一位置。
双模式分束器600、线性运算模块100、第二延时模块500、第一延时模块200、非线性运算模块700以及过滤模块400的作用可参见前述说明，此处在再赘述。
耦合器主要用于合并不同模块或器件输出的光信号。例如，第一耦合器810能够耦合输入光计算装置的光信号(如第三组光信号)以及过滤模块400输出的光信号(第二组光信号、第八组光信号或第十一组光信号)。第二耦合器820用于对线性运算模块100输出的光信号(第四组光信号)和第二延时模块500输出的光信号(调整了时延后的第二组光信号)进行合并，第三耦合器830可以对第二耦合器820输出的光信号以及非线性运算模块700输出的光信号进行合并。
光开关可以改变光信号的传输方向。例如，第一光开关910能够将第一延时模块200输出的信号传输至过滤模块400或非线性运算模块700。第二光开关920能够将非线性运算模块700输出的光信号传输至第三耦合器830或将非线性运算模块700输出的光信号输出。
下面结合图7，对光计算装置在实现运算的过程中光信号在光计算装置中的传输过程进行说明，在光计算装置可实现两种运算过程，一种为线性运算的运算过程，一种为和非线性运算的运算过程，下面分别为这两种运算过程中光信号的传输过程进行说明：
1、线性运算的运算过程。
以线性运算模块100对第一组光信号和第三组光信号执行线性运算为例，图8a中用黑色的实线标注了第一组光信号经过线性运算模块100变为第二组光信号的传输路径，由图8a可以看出第七组光信号经过第一耦合器810进入光计算装置，之后通过双模式分束器600，双模式分束器600将第十二组光信号分为第一组光信号和第三组光信号，将第一组光信号传输至线性运算模块100，线性运算模块100根据接收的第一电信号对第一组光信号进行调制， 输出第二组光信号，第二组光信号经过第二耦合器820传输至第三耦合器830，第三耦合器830将第二组光信号传输至第一延时模块200，第一延时模块200调整第二组光信号的时延，调整了时延后的第二组光信号在第一光开关910的作用下传输至过滤模块400，过滤模块400可以不进行任何剔除操作，直接将调整了时延后的第二组光信号传输至第一耦合器810。
第三组光信号在光计算装置中不经过线性运算模块100进行传输，传输路径为第二延时模块500->第二耦合器820->第三耦合器830->第一延时模块200->第一光开关910->过滤模块400第一耦合器810->双模式分束器600，当第一组光信号中最后一个光信号输入到线性运算模块100后，双模块分束器600将第三组光信号传输至线性运算模块100。
图8a中用黑色的虚线标注了第三组光信号经过线性运算模块100变为第四组光信号后第四组光信号的传输路径，由图8a可以看出双模式分束器600将第三组光信号传输至线性运算模块100，第三组光信号输入至线性运算模块100，线性运算模块100根据接收的第二电信号对第三组光信号进行调制，输出第四组光信号。
图8a中用灰色的实线标注了第二组光信号经过第一耦合器810后的传输路径，第二组光信号通过双模式分束器600，双模式分束器600将第二组光信号传输至第二延时模块500，第二延时模块500进一步调整第二组光信号的时延，使得第二组光信号从第二延时模块500输出后，能够与第四组光信号同时到达第二耦合器820。
第二耦合器820耦合第二组光信号和第四组光信号，输出第五组光信号，将第五组光信号传输至第三耦合器830。
若后续光计算设备无需要继续接收指示第一组数据的光信号，也即光计算装置所执行的计算已全部完成，第三耦合器830输出第五组光信号可以通过第一延时模块200(第一延时模块200不执行操作)、第一光开关910、非线性运算模块700(非线性运算模块700不执行操作)、第二光开关920，从光计算装置输出，第五组光信号可指示最终的计算结果。
若后续光计算设备还需要继续接收指示第一组数据的光信号，第三耦合器830可以将第五组光信号继续传输至第一延时模块200。第五组光信号的传输过程与第二组光信号从第三耦合器830输出后的传输过程类似，具体可参见前述内容，此处不再赘述，第三耦合器830可以对延时后的第五组光信号以及线性运算模块100调制后的其他组光信号合并输出第六组光信号，第三耦合器830输出第六组光信号可以通过第一延时模块200(第一延时模块200不执行操作)、第一光开关910、非线性运算模块700(非线性运算模块700不执行操作)、第二光开关920，从光计算装置输出，第五组光信号可指示最终的计算结果。
图8a所示的光信号的传输过程仅涉及线性运算过程，当光计算装置需要实现池化操作时，进行了线性运算后第三耦合器830耦合后的光信号还需要继续在光计算装置中进行传输，进行进一步处理。
以第六组光信号需要进行池化操作为例，对如图7所示的光计算装置在进行池化操作时，光计算装置中光信号的传输过程进行说明。第三耦合器830输出第五组光信号可以通过第一延时模块200(第一延时模块200不执行操作)、第一光开关910、过滤模块400(过滤模块400不执行操作)、第一耦合器810、双模式分束器600，最终到达线性运算模块100，线性运算模块100根据接收到指示第二权重的第三电信号对第六组电信号进行调制输出第七组光信号，第七组光信号经过第二耦合器820、第三耦合器830、第一延时模块200(第一延时模块200不执行操作)、第一光开关910，到达过滤模块400，过滤模块400过滤第七组光信号中的无效信号，输出第八组光信号，第八组光信号经过第二耦合器820、第三耦合器830、第一 延时模块200、第一光开关910、非线性运算模块700(非线性运算模块700不执行操作)、第二光开关920，从光计算装置输出。
以第九组光信号需要进行卷积运算为例，对如图7所示的光计算装置在进行卷积运算时，光计算装置中光信号的传输过程进行说明。线性运算模块100接收第九组光信号，线性运算模块100根据接收的第四电信号对第九组光信号进行调制，输出第十组光信号(这里仅是示例性的以线性运算模块100输出指示第二组数据与第二权重乘加运算结果的第十组光信号仅进行一次调制，在实际运算中，第十组光信号的产生也可以类似第五组光信号或第六组光信号产生的过程，也即线性运算模块100、第一延时模块200、第二延时模块500、第二耦合器820以及第三耦合器830等模块可配合产生第十组光信号)，第十组光信号经过第二耦合器820传输至第三耦合器830，第十组光信号经过第三耦合器830、第一延时模块200(第一延时模块200不执行操作)、第一光开关910，到达过滤模块400，过滤模块400过滤第十组光信号中的无效信号，输出第十一组光信号，第十一组光信号经过第二耦合器820、第二耦合器820、第三耦合器830、第一延时模块200、第一光开关910、非线性运算模块700(非线性运算模块700不执行操作)、第二光开关920，从光计算装置输出。
2、非线性运算的计算过程。
仍以光计算装置对第六组光信号进行非线性运算为例进行说明。
图8b中用黑色的实线标注了第五组光信号传输路径，若后续光计算设备无需要继续接收指示第一组数据的光信号，也即光计算装置所需执行的线性运算已全部完成，这第三耦合器830将第五组光信号传输至第一延时模块200，第一延时模块200可以不对第五组光信号进行处理，将第五组光信号传输至第一光开关910，通过第一光开关910，第五组光信号可以被传输至非线性运算模块700，非线性运算模块700可以对第五组光信号进行非线性操作，之后将执行了非线性操作后的第五组光信号传输至第二光开关920，通过第二光开关920，执行了非线性操作后的第五组光信号从光计算装置输出，执行了非线性操作后的第五组光信号可指示最终的计算结果。
图8b中用黑色的虚线标注了第五组光信号另一种的传输路径，若后续光计算设备还需要继续对第五组光信号进行处理，第三耦合器830可以将第五组光信号继续传输至第一延时模块200。第五组光信号的传输过程与第二组光信号从第三耦合器830输出后的传输过程类似，具体可参见前述内容，此处不再赘述。直至线性运算模块100所需执行的线性运算已全部完成，在完成所有线性运算后，第三耦合器830从第二耦合器820可以接收合并后的光信号，这里以当线性运算模块100执行所有线性运算后，第三耦合器830从第二耦合器820接收的光信号为第六组光信号。图8b中用灰色的虚线标注了第六组光信号的传输路径，第二耦合器820将接收第六组光信号传输至第一延时模块200，第一延时模块200可以不对第六组光信号进行处理，将第六组光信号传输至第一光开关910，通过第一光开关910，第六组光信号可以被传输至非线性运算模块700，非线性运算模块700可以对第六组光信号进行非线性操作，之后将执行了非线性操作后的第六组光信号传输至第二光开关920，通过第二光开关920，从光计算装置输出，执行了非线性操作后的第六组光信号可指示最终的计算结果。
需要说明的是，图8a～图8b中未示出放大器1000，放大器1000可以对第三耦合器830输出的光信号进行放大。
下面对光计算装置中各个模块或器件的构成以及对光信号的处理过程进行说明：
(1)、线性运算模块100。
线性运算模块100能够根据接收到的电信号(第一电信号、第二电信号或第三电信号) 对接收到的光信号(第一组光信号、第三组光信号以及第六组光信号)进行调制，改变光信号的光强，实现线性运算。
为了实现线性运算，线性运算模块100在接收到光信号后，可以将光信号分为N个时延不同的光信号，之后在对N个时延不同的光信号进行调制。
线性运算模块100中包括多个分束器、多个延时波导、多个耦合器以及N个光电调制器，可选的，还可以包括延时控制模块。
其中，延时控制模块接收输入至线性运算模块的光信号，校准光信号的时延。延时控制模块的构成、第一延时模块200以及第二延时模块500的作用均为调整光信号的时延，延时控制模块的构成、第一延时模块200以及第二延时模块500的结构也类似，关于延时控制模块的结构可参见下文中第一延时模块200以及第二延时模块500的说明，此处不再赘述。
分束器可以将一路光信号分为两路相同的光信号，分束器可以是Y形分束器，也可以是多模干涉耦合器。凡是能够将一路光信号分为两路相同的光信号的方式均适用于本申请实施例。延时波导可以传输光信号，还可以调整所传输的光信号的时延。
多个分束器和多个延时波导配合将延时控制模块输出的光信号分为N个时延不同的光信号。
光电调制器用于根据接收到的电信号对光信号进行调制，光电调制器可以是MZI，也可以可调谐微环谐振腔。每个光电调制器用于对多个分束器和多个延时波导输出的N个不同的光信号中的一个光信号进行调制。
多个耦合器配合用于将N个光电调制器输出的光信号合并为一个光信号。耦合器可以是Y分束器，通过反向Y分束器实现耦合器的功能。
如图9a所示，以N等于4为例，为线性运算模块100的结构示意图，线性运算模块100中包括时延控制模块1100、三个分束器(分别用分束器1101、分束器1102、分束器1103区分)、三个延时波导(分别用延时波导1104、延时波导1105、以及延时波导1106区分)、四个光电调制器(分别用光电调制器1107、光电调制器1108、光电调制器1109、以及光电调制器1110区分)、以及三个耦合器(分别用耦合器1111、耦合器1112、以及耦合器1113区分)。
以如图9a所示的线性运算模块100接收第一组光信号为例进行说明，第一组光信号输入至延时控制模块1100，延时控制模块1000调整第一组光信号的时延。
调整了时延后的第一组光信号传输至分束器1101，被分为两路相同的第一组光信号，为方便说明，该两路第一组光信号分别为第一组光信号1以及第二组光信号2。第一个光信号1不需要经过延时波导，传输至分束器1102，被分为两路相同的第一组光信号，为方便说明，该两路第一组光信号分别为第一组光信号3以及第二组光信号4。第一组光信号3传输至光电调制器1107。第二组光信号4经过延时波导1105，延时波导1105对第二组光信号4的时延进行调整，调整了时延的第一组光信号4传输至光电调制器1108。
第一个光信号2经过延时波导1104，传输至分束器1103，被分为两路相同的第一组光信号，为方便说明，该两路第一组光信号分别为第一组光信号5以及第二组光信号6。第一组光信号5传输至光电调制器1109。第二组光信号6经过延时波导1106，延时波导1106对第二组光信号6的时延进行调整，调整了时延的第一组光信号4传输至光电调制器1110。
线性运算模块100接收的第一电信号可加载在四个光电调制器上，以使得四个光电调制器可以基于第一电信号分别对接收到的第一组光信号进行调制。
这里需要说明的是，第一电信号可以包括多个子信号，每个光电调制器上加载一个子信号，每个光电调制器根据加载的一个电信号对接收到的第一组光信号进行调制。
线性运算模块100还可以包括N个相位调制器，一个相位调制器连接一个光电调制器，相位调制器能调制光电调制器输出的光信号的相位。线性运算模块100还可以包括至少N-1个锁相模块，锁相模块用于稳定光信号的相位。
参见图9b，以N等于4时，本申请实施例提供的一种线性运算模块，与图9a不同，线性运算模块100中包括四个相位调制器，分别为相位调制器1114、相位调制器1115、相位调制器1116以及相位调制器1117，还包括三个锁相单元，分别为锁相单元1118、锁相单元1119以及锁相单元1120。
相位调制器1114与光电调制器1107连接，调节光电调制器1107输出的光信号的相位。相位调制器1115与光电调制器1108连接，调节光电调制器1108输出的光信号的相位。相位调制器1116与光电调制器1109连接，调节光电调制器1109输出的光信号的相位。相位调制器1117与光电调制器1110连接，调节光电调制器1107输出的光信号的相位。
锁相单元1118与相位调制器1114连接，根据耦合器1111输出的光信号锁定相位调制器1114输出的光信号的相位，锁相单元1119与相位调制器1116连接，根据耦合器1112输出的光信号锁定相位调制器1116输出的光信号的相位，锁相单元1120与耦合器1112连接，根据耦合器1113输出的光信号锁定耦合器1112输出的光信号的相位。
(2)、第一延时模块200、第二延时模块500。
在光计算装置中第一延时模块200、第二延时模块500的作用均为调整光信号的时延，本申请实施例并不限定第一延时模块200以及第二延时模块500的结构，凡是能够调整光信号时延的器件均适用于本申请实施例。
下面对本申请实施例提供的一种第一延时模块200的结构进行说明，参见图10，第一延时模块200可以采用级联结构，该第一延时模块200可以包括多个串联的光开关(在图10中分别用光开关1、光开关2…光开关n区分)，两个光开关之间设置有两个光信号的传输路径，一个为长时延传输路径，该传输路径上设置有可调整时延的器件(如延时波导1)，该器件能够对光信号增加较长的时延，另一个为短时延传输路径，该传输路径上可设置可调整时延的器件(如延时波导2)，该器件能够对光信号增加较短的时延，该器件对光信号增加的时延相较于长时延传输路径上设置的可调整时延器件对光信号增加的时延较短，作为一种可能的传输时延，该短时延传输路径上也可以不设置可调整时延的器件。在第一延时模块200中，每个光开关可以通过电信号来控制，用于选择光信号的传输路径。
(3)、过滤模块400。
过滤模块400可以将串行的光信号中的部分光信号剔除，还能够调整所剔除的光信号前传输的光信号的时延，以保证过滤模块400输出一组光信号中无空缺。
如图11所示，为本申请实施例提供的一种过滤模块的结构示意图。该过滤模块400包括两个光开关(为方便说明用光开关501和光开关502区分)以及第三延时模块503。
以过滤模块400接收的光信号为第五组光信号为例，下面对过滤模块400中光信号的传输过程进行说明：
第七组光信号输入至过滤模块400后，先进入光开关501，在光开关501的作用下，传输至第三延时模块503，延时模块503可不进行处理直接将第七组光信号传输至光开关502，光开关502将第七组光信号中无效信号输出到光计算装置之外，将第七组光信号中剩余的光信号(第八组光信号)传输至光开关501，光开关501将第八组光信号传输至第三延时模块503。由于第七组光信号中包括的信号为串行的光信号，当将其中一部分光信号剔除后，会导致在第八组光信号中存在空缺，为了减少该空缺，第三延时模块503可以调整第八组光信号 中传输顺序位于无效信号之前的光信号的时延，而不调整传输顺序位于无效信号之前的光信号的时延，以保证传输顺序位于无效信号之前的光信号与传输顺序位于无效信号之后的光信号不再存在空缺。
(4)、双模式分束器600、光开关(第一光开关910以及第二光开关920)。
双模式分束器600、光开关本质上为带有分光比例调节功能的分束器。
以双模式分束器600为例，对双模式分束器600和光开关的结构说明，如图12所示，双模式分束器600包括分束器601、两个移相器(为方便说明，分别用移相器602以及移相器603区分)以及耦合器604。分束器601可以将光信号分为两路相同的光信号，移相器602以及移相器603可以对光信号的相位补偿，实现分光比例的调整。
(5)、耦合器(如耦合器300、第一耦合器810和第二耦合器820、以及耦合器604)。
本申请实施例中耦合器可以为Y型耦合器，用以将两路光信号耦合为一路光信号。
(6)、非线性运算模块700。
本申请实施例并不限定非线性运算模块700的构成，例如，非线性运算模块700中可以包括具备光学非线性效应的器件,如石墨烯层。
结合图7所示的光计算装置，对该光计算装置实现全连接层、卷积层、池化层的运算过程的方式进行说明。
1、全连接层。
为了实现全连接层的运算，可以利用光计算装置实现乘加运算。光计算装置可以多次接收串行的光信号(如第一组光信号或第三组光信号)，为方便说明这里用Xc[1]～Xc[n]表示串行的光信号，该串行的光信号由n个子信号(如Xc[1]、Xc[n])构成。
串行的光信号Xc[1]～Xc[n]进入光计算装置后，依次传输至线性运算模块100，在线性运算模块100中，针对任一子信号，线性运算模块100可以根据接收到的电信号(如第一电信号以及第二电信号)调制子信号，调制后的子信号继续在光计算装置中传输，调制后的子信号经过第一延时模块200(调整调制后的子信号的时延，以使得调整后的子信号与下一次输入光计算装置中的串行的光信号Xc[1]～Xc[n]中经调制后的子信号在时延上可保持对齐)、过滤模块400(过滤模块400不进行任何处理)、双模式分束器600、第二延时模块500、第二耦合器820以及第三耦合器830。
光计算装置可以继续接收串行的光信号Xc[1]～Xc[n]执行上述操作，直至完成所有线性运算。
下面以n等于8，线性运算模块100中包括4个调整器为例，对光计算装置所执行的运算过程进行说明：
参见图13，在时间段1～8，线性运算模块100依次接收Xc[1]～Xc[8]，线性运算模块100中的四个调制器可以调整Xc[1]～Xc[8]的强度，其中，w ij可以指示线性运算模块100中的调整器对子信号光强的调制程度(也就是说，w ij可指示权重)，其中，i指示调制器所调整的子信号的编号，j指示光电调制器的编号。
以Xc[1]为例，Xc[1]在时间段1输入至线性运算模块100后，分束为四个延时不同的子信号Xc[1]，分别为时延在时间段1的Xc[1]、时延在时间段2的Xc[1]、时延在时间段3的Xc[1]以及时延在时间段4的Xc[1]。第一个光电调制器对时延在时间段1的Xc[1]调制后的子信号为w 11Xc[1]。第二个光电调制器对时延在时间段2的Xc[1]调制后的子信号为w 12Xc[1]。第三个光电调制器对时延在时间段3的Xc[1]调制后的子信号为w 13Xc[1]。第四个光电调制器对时延在时间段4的Xc[1]调制后的子信号为w 14Xc[1]。
调制后的子信号在光计算装置中继续传输，传输路径为第二耦合器820->第三耦合器830->第一延时模块200->过滤模块400->双模式分束器600->第二延时模块500，直至串行的光信号中的最后一个子信号经过线性运算模块100。
之后，第一延时模块200调整经过调制后的所有子信号的时延，增加的时延为2*4θ(θ为单信号脉冲的时间长度)，此时，光计算装置继续接收串行的光信号Xc[1]～Xc[8]，后续接收的串行的光信号Xc[1]～Xc[8]在光计算装置中的运算过程与前一个串行的光信号的Xc[1]～Xc[8]类似。
串行的光信号中的最后一个子信号经过线性运算模块100后，经过第二耦合器820和第三耦合器830，光信号被合并为串行的8个光信号，可以指示最终的计算结果。
2、卷积层。
卷积层与全连接层的计算过程类似，区别在于在执行卷积层的运算时，过滤模块400可以从经过线性运算模块100调制后的光信号中剔除一些不需要参与运算的光信号，需要说明的是，过滤模块400剔除的光信号与所需完成的运算有关。
以卷积层所需要完成的运算为3x3的矩阵和2x2的卷积核的卷积运算为例，3x3的矩阵和2x2的卷积核的卷积运算如图14所示，其中，Xc 1～Xc 9为待输入的串行的光信号，a、b、c、d为卷积核所指示的权重。Xn[1]～Xn[4]为所需输出的光信号，指示最终的计算结果。
下面以n等于9，线性运算模块100中2个调整器参与计算为例，对光计算装置所执行的卷积运算进行说明：
参见图15a～15d，线性运算模块100依次接收Xc 1～Xc 9，线性运算模块100中的2个调制器可以调整Xc[1]～Xc[9]的强度，其中，d和c分别指示线性运算模块100第一次线性运算过程中调整器对子信号光强的调制程度(也即卷积核所指示的权重)。a和b分别指示线性运算模块100第二次线性运算过程中调整器对子信号光强的调制程度(也即卷积核所指示的权重)。Xn 1～Xn 9为一次线性运算过程中线性运算模块100输出的光信号。
其中，15a图为线性运算模块100第一次线性运算过程中的输入的串行光信号，以及输出的光信号。15b图为线性运算模块100第二次线性运算过程中的输入的串行光信号，以及输出的光信号。15c图为过滤模块400从线性运算模块100输出的光信号中剔除的光信号(线性框中的光信号即为所剔除的信号Xn 1～Xn 4、Xn 7)以及剩余的光信号。15d图为光计算装置最终输出的光信号。
3、池化层。
池化层与卷积层的运算过程类似，区别在于，池化层中的卷积核的作用为保留特定数据的最大值(最大池化)或平均值(平均池化)。
下面以光计算装置实现4x4矩阵到2x2矩阵的平均池化运算过程为例，对光计算装置所执行的运算过程进行说明：
4x4矩阵到2x2矩阵的平均池化运算如图16所示，经过平均池化运算，可以实现数据降维。
参见图17，图17为线性运算模块100执行线性运算过程中的输入的串行光信号，以及输出的光信号。线性运算模块100依次接收Xc 1～Xc 16，其中，0.25指示线性运算模块100在执行线性运算过程中调整器对子信号光强的调制程度,空白处指示性运算过程中调整器将子信号光强调制为0。Xn 1～Xn 16为线性运算模块100输出的光信号。从图17中可见，有效信号为Xn6,Xn8,Xn14,Xn16。Xn1～Xn5,Xn7,Xn9～Xn13,Xn15均为无效信号。光计算装置中的过滤模块400可以过滤无效信号。
为了使方案描述更加清楚，下面将结合前面的实施例，以图18所示的计算方法为例，对本发明实施例中提供的光计算装置所执行的光计算方法概括介绍。
线性运算模块100接收第一组光信号以及第一电信号。线性运算模块100根据第一电信号对第一组光信号进行调制以输出第二组光信号。
线性运算模块100可以接收第三组光信号以及第二电信号，线性运算模块100根据第二电信号对第三组光信号进行调制以输出第四组光信号，其中，第四组光信号用于指示第一组数据与第二部分权重值的计算结果。
第一延时模块200可以调整第二组光信号的时延；之后，耦合器300可以将延时后的第二组光信号与第四组光信号合并为第五组光信号，其中，第五组光信号指示第一组数据与第一部分权重以及第二部分权重进行乘加运算后的第一计算结果。
在一种可能实现方式中，光计算装置中还可以包括过滤模块，以光计算装置对第六组光信号进行池化操作为例进行说明。线性运算模块100可以接收第六组光信号以及第三电信号。
线性运算模块100根据第三电信号对第六组光信号进行调制以输出第七组光信号，过滤模块可以接收第七组光信号，并过滤第七组光信号中的无效信号后输出第八组光信号。
在一种可能实现方式中，光计算装置包括过滤模块的情况下，光计算装置还可以借助过滤模块实现卷积运算。以光计算装置对第九组光信号进行卷积运算为例进行说明。
线性运算模块100可以接收第九组光信号以及第四电信号，根据第四电信号对第九组光信号进行调制以输出第十组光信号；
过滤模块接收第十组光信号，并过滤第十组光信号中的无效信号后输出第十一组光信号，第十一组光信号用于指示对第三计算结果进行卷积操作的计算结果。
本申请实施例还提供了一种光计算系统，该光计算系统包括处理器以及如前述内容所提及的光计算装置，参见图19，光计算系统10中包括光计算装置20以及处理器30。处理器30和光计算装置20可以通过标准的主机接口或网络接口(network interface)等进行连接。例如，主机接口可以包括快捷外设互联标准(Peripheral Component Interconnect Express,PCIE)接口。处理器30可以向光计算装置20发送待执行计算的数据，其中，所述数据包括第一组数据，还可以包括第一权重。所述数据也可以是第二组数据以及第二权重。当光计算装置20接收到所述数据后，可以根据所述数据确定对应的一组或多组光信号(如第一组光信号、第三组光信号、第九组光信号等)以及电信号(第一电信号、第二电信号以及第四电信号等)，完成相应的计算过程。
本申请是参照根据本申请的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编 程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然，本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (13)

  1. 一种光计算装置，其特征在于，包括：
    线性运算模块，用于分别接收第一组光信号和第三组光信号，其中，所述第一组光信号和所述第三组光信号用于指示第一组数据；
    接收用于指示第一权重中的第一部分权重值的第一电信号，根据所述第一电信号对所述第一组光信号进行调制以输出第二组光信号，其中，所述第二组光信号用于指示所述第一组数据与所述第一部分权重值的计算结果，所述第一权重为第一神经网络层的权重；
    接收用于指示所述第一权重中的第二部分权重的第二电信号，并根据所述第二电信号对所述第三组光信号进行调制以输出第四组光信号，其中，所述第四组光信号用于指示所述第一组数据与所述第二部分权重值的计算结果；
    第一延时模块，用于调整所述第二组光信号的时延；
    耦合器，用于将延时后的所述第二组光信号与所述第四组光信号合并为第五组光信号，其中，所述第五组光信号指示所述第一组数据与所述第一部分权重以及所述第二部分权重进行乘加运算后的第一计算结果。
  2. 如权利要求1所述的光计算装置，其特征在于：
    所述线性运算模块，还用于接收第六组光信号，所述第六组光信号用于指示所述第一组数据与所述第一权重进行乘加运算后的第二计算结果，其中，所述第二计算结果包含所述第一计算结果；
    接收用于指示第二权重的第三电信号，并根据所述第三电信号对所述第六组光信号进行调制以输出第七组光信号，其中，所述第二权重为第二神经网络层的权重；
    所述光计算装置还包括：
    过滤模块，用于接收所述第七组光信号，并过滤所述第七组光信号中的无效信号后输出第八组光信号，所述第八组光信号用于指示对所述第二计算结果进行池化操作的计算结果。
  3. 如权利要求1或2所述的光计算装置，其特征在于：
    所述线性运算模块，还用于接收第九组光信号，所述第九组光信号用于指示第二组数据；
    接收用于指示第三权重的第四电信号，并根据所述第四电信号对所述第九组光信号进行调制以输出第十组光信号，所述第十组光信号用于指示所述第二组数据与所述第三权重进行乘加运算后的第三计算结果，所述第三权重为第三神经网络层的权重；
    所述过滤模块，还用于接收所述第十组光信号，并过滤所述第十组光信号中的无效信号后输出第十一组光信号，所述第十一组光信号用于指示对所述第三计算结果进行卷积操作的计算结果。
  4. 如权利要求1～3任一所述的光计算装置，其特征在于，所述装置还包括：
    非线性运算模块，用于对所述第六组光信号、第五组光信号、所述第八组光信号或所述第十一组光信号进行非线性操作，所述非线性操作为激活函数操作。
  5. 如权利要求1～4任一所述的光计算装置，其特征在于，所述装置还包括：
    双模式分束器，
    用于接收用于指示所述第一组数据的第十二组光信号，将所述第十二组光信号分为所述第一组光信号以及所述第三组光信号。
  6. 如权利要求1～5任一所述的光计算装置，其特征在于，所述装置还包括第二延时模块，
    所述双模式分束器，还用于将所述延时后的第二组光信号传输至所述第二延时模块；
    所述第二延时模块，还用于调整所述第二组光信号的时延，以使得所述第二组光信号与从所述第四组光信号同时传输至所述耦合器。
  7. 如权利要求1～6任一所述的光计算装置，其特征在于，所述线性运算模块包括N个光电调制器，
    N个光电调制器中的每一个光电调制器，用于接收所述第一组光信号中的一个光信号以及接收用于指示第一部分权重的第一电信号中的一个电信号，并根据接收的电信号对接收的光信号进行调制。
  8. 如权利要求7所述的光计算装置，其特征在于，所述线性运算模块还包括N个相位调制器以及至少N-1个锁相模块；
    一个所述相位调制器连接一个所述光电调制器，用于调整经过所述调制器调制后的第一组光信号或第二组光信号的相位；
    一个所述锁相模块连接所述N个相位调制器中的两个相位调制器，用于锁定经过所述两个相位调制器调制后的第一组光信号或第二组光信号的相位。
  9. 如权利要求1～8任一所述的光计算装置，其特征在于，所述线性运算模块还包括：
    放大器，用于放大所述第五组光信号、所述第六组光信号、所述第八组光信号或所述第十一组光信号的功率。
  10. 一种计算方法，其特征在于，应用于光计算装置，所述光计算装置包括线性运算模块、第一延时模块以及耦合器，该方法包括：
    所述线性运算模块接收第一组光信号以及第一电信号，其中，所述第一组光信号用于指示第一组数据，所述第一电信号用于指示第一权重中的第一部分权重值；根据所述第一电信号对所述第一组光信号进行调制以输出第二组光信号，其中，所述第二组光信号用于指示所述第一组数据与所述第一部分权重值的计算结果，所述第一权重为第一神经网络层的权重；
    所述线性运算模块接收第三组光信号以及第二电信号，其中，所述第三组光信号用于指示第一组数据，所述第二电信号用于指示所述第一权重中的第二部分权重值，并根据所述第二电信号对所述第三组光信号进行调制以输出第四组光信号，其中，所述第四组光信号用于指示所述第一组数据与所述第二部分权重值的计算结果；
    所述第一延时模块调整所述第二组光信号的时延；
    所述耦合器将延时后的所述第二组光信号与所述第四组光信号合并为第五组光信号，其中，所述第五组光信号指示所述第一组数据与所述第一部分权重以及所述第二部分权重进行乘加运算后的第一计算结果。
  11. 如权利要求10所述的计算方法，其特征在于，所述光计算装置还包括过滤模块，所述方法还包括：
    所述线性运算模块接收第六组光信号，所述第六组光信号用于指示所述第一组数据与所述第一权重进行乘加运算后的第二计算结果以及用于指示第二权重的第三电信号，其中，所述第二计算结果包含所述第一计算结果；其中，所述第二权重为第二神经网络层的权重；
    所述线性运算模块根据所述第三电信号对所述第六组光信号进行调制以输出第七组光信号；
    所述过滤模块接收所述第七组光信号，并过滤所述第七组光信号中的无效信号后输出第八组光信号，所述第八组光信号用于指示对所述第二计算结果进行池化操作的计算结果。
  12. 如权利要求10或11所述的计算方法，其特征在于，还包括：
    所述线性运算模块接收第九组光信号，所述第九组光信号用于指示第二组数据；
    所述线性运算模块接收用于指示第三权重的第四电信号，并根据所述第四电信号对所述第九组光信号进行调制以输出第十组光信号，所述第十组光信号用于指示所述第二组数据与所述第三权重进行乘加运算后的第三计算结果，所述第三权重为第三神经网络层的权重；
    所述过滤模块接收所述第十组光信号，并过滤所述第十组光信号中的无效信号后输出第十一组光信号，所述第十一组光信号用于指示对所述第三计算结果进行卷积操作的计算结果。
  13. 一种光计算系统，其特征在于，包括处理器以及与所述处理器连接的如权利要求1-9中任意一项所述的光计算装置，其中，所述处理器用于向所述光计算装置发送待执行计算的数据，其中，所述数据包括所述第一组数据。
PCT/CN2021/078233 2020-02-28 2021-02-26 一种光计算装置、系统以及计算方法 WO2021170107A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21760588.0A EP4099232A4 (en) 2020-02-28 2021-02-26 OPTICAL COMPUTING DEVICE AND SYSTEM AND COMPUTING METHOD
US17/896,143 US20220414442A1 (en) 2020-02-28 2022-08-26 Optical computing apparatus and system, and computing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010127941.7 2020-02-28
CN202010127941.7A CN113325917A (zh) 2020-02-28 2020-02-28 一种光计算装置、系统以及计算方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/896,143 Continuation US20220414442A1 (en) 2020-02-28 2022-08-26 Optical computing apparatus and system, and computing method

Publications (1)

Publication Number Publication Date
WO2021170107A1 true WO2021170107A1 (zh) 2021-09-02

Family

ID=77412619

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/078233 WO2021170107A1 (zh) 2020-02-28 2021-02-26 一种光计算装置、系统以及计算方法

Country Status (4)

Country Link
US (1) US20220414442A1 (zh)
EP (1) EP4099232A4 (zh)
CN (1) CN113325917A (zh)
WO (1) WO2021170107A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116339444A (zh) * 2021-12-23 2023-06-27 华为技术有限公司 光计算装置、方法和系统
CN117892784A (zh) * 2022-10-13 2024-04-16 华为技术有限公司 光电计算系统、加速卡和光电计算的调度方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107196713A (zh) * 2017-05-27 2017-09-22 东南大学 一种基于光信号延时的光接收机和接收方法
CN109981172A (zh) * 2019-03-01 2019-07-05 上海交通大学 基于多波长调制和色散时延的全光矩阵乘加实现方法
US20190370652A1 (en) * 2018-06-05 2019-12-05 Lightelligence, Inc. Optoelectronic computing systems

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107196713A (zh) * 2017-05-27 2017-09-22 东南大学 一种基于光信号延时的光接收机和接收方法
US20190370652A1 (en) * 2018-06-05 2019-12-05 Lightelligence, Inc. Optoelectronic computing systems
CN109981172A (zh) * 2019-03-01 2019-07-05 上海交通大学 基于多波长调制和色散时延的全光矩阵乘加实现方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4099232A4 *

Also Published As

Publication number Publication date
CN113325917A (zh) 2021-08-31
EP4099232A1 (en) 2022-12-07
EP4099232A4 (en) 2023-08-09
US20220414442A1 (en) 2022-12-29

Similar Documents

Publication Publication Date Title
WO2021170107A1 (zh) 一种光计算装置、系统以及计算方法
CN112001487A (zh) 一种光子神经网络
CN113872697B (zh) 光发送机和光调制的方法
CN113805641B (zh) 一种光子神经网络
CN112506265A (zh) 一种光计算装置以及计算方法
CN114815959B (zh) 一种基于波分复用的光子张量计算加速方法及装置
CN112232503A (zh) 计算装置、计算方法以及计算系统
CN112101540B (zh) 光学神经网络芯片及其计算方法
CN115905792A (zh) 一种用于光学实数矩阵计算的马赫曾德尔干涉仪网络
WO2022160784A1 (zh) 一种光计算装置、系统及卷积计算方法
WO2008027640B1 (en) Selecting optical waves
US11934943B1 (en) Two-dimensional photonic neural network convolutional acceleration chip based on series connection structure
CN115222035B (zh) 一种光子神经网络卷积加速芯片
CN117077750A (zh) 一种并行光子卷积运算芯片及系统
CN103149633A (zh) 一种双偏振态信号处理集成芯片
Wu et al. SiP architecture for accelerating collective communication in distributed deep learning
CN112232487B (zh) 光学神经网络芯片及其计算方法
CN113452449B (zh) 对时域信号进行酉矩阵计算的光学装置
CN112099565B (zh) 通用线性光计算模块及其控制方法
CN104579464A (zh) 一种进行数据传输的方法和装置
CN116266771A (zh) 基于多模干涉的卷积运算装置和卷积运算方法
CN116266065A (zh) 光学矩阵运算装置和系统
CN117217321B (zh) 一种量子图态产生装置及产生方法
CN116306858A (zh) 基于多模干涉的神经网络装置及其运算方法
CN111240400B (zh) 光除法模块、光子神经网络芯片、芯片系统及光除法器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21760588

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021760588

Country of ref document: EP

Effective date: 20220901

NENP Non-entry into the national phase

Ref country code: DE