CN114723019A - Convolution calculation method and device based on photon calculation chip - Google Patents
Convolution calculation method and device based on photon calculation chip Download PDFInfo
- Publication number
- CN114723019A CN114723019A CN202210475719.5A CN202210475719A CN114723019A CN 114723019 A CN114723019 A CN 114723019A CN 202210475719 A CN202210475719 A CN 202210475719A CN 114723019 A CN114723019 A CN 114723019A
- Authority
- CN
- China
- Prior art keywords
- vector
- calculation
- matrix
- output
- photon
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 111
- 239000013598 vector Substances 0.000 claims abstract description 145
- 239000011159 matrix material Substances 0.000 claims abstract description 119
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000010586 diagram Methods 0.000 claims description 20
- 238000004891 communication Methods 0.000 claims description 19
- 230000003287 optical effect Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 abstract description 4
- 238000013527 convolutional neural network Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
- G06N3/0675—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means using electro-optical, acousto-optical or opto-electronic means
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Complex Calculations (AREA)
Abstract
The application discloses a convolution calculation method and device based on a photon calculation chip. The method is suitable for a convolution calculation device of a photon calculation chip, and comprises the following steps: acquiring a weight matrix, and loading the weight matrix to an interferometer array in a photon calculation chip; acquiring a target input vector to be processed; inputting target input data to an input end, and performing linear calculation of a vector-weight matrix by using the target input data through the input end to obtain an output vector; and extracting elements belonging to the same row from each output vector, and summing to obtain a convolution calculation result between the convolution kernel and the target characteristic graph. The method and the device have the advantages that the linear calculation of the vector and the weight matrix is realized by utilizing the photon calculation chip, the operation between the matrix and the matrix in the convolutional neural network can be simulated, the operation times of the photon calculation chip are reduced by optimizing the weight matrix extraction method, and the calculation efficiency is improved.
Description
Technical Field
The application relates to the technical field of photonic chips, in particular to a convolution calculation method and device based on a photonic calculation chip.
Background
An Artificial Neural Network (ANN) algorithm includes a large number of matrix linear multiplication operations. In order to realize massive data training and reasoning, AI chips such as GPU, TPU, FPGA, ASIC and brain-like chip are produced. However, the current AI chip still relies on electronic computing technology, and with the failure of moore's law, the electronic AI chip is difficult to meet the ever-increasing computing demands in the future. Therefore, researchers propose a non-von neumann photonic computing chip to replace an electrical signal with an Optical signal to realize high-speed, high-parallelism, and low-power Optical Neural Network (ONN) operation.
In the conventional convolution operation, a convolution kernel of 3 × 3 is taken as an example for explanation, as shown in fig. 1. The feature matrix [ a11, a12, a13], [ a21, a2, a23], [ a31, a32, a33] in the sliding window is convolved with convolution kernels [ w11, w12, w13], [ w21, w22, w23], [ w31, w32, w33] to obtain (a11w11+ a12w12+ a13w13) + (a21w21+ a22w22+ a23w23) + (a31w31+ a32w32+ a33w 33). The 3 x 3 convolution kernel W is implemented using a photon computation chip. In this case, the U, Σ, and V × matrices are all 3 × 3, and 3 interferometers (3 × (3-1)/2) are required for a total of 9 interferometers.
Since the input of the photon computing chip is a one-dimensional vector, the feature map in the sliding window needs to be input in three times, namely [ a11, a12, a13], [ a21, a22, a23] and [ a31, a32, a33 ]. The calculation results are shown in formulas (1) - (3).
Formula (1):
formula (2):
formula (3):
comparing the above results, it can be seen that the output of equation (1) is only valid in line 1, corresponding to detector No. 1 of the photon counting chip. The output of the formula (2) is only valid in the 2 nd line, and corresponds to the No. 2 detector of the photon calculation chip. And (4) outputting a No. 3 detector which is only valid in the 1 st line and corresponds to the photon calculation chip. If there are n convolution kernels (usually n > m), the photon counting chip processes 3n times in the sliding window, resulting in too many counting times.
Disclosure of Invention
In order to solve the technical problem or at least partially solve the technical problem, the present application provides a convolution calculation method and apparatus based on a photon calculation chip.
According to an aspect of the embodiments of the present application, there is provided a convolution calculation method based on a photon calculation chip, the method is applied to a convolution calculation apparatus of the photon calculation chip, the convolution calculation apparatus includes an optical signal input end and a photodetector output end, the input end and the output end are connected through an interferometer array arranged in a cascade manner, the method includes:
acquiring a weight matrix, and loading the weight matrix to an interferometer array in a photon computing chip;
acquiring a target input vector to be processed, wherein the target input vector is obtained from a target characteristic diagram;
inputting the target input vector to the input end, and performing linear calculation of a vector-weight matrix by using the target input vector through the input end to obtain an output vector;
and extracting elements belonging to the same row from each output vector, and summing to obtain a convolution calculation result between the convolution kernel and the target characteristic graph.
Further, the obtaining the weight matrix includes:
extracting an ith row parameter from n m-by-m convolution kernels, and generating an m-by-n weight matrix corresponding to the ith row parameter based on the ith row parameter, wherein the number of the weight matrix corresponding to the ith row parameter is m;
and loading the weight matrix corresponding to the ith row parameter to an interferometer array of a photon calculation chip, wherein m, n and i are integers which are more than or equal to 1.
Further, the obtaining a target input vector to be processed includes:
acquiring a target characteristic diagram to be processed;
analyzing the target characteristic diagram by using a preset sliding window to obtain m characteristic line vectors to be processed, wherein each characteristic line vector comprises m elements;
and extracting a characteristic line vector of the ith line, converting the characteristic line vector into a characteristic column vector corresponding to the ith line, and determining the characteristic column vector as the target input vector.
Further, the inputting the target input vector to the input end, and performing linear calculation of a vector-weight matrix by using the target input vector through the input end to obtain an output vector includes:
inputting the characteristic column vector corresponding to the ith row into the input end so as to enable the input end and the weight matrix corresponding to the ith row to perform linear multiplication operation to obtain an output vector corresponding to the ith row;
and repeatedly operating the photon computing chip until m output vectors are obtained, wherein the output vectors comprise n elements.
Further, the extracting elements belonging to the same row from each output vector and summing the elements to obtain a convolution calculation result between the convolution kernel and the target feature map includes:
extracting elements belonging to the same row from the m output vectors, and summing to obtain output values corresponding to the m output vectors;
and determining output values corresponding to the m output vectors as the calculation result.
According to another aspect of the embodiments of the present application, there is also provided a convolution calculating apparatus based on a photon calculating chip, including: the optical fiber coupling device comprises an optical signal input end and an optical detector output end, wherein the input end is connected with the output end through an interferometer array which is arranged in a cascade mode, and a weight matrix is loaded in the interferometer array;
the input end is used for obtaining a target input vector and performing linear calculation of a vector-weight matrix by using the target input vector to obtain an output vector;
and the output end is used for extracting elements belonging to the same row from each output vector to sum so as to obtain a convolution calculation result between the convolution kernel and the target characteristic graph.
Further, the weight matrix is generated according to a unitary matrix, a diagonal matrix and a complex conjugate matrix corresponding to the unitary matrix, where a dimension of the weight matrix is mxn, a dimension of the unitary matrix is mxm, a dimension of the diagonal matrix is mxn, and a dimension of the complex conjugate matrix is nxn.
Further, the number of interferometers in the unitary matrix corresponding interferometer array is calculated based on the following formula:
m1n (n-1)/2, wherein m1The unitary matrix corresponds to the number of interferometers in the array of interferometers.
Further, the number of interferometers in the array of corresponding interferometers of the diagonal matrix is calculated based on the following formula:
m2(ii) 3n (3n-1)/2, wherein m is2The diagonal matrix corresponds to the number of interferometers in the interferometer array.
According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program that executes the above steps when the program is executed.
According to another aspect of the embodiments of the present application, there is also provided an electronic apparatus, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus; wherein: a memory for storing a computer program; a processor for executing the steps of the method by running the program stored in the memory.
Embodiments of the present application also provide a computer program product containing instructions, which when run on a computer, cause the computer to perform the steps of the above method.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: the method and the device have the advantages that the linear calculation of the vector and the weight matrix is realized by utilizing the photon calculation chip, the operation between the matrix and the matrix in the convolutional neural network can be simulated, the operation times of the photon calculation chip are reduced by optimizing the weight matrix extraction method, and the calculation efficiency is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a diagram illustrating a conventional convolution calculation provided in an embodiment of the present application;
fig. 2 is a schematic diagram of a convolution calculating apparatus based on a photon calculating chip according to an embodiment of the present application;
fig. 3 is a flowchart of a convolution calculation method based on a photon calculation chip according to an embodiment of the present disclosure;
FIG. 4 is a diagram illustrating convolution calculations provided by an embodiment of the present application;
FIG. 5 is a diagram illustrating convolution calculations provided by an embodiment of the present application;
FIG. 6 is a diagram illustrating convolution calculations provided in an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments, and the illustrative embodiments and descriptions thereof of the present application are used for explaining the present application and do not constitute a limitation to the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another similar entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Based on the above technical problem, the embodiments of the present application provide a convolution calculation method and apparatus based on a photon calculation chip. The method provided by the embodiment of the invention can be applied to any required electronic equipment, for example, the electronic equipment can be electronic equipment such as a server and a terminal, and the method is not particularly limited herein, and is hereinafter simply referred to as electronic equipment for convenience in description.
Fig. 2 is a schematic diagram of a convolution calculation apparatus based on a photon calculation chip according to an embodiment of the present application, and the apparatus may be implemented as part of or all of an electronic device through software, hardware, or a combination of the two. As shown in fig. 2, the apparatus includes:
the optical signal input end and the optical detector output end are connected through an interferometer array which is arranged in a cascade mode, and a weight matrix is loaded in the interferometer array;
the input end is used for obtaining a target input vector and performing linear calculation of a vector-weight matrix by using the target input vector to obtain an output vector;
and the output end is used for extracting elements belonging to the same row from each output vector and summing the elements to obtain a convolution calculation result between the convolution kernel and the target characteristic graph.
In the embodiment of the present application, the number of interferometers in the interferometer array is determined according to a weight matrix, the weight matrix is generated according to a unitary matrix, a diagonal matrix, and a complex conjugate matrix corresponding to the unitary matrix, and the dimensions corresponding to the unitary matrix, the diagonal matrix, and the complex conjugate matrix are respectively determined according to the number of convolution kernels in the input layer.
In the embodiment of the present application, a3 × 3 convolution kernel is taken as an example, the dimension of the weight matrix is n × 3, the dimension of the unitary matrix is n × n, the dimension of the diagonal matrix is n × 3, the dimension of the complex conjugate matrix is 3 × 3, and n is the number of convolution kernels.
It should be noted that, in the embodiment of the present application, when designing a photon computing chip, the number of interferometers in each corresponding interferometer array is determined based on the dimensions corresponding to the unitary matrix, the diagonal matrix, and the complex conjugate matrix. In the operation process of the photon calculation chip, the conventional convolution calculation can be realized through the set interferometer array, and the calculation times of the photon calculation chip can be reduced.
In the embodiment of the present application, the number of interferometers in the interferometer array corresponding to the unitary matrix is calculated based on the following formula: m is1N (n-1)/2, wherein m1The unitary matrix corresponds to the number of interferometers in the array of interferometers.
In the embodiment of the present application, the number of interferometers in the array of corresponding interferometers of the diagonal matrix is calculated based on the following formula: m is2(ii) 3n (3n-1)/2, wherein m is2The diagonal matrix corresponds to the number of interferometers in the interferometer array.
It should be noted that, in order to reduce the amount of calculation, if the dimension of the matrix implemented by the photon calculation chip in the embodiment of the present application is n × 3, the U matrix is n × n, and n (n-1)/2 interferometers are required. The Σ matrix is n × 3, 3n (3n-1)/2 interferometers are required, the V × matrix is 3 × 3, 3 interferometers are required, n is the number of convolution kernels, and the Interferometer in the embodiment of the present application is a Mach-Zehnder Interferometer (Mach-Zehnder Interferometer, abbreviation: MZI).
According to an aspect of embodiments of the present application, there is provided an embodiment of a method for convolution calculation based on a photon calculation chip. Fig. 3 is a flowchart of a convolution calculation method based on a photon calculation chip according to an embodiment of the present application, and as shown in fig. 3, the method includes:
step S11, obtaining a weight matrix, and loading the weight matrix to an interferometer array in the photon calculation chip.
The convolution calculation device comprises an optical signal input end and an optical detector output end, and the input end is connected with the output end through an interferometer array which is arranged in a cascade mode. The photonic computing chip may determine the received data sent by the requester device as target input data to be processed, where the target input data to be processed may be text data, image data, or the like.
Note that, since the matrix dimension of the weight matrix is m × n, it is necessary to insert weight values into the interferometer array in the optical neural network. The weight value may be a weight value preset by a developer. Specifically, convolution kernel parameters corresponding to the input layer may be obtained, corresponding weight values may be obtained based on the convolution kernel parameters, and a weight matrix may be constructed based on the weight values.
In the embodiment of the present application, obtaining the weight matrix includes the following steps a1-a 2:
step a1, extracting the ith row parameter from n m × m convolution kernels, and generating an m × n weight matrix corresponding to the ith row parameter based on the ith row parameter, where the number of the weight matrix corresponding to the ith row parameter is m.
And A2, loading the weight matrix corresponding to the ith row parameter to an interferometer array of the photon calculation chip, wherein m, n and i are integers which are more than or equal to 1.
Step S12, a target input vector to be processed is obtained, where the target input vector is obtained from the target feature map.
In the embodiment of the present application, in step S12, acquiring a target input vector to be processed includes the following steps B1-B3:
and step B1, acquiring a target characteristic diagram to be processed.
And step B2, analyzing the target characteristic diagram by using a preset sliding window to obtain m characteristic line vectors to be processed, wherein each characteristic line vector comprises m elements.
And step B3, extracting the characteristic line vector of the ith row, converting the characteristic line vector into a characteristic column vector corresponding to the ith row, and determining the characteristic column vector as a target input vector.
In this embodiment of the present application, the photon computing chip determines a preset sliding window during operation, where the length of the preset sliding window is the same as the number of convolution kernels in the output layer, for example, when the number of convolution kernels in the output layer is 3, the length of the preset sliding window is 3, and the width of the preset sliding window is 1. Extracting a plurality of characteristic row vectors from the target characteristic diagram by using a preset sliding window, for example: the target feature map comprises a plurality of feature vectors, the length of a preset sliding time window is 3, therefore, the preset sliding window is utilized to traverse each line of feature vectors, 3 feature vectors (elements) are extracted from the plurality of feature vectors according to the sequence, the 3 feature vectors (elements) are used for determining a first line of feature row vectors, and m feature row vectors are obtained until all feature vector extraction is completed.
As an example, when there are n convolution kernels, the feature K may be extracted from the target feature map through the n convolution kernels, convolution calculation is performed on the feature K, and finally m feature row vectors are obtained, and then each feature row vector is converted to obtain m feature column vectors, and the feature column vectors are determined as the target input vectors.
And step S13, inputting the target input vector to the input end, and performing linear calculation of the vector-weight matrix by using the target input vector through the input end to obtain an output vector.
In this embodiment, step S13, inputting the target input vector to the input end, and performing linear calculation of a vector-weight matrix by using the target input vector through the input end to obtain an output vector, includes the following steps C1-C2:
and step C1, inputting the characteristic column vector corresponding to the ith row into the input end, so that the input end and the weight matrix corresponding to the ith row perform linear multiplication to obtain an output vector corresponding to the ith row.
And step C2, repeatedly operating the photon calculation chip until m output vectors are obtained, wherein the output vectors comprise n elements.
In this embodiment of the application, when the photon computing chip is running, first, the weighted value of the 1 st line of n convolution kernels is loaded into the interferometer array of the photon computing chip, as shown in fig. 4, an input vector of the photon computing chip is a1 st line vector [ a11, a12, a13] in a preset sliding window, and an obtained matrix operation result is shown in a formula.
The line 1 result of the formula corresponds to the dot-product accumulation of the target input data with line 1 of the 1 st convolution kernel. The nth row result corresponds to the dot-product accumulation of the target input data and the 1 st row of the nth convolution kernel. The results of n lines are all valid, stored in the memory and called subsequently.
Next, as shown in fig. 5, the 2 nd row of data of n convolution kernels is loaded into the interferometer array of the photon computing chip, and the input vector of the photon computing chip is the 2 nd row [ a21, a22, a23] in the preset sliding window, and is computed according to [ a21, a22, a23] and the weight matrix, so as to obtain the computing result of the 2 nd row.
Finally, as shown in fig. 6, the 3 rd row of data of n convolution kernels is loaded into the interferometer array of the photon computing chip, the input vector of the photon computing chip is the 3 rd row [ a31, a32, a33] in the preset sliding window, and the calculation is performed according to [ a31, a32, a33] and the weight matrix, so as to obtain the calculation result of the 3 rd row.
And step S14, extracting elements belonging to the same row from each output vector, and summing the elements to obtain a convolution calculation result between the convolution kernel and the target feature map.
In this embodiment of the present application, in step S14, extracting elements belonging to the same row from each output vector, and summing the extracted elements to obtain a convolution calculation result between a convolution kernel and a target feature map, including the following steps D1-D2:
and D1, extracting elements belonging to the same row from the m output vectors, and summing to obtain output values corresponding to the m output vectors.
And D2, determining the output values corresponding to the m output vectors as convolution calculation results.
In the embodiment of the present application, after the output vector corresponding to each feature row vector is obtained, a plurality of elements belonging to the same row in the output vector of the row are accumulated and summed, so as to obtain an output value. And finally, calculating through the output vector corresponding to each characteristic row vector to obtain a convolution calculation result corresponding to the target characteristic graph. Therefore, the linear calculation of the vector and the weight matrix is realized by utilizing the photon calculation chip, the operation between the matrix and the matrix in the convolutional neural network can be simulated, the operation times of the photon calculation chip are reduced by optimizing the weight matrix extraction method, and the calculation efficiency is improved.
In the embodiment of the present application, after the interferometer array completes the calculation, the calculation result is transmitted to the output layer, and the output layer outputs the calculation result.
According to the embodiment of the application, the dimensionalities corresponding to the unitary matrix, the diagonal matrix and the complex conjugate matrix are determined according to the number of convolution kernels when a photon computing chip is designed, and the number of interferometers in the interferometer array corresponding to each of the unitary matrix, the diagonal matrix and the complex conjugate matrix is determined based on the dimensionalities corresponding to the unitary matrix, the diagonal matrix and the complex conjugate matrix. In the operation process of the photon computing chip, the conventional convolution calculation can be realized through the set interferometer array, the calculation times of the photon computing chip can be reduced, compared with the prior art, the calculation times of the photon computing chip in the embodiment of the application are 1/n of the calculation times of the prior art, and meanwhile, the calculation efficiency of the photon computing chip is improved.
In the embodiment of the present application, the method may also implement convolution kernel operations of other sizes, for example, n m × m convolution kernels, and then a photon calculation chip with n × m dimensions needs to be run m times, where a one-dimensional vector input by the photon calculation chip includes m elements. Specifically, when n is an integer multiple of m, the n × m dimensional photon counting chip is decomposed into n/m photon counting chips of m × m dimensions. The input vector of each photon computing chip is still m elements, but a beam splitting device is needed to be used, and m paths of input light are firstly split into 2m paths.
In the embodiment of the present application, the target input data graph may be traversed first, and then the convolution kernel is refreshed. For example, when the photon computation chip loads the 1 st line weight of N convolution kernels, let step be 1, and the input vector is [ a11, a12, a13], [ a12, a13, a14], [ a13, a14, a15] … [ aN (N-2), aN (N-1), aNN ] in this order. And after the input signal traverses, updating the photon computing chip to be the 2 nd row of the n convolution kernels.
An embodiment of the present application further provides an electronic device, as shown in fig. 7, the electronic device may include: a processor 1501, a communication interface 1502, a memory 1503 and a communication bus 1504, wherein the processor 1501, the communication interface 1502 and the memory 1503 complete communication with each other through the communication bus 1504.
A memory 1503 for storing a computer program;
the processor 1501 is configured to implement the steps of the above embodiments when executing the computer program stored in the memory 1503.
The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the terminal and other equipment.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In yet another embodiment provided by the present application, there is further provided a computer-readable storage medium having stored therein instructions, which when run on a computer, cause the computer to perform the convolution calculation method based on a photonic computing chip as described in any one of the above embodiments.
In yet another embodiment provided herein, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform the following method of convolution calculation based on a photonic computing chip, the method comprising:
acquiring a weight matrix, and loading the weight matrix to an interferometer array in a photon calculation chip;
acquiring a target input vector to be processed, wherein the target input vector is obtained from a target characteristic diagram;
inputting a target input vector to an input end, and performing linear calculation of a vector-weight matrix by using the target input vector through the input end to obtain an output vector;
and extracting elements belonging to the same row from each output vector, and summing to obtain a convolution calculation result between the convolution kernel and the target characteristic diagram.
Further, obtaining a weight matrix includes:
extracting ith row parameters from n m convolution kernels, and generating m × n weight matrixes corresponding to the ith row parameters based on the ith row parameters, wherein the number of the weight matrixes corresponding to the ith row parameters is m;
and loading the weight matrix corresponding to the ith row parameter to an interferometer array of the photon calculation chip, wherein m, n and i are integers which are more than or equal to 1.
Further, obtaining a target input vector to be processed includes:
acquiring a target characteristic diagram to be processed;
analyzing the target characteristic diagram by using a preset sliding window to obtain m characteristic line vectors to be processed, wherein each characteristic line vector comprises m elements;
and extracting the characteristic line vector of the ith line, converting the characteristic line vector into a characteristic column vector corresponding to the ith line, and determining the characteristic column vector as a target input vector.
Further, inputting a target input vector to the input end, and performing linear calculation of a vector-weight matrix by using the target input vector through the input end to obtain an output vector, including:
inputting the characteristic column vector corresponding to the ith row into an input end to enable the input end and the weight matrix corresponding to the ith row to carry out linear multiplication operation to obtain an output vector corresponding to the ith row;
and repeatedly operating the photon computing chip until m output vectors are obtained, wherein the output vectors comprise n elements.
Further, extracting elements belonging to the same row from each output vector for summation to obtain a convolution calculation result between a convolution kernel and the target feature map, including:
extracting elements belonging to the same row from the m output vectors, and summing to obtain output values corresponding to the m output vectors;
and determining output values corresponding to the m output vectors as convolution calculation results.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk), among others.
The above description is only for the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.
The above description is merely exemplary of the present application and is presented to enable those skilled in the art to understand and practice the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A convolution calculation method based on a photon calculation chip is characterized in that the method is suitable for a convolution calculation device of the photon calculation chip, the convolution calculation device comprises an optical signal input end and an optical detector output end, the input end and the output end are connected through an interferometer array which is arranged in a cascade mode, and the method comprises the following steps:
acquiring a weight matrix, and loading the weight matrix to an interferometer array in a photon computing chip;
acquiring a target input vector to be processed, wherein the target input vector is obtained from a target characteristic diagram;
inputting the target input vector to the input end, and performing linear calculation of a vector-weight matrix by using the target input vector through the input end to obtain an output vector;
and extracting elements belonging to the same row from each output vector, and summing to obtain a convolution calculation result between the convolution kernel and the target characteristic graph.
2. The method of claim 1, wherein obtaining a weight matrix comprises:
extracting ith row parameters from n m-m convolution kernels, and generating m-n weight matrixes corresponding to the ith row parameters based on the ith row parameters, wherein the number of the weight matrixes corresponding to the ith row parameters is m;
and loading the weight matrix corresponding to the ith row parameter to an interferometer array of a photon calculation chip, wherein m, n and i are integers which are more than or equal to 1.
3. The method of claim 1, wherein the obtaining a target input vector to be processed comprises:
acquiring a target characteristic diagram to be processed;
analyzing the target characteristic diagram by using a preset sliding window to obtain m characteristic line vectors to be processed, wherein each characteristic line vector comprises m elements;
and extracting a characteristic line vector of the ith line, converting the characteristic line vector into a characteristic column vector corresponding to the ith line, and determining the characteristic column vector as the target input vector.
4. The method of claim 3, wherein inputting the target input vector to the input end, and performing linear computation of a vector-weight matrix by using the target input vector through the input end to obtain an output vector comprises:
inputting the characteristic column vector corresponding to the ith row into the input end so as to enable the input end and the weight matrix corresponding to the ith row to perform linear multiplication operation to obtain an output vector corresponding to the ith row;
and repeatedly operating the photon computing chip until m output vectors are obtained, wherein the output vectors comprise n elements.
5. The method according to claim 4, wherein the extracting elements belonging to the same row from each output vector and summing the elements to obtain a convolution calculation result between the convolution kernel and the target feature map comprises:
extracting elements belonging to the same row from the m output vectors, and summing to obtain output values corresponding to the m output vectors;
and determining output values corresponding to the m output vectors as the convolution calculation result.
6. A convolution calculation apparatus based on a photon calculation chip, comprising: the optical signal input end and the optical detector output end are connected through an interferometer array which is arranged in a cascade mode, and a weight matrix is loaded in the interferometer array;
the input end is used for obtaining a target input vector and performing linear calculation of a vector-weight matrix by using the target input vector to obtain an output vector;
and the output end is used for extracting elements belonging to the same row from each output vector to sum so as to obtain a convolution calculation result between the convolution kernel and the target characteristic graph.
7. The apparatus of claim 6, wherein the weight matrix is generated according to a unitary matrix, a diagonal matrix and a complex conjugate matrix corresponding to the unitary matrix, wherein the dimension of the weight matrix is mxn, the dimension of the unitary matrix is mxm, the dimension of the diagonal matrix is mxn, and the dimension of the complex conjugate matrix is nxn.
8. The apparatus of claim 7, wherein the number of interferometers in the unitary matrix-corresponding interferometer array is calculated based on the following formula:
m1n (n-1)/2, wherein m1The unitary matrix corresponds to the number of interferometers in the array of interferometers.
9. The apparatus of claim 7, wherein the number of interferometers in the array of diagonal matrix corresponding interferometers is calculated based on the formula:
m23n (3n-1)/2, wherein m is2The diagonal matrix corresponds to the number of interferometers in the interferometer array.
10. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus; wherein:
a memory for storing a computer program;
a processor for performing the method steps of any of claims 1 to 5 by executing a program stored on a memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210475719.5A CN114723019A (en) | 2022-04-29 | 2022-04-29 | Convolution calculation method and device based on photon calculation chip |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210475719.5A CN114723019A (en) | 2022-04-29 | 2022-04-29 | Convolution calculation method and device based on photon calculation chip |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114723019A true CN114723019A (en) | 2022-07-08 |
Family
ID=82246546
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210475719.5A Pending CN114723019A (en) | 2022-04-29 | 2022-04-29 | Convolution calculation method and device based on photon calculation chip |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114723019A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117850726A (en) * | 2024-03-08 | 2024-04-09 | 深圳市光科全息技术有限公司 | Method, system and equipment for carrying out convolution calculation by utilizing DOE photonic crystal film |
| WO2024168514A1 (en) * | 2023-02-14 | 2024-08-22 | 北京大学 | Data processing method and apparatus applied to in-memory computing chip, and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210036783A1 (en) * | 2019-07-29 | 2021-02-04 | Lightmatter, Inc. | Systems and methods for analog computing using a linear photonic processor |
| CN113592069A (en) * | 2021-07-20 | 2021-11-02 | 东南大学 | Photon neural network for four-input logic operation |
| CN113961035A (en) * | 2021-10-15 | 2022-01-21 | 北京理工大学 | Deep learning-oriented high-efficiency reconfigurable all-optical neural network computing chip architecture |
| CN114037070A (en) * | 2022-01-07 | 2022-02-11 | 苏州浪潮智能科技有限公司 | Optical signal processing method, photonic neural network chip and design method thereof |
-
2022
- 2022-04-29 CN CN202210475719.5A patent/CN114723019A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210036783A1 (en) * | 2019-07-29 | 2021-02-04 | Lightmatter, Inc. | Systems and methods for analog computing using a linear photonic processor |
| CN113592069A (en) * | 2021-07-20 | 2021-11-02 | 东南大学 | Photon neural network for four-input logic operation |
| CN113961035A (en) * | 2021-10-15 | 2022-01-21 | 北京理工大学 | Deep learning-oriented high-efficiency reconfigurable all-optical neural network computing chip architecture |
| CN114037070A (en) * | 2022-01-07 | 2022-02-11 | 苏州浪潮智能科技有限公司 | Optical signal processing method, photonic neural network chip and design method thereof |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024168514A1 (en) * | 2023-02-14 | 2024-08-22 | 北京大学 | Data processing method and apparatus applied to in-memory computing chip, and device |
| CN117850726A (en) * | 2024-03-08 | 2024-04-09 | 深圳市光科全息技术有限公司 | Method, system and equipment for carrying out convolution calculation by utilizing DOE photonic crystal film |
| CN117850726B (en) * | 2024-03-08 | 2024-06-21 | 深圳市光科全息技术有限公司 | Method, system and equipment for carrying out convolution calculation by utilizing DOE photonic crystal film |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108701250B (en) | Data fixed-point method and device | |
| CN114723019A (en) | Convolution calculation method and device based on photon calculation chip | |
| US11734553B2 (en) | Error allocation format selection for hardware implementation of deep neural network | |
| CN112085172B (en) | Method and device for training graph neural network | |
| US20240346314A1 (en) | End-to-end data format selection for hardware implementation of deep neural network | |
| GB2568083A (en) | Histogram-based per-layer data format selection for hardware implementation of deep neutral network | |
| US11580194B2 (en) | Information processing apparatus, information processing method, and program | |
| US20190236436A1 (en) | Hierarchical Mantissa Bit Length Selection for Hardware Implementation of Deep Neural Network | |
| CN111052111A (en) | Arithmetic circuit, arithmetic method, and program | |
| CN110337636A (en) | Data transfer device and device | |
| CN112541575A (en) | Method and device for training graph neural network | |
| CN112835551A (en) | Data processing method for processing unit, electronic device, and computer-readable storage medium | |
| CN113609445A (en) | Multi-source heterogeneous monitoring data processing method, terminal device and readable storage medium | |
| CN114974421A (en) | Single-cell transcriptome sequencing data interpolation method and system based on diffusion-noise reduction | |
| CN115952391A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN110009091B (en) | Optimization of learning network in equivalence class space | |
| CN116166217A (en) | System and method for performing floating point operations | |
| JP2020067897A (en) | Arithmetic processing unit, learning program, and learning method | |
| US20220180171A1 (en) | Four-bit training for machine learning | |
| CN111275308B (en) | Resource allocation method and device | |
| CN112667905A (en) | Behavior event sequence generation method, device, equipment and storage medium | |
| CN114266269A (en) | Bearing fault diagnosis method and system, storage medium and equipment | |
| CN114549945A (en) | Remote sensing image change detection method and related device | |
| CN110059757A (en) | Classification method, device and the electronic equipment of mixed signal | |
| CN117495571B (en) | Data processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |