CN117200929A - Optical device for two-dimensional convolution processing of multichannel video stream - Google Patents
Optical device for two-dimensional convolution processing of multichannel video stream Download PDFInfo
- Publication number
- CN117200929A CN117200929A CN202311071985.2A CN202311071985A CN117200929A CN 117200929 A CN117200929 A CN 117200929A CN 202311071985 A CN202311071985 A CN 202311071985A CN 117200929 A CN117200929 A CN 117200929A
- Authority
- CN
- China
- Prior art keywords
- signals
- optical
- wavelength
- video stream
- wavelength division
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003287 optical effect Effects 0.000 title claims abstract description 81
- 238000004364 calculation method Methods 0.000 claims abstract description 15
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000003491 array Methods 0.000 claims description 2
- 239000012782 phase change material Substances 0.000 claims description 2
- 230000008901 benefit Effects 0.000 abstract description 5
- 238000000034 method Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The optical device for two-dimensional convolution processing of the multichannel video stream adopts a photoelectric cooperative mode, and realizes large-scale two-dimensional calculation of the multichannel video stream by means of front-end electrical shift copying or optical copying and electrical back-end shift. Meanwhile, the device utilizes the advantage of ultra-high parallelization of optical computation, and the computation of the single computing platform to the multi-channel video stream is realized in a wavelength multiplexing mode.
Description
Technical Field
The invention relates to the field of optical computation, in particular to an optical device for two-dimensional convolution processing of a multichannel video stream.
Background
With the rapid development of sensor networks, mobile internet, edge data processing and other technologies, a large amount of data is continuously generated and transmitted in various application fields in a streaming media form. Therefore, there is increasing interest in how to reasonably handle large amounts of stream data in different situations. In particular, the requirements of low time delay and high flux in applications such as traffic video processing and vehicle-mounted video recognition in intelligent driving in urban management at present are considered. The method is characterized in that multipath video stream data are processed on a distributed computing platform, so that the identification, tracking and searching of motor vehicles, non-motor vehicles and pedestrians at multiple intersections are realized, and the problems which are needed to be solved at present are solved. In order to meet urgent demands for processing multi-channel video stream data, a plurality of computing architectures exist in the market, but the traditional data stream analysis has the problems of high time delay, weak expansibility, poor adaptability and the like in the process of streaming data, so that the increasingly-large data volume demands are difficult to meet. For this reason, it is urgently required to introduce new stream data processing means.
In this context, light computing is an effective way to break through the bottleneck of electronic technology. Optical computing has been rapidly developed in recent years, and is most characterized by no energy consumption and an ultra-high time rate in data transmission, compared with electrical computing, so that it has advantages of high speed, large bandwidth, low power consumption, and the like. These advantages facilitate acceleration of multi-channel video stream data processing. Meanwhile, the core of the multi-channel video stream data, such as traffic video processing, vehicle-mounted video recognition and the like, has similarity with the algorithm required to be calculated. The optical computation has the advantage of high parallelization, and can realize multi-dimensional multiplexing of time, space, wavelength, mode and the like, so that the optical computation can realize real-time computation through multiplexing when processing multi-channel video stream data. However, the work of combining optical computing with applications, especially with multi-channel video streams, is extremely missing, and few work is focused on solving the problems encountered by optical computing when applied. However, from the development point of view of optical computing technology and multi-channel video stream processing technology, it is extremely important to apply optical computing to multi-channel video streams. Therefore, an optical architecture for multi-channel video streams is imperative.
Patent document CN114841334a discloses an optical computing device, system and convolution computing method, specifically, data are loaded onto different optical wavelengths, different weights are loaded by different modulators through a wavelength routing method, and finally, convolution computation is finally completed through addition and addition of different wavelengths, however, the convolution processing input data size realized by the scheme is limited to the number of wavelengths and the complexity of a wavelength router, when standard high definition video 1920×1080 resolution is processed, the number of wavelengths is more than 2 million, and the complexity of the wavelength router is in a square relationship, which cannot be realized.
Patent document CN109639359a discloses a scheme for performing convolution operation by using a micro-ring resonator, performing multiplication operation by using the micro-ring resonator, and performing addition operation by incoherent light intensity accumulation, so as to further implement convolution operation, however, the optical path structure of the scheme can only implement one-dimensional convolution, and when two-dimensional convolution is performed, the two-dimensional convolution needs to be decomposed into one-dimensional convolution in a computer to perform sequential operation, thereby greatly reducing the execution efficiency of the scheme. The video stream processing is typically a two-dimensional convolution processing, and thus the scheme disclosed in this document is not optimally adapted to the video stream processing.
Disclosure of Invention
The object of the present invention is to address the above-mentioned deficiencies of the prior art by proposing an optical device for convolutionally processing a multi-channel video stream. The device adopts a photoelectric cooperative mode, realizes the calculation of the multichannel video stream through wavelength multiplexing, and simultaneously realizes the two-dimensional calculation of video stream data through front-end electrical shift copying or optical copying and data copying of electrical rear-end shift (hereinafter respectively called as 'front storage' and 'rear storage').
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the invention provides an optical device for two-dimensional convolution processing of multichannel video streams in a 'pre-storage' configuration, which is used for processing N multiplied by M video stream output signal sources and is characterized by comprising the following components:
the front-end electrical processor array consists of M electrical processors and is used for integrating N video stream signals (single signal size A multiplied by B, A > B) into a group of signals NA multiplied by B in a way of connecting short sides in sequence, so that N multiplied by M video stream output signal sources are integrated into M paths of signals, and each path of signal is duplicated to output M multiplied by K groups of signals, wherein each electrical processor outputs K groups of signals, the K groups of signals are respectively the 1 st row to the (B-K+1) th row, and the 2 nd row to the (B-K) th row … K group signals are the K th row to the last row of original data;
the light source array consists of M light sources with different wavelengths, and each light source is equally divided into K parts;
the modulator array is composed of M multiplied by K modulators and is used for modulating the electric signals to light, wherein each K modulators are in a group, the M groups of modulators are respectively connected with light sources with different wavelengths and an electrical processor, namely, each modulator in each group of modulators is respectively connected with the corresponding light source and the electrical processor;
the wavelength division multiplexer array consists of K wavelength division multiplexers, and each wavelength division multiplexer is respectively connected with a corresponding modulator in each group of modulators and is used for combining light with different wavelengths together to realize wavelength multiplexing;
the optical computing device array consists of K optical computing devices, and each optical computing device comprises a series of optical computing operations such as matrix multiplication, convolution computation and the like and can process the input M wavelength signals simultaneously. K optical computation devices are connected to different wavelength division demultiplexers, respectively.
A wavelength-division-demultiplexer array composed of K wavelength-division demultiplexes M different wavelength optical signals in each optical computation transpose. Each wavelength division demultiplexer is connected with M photodetectors respectively.
The photoelectric detector array consists of M multiplied by K photoelectric detectors. For converting the processed optical signal into an electrical signal. And transmits K sets of signals belonging to the same wavelength to a back-end electrical processor. Every K groups of photoelectric detectors are respectively connected with 1 rear-end electrical processor
The rear-end electrical processor array consists of M electrical processors, and the two-dimensional convolution operation with the size of J multiplied by K is realized by directly adding the signals of the same wavelength output by the K photoelectric detectors.
Meanwhile, the invention also provides an optical device for two-dimensional convolution processing of multi-channel video streams in a 'post-storage' configuration, which is used for processing N multiplied by M video stream output signal sources and is characterized by comprising:
the front electrical processor array consists of M electrical processors and is used for integrating N video stream signals (single signal size A multiplied by B, A > B) into a group of signals NA multiplied by B in a mode of connecting short sides in sequence, so that N multiplied by M video stream output signal sources are integrated into M paths of signal outputs;
the light source array consists of M light sources with different wavelengths;
the modulator array consists of M modulators, corresponds to M paths of signals output by the pre-electric processor array, and copies K paths of signals after each path of signals are modulated on light; the modulator is connected with the wavelength division multiplexer and corresponds to the K wavelength division multiplexers;
the wavelength division multiplexer array consists of K wavelength division multiplexers, and each wavelength division multiplexer is respectively connected with M modulators;
an array of light computing devices, consisting of K light computing devices. Each optical computing device comprises a series of optical computing operations such as matrix multiplication, convolution computation and the like, and can process the input M wavelength signals simultaneously. K optical computation devices are connected to different wavelength division demultiplexers, respectively.
A wavelength-division-demultiplexer array composed of K wavelength-division demultiplexes M different wavelength optical signals in each optical computation transpose. Each wavelength division demultiplexer is connected with M photodetectors respectively.
The photoelectric detector array consists of M multiplied by K photoelectric detectors. For converting the processed optical signal into an electrical signal. And transmits K sets of signals belonging to the same wavelength to a back-end electrical processor. Every K groups of photoelectric detectors are respectively connected with 1 rear-end electrical processor
The rear-end electrical processor array consists of M electrical processors, and the same wavelength signals output by the K photoelectric detectors are processed according to the following operations: the first set of signals selects the first through (B-K+1) th rows, the second set of signals selects the second through (B-K) th rows …, and the K-th through last rows. And then adding the processed K groups of signals to realize two-dimensional convolution operation.
Further, the array of optical computing devices of the present invention can optically perform convolution operations. The internal modulation devices may include micro-rings, mach-Zehnder modulators, mach-Zehnder interferometers, phase change materials, and the like.
The front-end and the back-end electrical processor arrays can integrate and rearrange two-dimensional data. The main functions are as follows: n signals (single signal size A multiplied by B, A > B) are integrated into a group of signals (NA multiplied by B) in a way of connecting the short sides in sequence; the signals of the (NA multiplied by B) are selected according to the K row to the K+L row (K, L is more than or equal to 0) and are integrated into 1 row according to the head-tail connection mode of each row; dividing one-dimensional data into a plurality of one-dimensional data according to a selected size and rearranging the one-dimensional data into two-dimensional data.
The invention has the technical advantages that:
(1) By integrating the multi-channel video stream data in the electrical processor, the photoelectric data processing speed matching is realized. The problem of excessive computing power resources caused by high instantaneous computing power of the optical computing device is relieved.
(2) The invention breaks through the limitation that photon convolution calculation can only realize one-dimensional calculation by designing a two-stage storage architecture (a front storage architecture or a rear storage architecture is two-stage storage architecture), so that two-dimensional data of video stream can be subjected to running water type calculation without decomposition, and the invention has important significance for vehicle identification and tracking search required by video data processing.
(3) The input data size of the invention only depends on the data size of the video stream, is not limited by the optical computing system, and provides an adaptation scheme for video stream processing.
Drawings
Fig. 1 is a schematic diagram of an embodiment of an optical computing device for multi-channel video stream processing in pre-use storage according to the present invention, showing one possible device connection scheme when n=4, m=3, k=3, and b=2080.
Fig. 2 is a schematic diagram of an embodiment of the optical computing device for multi-channel video stream processing after use according to the present invention, showing one possible device connection scheme when n=4, m=3, k=3, and b=2080.
FIG. 3 illustrates an example graph of data partitioning during front storage and an example graph of data stacking during back storage.
Detailed Description
The following describes the technical solution of the present invention in detail with reference to the drawings and examples, and gives detailed embodiments and structures, but the scope of protection of the present invention is not limited to the examples described below. The "left" and "right" described below are all based on the placement positions described in the figures, and do not describe the positions of the actual systems, which are guaranteed to be in the same order as the connection of the figures.
An optical device for multi-channel video stream processing includes two architectural schemes (front storage architecture and rear storage architecture). In the "pre-storage" configuration, the apparatus includes n×m video stream output signal sources, and N video stream signals (single signal sizes a×b, a > B) are integrated into a group of signals (na×b) in a manner of sequentially connecting short sides in M electrical processors, respectively, so that n×m input signal source signals are integrated into M signals. The electrical processor copies and outputs the integrated signals K times, wherein the first group of signals output by the electrical processor are the first row to the (B-K+1) th row of the integrated signals, the second group of signals are the second row to the (B-K) th row …, and the K group of signals are the K row to the last row. The same path of integrated signals are modulated onto light with the same wavelength, and the K paths of integrated signals are transmitted onto different paths of K paths according to the copied K paths of integrated signals, the integrated signals are combined with light with other wavelengths through a wavelength division multiplexer, and then convolution calculation is carried out on the K optical calculation devices respectively, so that one-dimensional convolution operation with the length of J is realized. After convolution calculation, the convolution is decomposed into different wavelengths by a wavelength division multiplexer and converted to the electrical domain by a photodetector. And directly adding the K groups of signals copied by the same integrated signal after convolution on a back-end electrical processor, thereby realizing two-dimensional convolution operation with the size of J multiplied by K. In a "post-storage" configuration, the device comprises n×m video stream output signal sources, and N video stream signals (single signal sizes a×b, a > B) are integrated into a group of signals (na×b) in M electrical processors, respectively, in a manner of sequentially connecting short sides, thereby integrating n×m input signal source signals into M paths of signals. The electrical processor modulates the integrated M paths of signals to M paths of light with different wavelengths, copies K paths of identical signals on the light, transmits the signals to K paths of different light paths, combines the signals with the light with other wavelengths through the wavelength division multiplexer, and then carries out convolution calculation on the K optical calculation devices respectively to realize one-dimensional convolution operation with the length J. After convolution calculation, the convolution is decomposed into different wavelengths by a wavelength division multiplexer and converted to the electrical domain by a photodetector. The K groups of signals copied from the same integrated signal after convolution are processed on the back-end electrical processor as follows: the first set of signals selects the first through (B-K-1) th rows, the second set of signals selects the second through (B-K) th rows …, and the K-th through last rows. The processed K sets of signals are then summed to achieve a two-dimensional convolution operation of size jxk.
The device comprises N x M video stream output signal sources, M electrical processors, K optical computing devices, K groups of wavelength division multiplexers, K groups of wavelength division demultiplexers and K x M photodetectors, M light sources, M x K modulators (in a 'pre-storage' configuration) or M modulators (in a 'post-storage' configuration), and a back-end electrical processor.
Referring to fig. 1, when n=4, m=3, k=3, b=2080, the optical device employing multi-channel video stream processing in the "pre-storage" configuration comprises 4×3=12 video stream output signal sources, 3 electrical processors, 3 optical computing devices, 3 sets of wavelength division multiplexers, 3 sets of wavelength division demultiplexers and 3×3=9 photodetectors, 3 light sources, 3×3=9 modulators. The specific connection modes and functions of the devices are described as follows:
first, the signals are output from the video stream output signal source 110, and the output ports are connected to the electrical processor array 120, and in the embodiment, each 4 cameras 111 form a set of output signals to one electrical processor 121. The output ports of the electrical processor array 120 are connected to the input ports of the modulator array 140, and in the embodiment, 3 output ports of each electrical processor 121 are respectively connected to the electrical input ports of 3 modulators 141, and the output 3 ports are respectively output as the 1 st to 2078 th, 2 nd to 2079 th and 3 rd to 2080 th rows of the integrated video stream in the order from top to bottom. The array of light sources 130 is used to generate a sequence of signal-bearing light, each light source 131 outputting light of a wavelength different from each other, in this embodiment 3 different wavelengths. The light power generated by the light source 131 is according to 1:1: the 1 scale is replicated to 3 parts using an optical splitter, and is connected to the optical input ports of the 3 modulators 141, respectively. An output port of the modulator 141 is connected to an input port of a wavelength division multiplexer 151 in the wavelength division multiplexer array 150. The output port of the wavelength division multiplexer 151 is connected to the input port of the optical computing device array 160, wherein the optical computing device 161 may be a convolution kernel computing unit or a matrix multiplication computing unit, etc. The output ports of the array of optical computation devices 160 are connected to an array of wavelength division demultiplexers 170, which wavelength division demultiplexers 171 divide the light into different optical paths according to different wavelengths, in the embodiment 3 paths. The output port of the wavelength demultiplexer array 170 is connected to the input port of the photodetector array 180, demultiplexes the optical signal into different wavelength signals and converts to the electrical domain using the photodetectors 181. The 3 sets of signals copied from the same integrated signal after convolution are directly summed on the back-end electrical processor 190 to achieve a two-dimensional convolution operation.
Referring to fig. 2, when n=4, m=3, k=3, and b=2080, the optical device for multi-channel video stream processing in the "post-storage" configuration includes 4×3=12 video stream output signal sources, 3 electrical processors, 3 optical computing devices, 3 sets of wavelength division multiplexers, 3 sets of wavelength division demultiplexers, and 3×3=9 photodetectors, 3 light sources, and 3 modulators. The specific connection modes and functions of the devices are also described as follows:
similar to the "pre-store", first the signals are output by a video stream output signal source 210, the output ports are connected 220 to an array of electrical processors, in the embodiment each 4 cameras 211 forming a set of output signals to one electrical processor 221. The output ports of the electrical processor array 220 are connected to the input ports of the modulator array 240, and in the embodiment each electrical processor 221 is connected to the electrical input ports of 1 modulator 241. The array of light sources 230 is used to generate signal-bearing light, each light source 231 outputting light having a wavelength different from each other, in this embodiment 3 different wavelengths. The light source 231 generates a light train that is coupled to the light input ports of 1 modulator 241. An output port of the modulator 241 is connected to an input port of a wavelength division multiplexer 251 in the wavelength division multiplexer array 250. The output port of the wavelength division multiplexer 251 is connected to the input port of the optical computing device array 260, wherein the optical computing device 261 may be a convolution kernel computing unit or a matrix multiplication computing unit, etc. The output port of the array of optical computation means 260 is connected to an array of wavelength division demultiplexers 270, wherein the wavelength division demultiplexers 271 divide the light into different optical paths according to different wavelengths, in the embodiment 3. The output port of the wavelength-division-demultiplexer array 270 is connected to the input port of the photodetector array 280, converting the signals to the electrical domain. The 3 sets of signals duplicated on the back-end electrical processor for the same integrated signal after convolution are processed as follows: the first group of signals selects rows 1 through 2078, the second group of signals selects rows 2 through 2079, and the 3 group of signals selects rows 3 through last. And then adding the processed 3 groups of signals, thereby realizing two-dimensional convolution operation.
Fig. 3 shows a specific operation procedure of data division in the front storage process and data superposition in the rear storage process when n=4, m=3, k=3, and b=2080. Wherein fig. 3 (a) is a process of dividing data into different groups in the pre-storing process, in an embodiment, into 3 groups, the first group is the 1 st to 2078 th rows of the picture, the second group is the 2 nd to 2079 th rows, and the third group is the 3 rd to 2080 th rows. And fig. 3 (b) is a diagram showing a process of data superimposition in the post-storage process. The data passing through different optical one-dimensional computing cores are rearranged into a two-dimensional picture matrix, and then the 1 st row to the 2078 st row, the 2 nd row to the 2079 th row and the 3 rd row to the 2080 th row of the 3 groups of signals are selected respectively for superposition. Thereby realizing a two-dimensional convolution process.
Claims (4)
1. An optical device for two-dimensional convolution processing of a multi-channel video stream for processing N x M video stream output signal sources, comprising:
the front-end electrical processor array consists of M electrical processors and is used for integrating N video stream signals (the single signal size A multiplied by B, A is larger than B) into a group of signals NA multiplied by B in a way of connecting the short sides in sequence, so that N multiplied by M video stream output signal sources are integrated into M paths of signals, and each path of signal is duplicated to output M multiplied by K groups of signals, wherein each electrical processor outputs K groups of signals, the K groups of signals are respectively the 1 st row to the (B-K+1) th row, and the 2 nd row to the (B-K) th column row and the K group of signals are the K row to the last row;
the light source array consists of M light sources with different wavelengths, and each light source is equally divided into K parts;
the modulator array is composed of M multiplied by K modulators and is used for modulating the electric signals to light, wherein each K modulators are in a group, the M groups of modulators are respectively connected with light sources with different wavelengths and an electrical processor, namely, each modulator in each group of modulators is respectively connected with the corresponding light source and the electrical processor;
the wavelength division multiplexer array consists of K wavelength division multiplexers, and each wavelength division multiplexer is respectively connected with a corresponding modulator in each group of modulators and is used for combining light with different wavelengths together to realize wavelength multiplexing;
the optical computing device array consists of K optical computing devices, and each optical computing device is respectively connected with a corresponding wavelength division demultiplexer and is used for realizing matrix multiplication and convolution computing optical computing operation and processing the input M wavelength signals;
the wavelength division multiplexing device comprises a wavelength division multiplexing device array and a plurality of optical calculation transposes, wherein the wavelength division multiplexing device array consists of K wavelength division multiplexing devices and is used for demultiplexing M different wavelength optical signals in each optical calculation transpose, and each wavelength division multiplexing device is respectively connected with M photoelectric detectors.
The photoelectric detector array is composed of M x K photoelectric detectors. For converting the processed optical signals into electrical signals and transmitting the signals belonging to the same wavelength K groups to a rear-end electrical processor, wherein each K groups of photodetectors are respectively connected with 1 rear-end electrical processor
The rear-end electrical processor array consists of M electrical processors, and the two-dimensional convolution operation with the size of J multiplied by K is realized by directly adding the signals of the same wavelength output by the K photoelectric detectors.
2. An optical device for two-dimensional convolution processing of a multi-channel video stream for processing N x M video stream output signal sources, comprising:
the front electrical processor array consists of M electrical processors and is used for integrating N video stream signals (single signal size A multiplied by B, A > B) into a group of signals NA multiplied by B in a mode of connecting short sides in sequence, so that N multiplied by M video stream output signal sources are integrated into M paths of signal outputs;
the light source array consists of M light sources with different wavelengths;
the modulator array consists of M modulators, corresponds to M paths of signals output by the pre-electric processor array, and copies K paths of signals after each path of signals are modulated on light; the modulator is connected with the wavelength division multiplexer and corresponds to the K wavelength division multiplexers;
the wavelength division multiplexer array consists of K wavelength division multiplexers, and each wavelength division multiplexer is respectively connected with M modulators;
the optical computing device array consists of K optical computing devices, and each optical computing device is respectively connected with a corresponding wavelength division demultiplexer and is used for realizing matrix multiplication and convolution computing optical computing operation and processing the input M wavelength signals;
a wavelength-division-demultiplexer array composed of K wavelength-division demultiplexes M different wavelength optical signals in each optical computation transpose. Each wavelength division demultiplexer is connected with M photodetectors respectively.
The photoelectric detector array consists of M multiplied by K photoelectric detectors and is used for converting the processed optical signals into electric signals and transmitting K groups of signals belonging to the same wavelength to a rear-end electric processor. Each K groups of photoelectric detectors are respectively connected with 1 rear-end electrical processor;
the rear-end electrical processor array consists of M electrical processors, and the same wavelength signals output by the K photoelectric detectors are processed according to the following operations: the first set of signals selects the first through (B-K+1) th rows, the second set of signals selects the second through (B-K) th rows …, and the K-th through last rows. And then adding the processed K groups of signals to realize two-dimensional convolution operation.
3. The optical device for two-dimensional convolution processing of a multi-channel video stream according to claim 1 or 2, wherein: the optical computing device array can realize convolution operation on light, and the internal modulation device is a micro-ring, a Mach-Zehnder modulator, a Mach-Zehnder interferometer or a phase change material.
4. The optical device for two-dimensional convolution processing of a multi-channel video stream according to claim 1 or 2, wherein: the front-end and back-end electrical processor arrays can integrate and rearrange two-dimensional data, and can integrate N signals (the single signal size A multiplied by B, A is larger than B) into a group of signals (NA multiplied by B) in a mode of connecting short sides in sequence; the signals of the (NA multiplied by B) are selected according to the K row to the K+L row (K, L is more than or equal to 0) and are integrated into 1 row according to the head-tail connection mode of each row; dividing one-dimensional data into a plurality of one-dimensional data according to a selected size and rearranging the one-dimensional data into two-dimensional data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311071985.2A CN117200929A (en) | 2023-08-24 | 2023-08-24 | Optical device for two-dimensional convolution processing of multichannel video stream |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311071985.2A CN117200929A (en) | 2023-08-24 | 2023-08-24 | Optical device for two-dimensional convolution processing of multichannel video stream |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117200929A true CN117200929A (en) | 2023-12-08 |
Family
ID=88982736
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311071985.2A Pending CN117200929A (en) | 2023-08-24 | 2023-08-24 | Optical device for two-dimensional convolution processing of multichannel video stream |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117200929A (en) |
-
2023
- 2023-08-24 CN CN202311071985.2A patent/CN117200929A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200019851A1 (en) | Optical convolutional neural network accelerator | |
| WO2020102204A1 (en) | Large-scale artificial neural-network accelerators based on coherent detection and optical data fan-out | |
| CN103678258B (en) | Method for improving data resolution ratio of silica-based optical matrix processor | |
| CN114819132B (en) | Photon two-dimensional convolution acceleration method and system based on time-wavelength interleaving | |
| CN114815959B (en) | Photon tensor calculation acceleration method and device based on wavelength division multiplexing | |
| US20230385627A1 (en) | Optical Computing Apparatus and System, and Convolution Computing Method | |
| Totovic et al. | WDM equipped universal linear optics for programmable neuromorphic photonic processors | |
| CN102147634B (en) | Optical vector-matrix multiplier based on single-waveguide coupling micro-ring resonant cavity | |
| Hu et al. | Batch processing and data streaming fourier-based convolutional neural network accelerator | |
| US20220414442A1 (en) | Optical computing apparatus and system, and computing method | |
| CN117077750A (en) | Parallel photon convolution operation chip and system | |
| CN117200929A (en) | Optical device for two-dimensional convolution processing of multichannel video stream | |
| CN117749275A (en) | Multichannel orbital angular momentum optical communication system and method based on Dammann grating | |
| Shi et al. | Parallel photonic convolutional processing on-chip with cross-connect architecture and cyclic AWGs | |
| Desmulliez et al. | Perfect-shuffle interconnected bitonic sorter: optoelectronic design | |
| CN115130666B (en) | Two-dimensional photon convolution acceleration method and system | |
| CN116258624A (en) | Optical tensor convolution computing system and method based on multi-imaging projection architecture | |
| CN110247728B (en) | Photon neural network chip | |
| CN111949067B (en) | Dammann convolution optical computer | |
| CN115358366A (en) | Acceleration system of convolution operation layer in optical neural network | |
| Zhang et al. | Time-stretch optical neural network with time-division multiplexing | |
| CN117709423B (en) | Deep neural network photon acceleration chip and operation system thereof | |
| RU2433445C1 (en) | Optical computing device for calculating function difference | |
| JP2778176B2 (en) | Optical arithmetic unit | |
| Zheng et al. | High-throughput optical neural networks based on temporal computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |